{"id":769,"date":"2023-02-07T20:00:00","date_gmt":"2023-02-07T19:00:00","guid":{"rendered":"https:\/\/www.anginf.de\/?p=769"},"modified":"2023-02-08T13:02:12","modified_gmt":"2023-02-08T12:02:12","slug":"image-tricks-with-stable-diffusion-and-clip-interrogator","status":"publish","type":"post","link":"https:\/\/www.anginf.de\/?p=769","title":{"rendered":"Image tricks with stable diffusion and clip-interrogator"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">When I got <a href=\"https:\/\/www.anginf.de\/?p=765\" data-type=\"post\" data-id=\"765\">stable-diffusion to work<\/a>, I really wanted to see what the AI would do on its own, letting it describe a given picture and re-create the picture from this description. How detailed can or should a description be?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">First install the &#8222;clip-interrogator&#8220;:<\/p>\n\n\n\n<pre class=\"wp-block-code has-dark-gray-color has-text-color\"><code lang=\"bash\" class=\"language-bash\">python3 -m pip install --upgrade pip setuptools wheel\nsudo apt install -y rustc cargo\npip install clip-interrogator\n<\/code><\/pre>\n\n\n\n<p class=\"wp-block-paragraph\">And here&#8217;s the sample python file. I got an error when I didn&#8217;t reassign\/clear the &#8222;pipe&#8220;.<\/p>\n\n\n\n<pre class=\"wp-block-code has-dark-gray-color has-text-color\"><code lang=\"python\" class=\"language-python\">import torch\nfrom torch import autocast\nfrom diffusers import StableDiffusionPipeline\nfrom PIL import Image\nfrom clip_interrogator import Config, Interrogator\n\nprint(\"### Starting Stable Diffusion Pipeline\")\npipe = StableDiffusionPipeline.from_pretrained(\"CompVis\/stable-diffusion-v1-4\")\npipe.to(\"cuda\")\n\nprompt = \"a dark environment, two warriors standing on a chess field with swords drawn\"\n# prompt = \"a laptop sitting on top of a wooden table\"\nsteps = 50\nwidth = 512\nheight = 512\n\nprint(\"### Creating images with Stable Diffusion\")\nwith autocast(\"cuda\"):\n  for i in range(1):\n    output = pipe(prompt, width=width, height=height, num_inference_steps=steps)\n    image = output[\"images\"][0]\n    file = prompt.replace(\" \", \"_\").replace(\",\", \"\")\n    image.save(f\"{file}-{i}.png\")\n\npipe = \"\" # Destroy?\nprint(\"### Initializing Interrogator\")\nci = Interrogator(Config(clip_model_name=\"ViT-L-14\/openai\"))\nfile = prompt.replace(\" \", \"_\").replace(\",\", \"\")\nfor i in range(1):\n  print(\"Loading file \", i)\n  image = Image.open(f\"{file}-{i}.png\").convert('RGB')\n  print(ci.interrogate(image))<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"<p>When I got stable-diffusion to work, I really wanted to see what the AI would do on its own, letting it describe a given picture and re-create the picture from this description. How detailed can or should a description be? First install the &#8222;clip-interrogator&#8220;: And here&#8217;s the sample python file. I got an error when &hellip; <a href=\"https:\/\/www.anginf.de\/?p=769\" class=\"more-link\"><span class=\"screen-reader-text\">Image tricks with stable diffusion and clip-interrogator<\/span> weiterlesen <span class=\"meta-nav\">&rarr;<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-769","post","type-post","status-publish","format-standard","hentry","category-allgemein"],"_links":{"self":[{"href":"https:\/\/www.anginf.de\/index.php?rest_route=\/wp\/v2\/posts\/769","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.anginf.de\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.anginf.de\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.anginf.de\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.anginf.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=769"}],"version-history":[{"count":4,"href":"https:\/\/www.anginf.de\/index.php?rest_route=\/wp\/v2\/posts\/769\/revisions"}],"predecessor-version":[{"id":787,"href":"https:\/\/www.anginf.de\/index.php?rest_route=\/wp\/v2\/posts\/769\/revisions\/787"}],"wp:attachment":[{"href":"https:\/\/www.anginf.de\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=769"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.anginf.de\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=769"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.anginf.de\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=769"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}