I'm sorry but , "unlock your face with your phone" just cracked me up..

I've been using Stable Diffusion to _deCGI_ images. Take a screenshot from a game, run it through SD with a low noise rate, give it a detailed description of everything in the picture and it produces pretty solid photo recreations of the images. Also, often, it gets possessed by Eldritch gods and spews out monsters.

If anyone is stuck with the code. The "i" should be a "t" in this line in the loop: ``` latents = scheduler.step(noise_pred, i, latents)["prev_sample"] ```

The very concept of embeddings is amazing to me. It's literally "organize concepts themselves into points in space, where similar things are closer together, in many many dimensions; now you can do arithmetic on *the meanings of words, phrases, and sentences.* " Want to add the meaning of "horse" and the meaning of "male"? Well, just add these vectors together and the resulting coordinates will point right at "stallion"! They amaze me so much that, when I watched Everything, Everywhere, All At Once for the first time, I completely geeked out when I realized their description of the organization of the multiverse is effectively a well-embedded latent space 😅

this guy makes sense. I want more of him teaching SD and how it works.

Mike asked himself what the use case for mixing two prompts is. I used this only yesterday, to produce a photorealistic painting of an owlbear from DnD... So it has practical uses!

"Simple, you just chip away all the stone that doesn't look like David."

I've been playing with Stable Diffusion (specifically the "InvokeAI" fork because I don't have 10gb VRAM), and I've found out that spamming the end with keywords like "realistic, 4k, trending on artstation, 8k, photorealistic, hyperrealistic" has more effect on how good the output image is than I thought.

I love it how you simplify and explain this heap of complexity that is in generative models like this. You gave me the impulse to play around with it, inspite of being pretty complicated code due to the depth of the abstraction. It's a lot of fun to fantasize about something and have the model come up with a visual representation.

SD is just outstanding. It can mimic the other projects and the 1.4/1.5 models will be public domain. You can't beat that.

I would like to see a version of the code where it shows the result of each step, so you can see the noise getting reduced with each iteration

I really liked the stable diffusion that came with the webui that you could install on your own computer, to avoid quotas or subscription costs, and it provided easy to use UI as well. With inpaint feature inside the UI as well. Shoutouts to people who make those applications from the rough code for regular people to use.

"there are questions of ethics, there are questions on how it's trained. Let's leave those for another time" well, if that doesn't just sum up the tech industry.

I like how your channel has adapted to the advent of the machine learning boom we are experiencing

We need an entire "Frogs on stilts" channel.

Mike is a legend, truly great videos with him

This video finally explained the code to me in a simple way! Now im less confused!!! Amazing extra documentation from you guys

So amazing ❤ I love stable diffusion Playing around the few last weeks

Love Mikes explanations, somehow he manages explain so complicated stuff in so simple and understandable way. It will be interesting to know Mikes opinion om Midjourney as it's seems like the winner for now among the picture creation AIs.

The current version of the reference notebook is already deprecated due to Hugging Face's API changes :) You try to operate on "image", which is now a DecoderOutput class: image = (image/ 2 + 0.5).clamp(0, 1) It is fixed by unpacking its tensor attribute with its sample method: image = (image.sample / 2 + 0.5).clamp(0, 1)

this is so interesting and has so many unexplored use cases

For anyone trying to get the notebook to work and is getting this error: "TypeError: unsupported operand type(s) for /: 'DecoderOutput' and 'int'" change "image = (image / 2 + 0.5).clamp(0, 1)" to "image = (image.sample / 2 + 0.5).clamp(0, 1)". As noted at the top of the notebook it seems the huggin API has changed.

Awesome explanation, thank you!

This was so helpful in understanding this new tech. thank you

Immediately recognized the book on Dr. Ponds desk - Prof. Paar was one of my teachers when I studied IT sec. Nice to see it outside of Germany too!

Excellent explanations, as always! Thanks!

great video. today SORA was launched, nad youvideos help to understand whats going on the background. many thanks!

I doubt DALL-E 2 is the “biggest” image generator. Stable Diffusion is probably bigger. In my circle, the biggest one is NovelAI, which is a Stable Diffusion variant specialized in anime-style images. Notably, its training data is probably the best image dataset out there in terms of detailed labels. It’s already been causing a lot of drama in the community. One notable case involved someone feeding a WIP drawing to img2img, posting it, claiming it as their own drawing. When the actual artist posts their finished image, this person then proceeds to accuse the artist of copying “their” art.

Mikes explanations Aretha best ❤

Wow this is actually pretty amazing. Fascinating stuff

Great video! Can anyone recommend any other videos that explain the text encoding and the whole clipping process used to guide the image generation based on input prompt?

this video just put me on a wonderful path, thank you!

great video and very educational I'd love to hear you guys talk about textual inversion

Excellent tutorial. Thank you.

Great explanation.

Seeing that GPT-2 vid reminded me: we haven't had Robert Miles on in a fair while. Is he just too busy?

Good timing with the NovelAI leaks

Thanks for the explanations of how AIs are being trained. I can see a slight hint of a neural network here. I think the advantage now is that companies like Bluewillow is utilizing discord to quickly gain testers free of charge even.

Stable Diffusion in code? More like “Super great explanation that’s solid gold!” 👍

I generated thousands of images with stable diffusion. It's really fun and inpiring.

Thank you for this video, it's really interesting!

I was waiting for this 🙏🙏🙏

Thanks for this video. So the Steps is actually the Noise Level.

Anyone else surprised that diffusion models are the clear winners for image generation? And GANs have almost completely fallen from favor? I haven’t seen them in any recent SOTA work..

Well, xkcd did pick the number 4 by die roll. Seems a random enough seed to me.

Amazing so stable diffusion helps un clutter all that extra pixel during the process of facial recognition.

On line 56, the image is coming from the sample property of the DecoderOutput, change to 55: with torch.no_grad(): 56: image = vae.decode(latents).sample

Thank you for the SCIENTIFIC video! It got outta control after the "novelaileak", which it is very important to leave some information as realistic as it can. I'm quite sad about the sub-culture but I still have hope on the artist / researcher to snap out from the chaos.

The notebook can still work with a few minor tweaks: The text prompt should be multiplied by the batch size; The scheduler step takes in "t" instead of "i", and now it prefers scaling via scheduler.scale_model_input(latent_model_input, t) rather than with explicit sigma. Also, torch.autocast did not work on my local machine for some reason. Anyway, thanks a lot for the code.

Hi Mike. This is the by far the most technically clear explanation of SD that I have seen so thank you for this! Now as you would be aware by now, the art community is up in arms against this tech and I would love to hear your opinion based on the factual knowledge you have. The main issue that keeps coming up is that SD tech is art theft because it steals copyrighted artwork then companies profit using the images. Another point artists are making is that SD is just a mish mash collage of original art so nothing generated by Ai is brand new. Would you agree or disagree with these points and why strictly based on from your technical knowledge.

I'll copy your transcript and feed it to open.ai's playground and ask him to re-interpret your addresss for images but for my own audio interpolation in music. Brilliant.

Great one again!

I didn't realise that this is basically the next evolution of the "AI Upscaling" technology that has been used to in videogame mods: Take an image and then add detail until it looks like what I think it's supposed to. It's still mind-bending how it results in what it does, but AI Upscaling wasn't so scary, so I suppose this feels a bit less scary now.

I would do just about anything for more Mike content!

Thank you for trying to fix the code after the API update broke it

So it is basically a morphing, blending and upscaling algorhythm of compressed/encoded data?

13:47 reminds me of the wave function collapse algorithm.

Fascinating.

I don't know if this is more amazing or more frightening. Brilliant stuff.

thanks for the video

3:07 earned my like. I need to go see that now. 😂

12:34 beautiful cityscapes 🏙️

[notebook error] Hello, Thanks for the fantastic video. I noticed that as of today the notebook does not run since there are some errors. I do not why, probably some library changed a bit.The first error is at line 50 of the cell with the first inference loop. Instead of 'i' there should be 't'. The second error appears at line 59. Now to access the image's tensor you have to write 'image["sample"]' instead of just 'image'.

If you can make images by removing noise from random noise. Can you make P solutions from NP solutions the same way by training on known P solutions having "noise" added to make them NP?

Great video. I would love to see a video about the recent controversy with GitHub copilot and GPL licenses.

On line 50, i should be changed to t (as we need the FloatTensor) 50: latents = scheduler.step(noise_pred, t, latents)["prev_sample"]

I love this

If you mention another video please also link it in the description!

Could you do a video about the different samplers? (eg. DDIM, Euler, Euler a, etc.) That part of the process is still a mystery for me

Great video. However, could you explain what this line "latent_model_input = latent_model_input / ((sigma**2 + 1) ** 0.5)" does?

Pretty cool!

love this tool but im having an error when trying to noise an image to run the AI over a guide image. the add_noise def returns an error of "AttributeError: 'int' object has no attribute 'to'". It come after the call line below any help would be amazing latents = scheduler.add_noise(encoded, noise, start_timestep)

Great video, really informative. I was hoping to try out your Google Colab code, although it seems broken at the moment. Are there any updates regarding this announcement regarding the known bugs? "Note: There might be a handful of bugs at the moment. The developers of this stable diffusion implementation keep changing the api. Everyone should know not to make breaking api changes so regularly! I'll do a pass over the code and fix bugs as soon as I can. Am away this week :) thanks to Michael d for bringing this to my attention."

hi, could you guys make a video on what kernels are please?

Another cool thing you can do is _negative prompts,_ that you can put in place of the "unconditioned" embedding.

A hybrid frog/snake is properly called a *SNOG*, obviously.

> There are questions about ethics. There are questions about how these were trained. Maybe we deal with them another time. I really hope there is a discussion of this at some point. As a discipline that skews very white/male and enjoys relatively posh working conditions, it's very easy to insulate ourselves from the very real problems of the world. And because computers are so powerful it's also simple to automate oppression of many kinds, helping it continue to run smoothly. I think we have a responsibility to talk about these issues and I would love to see this channel model that in a constructive way.

I wanted to ask if anyone got a simmilar error when running this line: "image = (image / 2 + 0.5).clamp(0, 1)" Error: "unsupported operand type(s) for /: 'DecoderOutput' and 'int'". Seems like to output of the vae decoding the latents is can not be used together with an int. Can anyone help?

so cool!

What surprises me is how primitive a lot of these techniques seem to be under the hood, and how much further it can obviously be taken. These techniques are still in their infancy. For instance, there seem to be a lot of potential image-generating procedures that might converge faster than random high frequency noise. What if there could be stages with simulated random brush strokes, or generating geometric shapes, or input to 3d modelling software. If the tools that humans use to create digital art could be algorithmically leveraged by an AI, if might be even more effective. Also, if you could spatially embed the tags in the source image in a way it could be coupled to the segmentation, maybe it could be used as a tool to 'compose' an image. A blob of one color is tagged as a dog, a blob of another is tagged as a bench, and the AI interprets it with those spatially defined weights to start.

Can we add annotations along with the image in an image2image model? The annotations to tell us which part of the image needs to be regenerated. Like I want to change the background with the annotations to that background so it gives exactly the same person with a different background? Something like Photoshop Generative AI

Mike said link to code in description!

So I know you briefly mentioned the ethics of using these in the previous video (Usually around the trained images as I understand) - does Stable Diffusion allow you to not just supply that original image like the rabbit image you provided there, but the *entire* training set for a local training process based *only* on images you've provided/made/created/got permission to train based off of?

Is there a way to use 2 image prompts instead of 2 text prompts to get a 50/50 blend?

Mind giving a quick review of Bluewillow and which software does it utiized? I think you guys break down the whole infrastructure which is actually very informative.

Well, done, I just don't understand how the guiding works. What if I instruct it to create a complex image that certainly wasn't in any training data with many complex relations what should be where in the inquiry? How it can be constructed as a whole instead of creating and merging the parts it may have encountered?

I just watched this video. Obtained a Colab error on this statement: image = (image / 2 + 0.5).clamp(0, 1) . The error was: TypeError: unsupported operand type(s) for /: 'DecoderOutput' and 'int'

Now Deep Dream Generator has just added a text to image diffusion generator too, and it's actually pretty decent.

Only a matter of time until someone adapts this to 3d models. I mean, there are millions of 3d models on the internet in form of assets for all kind of engines and frameworks, all with a description to them, too.

This is literally the first episode of Computerphile ever that I didn't understand anything of what was explained. And judging from the comments I'm the only one. Looks like I totally missed the boat on this topic.

@14:47 - idea: hand draw your animation sequence.Give the first to image and text to AI and get the result. Then hand the resulting image, your next hand drawn frame and the text to generate the 2nd frame. Continue the process so that each new frame is a combination of the last and what you want it to look like combined. In this way the "flicker" might be reduced. But I haven't seen what you're talking about. I may be off.

When trying to run "pil_to_latent(im)", in the "Scheduling and Visualization" section, I'm getting "AttributeError: 'AutoencoderKLOutput' object has no attribute 'sample'". I've tried changing latent.mode() to latent.sample(), with no change.

I think there is an error in line "image = vae.decode(latents)" It should be "image = vae.decode(latents).sample" ????

I don't understand at all how the result of this reconstruction process (remove noise) is stored. Sounds a bit like witchcraft to me. Remove some noise, here we go. I mean in which form is the noise reduction saved? In a database? Does it save pixels or what exactly?

what algorithms used in pakage managers? .

Would love to see a test to see how it works when it's trained with a limited dataset.

Such a fun and interesting tool. Wish it wasn't used to do bad things, like stealing people's artworks

this whole topic blows my mind even more than when i first heard of deepfakes

The code might have a bug, "TypeError: unsupported operand type(s) for /: 'DecoderOutput' and 'int'" on the line "image = (image / 2 + 0.5).clamp(0, 1)"

7:18 is clearly a reference to xkcd 221