love to see more good explaination for other model, your explaination is soo good

Hey! I am start my CompSci Masters program in the Fall, and just wanted to say that I love this video. I've never really had time to sit down and learn PyTorch, so the brevity of this video is greatly appreciated! It gives me a fantastic starting point that I can tinker around with, and I have an idea on how I can apply this in a non-conventional way that I haven't seen much research on... Thanks again!

Your explanations are simply great! I do recommend you to return back to KZfaq covering latest papers in this field :)

one CRAZY thing to take from this code (and video) GREEK LETTERS ARE CAN BE USED AS VARIABLE NAME IN PYTHON

These implementation videos are marvelous. You really should do more of them. Big fan of your channel!

The Under rated OG channel

At 13:20, the formula is not correct. It is not approximate, not strictly equal. It should have been made clear.

Great video! one question: at 22:40 you say "the authors decided to use a simple mean-squared error ..". That part isn't clear to me, at this point we are already considering the loss, we need to minimize the given KL divergence. Why is there a new loss being introduced at this point and how is that justified?

insance

Awesome video.

Great explanation. Thanks.

Nice explanation handsome dude, you just got a new subscriber.

So far the best model, fastest, high quality image generator on my 3070 gpu. Very very great !!! I used transformers encoders for "denoising". 95% noise on first iteration, no pure noise. No signal in the denoiser's inputs. tested on cifar 100 ! temperature=0.7, top k=40 only 4 steps for denoising !!! Very impressive ! It is the first time i get so confident about the power of a generative model !!!

Amazing video; thanks a lot for going in depth on the math with simplified animations!

Great video buddy, if you can share some more resources to understand the underlying math, that would be great.

Appreciate the effort you put into this. You definitely can teach. If only I have a brain to understand math... still got some bits here and there. Thanks

thanks for your detailed introduction

That's a really nice video. You summarised so much information in a concise video. And the explanations were crystal clear. Thanks a lot.

Usually the KL divergence is expressed as DKL(q||p)=q*log(q/p), but in the slice @16:48, DKL(q||p)=log(q/p), Why q is ignored here?

Thanks for such amazing illustration for Diffusion. One question is about the equation in slice @ 13:16, how to get t-2 and t-3? x_t=sqrt(a_t)*x_t-1+sqrt(1-a_t)*e x_t-1=sqrt(a_t-1)*x_t-2+sqrt(1-a_t-1)*e x_t=sqrt(a_t)*[sqrt(a_t-1)*x_t-2+sqrt(1-a_t-1)*e]+sqrt(1-a_t)*e=sqrt(a_t*a_t-1)*x_t-2+[sqrt(a_t-a_t*a_t-1)+sqrt(1-a_t)]*e The rightmost term doesn't equal or close to sqrt(1-a_t*a_t-1)*e Dis I misunderstand something? Thanks again. @Outlier

Very Bad Video

Thanks for the explanation, it's awesome! But I have a question. In cross attention, if we set the text as V, the final attention matrix could be viewed as a weighted sum of each word in V itself (the "weighted" part comes from the Q, K similarity). If I understand correctly, the final attention matrix should contain the values in the text domain, why can we multiply by a W_out projection and get the result in the image domain (add it to the original image)? Will it make more sense if we set the text condition as Q, and the image as K, V?

U really liked that you showed the derivation in an understandable way

Amazing explanation thanks a lot! Minor improvement suggestion: add a pop-protection to your microphone :)

I think it is not exactly true statement that N(mu, sigma) = mu + sigma*N(0, 1). Just try that transformation, mu plays a role of translation in the value axis. However, what is correct, that if you sample from the left side, it acts the same as if you sample from the right side. I am pointing this out because I got stuck with that for a while. But I still also might got it completely wrong.

Love this - what a fantastic achievement!

Brilliant work!

11:57

This is a really great video, thanks for your big effort explaining!

Wonderful explanation! Not sure if this is in the original papers, but I find it very odd that there is no nonlinear function after V and before W_out. It seems like a waste to me since Attention@V is itself a linear function, so w_out wont necessarily change content of the data beyond what Attention@V already would have done through training.

The strange patten in the reconstructed image and the generated image is likely to be caused by the perceptual loss, I have no idea why but the disappears when I take the perceptual loss away.

Thank you so much . But please i prefer that you go to the maths and operations more detailly being training of Würstchen 🎉🎉 thank you

really nice video, helps me understand a lot❗

Great video!! At 13:34, does multiplying with a_0 correct?

best explanation

This is an amazing video 🔥

12:12 does anyone understand here why when \beta is linear, \sqrt{1-\beta} is linear as well?

the statement at 15:33 isnt right ... is it ? cause i have a counter f(x) = x^2 , g(x) = -x^2 here f(x) >= g(x) but thier derivatives are negatives of each other. Please help i dont really understand the concept of ELBO

At inference. Input of State A( VQGAN decoder) is discrete latents. Continuous latents needs to be quantize to discrete latents( discrete latents is also choosen from codebook, by which vector in Continuous latents nearest to vector in codebook). But Output of State B is Continuous latents. And Output of State B is directly for Input of State A..... if it right ? How State A( VQGAN decoder) handle Continuous latents . I check VQGAN paper and this Wurchen paper. That is not clear. Please help me that. Thank you

So, apparently, it sounds like Wurchen is exactly at Stage C. am i right?

Hey great video. Can you tell me why random sampling of codebook vectors doesn't generate a meaningful images. In Vae we random sample from std gaussian, why the same doesn't work for vq auto encoders.

How did you manage to get 128 A100 for 3 weeks? I think the cost is about 100k USD for one run. Assuming you did multiple iterations, the overall cost can be easily 200k for this project.

Just completed writing a 24 paged handwritten note based on this video and a few other sources

Amazing explanation and thank you for taking the time to properly visualize everything

Sorry if I am misunderstanding, but at 19:10, shouldn't the code be: "uncond_predicted_noise = model(x, t, None)" instead of "uncond_predicted_noise = model(x, labels, None)" Also, according to the CFG paper's formula, shouldn't the next line be: "predicted_noise = torch.lerp(predicted_noise, uncond_predicted_noise, -cfg_scale)" under the definition of lerp? One last question: have you tried using L1Loss instead of MSELoss? On my implementation, L1 Loss performs much better (although my implementation is different than yours). I know the ELBO term expands to essentially an MSE term wrt predicted noise, so I am confused as to why L1 Loss performs better for my model. Thank you for your time.

Hey @Outlier , on 12:44 looks like you mistakenly use "a" instead of "alpha" notation symbol inside 'product notation (Pi Notation)'. Since you mentioned the example multiplication below using "alpha notation". e.g. t = 8 "alpha_8" = "alpha_1" x "alpha_2" x "alpha_3" x "alpha_4" x "alpha_5" x "alpha_6" x "alpha_7" x "alpha_8" ======================================================= Is it intentional, though ? Please let me know. Thanks

This is dumb, but can anyone explain why when beta increases linearly, the square root of 1 - beta decreases linearly, at 12:13? Shouldn't it have some curve to it, given the square root?

@22:32 ur missing a bar over the alpha on the bottom line. Plz correct me if im wrong

Hi! Can you please explain why the output is getting two stitched images?

How did the KL divergence expand to log(q/p)? You yourself mentioned it to be the integral of q*log(q/p) for D_kl(q||p) Perhaps too much of a simplification.