Variational Autoencoder - Model, ELBO, loss function and maths explained easily!

No video

Variational Autoencoder - Model, ELBO, loss function and maths explained easily!

Рет қаралды 24,149

Күн бұрын

A complete explanation of the Variational Autoencoder, a key component in Stable Diffusion models. I will show why we need it, the idea behind the ELBO, the problems in maximizing the ELBO, the loss function and explain the math derivations step by step.
Link to the slides: github.com/hkproj/vae-from-sc...
Chapters
00:00 - Introduction
00:41 - Autoencoder
02:35 - Variational Autoencoder
04:20 - Latent Space
06:06 - Math introduction
08:45 - Model definition
12:00 - ELBO
16:05 - Maximizing the ELBO
19:49 - Reparameterization Trick
22:41 - Example network
23:55 - Loss function

Пікірлер: 55

@Koi-vv8cy 9 ай бұрын

It's the clearest explanation about VAE that I have ever seen.

@umarjamilai 9 ай бұрын

If you're up to the challenge, watch my other video on how to code Stable Diffusion from scratch, which also uses the VAE

@sandahk7173 29 күн бұрын

It was the best video I found to explain VAE. Thank you so much!

@lucdemartre4738 4 ай бұрын

I would pay so much to have you as my teacher, that's not only the best video i've ever seen on deep leanring, but probably the most appealing way anyone ever taught me CS !

@chenqu773 7 ай бұрын

The peps starting from 06:40 are the gem. Totally agree.

@JohnSmith-he5xg 9 ай бұрын

Getting philosophical w/ the Cave Allegory. I love it. Great stuff.

@vipulsangode8612 4 ай бұрын

This is the best explanation on the internet!

@user-sz5fg2sn7y 5 ай бұрын

I love this so much, this channel lands in my top 3 ML channels ever

@greyxray 5 ай бұрын

so clear! so on point! love the way you teach!

@shuoliu3546 4 ай бұрын

You solved my confusion since long! Thank you !

@desmondteo855 3 ай бұрын

Incredible explanation. Thanks for making this video. It's extremely helpful!

@shajidmughal3386 2 ай бұрын

Great explanation. Clean!!! Reminds me of school where our physics teacher taught everything practical and it felt so simple. subs+1👍

@xm9086 5 ай бұрын

You are a great teacher.

@vikramsandu6054 3 ай бұрын

Simply amazing. Thank you so much for explaining so beautifully. :)

@awsom 5 ай бұрын

Great Explanation!!

@dannyjiujitsu Ай бұрын

Phenomenal teaching.

@lucdemartre4738 4 ай бұрын

PLATO MENTIONED PLATO MENTIONED I LOVE YOU THAT'S THE BEST VIDEO I'VE EVER SEEN !!!

@miladheydari7916 4 ай бұрын

this is the best video on the Internet

@hainguyenthien4225 28 күн бұрын

thank you very much

@oiooio7879 Жыл бұрын

Wow thank you very informative

@mahsakhalili5042 2 ай бұрын

Appreciate it! it helped me a lot

@Kiara-h2o 27 күн бұрын

Thanks a lot

@nassimlamrini-b5i 22 күн бұрын

🎯 Key points for quick navigation: 00:13 *understand fundamental ideas* 00:41 *explain autoencoder concept* 02:36 *issues with traditional autoencoders* 03:02 *introduce variational autoencoder* 04:13 *sampling from latent space* 25:02 *Model and prior choice.* 25:17 *Element-wise noise multiplication.* 25:32 *Learning log variance.* 26:00 *Maximizing ELBO for space learning.* 26:15 *Derivation of loss function.* Made with HARPA AI

@pauledam2174 29 күн бұрын

great video ( am about half way through). I think at minute 13 there is a misstatement (I think). in general if g(x) is greater than h(x), if we find the max of h it doesn't mean we have located the max for g

@user-sz5fg2sn7y 5 ай бұрын

Thanks!

@isaz2425 4 ай бұрын

Thanks, this video have many explanations that are missing from other tutorials on VAE. Like the part from 22:45 onwards. I saw a lot of other videos that didn't explain how the p and q functions were related to the encoder and decoder. (every other tutorial felt like they started talking about VAE, and then suddenly changed subject to talk about some distribution functions for no obvious reason).

@umarjamilai 4 ай бұрын

Glad you liked it!

@TheBlackNight971 2 ай бұрын

Both VAE and AE map the input over a latent space, the difference lies on the **structure** of this latent space. The AE latent space is not "well-organized" as well as the VAE's latent space.

@shashankgsharma0901 3 ай бұрын

thanks UMAR!

@umarjamilai Жыл бұрын

Link to the slides: github.com/hkproj/vae-from-scratch-notes

@user-wy1xm4gl1c Жыл бұрын

thx for the video, this is awesome!

@lifeisbeautifu1 4 ай бұрын

You rock!

@waynewang2071 5 ай бұрын

Hey, thank you for the great video. Curious if there is any plan to have a session for code for VAE? Many thanks!

@morgancredib-ai2501 7 ай бұрын

A normalizing flow video would complement this nicely

@nadajonidi9691 6 ай бұрын

Would you please give the url for normalizing flows

@user-hd8mi1bt2f 5 ай бұрын

Thanks for sharing . In the chicken and egg example, will p(x, z) be trackable? if x, z is unrelated, and z is a prior distribution, so p(x, z) can be writen in a formalized way?

@sohammitra8657 Жыл бұрын

Hey can you do a video on SWin transformer next??

@oiooio7879 Жыл бұрын

Can you do more explanations with coding walk through that video you did on transformer with the coding helped me understand it a lot

@umarjamilai Жыл бұрын

Hi Oio! I am working on a full coding tutorial to make your own Stable Diffusion from scratch. Stay tuned!

@huuhuynguyen3025 Жыл бұрын

@@umarjamilai i hope to see it soon, sir

@GrifinsBrother 6 ай бұрын

Sad that you have not released video "Hot to code the VAE"(

@martinschulze5399 8 ай бұрын

14:41 you dont maximiye log p(x), that is a fixed quantity.

@DikkeHamster 28 күн бұрын

Don't AEs also learn a latent space representation? I'm not sure this is the largest difference with VAEs. The sampling is, however.

@umarjamilai 28 күн бұрын

They do, but nobody forces the latent space of AEs to be a gaussian or any other known distribution.

@prateekpatel6082 6 ай бұрын

why does learning distribution via a latent variable capture semantic meaning. ? can you please elaborate a bit on that

@quonxinquonyi8570 5 ай бұрын

Latent variable is of low dimension compare to input which is of high dimension…so this low dimension latent variable contains features which are robust, meaning these robust features survive the encoding process coz encoding process removes redundant features….imagine a collection had images of cat and a bird image distribution, what an encoder can do in such a process is to outline a bird or cat by its outline without going into details of colours and texture….these outlines is more than enough to distinguish a bird from a cat without going into high dimensions of texture and colors

@prateekpatel6082 5 ай бұрын

@@quonxinquonyi8570 that doesnt answer the question. Latent space in autoencoders dont capture semantic meaning , but when we enforce regularization on latent space and learn a distribution thats when it learns some manifold

@quonxinquonyi8570 5 ай бұрын

@@prateekpatel6082 learning distribution means that you could generate from that distribution or in other words sample from such distribution…but since the “ sample generating distribution “ can be too hard to learn, so we go for reparametrization technique to learn the a standard normal distribution so that we can optimize

@quonxinquonyi8570 5 ай бұрын

I wasn’t talking about auto encoder,I was talking about variational auto encoder…

@quonxinquonyi8570 5 ай бұрын

“ learning the manifold “ doesn’t make sense in the context of variational auto encoder….coz to learn the manifold, we try to approach the “score function” ….score function means the original input distribution….there we have to noised and denoised in order to get some sense of generating distribution….but the problem still holds in form of denominator of density of density function….so we use log of derivative of distribution to cancel out that constant denominator….then use the high school level first order derivative method to learn the noise by using the perturbed density function….