Autoencoder In PyTorch - Theory & Implementation

Рет қаралды 65,313

Күн бұрын

In this Deep Learning Tutorial we learn how Autoencoders work and how we can implement them in PyTorch.
Get my Free NumPy Handbook:
www.python-engineer.com/numpy...
✅ Write cleaner code with Sourcery, instant refactoring suggestions in VS Code & PyCharm: sourcery.ai/?... *
⭐ Join Our Discord : / discord
📓 ML Notebooks available on Patreon:
/ patrickloeber
If you enjoyed this video, please subscribe to the channel:
▶️ : / @patloeber
Resources:
www.cs.toronto.edu/~lczhang/3...
Code: github.com/patrickloeber/pyto...
More PyTorch Tutorials:
Complete Beginner Course: • Deep Learning With PyT...
Dataloader: PXOzkkB5eH0
Transforms: • PyTorch Tutorial 10 - ...
Model Class: • PyTorch Tutorial 06 - ...
CNN: • PyTorch Tutorial 14 - ...
~~~~~~~~~~~~~~~ CONNECT ~~~~~~~~~~~~~~~
🖥️ Website: www.python-engineer.com
🐦 Twitter - / patloeber
✉️ Newsletter - www.python-engineer.com/newsl...
📸 Instagram - / patloeber
🦾 Discord: / discord
▶️ Subscribe: / @patloeber
~~~~~~~~~~~~~~ SUPPORT ME ~~~~~~~~~~~~~~
🅿 Patreon - / patrickloeber
#Python PyTorch
Timeline:
00:00 - Theory
02:58 - Data Loading
05:30 - Simple Autoencoder
15:02 - Training Loop
17:00 - Plot Images
19:00 - CNN Autoencoder
29:12 - Exercise For You
----------------------------------------------------------------------------------------------------------
* This is an affiliate link. By clicking on it you will not have any additional costs, instead you will support me and my project. Thank you so much for the support! 🙏

Пікірлер: 81

@patloeber 3 жыл бұрын

Let me know if you enjoyed the new animations in the beginning and want to see this more in the future :)

@sumithhh9379 3 жыл бұрын

Hi Patrick, Any plans to have a series on NLP state of the art models?

@MrDeyzel 3 жыл бұрын

They're great

@md.musfiqurrahaman8612 2 жыл бұрын

Love the animations and want more. Learning pytorch and following your tutorials.

@jh-pq9tp 3 жыл бұрын

big thanks to you. i cannot imagine how could i learn my dl course without your tutorial. Your work is the best in youtube so far!

@starlite5097 3 жыл бұрын

I love all your PyTorch videos, please do more :D

@patloeber 3 жыл бұрын

Thanks! I will :)

@salimibrahim459 3 жыл бұрын

Nice, was waiting for this :)

@patloeber 3 жыл бұрын

Hope you enjoyed it!

@pleasedontsubscribeme4397 3 жыл бұрын

Great work!

@saeeddamadi3823 3 жыл бұрын

Thank you so much for clear presentation of Autoencoder!

@patloeber 3 жыл бұрын

glad you like it!

@markavilin5020 2 жыл бұрын

Very clear, thank you very much

@harshkumaragarwal8326 3 жыл бұрын

Great work!! Thanks :))

@ingenuity8886 Ай бұрын

Thank you so much , you explained it really good.

@Jerrel.A Жыл бұрын

TopNotch explanation! Thx.

@satpalsinghrathore2665 Жыл бұрын

Very cool. Thank you.

@ayankashyap5379 3 жыл бұрын

at 22:17 when calculating the shape of the conv output, it should be 128*128*1 => 64*64 * 16 and the rest should also be different accordingly

@adityasaini491 3 жыл бұрын

Hey Patrick a really informative and concise video! Thoroughly enjoyed it :DD Just a small correction at 12:51, you used the word dimension while explaining the Normalize transform, whereas the two attributes are just the mean and standard deviation of the resultant normalized data.

@patloeber 3 жыл бұрын

thanks for the hint!

@anarkaliprabhakar6640 5 ай бұрын

Nice explanation

@saadmunir1467 3 жыл бұрын

Its reaallly nice but it would be a very nice addition to include variational autoencoders and Generative adversial networks as well :). Maybe they can be helpful to many struggling with class imbalance during classification

@patloeber 3 жыл бұрын

Great suggestion!

@shahinghasemi2346 3 жыл бұрын

Thank you for your nice tutorials please do the same for a non-image data. I'm curious to see CNN auto-encoders with non-image data.

@garikhakobyan3013 3 жыл бұрын

hello, nice videos you have. looking forward new videos on paper review and implementations.

@huoguo7426 2 жыл бұрын

Great video! Could you provide the same walkthrough for a variational autoencoder? Or point point me to a good walkthrough on the theory and implementation of a variational autoencoder?

@CodeWithTomi 3 жыл бұрын

Yet another Pytorch video🔥

@devadharshan6328 3 жыл бұрын

Can u help to implement pytorch with django

@patloeber 3 жыл бұрын

man you are fast :D

@devadharshan6328 3 жыл бұрын

Can u upload Ur GUI chat bot code in GitHub I tried code along approach I was able to learn the concept but I got some few bugs . Can u upload it

@patloeber 3 жыл бұрын

I added the code here: github.com/python-engineer/python-fun

@devadharshan6328 3 жыл бұрын

@@patloeber thanks

@DiegoAndresAlvarezMarin 2 жыл бұрын

Beautifully explained!! thank you!

@astridbrenner2957 3 жыл бұрын

This channel is so underrated.Please upload tutorials about Django

@Mesenqe 3 жыл бұрын

This channel is really good, I learned PyTorch from this channel. Guys I assure you subscribe to this channel.

@patloeber 3 жыл бұрын

Thanks so much:) appreciate the nice words

@3stdv93 Жыл бұрын

Thanks ❤

@ujjwalkumar-uf8nj Жыл бұрын

Hey Patrick I used your exact code to train the CNN based autoencoder but couldn't get it to converge without Batch Normalization, after adding BatchNorm2d after every ReLU it works fine, but without it, it doesn't, tried different values for lr from 1e-2 to 1e-5. I was training on MNIST dataset only. the loss becomes NaN or ranges between 0.10 to 0.09.

@saurrav3801 3 жыл бұрын

Bro always waiting for your pyt🔥rch video ....🤙🏼🤙🏼🤙🏼

@patloeber 3 жыл бұрын

🙌

@mojojojo890 Жыл бұрын

which one is the link that explains how you make the pytorch classes please?

@user-ie1cv8su2f 2 жыл бұрын

thank you!

@patloeber 2 жыл бұрын

You're welcome!

@maharshipathak 2 ай бұрын

For python 3.11+, pytorch 2.3+ change the dataiter.next() to next(dataiter)

@user-wr4yl7tx3w Жыл бұрын

But how do we leverage the low dimensional embedding given that they represent the PCA?

@falklumo Жыл бұрын

It should be noted that the performance difference between Linear and CNN as shown here comes from the chosen compression factor. Linear chose 12 Byte per image, CNN chose 256 Byte per image, where an original image is 784 Byte. So, the CNN code does not compress enough, less than PNG actually! You need two more linear layers to compress 64 down to 16 and then 4.

@aisidamayomi8534 11 күн бұрын

Please can you do for a network intrusion detection

@hadisizadiyekta125 2 жыл бұрын

you used recons and img as input for loss function, however if we want to train my model and test it we should use "recon" and "labels" as an input for loss function. but the labels are 3D, how we can do that?

@amzadhossain8118 3 жыл бұрын

Can u make a video on DNCNN

@lankanathaekanayake7680 2 жыл бұрын

is it possible to use sentences as input data?

@devadharshan6328 3 жыл бұрын

Great animations my suggestion is to add in more animations not only in theory but also in the working of the code . Just my suggestion but great video thanks for Ur teaching.

@patloeber 3 жыл бұрын

thanks! definitely a good idea

@marinacarnemolla5515 2 жыл бұрын

hi, I have a question: if we pass the image as input of the model, it will put the weights to zero and then the output will be exactly the same of the input image. So, why the image is given as input of the model? It doesn't make sense to me. Could yu explain this to me?

@marytajz1814 3 жыл бұрын

your tutorials are amazing! Thank you so much... Could you please make a video for nn.embedding as well?

@patloeber 3 жыл бұрын

I'll have a look at it

@martinmayo8197 2 жыл бұрын

I don't understand a little bit the sintaxis. Why do you define the method 'forward' but never call it explicitly ? Maybe the line "recon = model(img)" is where you are using it, but I didn't know that it could be done like this. I would had written "recon = model.forward(img)", is it the same ?

@haikbenlian5466 2 жыл бұрын

How you found that the image size was decreased from 28 to 14?

@khushpatelmd 2 жыл бұрын

If you normalize the input image which is also the label, the values will be between -1 to +1 but your output since passed through sigmoid will be between 0 and 1. How will you decrease loss for pixels that are between -1 to 0 as your predictions will never be less than 0?

@anonim5052 2 ай бұрын

you need to change sigmoid function at the end to tanh, to output will also be betweet -1 to 1

@teetanrobotics5363 3 жыл бұрын

youre the best

@patloeber 3 жыл бұрын

thanks!

@teetanrobotics5363 3 жыл бұрын

Could you please make GANs, VAE , transformers and BERT in pytorch

@ArponBiswas-wq3sh 3 ай бұрын

Very nice but need more

@avivalviannur5610 8 ай бұрын

I tried to rerun your code in the part of Autoencoder CNN, but then I got Loss = nan in each epoch. Do you know what is wrong?

@736939 2 жыл бұрын

Can you please show how to work with variational autoencoders and applications such as Image segmentation.

@patloeber 2 жыл бұрын

will look into this!

@736939 2 жыл бұрын

@@patloeber Thank you. Because for me it's hard to program it.

@Saens406 2 жыл бұрын

why there is no require_grad there?

@YounasKhan-vm8nr 3 ай бұрын

Do you have anything specific for face images, this won't work on face images.

@vallisham1756 2 жыл бұрын

module 'torch.nn' has no attribute 'ReLu' Is anyone else getting the same error

@theupsider 2 жыл бұрын

its ReLU

@vallisham1756 2 жыл бұрын

@@theupsider Thanks a lot!

@AshishSingh-753 3 жыл бұрын

Next video is on GAN

@roshinroy5129 Жыл бұрын

Am I the only one encountering nan values during training this ?

@pingpong3904 10 ай бұрын

On one of my virtual machines i also get nan values, when using torch 2.0.1. I tried a couple of stuff, but only with torch 1.12.1 it works fine. On my desktop pc it works with torch 2.0.1 though. But i do not know why.

@anirudhjoshi1607 2 жыл бұрын

dude my CNN autoencoder is doing worse than the linear autoencoder, lmao

@marc2911 11 ай бұрын

me too the ouputs show strange padding artefacts as well

@pingpong3904 10 ай бұрын

@@marc2911 On one of my virtual machines i also get nan values or strange artifacts, when using torch 2.0.1. I tried a couple of stuff, but only with torch 1.12.1 it works fine. On my desktop pc it works with torch 2.0.1 though. But i do not know why.