The Most Important Algorithm in Machine Learning

Рет қаралды 286,668

Күн бұрын

Shortform link:
shortform.com/artem
In this video we will talk about backpropagation - an algorithm powering the entire field of machine learning and try to derive it from first principles.
OUTLINE:
00:00 Introduction
01:28 Historical background
02:50 Curve Fitting problem
06:26 Random vs guided adjustments
09:43 Derivatives
14:34 Gradient Descent
16:23 Higher dimensions
21:36 Chain Rule Intuition
27:01 Computational Graph and Autodiff
36:24 Summary
38:16 Shortform
39:20 Outro
USEFUL RESOURCES:
Andrej Karpathy's playlist: • Neural Networks: Zero ...
Jürgen Schmidhuber's blog on the history of backprop:
people.idsia.ch/~juergen/who-...
CREDITS:
Icons by www.freepik.com/

Пікірлер: 365

@ArtemKirsanov 2 ай бұрын

Join Shortform for awesome book guides and get 5 days of unlimited access! shortform.com/artem

@TNTsundar 2 ай бұрын

Can you talk about liquid neural networks? I’m interested to know if that’s a revolutionary work that deserves more recognition and following. arxiv.org/pdf/2006.04439.pdf

@Mutual_Information 2 ай бұрын

Back prop is a hard, heavy thing to explain, and this video does it extremely well. I mean, that section 'Computational Graph and Autodiff' might be the best explanation of that subject on the internet. I'm very impressed - well done!

@33gbm 2 ай бұрын

You two are the best channels I have found in the SoME episodes. It's great to see this interaction between you guys.

@dprophecyguy 2 ай бұрын

Love your videos

@michaelcharlesthearchangel 2 ай бұрын

If there is no mention of sine waves in neural networks then it won't be total.

@ExtantFrodo2 7 күн бұрын

Where is that section 'Computational Graph and Autodiff' ?

@priteshtadvi4946 13 күн бұрын

I knew that calculus is important for machine learning but never knew that 12th grade derivatives are that much important. When you said about chain rule, that bring me back to my school days , I never thought that derivatives, integration and probabilities will be used this way in future. Well explained video. Thanks for sharing this knowledge and conveying process much simply.

@undertheshadow 2 ай бұрын

"Wait, It's all derivatives?" "Always has been" Great work pal. Provides excellent clarity. Looking forward to the second part.

@rad6626 Ай бұрын

😂 Turns out back propagation isn’t just magic

@CuriousLad 2 ай бұрын

Funnily enough, the calculus portion of the video is probably one of the best explained I've seen

@George70220 2 ай бұрын

Why would that be 'funnily enough'? What a diss lmao.

@balu6923 2 ай бұрын

@@George70220 I don't think CuriousLad meant it as a diss, it's just that when Artem made the video, he explained the Calculus section as a background information. The partial derivates and gradient descent wasn't the main topic of the vid, yet you could show this to Calculus I student and they would be thanking him for the explanation, even if they have not interest in learning back propagation! That's why funnily enough, while the intro Calc topics wasn't the main part of the video, that portion would be very helpful to anyone starting out int Calc!

@veritas7010 2 ай бұрын

I dont agree for example the act of minimizing loss function and gradient descend were not properly linked there were just two pieces of information unprocessed dumped in series

@keithwallace5277 Ай бұрын

This has to be one of the greatest explanation of the inner working of learning in ML, I love it!

@dianasong4594 14 күн бұрын

indeed

@vastabyss6496 2 ай бұрын

It makes sense that you would cover both computational neuroscience AND machine learning since they both play a significant role in AI research. The sort of content you're making is definitely 3Blue1Brown level. Keep up the good work!

@nickwissler6811 22 күн бұрын

He also managed to squeeze an entire calc 1 course into this single video. It's amazing

@Redant1Redant 20 күн бұрын

That was an outstanding explanation. Your ability to explain higher mathematical concepts in such simple terms is really an amazing service to the rest of us who wanna understand these subjects but don’t have a mathematics degree. Thank you.

@matheusmendonca1332 2 ай бұрын

By far the best ML explanation I have seen on internet.

@maheshwaransivagnanam6452 15 күн бұрын

I've been trying to get into ML for quite a while now. This is by far the best explanation of gradient descent and back propagation hands down!!! Amazing work!!!

@asdasd-yr7wi 2 ай бұрын

31 years now, had like 13 years of math in school and another 5 years at university, first time i really understood how derivatives work, bcs visualisation instead of "you calculate it this way and derive it that way, now memorize"

@ArnaudMEURET 7 күн бұрын

May I ask which university you went to?

@krishnagupta31 Сағат бұрын

this is the most intuitive video I have ever come across. Amazing work!!!!!

@gianlucanordio7200 2 ай бұрын

I just have to say this goes way beyond the quality of the many chainrule videos I've seen so far. Good job man, you've got some impressive skills to keep me watching a math video and take notes past my usual bedtime

@marc_frank Ай бұрын

you take notes?

@f_pie 2 ай бұрын

This is the best ML explanation I have seen on YT

@ReighKnight 2 ай бұрын

The visuals on this video is from another planet . So Good !!!!!!!!

@shikhargairola5815 2 ай бұрын

It’s probably the best explanation of backward propagation. Hats off to your hard work and saving this so valuable content.

@Anonymous-fr2op 2 ай бұрын

Damn, I was wondering where you've been since over half a year, whilst I was stuck in backpropagation😂 and here you came back like a true mind reader. Glad to see you back❤

@highchiller 2 ай бұрын

He was calculating your backward step so you can make your next forward step (sorry, couldnt resist) XD

@bungerwow7963 Ай бұрын

I've seen probably 20 videos on this and your explanation of the derivatives for someone not in calculus was really helpful. thanks.

@pradhumnkanase8381 2 ай бұрын

There could not have been a better explanation. Hats off to you

@danielgsfb Ай бұрын

What an amazing video. I hope one day they come up with some world prize for 'free education heroes'. 173k views for a video like this is simply disgusting. This guy deserves maybe 2 billion views. God damn it, that makes me mad.

@user-cm5fj8qz8k 2 ай бұрын

this's by far the most clearer explaination and simplification of backpropagation i have watched

@Alwaysiamcaesar Ай бұрын

I actually pictured this all in my head successfully where I thought I had everything in a canonical deep neural network figured out the other day. It’s one thing to hold it, it’s another to do the detailed, gritty work of explaining it in video format. Very well done.

@moralboundaries1 2 ай бұрын

So clear and concise! Thank you for creating this.

@AlexKelleyD 2 ай бұрын

This is one of, if not the, best videos I’ve seen that throughly explains back propagation. It will definitely help me to be able to better explain the algorithm to others, so thank you for creating it.

@kakandeemmanuel7410 9 күн бұрын

I cannot tell how much excited this video has got me once I realized I am understanding every single step effortlessly.😂😂😂 Thanks so much for the explanation. God bless you!🙏🙏🙏🙏🙏🙏

@MrRhainer Ай бұрын

The best explanation about Deep Learning. Grateful.

@aabiddd 2 ай бұрын

all these basic concepts such as derivatives, least square method, I'm learning it in my college. watching these kind of machine learning videos has made me understand the practical applications of these theoretical concepts a bit better now 😌

@Master_of_Chess_Shorts 2 ай бұрын

This has to be the best explanation of the chain rule ever! Thanks

@K9Megahertz 2 ай бұрын

This is a visual masterpiece! Well done! Much of this was a review for me as I took the time to go through all this last year. I did an implementation of the MNIST handwritten number neural network and had to learn all the calculus covered here to work out the backpropagation math. You really do have to dig in to it to get a good handle on it but it's fun stuff.

@user-dw8sz5mw3m 2 ай бұрын

It's very very nice to see that are you updating.

@Maverick99xx 2 ай бұрын

This is incredibly well done and helped me visualize derivatives comprehensively. Thank you.

@cachegrk 2 ай бұрын

This is the best ever explanation I have seen. Thanks for taking the time and doing something extraordinary.

@black_crest 2 ай бұрын

This just might be the most underrated video on Back Propagation that I've ever seen! I hope more people come across this

@nayanahgowda3219 Ай бұрын

Hands down the best explanation I have seen so far! So clear and easy to understand!!

@chilledpepsi Ай бұрын

Hands down the best explanation there is to backprop

@tonsetz 2 ай бұрын

He is back! Greetings from Brazil, we've all been waiting for this release!

@naveen_malla Ай бұрын

Dude, this is the most beautiful ML video i've ever seen. Highly informative yes, but also beautifully made. Thank you for your work.

@TysonPower 2 ай бұрын

Always impressive! Looking forward to the second one.

@martonbalassa8128 2 ай бұрын

This is the best youtube channel in my feed, and I have many.

@brahmatejachilumula2668 2 ай бұрын

Beat graphical experience with a clear information, Really enjoyed throughout the video !!!

@user-yb2sx4zz4y Ай бұрын

The world needs more of you bro

@kltr007 2 ай бұрын

This video explains the mathematical base of neural networks in a way I understood it the frist time enough to be able to explain it to somebody else. Thank You for that. I can't even imagine how much work you put into the animations. A master piece!

@MaitreJedi19 2 ай бұрын

Animation is great, but more and more people are doing it now. What make this special is the story, the complexity build-up is perfect and efficient. One needs a deep understanding of the subject and strong teaching skills to produce this.

@mdrumi629 16 күн бұрын

Great explanation! Waiting for the second part.

@winterknight1159 2 ай бұрын

I have been doing ML research for a few years now but somehow I was drawn to this video. I am glad to say that it did not disappoint! You have done an amazing job, putting things in perspective and showing respect to calculus where it is due. We forget how a simple derivatives powers all of ML. Thank you for reminding that!

@ArtemKirsanov Ай бұрын

Thank you! That’s really nice to hear!

@XxIgnirFirexX 2 ай бұрын

I think I just found my favourite channel of all times. I've been on YT since 2011 and never had a crush for a YT channel before today é.è

@RohitKumar-pu4nm Ай бұрын

Спасибо, это лучший канал связок, все работает, буду это пробовать.

@kalebnegussie8140 11 күн бұрын

Excellent explanation. I am going to rewatch this a few more times. Well done and thank you.

@ChPonsard 2 ай бұрын

Excellent video, thank you. I'm already looking forward to the synaptic plasticity video!

@HeatherRoberson-vx5eh 20 күн бұрын

As a student in this business, who has passed through a bunch of professors, I can say with confidence! With this trader, you will both learn and earn and, importantly, receive advice. Everything is competent and clear, without a bunch of any unnecessary movements! Keep up the good work!🤣

@ahumanperson3649 2 ай бұрын

Great video! Very elegant explanation of back propagation, and I’m super excited to see the different mechanics of biological neural networks! Keep up the good work.

@SilentderLaute 12 күн бұрын

These Animations are Insane :3 Great Video.

@chakravarthyelumalai8408 2 ай бұрын

A million dollar explanation. Thank you @Artem

@darkyz543 Ай бұрын

Absolutely marvelous!!!! Thank you.

@andrewshort6440 24 күн бұрын

Magnificent work, from the beautiful, creative, elegant design, to the mastery in teaching. Thank you!

@OscarGGL 2 ай бұрын

Artem back with another masterclass!

@ks0ni 2 ай бұрын

Wow, hats off to you! Can't even imagine how long it takes to make something like this

@slk627 2 ай бұрын

thank you so much! The most clear explanation of the topic i've seen so far, amazing job! I wish i had this kind of videos during school education.

@eurob12 11 күн бұрын

Very well explained how backpropagation and how the loss function helps in determining the optimal minimum by using calculus, great detail which helps newbies like me understand this complex topic much better.

@_1jay 2 ай бұрын

criminally underrated

@ac695 Ай бұрын

Amazing video. Underrated channel.

@francescobranca653 2 ай бұрын

Very insightful video. Can't wait to see the second part. I would really love to see a video from you on spiking neural networks too!

@pentachronic 3 күн бұрын

Excellent video. Thanks for this.

@dwinsemius 18 күн бұрын

In traditional statistics (which preceded machine learning by many decades) the "loss function" was called the "deviance" or "the variance"

@qoobes 2 ай бұрын

This is insane. I loved the video, keep it up!

@The-Martian73 Ай бұрын

If you couldn’t understand this explanation, visualization, clearness … there’s nothing else can work with you I swear

@woddenhorse 6 күн бұрын

Amazing Visualisation 🔥🔥

@myelinsheathxd 2 ай бұрын

Thank you for illustration!

@DB-nl9xw 2 ай бұрын

Make more videos like this. I learned so much. Thank you for making this great videos.

@halilzabun 22 күн бұрын

One of the best visual explanations of the backpropagation algorithm I've seen! The animations are really good.

@javastream5015 20 күн бұрын

Sure that it was the back propagation algorithm?

@TurinBeats 4 күн бұрын

Waiting patiently for the second video 🫰♥️. Much love from Kenya, thank you for making me understand back propagation. Started watching your channel because of Obsidian, stayed for the AI lessons 🫰.

@shizzm1990 2 ай бұрын

Some people just want to see the world learning. Great Video Artem!

@ususa1985 Ай бұрын

Very simple explanation and easy to understand, thank you very much Artem :)

@Ant3_14 2 ай бұрын

You are the best source of understanding computation that is biological and organic (all ml stuff), thank you.

@benmuller6103 2 ай бұрын

Excellent explanation - I already understood this conceptually but this video gives a very good intuition for the repeated chain rule application

@phoneix24886 25 күн бұрын

You are a great teacher. Keep up the good work.

@gersonrodriguez9005 Ай бұрын

Ya ví el video completo como 5 veces en estas semanas, este tema me fascina

@arvindnanjundaswamy2442 2 ай бұрын

Outstanding explanation. Thanks

@EDM179 Ай бұрын

Great job Artem

@ram-my6fl 2 ай бұрын

Most Comprehensive Explanation EVER my opinion : better than 3b 1b, No offence to 3b 1b Hes great at it and one of the pioneers who did these kind kf visual explanations. But i like your explanation as it is slow paced & comprehensive

@domorobotics6172 27 күн бұрын

Yeah 3b1b definitely deserves respect from me, but I think he will to recognize this video is very carefully done. I like that these people just care about the truth and the perfection, and even with a little bit of envy, care about the best product being done.

@philipm3173 2 ай бұрын

This is just superb, thank you Artem! Timing couldn't be any better as the gradient descent algorithm was mentioned in Grahaene's "How We Learn" which I'm currently reading.

@Nico-eq3rf 29 күн бұрын

What a banger, this helped alot. Thanks for making this video :)

@trbt177 10 күн бұрын

this is fantastic material even more fantastic to post docs in this field

@puffinjuice 23 күн бұрын

Very well explained!

@syedzulqarnain1002 29 күн бұрын

Great video on this topic. Very well organized briefly touching all related concepts

@persevere1052 2 ай бұрын

Fantastic explanation and animations!

@pcwalid 2 ай бұрын

Thank you for this excellent explanations !

@MissPiggyM976 2 ай бұрын

Wonderful video, many thanks!

@TrendTrackers111 22 күн бұрын

Thank you very much. Lot of love from India !

@kaminenianirudh 2 ай бұрын

Yo, I'm hyped for the next video

@visunashokkumar1782 4 күн бұрын

Man this knowledge is Gold for me

@kleytondacosta8228 2 ай бұрын

Really nice work! Congrats.

@mohanbhosale5890 2 ай бұрын

omg, what an explanation. You legend, more power to you !!!

@mehranshafieecheyki156 2 ай бұрын

I enjoy watching your videos, thank you .

@y5mgisi 2 ай бұрын

Man this is such a great channel.

@EllisKhloe Ай бұрын

Please update new video soon! I am really looking forward to it!

@-mwolf 2 ай бұрын

Amazing explanation!

@aminyazdanpanah2940 29 күн бұрын

Wow such great video!!!! Keep up the great work

@4th_wall511 22 күн бұрын

bro im 2 minutes in and your graphics are insanely good I can already tell this is going to be a treat. Holy smokes man I'm having a graphicgasm

@sirousmohseni4 Ай бұрын

I will need to watch this again.