In the backward function of the dense class you're returning a matrix which uses the weight parameter of the class after updating it, surely you'd calculate this dE/dX value before updating the weights, and thus dY/dX?

This video, instead of the plethora of other videos on "hOw tO bUiLd A NeUrAl NeTwOrK fRoM sCraTcH", is the literal best. It deserves 84 M views, not 84 k views. It is straight to the point, no 10 minutes explanation of pretty curves with zero math, no 20 minutes introduction on how DL can change the world I truly mean it, it is a refreshing video.

I like how he said he wouldn’t explain how a neural network works, then proceeds to explain it

I love the 3b1b style of animation and also the consistency with his notation, this allows people to learn the matter with multiple explanations while not losing track of the core ideas. Awesome work man

Not only was the math presentation very clear, but the Python class abstraction was elegant.

This is basically ASMR for programmers

Probably the best explaination of neural network of KZfaq ! The voice and the musique backside is realy soothing !

This might be the most intuitive explanation of the backpropagation algorithm on the Internet. Amazing!

Thank you so very, very, very much for this video. I have been wanting to do Machine Learning, but without "Magic". It drives me nuts when all the tutorials say "From Scratch" and then proceed to open Tensor Flow. Seriously, THANK you!!!

This could be 3Blue1Brown for programmers! You got yourself a subscriber! Great video!

The best tutorial on neural networks I've ever seen! Thanks, you have my subscription!

THANK YOU ! This is exactly the video I was looking for. I always struggled with making a neural network, but following your video, I made a model that I can generalize and it made me understandexactly the mistakes I made in my previous attempts. It's easy to find on youtube videos of people explaining singular neurons and backpropagation, but then quickly going over the hard part: how do you compute the error in an actual network, the structural implementation and how it all ties together. This approach with separating the Dense layer from the activation layer also makes things 100x clearer, and many people end up smacking them both in the same class carelessly. The visuals make the intuition for numpy also much much easier. It's always a thing I struggled with and this explained why we do every operation perfectly. even though I was only looking for one video, after seeing such quality, I HAVE to explore the rest of your channel ! Great job.

Thanks for making such great quality videos. I'm working on my Ph.D., and I'm writing a lot of math regarding neural networks. Your nomenclature makes a lot of sense and has served me a lot. I'd love to read some of your publications if you have any.

This video should be the first video you see when you search neural network.

Best tutorial video about neural networks i've ever watched. You are doing such a great job 👏

Very clean and pedagogical explanation. Thanks a lot!

This was the best mathematical explanation on KZfaq. By far.

jesus christ this is a good video and shows clear understanding. no "i've been using neural networks for ten years, so pay attention as i ramble aimlessly for an hour" involved

This video really saved me. From matrix representation to chain rule and visualisation, everything is clear now.

This video is the best on KZfaq for Neural Networks Implementation!

This is the best channel for learning deep learning!

This is a so high quality content. I have only basic knowledge of linear algebra and being a non-native speaker I could fully understand this

this has to be the single best neural network explaining video I have ever watched

Man, I love you. How many times i tried too do the multilayer nn on my own, but always faced thousand of problems. But this video explained everything. Thank you

Absolutely astonishing quality sir. Literally on the 3b1b level. I hope this will help me pass the uni course. SUB!

You are the only youtuber I sincierly want to return. We miss you!

This is a very good approach to building neural nets from scratch.

Thank you very much for your videos explaining how to build ANN and CNN from scratch in Python: your explanations of the detailed calculations for forward and backward propagation and for the calculations in the kernel layers of the CNN are very clear, and seeing how you have managed to implrment them in only a few lines of code is very helpful in 1. understanding the calculations and processes, 2. demistifying the what is a black box in tensorflow / keras.

There are many solutions on the internet...but i must say this one is the best undoubtedly...👍 cheers man...pls keep posting more.

Amazing approach ! Very well explained. Thanks!

best video, very clear-cut. Finally I got the backpropagation and derivatives.

It is the best one I've seen among the explanation videos available on KZfaq! Well done!

one of the best video i have ever seen. struggled alot to understand this and you have explained so beautifully you made me fall in love with the neural network which i was intimidating from. thank you so much.

This is such an elegant and dynamic solution. Subbed!

I think the last row's indices of the W^T matrix at 17:55 must be (w1i, w2i,...,wji). Still the best explannation i have ever seen btw, thank you so much. I dont know why this channel is still so underrated, looking forward to seeing your new videos in the future

Thank you, that's the best video I have ever seen about neural networks!!!!! 😀

Such a great video. Really helped me to understand the basics.

Impressive, lot of information but remains very clear ! Good job on this one ;)

I think most of the ML PhDs dont aware of this abstraction. Simply the best.

Very well-done. I appreciate the effort you put into this video. Thank you.

I developed my first neural network in one night yesterday. that could not learn because of backward propagation, it was only going through std::vectors of std::vectors to get the output. I was setting weights to random values and tried to guess how to apply backward propagation from what i have heard about it. But it failed to do anything, kept guessing just as I did, giving wrong answers anyway. This video has a clean comprehensive explanation of the flow and architecture. I am really excited how simple and clean it is. I am gonna try again. Thank you.

your voice is calming and relaxing, sorry if that is weird

I loved the background music. It gives peaceful mind. I hope, you will continue to make videos, very clear explanation

Whyyyy you don't have 3Million subscriptions you deserve it ♥️♥️

Only 4 video and you have avove 1k subs, Please continue your work 🙏🏼

Thank you so much, my assignment was so unclear, this definitely helps!

Content at it's peak

This is the best video i have seen so far ❤

actually,you saved my life, thanks for doing these

Wonderful, informative, and excellent work. Thanks a zillion!!

I noticed that you are using a batch size of one. make a separate Gradiant variable and ApplyGradiants function for batch sizes > 1 Note 1: also change "+ bias" to "np.add(stuff, bias)" or "+ bias[:,None] Note 2: in backpropagation, sum up the biases on axis 0 (I'm pretty sure that the axis is 0) and divide both weights and biases by batch size

In your code you compute the gradient step for each sample and update immediately. I think that this is called stochastic gradient descent. To implement full gradient descent where I update after all samples I added a counter in the Dense Layer class to count the samples. When the counter reached the training size I would average all the stored nudges for the bias and the weights. Unfortunately when I plot the error over epoch as a graph there are a lot of spikes (less spikes than when using your method) but still some spikes. My training data has (x,y) and tries to find (x+y).

Thank you for really great explanation! Wish you will make even more 😉

Finally found the treasure. Please do more video bro. SUBSCRIBED

That was incredibly explained and illustrated. Thanks

This is so ASMR and well explained!

That was helpful, thank you so much.

amazing video. one thing we could do is to have layers calculate inputs automatically if possible. Like if I give Dense(2,8), then the next layer I dont need to give 8 as input since its obvious that it will be 8. Similar to how keras does this.

This is really dope. The best by far. Subscribed right away

In tensorflow they use weight matrix W dimensions i x j then take transpose in calculation.

you are the best 🥺❤️..wow.. finally i able to understand the basics thanks

This is one of the best videos to really understand the vectorized form of neural networks! Really appreciate the effort you've put into this. Just as a clarification, the video is considering only 1 data point and thereby performing SGD, so during the MSE calculation Y and Y* are in a way depicting multiple responses at the end for 1 data point only right? So for MSE it should not actually be using np.mean to sum them up?

Thank you! Well done! Absolutely wonderful video.

Thanks you so much for your contribution in this field.

Clear, to the point. Thank you. Like (because there are just 722, and have to be a lot more)

after 1000 videos watched, i think i get it now, thanks

whiteout any doubt best explanation of NN ive ever seen - why you stop your productivity my friend ?

I would like alot if u continue your channel bro

Amazing explanation!!

Keep it up .please make a deep learning and ml series for future.

Great stuff! I find it even better than the one from 3b1b. Can you think of any way the code can be checked with matrices outside the learning set?

Very nice and clean video, keep it up

Dude this is amazing

Hi there, great video, super helpful, but at 19:21 line 17 the gradient is computed with the updated weights instead of the original weights which (I believe) caused some exploding/vanishing gradient problems for my test data (iris flower dataset). Fixing that solved all my problems. If I am wrong please let me know. Note: I used leaky RELU as activation function

Big Fan of you from today !

My brain is smoking i dont know what the hell is going on but that is kind of cool Keep it up

This is an amazing tutorial! way better than any textbook I have read. I have a question, let's say you want to convert this to a regression model, thus you don't use an activation function and instead use a dense layer as your final layer. If I want to output two regressed values should the loss be a vector of two values in order to back propagate?

Thank you very very much for this video....

That is so satisfying

I love u , best ML video ever

Very educational

Awesome man!!

Amazing tutorial!

Yeah, this is awesome

this is a great video thank you so much

Well done

Best tutorial💯💯💯💯

Thank you so much for the video!!!

Brother. could you please do a series on neural networks basics to advanced..including cnn

Is there a practical reason why the activation functions are implemented as layers, rather than the other layers, such as Dense, taking the activation function as an argument & applying it internally?

Really great video, it reminds me of 3blue1brown style. I plan on revisiting it tomorrow. What software do you use to create these videos/graphics.

How output gradient is calculated and passed into the backward function?

shouldnt the gradient of the input vectors involve some sort of sum along one axis of the weights matrix? for example if your input vector is shape (2,1) and your weights matrix is (4,2), you do np.matmul(weights,input) and create a vector of shape (4,1), the gradient of the input vector with respect to the output vector is the weights matrix summed along axis 0

Tanh is one of those things that sounds great in principle but in practice people use linear. Frankly I just clamp it these days. Curves kill the spread, clump it all up and lines work fine. They're close enough to curves.

This indeed is the better explanation of the math behind the neural networks I've found on the internet, could I please use your code on github in my final work for college?

Is there a way to train the model, save the training network and use it to make predictions? If you can explain would be very much appreciated. Great video by the way!!

BEST OF BEST THANK YOU

Amazing!!

It is the best It is the Beauty because the explanation is great

Hey. Amazing video man. I just have one question about the mnist Convolutional file in your github. So you only trained your neural network to identify 2 out of 10 classes ie. 0 and 1 but if I were to extend it to all 10 classes with 100 cases each when I preprocess the data, would all the dimensions in your original network layer be the same? like the Reshape and Dense layers? Basically, I was wondering if there is a way I can use the convolutional layer that would account for all classes in the dataset.