“Do neural networks work because they reason like a human. No. They work because they fit the data.” You should have added “boom. mic drop.”. Excellent video!

DUUUUUUDE. I watched all the people you'd expect, 3Blue1Brown, StatQuest etc. I was so lost. Then I was ranting to a friend about how exasperated I was, and he sent me this video. BAM. Everything clicked. You're the man. Also, love how you ended it with a banger.

BRUH. This video gave me that amazing feeling when something clicks in your brain and everything all of a sudden makes sense! Thank you I have never seen neural networks explained in this way before.

I was expecting a duplicate of many other nueral network videos, but this was a perspective that I have not seen before! Awesome Video!

This is THE best intuition behind neural networks I have ever seen. Thanks for the great video!

Getting some bit of math history context makes these extra enjoyable. Great video, explanation, and visualization.

Oh my lord. I've been struggling with neural networks for awhile and I've always felt like I have a decent grasp on them but this video finally brought everything together. Beautiful introduction

Simply and elegantly explained. The bit at the end was superb.

Brilliant explanation! Very glad I stumbled on your channel!

amazing viewpoint of explanation. Would've loved an additional segment using this viewpoint to do the MNIST image recognition

Best explanation of parametric calcs ever! Bias & weights has new meaning

This is the best video I have ever seen on the internet that describes what a neural network is actually The best and most powerful explanations are those that give you the intuitive meaning behind the math and this video does it perfectly When a video describes a neural network by jumping into matrices and talking about subscripts i's and J's, they're just talking about the mechanics and do absolutely nothing about making you understand what you're reading Unfortunately, this is how most textbooks approach the subject and it's also how many content creators approach the subject as well This type of video only comes from someone who understands things so deeply that they're able to explain it in a way that involves almost zero math I consider this video one of the true treasures of KZfaq involving artificial intelligence education

Fantastic video! I have been working with econometrics, data science, neural networks, and various kind of ML for 20 years but never thought of the ReLU neural networks as just a series of linear regressions until now!

Building an intuitive understanding of the math behind Neural Network is so important. Understand the application of NN gets the job done; understand the math behind of NN makes the job fun. This video helps the latter! Nice video!

Great video! I think this video finally shows what I was waiting for, namely what is the purpose of multiple neurons / layers in a neural network intuitively. This is the first time i have actually seen it explained clearly, good job!

Made my brain goes boom. Seriously thanks for sharing this perspective!

One of the best introductory explanations about the foundational principles of neural networks. Well done and keep up the good work!

I love to get the history lesson first always, Excellent.

This might actually be the clearest perspective on neural networks I have seen yet!

This is such a cool way of thinking about it! You did an amazing job discussing a popular topic in a refreshing way. I can't believe I just found your channel - as a video creator myself, I understand how much time this must have taken. Liked and subscribed 💛

IMO one of the best explanation when it comes the idea/fundamental concept of the NN, Please make more 🙏

Thank you! That is a really brilliant video! I have been using regressions often, but never knew that Neural Network is kinda the same idea. Very enlightening!

This is flat out the best video on neural networks on the internet, provided you are not a complete newbie. Never have I had such an "ahaaa" moment. Clear, consize, easy to follow, going from 0 to hero effortlessly. Bravo.

And with this single video, you earned my subscription.

The best video I have seen in giving one an understanding of neural nets. Thank you. Excellent, looking for more from you.

This is a really great video!! Love the approach with parametric regression.

Just watched Sebastian Lague's video on Neural Networks the other day, and whilst great as always, it was _such_ a standard method of explaining them. Because mostly I just see this explained in the same way each time. This was such a nice change, and really provided me with a different way to look at this. Seeing 'no lin-alg, no calc, no stats' really concerned me, but, you did a great job, just by trying to explain different parts. Such a great explanation - would recommend to others.

Thank you thousands of times, you excellent teacher. Finally, I saw a high-quality and clear explanation of neural networks.

This the clearest video I've ever seen on KZfaq on what a Neural Network is. Thank you so much... you are a star. Could I perhaps ask or encourage you to please create for many of us keen on learning Neural networks on our own, a video practically illustrating the fundamental difference between supervised, unsupervised and reinforcement learning.

I was wondering myself exactly what a simulated NN actually is doing (not what it is, but what it is doing) and this explanation is the best by far, if not THE answer. One adjustment I will suggest, is at the end explain that a simulated NN is not required at all, and explain that alternative systems can also perform the same function, which begs the question, what exactly are the fundamental requirements needed for line fitting to occur? Yes I like to generalize and get to the fundamentals.

Like someone else said, I expected the video to be similar to all the others, but this one gave me so much more, very nice.

Where has this video been all my life! amazing simply amazing! we need more please

THIS was the missing piece of the puzzle I was looking for. This video helped me a lot. Thanks.

Your channel is really amazing! Thanks for making videos.

Great video, this is an explanation I have not heard before. Also I don't know if that abrupt ending was purposefully sarcastic, but I thoroughly enjoyed it lol

I've loved all three of your videos, looking forward to more!

Amazing explanations!! Thank you!!

This straight forward explaining method can save thousands of kids from dropping school "due to math"

good video! very easy to follow.

This EXCELLENT and the best video explaining intuitively what a neural network does. You are seriously brilliant

Another explanation that's needed is to explain the concept of gradient descent (GD), that's the generalized method used to figure out the best fit. Lot's of systems use GD including natural evolution, it's basically trial and error with adjustments, although there are various ways to make it work more efficiently which can become quite complicated. You can even use GD to figure out better forms of the GD algorithm, that is it can be used recursively on itself.

Another great concise visual explanation! Thank you!👍

Thank you! Really beautiful point about layers and exp growth of number of a segments one can make!

Good one! Nice video. In the regression line of Gauss one is not taking the perpendicular distances, though. But very cool video!

wow. very good. keep posting great content like this. you have an better potential to explain complex topics to simpler versions. and there are people who just post content just for the sake of posting and minting money. we need more people like you.

wow, ReLU is an unexpected starting point to explain NNs, but nicely demonstrates the flexibility of summing up weighted non-linear functions.such a refreshing way!

Great concise explanation, and it does works: it fits at least my brain's data like a glove! Not that I have a head shaped like a hand (or do I?), but you did light up some bulbs in there after watching those lines animations fitting better and better. However, what happens when the neural network fits too well? If you can briefly mention the overfitting problem in one of your next episodes, I''d greatly appreciate. Looking forward to the CNNs and transformer ones! 🦾🤖

i'm positive I only got this recommended because of Veritasium's FFT video, but thank you youtube algorithm nonetheless. What a brilliant explanation!

I wish I could like the video more than once.. Great job buddy

480p in 2022 surely takes me back in time. i love it!!

most intuitive explanation ive seen

Excellent! First time I've seen it explained like this.

Wow this is a really interesting view to the neural networks and what role do layers play in it.

god bless you for this work

Hands down the most enlightening ANN series on the net from my perspective, afaik. I'd be happy to pay 5 USD for the next video in the series.

8:47 --> 9:21 Is like watching my brain while I predict some trades. 🤣🤣🤣 "The reason why neural networks work....is that they fit the data" sweet stuff.

Might be the best and most rational neural networks video on KZfaq that I've seen 🤘🤘

Brilliant! Lots to unload from the concluding sentence: neural networks works because they fit the data. Sounds like an even deeper issue than misalignment due to proxy-based training.

Great video. Special thanks for the historical background.

I like this video so much It really shows that ANNs are really just, at the end of the day, glorified multivariate regression models.

Oh wow. This was so much more than I was expecting. And then it all clicked right in at about 9:45

Nicely explained!

This makes total sense thank you. With the last observation of the video, how does that reconcile with statements from the openai team regarding emergent properties of GPT4, that they didn't expect, or don't comprehend. I might be mixing apples and oranges, but if it's just curve fitting then why has some thing substantially changed?

Actually adding a bit of math to this video won't hurt while you add to them visual representation of graphs and formulas. But any way one of the most accessible explanation i have ever seen.

This is such a good explanation

This is a really good explanation

This is the second video of yours that I watch that gives me an eureka moment. Fantastic content. One thing I don't get is, people used to use the sigmoid function before ReLU, right? Was it just because natural neurons work like that and artificial ones were inspired by them?

👍 Super demonstration! how did you generate the graphics? Thanks!

Careful there! You should explicitly mention that you are taking the absolute values of the errors. (Usually we use squares). Without the squares (or abs), the positive and negative errors will kill each other off, and the simple regression does not have a unique solution. Without the squares (or abs), you can start with any intercept, and find a slope that will give you ZERO total error!!

Damn you need and deserve more subscribers!

I loved the video, would you please make another one explaining the back propagation?

beautiful! could you do this type of videos on other machine learning models such as convolution?

That's a great explanation

Love this!

Great video!

Having worked with a simple single-layer 2-synapse neuron in a spreadsheet, I find this video vastly overexplains the topic at a high level, while not going into enough detail. It does, however, go over the linear regression needed for the synapse weight updates. Also it treats the massive regression testing as a benefit instead of a cost. One synapse per neuron in the layer above, or per input if the top layer. One neuron per output if the bottom layer. Middle layers define resolution, from this video at a rate of (neurons per layer)^(layers). Fun fact: Neural MAC (multiply-accumulate) chips can perform whole racks worth of computation. The efficiency gain here isn't so much in speed as it is reduction of power and space, by rearranging the compute units and using analog accumulation. In this way the MAC units more closely resemble our own neurons too.

Thanks for the vid! At about 10:30 you say a NN with n neurons in each of L layers expresses ~ n^L linear segments. Could this be a mistake? I think it's more like n^2 * L

Nice interpretation 😊, please can you make a video explaining how neural networks used in for example digit recognition

This is a really cool explanation I haven't seen before. But I have two questions: Where does overfitting fit in here? more neurons would mean higher risk of overfitting? do layers help or are they unrelated? And where would co-activation of multiple neurons fit in this explanation? e.g., combination of information from multiple sensory sources?

great video!

Really nice video!

Amazing!

Is this piecewise linear approximation of a network a feature of using the ReLU activation function? What if we use a sigmoid activation function instead?

Thanks needed this

Great video! I am left with a question though. If the number of straight line segments in an NN with n neurons in each of the L layers is n^L, then why would we ever use n > 2? If we are constrained by the total number of neurons n*L, than n = 2 maximizes n^L. I have two guesses why use n>2: 1. (Hardware) Linear algebra is fast, especially on a GPU. We want to use vectors of larger sizes to make use of the parallelism. 2. (Math) Maybe the number of gradient descent steps needed to fit a deeper neural network is larger than to fit a shallower NN with wider layers? If you plan to make any more videos about this, this question would be great to address. If not, maybe you can reply here with your thoughts? Thank you!

Top notch video, one thing you might have mentioned is that you only deal with the one-dimensional case here.

phenomenal

Is it right that, at 8:19, the input x is a 2d input (namely x and y) ? Otherwise the points could be plot in 2d ?

Omg great video

Thank you. This is far more intuitive than the usual interpretation with nodes and edges graph with the inputs bouncing back and forth between layers of the graph until it finally gets an output. What are the advantages and disadvantages between this method of approximating a function and polynomial interpolation?

You are math God! . . . subscribed

Very good video

Do you have a video of how to apply this to train a neural network? Would be awesome.

Neural network "training" is just model fitting. It is just that the proposed structure of it is just quite versatile.

Whoa, easy to understand for me.

extremely underrated

Brilliant.

Might have been useful to explain why a nonlinear function like the ReLU is used as the transfer function, or that it's not the only common transfer function, which is kind of implied since you use the identity in the beginning, though that isn't nonlinear. Also the link to the human brain is in the structure of neurons, the conceptual foundation of a weighted sum transformed into a single value by a nonlinear process, dendritic signals combined into a single signal down the axon, at the end of which lie new connections to further neurons. Furthermore in the case of computer vision the structure of the visual cortex served as an inspiration for the neural networks in that field. If memory serves the neocognitron was one such network, not a learning network as it's parameters were tweeked by humans till it did as desired, but foundational to convolutional neural networks. Otherwise interesting enough, stressing the behind the scenes nature of neural networks, though maybe mentioning how classification relates to those regressions would have been cool too. Btw what learning scheme were you using, as far as I could tell it was some small random jump first added and then subtracted if the result got worse? I assume if the substraction is even worse it rolls back the entire thing and rolls a new jump? I ask as it neither sounded nor looked like back propagation was used. The thumbnail still bugs me, the graph representation of the network isn't wrong it just shows something different, the nature of the interconnections between neurons in the network that are hard to see in the graph representation of the resulting regression. It's like saying the map of the route is wrong because it's not a photo of the destination.

Thanks.

this is gonna blow up