Hooray!!! Thank you very much! BAM! :)

Muito bem!!! (Muito BAM!!! :)

@@statquest Another Brazilian watching this amazing channel: TRIPLE BAM! This Brazilian is a teacher and recommends this channel in all classes: SUPER BAM!

@@jpmagalhaes6645 Muito obrigado!!! :)

I even google to confirm if Itaú was actually a Brazilian company LOL

@@statquest that was amazing, as a brazilian I appreciate this joke very much

Thank you very much and thank you for your support!!! :)

Muito obrigado!!! :)

@@statquest In Portuguese, "Muito obrigado!" - = Thank you very much! - can be replicated in the end by "Eu que agradeço", which brings a sense that the person who received great services or experienced great experiences is thankful in the end. As this seems to be the case...Eu que agradeço, Mr. Josh. Abraços fortes.

@@cesarbarros2931 VERY COOL! I've been to Brazil twice and hope I can return one day to learn (and speak) more Portuguese.

Thank you very much!!! :)

BAM! :)

Thank you very much! :)

Yes, if we have 3 inputs, then we end up with a 4D shape and it's much harder to draw! :)

Glad you like them!

Good luck! :)

Thank you so much 😀!

Hooray!!! Thank you Mikael!!! And thank you for helping me edit the first draft of this video!!! I'm looking forward to talking with you about Convolutional Neural Networks soon. :)

believe it or not these days I'm watching alot of vids on Svalbard..... whatta fantastic place and history... my fav vid is one on seed vault....amazing kzfaq.info/get/bejne/aMV_eNaXkpfVl40.html

Wow, thanks!

Hooray!!! I'm glad you appreciate the work! I spent a lot of time on this video. :)

Thanks!

Thanks a lot 😊!

Thank you! :)

Muito obrigado!

Thanks! There is already a CNN video kzfaq.info/get/bejne/fq2ndbt1sKzPaX0.html and I hope to have an RNN video out soon.

Thanks, will do!

Awesome, thank you!

:)

Awesome! Thank you!

Thank you very much! :)

Thank you very much and thank you for your support!! BAM! :)

Thank you! :)

Thanks! :)

Thank you! :)

We use backpropagation to determine the weights and biases, and backpropagation is covered in these videos: kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html kzfaq.info/get/bejne/n9-eZd2VprLNmWw.html kzfaq.info/get/bejne/fbGKorJ5va3HfKM.html kzfaq.info/get/bejne/rqh1m5lnu5_LiqM.html

@@statquest Great! Thanks a ton.

Glad you liked it!

It takes a lot of time, but it's fun! :)

Bam!

You're welcome!

Thanks! :)

"predominant color" is a categorical value. Therefore you would need to do a one hot encoding before. e.g. blue = (0,1), red = (1,0) for 2 possible colors.

That's correct. The inputs for a categorical input would just be 0 or 1 (rather than values between 0 and 1).

Yes, I can do that.

:)

Probably. And that is, in general, a limitation of all machine learning methods. If new data is outside of the range of the original training data, your predictions are probably going to be pretty random.

Glad to hear that!

Thanks!

1) In this case, the outputs are not probabilities (you can tell because they don't add up to 1). The next video in this series shows that it is very common to add "SoftMax" to this sort of Neural Network to give "probabilities". I put the "probabilities" in quotes because their interpretation comes with some caveats. For more details, see: kzfaq.info/get/bejne/gdZ7ospesZ_alZs.html

2) The inputs don't really affect the loss function or activation function. However, it might effect the number of hidden layers and nodes in each layer.

Welcome!

Thank you!

To learn why the y-axis values are different, see: kzfaq.info/get/bejne/edd_mcxllrLKdKs.html

I didn't add a final ReLU because I didn't need it. There are no rules for building neural networks and you can just build them the way that works best with whatever data you have.

@@statquest Always a Fan of your content and have suggested others to understand the basics of ML through your videos. Really appreciate your quick response on @abdullahalmussawi5291 query.... God bless you brother... :)

Thank you so much! BAM! :)

Thanks!

Nope, it's crinkled surface.

Bam!

Changing the scale on the y-axis just means zooming in on the part we are interested. The values remain the same, we just zooming in on them.

Thank you!

Thanks!

Thanks! I'm pretty sure that items in my store can be shipped pretty much anywhere in the world, including Canada.

Not much. The numbers would be larger and wouldn't fit so nicely in the small boxes I created.

Bam! :)

That's coming up in a few weeks. We'll do an example using cross entropy and softmax.

BAM! :)

Noted

Josh I found the answer: "I started out by creating a neural network with 3 outputs, one for setosa, one for versicolor and one for virginica. I then trained the neural network with backpropagation to get the full neural network used in this video. I then ran the same training data through the network to see which output gave high values for setosa and I then labeled that output "setosa"." Thanks

yep

:)

All of the weights and biases in a neural network are always determined using something called Backpropagation. To learn more, see: kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html

I'm working on a new book focused just on neural networks that covers the theory (like this video) but also has worked out code examples. However, it's still at least a year away. :(

Yes. However, if you'd like to look at examples of how the derivatives are calculated, see: kzfaq.info/get/bejne/gdZ7ospesZ_alZs.html kzfaq.info/get/bejne/g5tpfaidqrbLeZs.html kzfaq.info/get/bejne/bKeihtykmtescYk.html kzfaq.info/get/bejne/rqh1m5lnu5_LiqM.html

I'm going on a real tangent here, so if there's nothing to worry about, a simple 'no' would be a fine answer :D thanks again!

I'm not really sure. My guess is that if you think you might run into this sort of problem, then you need to be careful with how you scale your data.

@@statquest will do, was only asking in case it was a known pitfall but too rare to put into the video. Thanks for your reply & all these videos, I'm sure you get a message every hour on this but you're really the best educator I've ever come across 🙏

I'll keep that in mind.

@@statquest Thank you! I have watched your videos through my undergrad and still watch a lot of them as I am getting my PhD (Epidemiology)!!! Thanks for all you do!

Can you provide me with a link that shows that I got the petals and sepals backwards? Pretty much very single page I found is consistent with what I present here. For example, see: images.app.goo.gl/iAbv954ML8dExUru9 images.app.goo.gl/ugN6JPWs6of1FBWj6 images.app.goo.gl/ZbKVgCkdnC5hdgBA9

The weights and biases are all optimized via backpopagation, just like they are for every other neural network. For details about backpopagation, see: kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html kzfaq.info/get/bejne/n9-eZd2VprLNmWw.html and kzfaq.info/get/bejne/fbGKorJ5va3HfKM.html

I have no idea.

The input for the activation function ends up being: input_1 * w_1 + input_2 * w_2 + ... * input_n * w_n + bias_i. So the derivative, with respect to the bias = 0 + 0 + ... + 0 + 1 = 1 regardless of the number of inputs.

The main idea of neural networks is that they are functions that fit shapes to your data, and by running values through a "trained" neural network, I can illustrate both the shape and how that shape came to be. If you'd like to learn about training a neural network, see: kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html

@@statquest This makes sense, thank you for your great work. I will give the video a watch, going through your series to learn NLP!

@@iReaperYo Here's the whole playlist: kzfaq.info/get/bejne/edd_mcxllrLKdKs.html

Thank you! :)

Designing neural networks is more of an art than a science - there are general guidelines, but generally speaking you find something that works on a related dataset and then train it with your own data. In other words, you rarely build your own neural network. However, if you are determined to build your own, the trade off is this - the more hidden layers and nodes within the hidden layers, the better your model will be able to fit any kind of data, no matter how complicated, but at the same time, you will increase the computation and training will be slow.

Because I didn't need to. There are no rules for building neural networks. You simply design something that works.

Yes, soon

If you scroll down on the login screen you should see a button that says Pay With Debt or Credit Card. It is possible you did not scroll down far enough, though. It's hard to see, however, I just tried it and it worked. However, let me know if you are still having trouble - you can contact me via: statquest.org/contact/

We actually train all 3 outputs at the same time - so those weights work with all 3.

@@statquest Thank you for a quick reply and clearing my confusion !!!

Thanks!

I'll keep that in mind!

All the weights and biases for all neural networks come from backpropagation applied a training dataset, so that's what I used here. If you'd like to learn more about back propagation, see: kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html

I'll keep that in mind.

I started out by creating a neural network with 3 outputs, one for setosa, one for versicolor and one for virginica. I then trained the neural network with backpropagation to get the full neural network used in this video. I then ran the same training data through the network to see which output gave high values for setosa and I then labeled that output "setosa".

See: kzfaq.info/get/bejne/rqh1m5lnu5_LiqM.html

@@statquest oh, really helpfull, tks

It is plotted below zero. But since the number is close to zero, it may not be easy to see.

@@statquest Noted! Thanks for the response Mr. Starmer. Know that you are an incredible teacher and greatly appreciated by us!

In a neural network, weights and biases start out as random number, but are then optimized using Gradient Descent and Backpropagation. For details, see: kzfaq.info/get/bejne/qaqmZ8ll2Ji3cmw.html and kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html

@@statquest BAM

Thank you! :)

I'm not sure I understand your question. In this example, the neural network is trained given the Iris dataset. If we trained it with different data (or even just a different random seed for the same data), we would probably get different values for the biases.

Unfortunately I have no idea.

The weights and biases were derived using backpropagation. For details, see: kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html kzfaq.info/get/bejne/n9-eZd2VprLNmWw.html and kzfaq.info/get/bejne/fbGKorJ5va3HfKM.html

@@statquest Thank you Josh! Will have a good look at them!

I'm currently working on a series of videos that show everything you need to know to create neural networks in PyTorch-Lightning.

That bias term, 0, is the result of backpropagation. For details, see: kzfaq.info/get/bejne/f7Rii9Bzza-wpGg.html

I'll keep that in mind.

Maybe one day I will. :)

Thanks!

Bam!

@@statquest This is really excellent. Do you allow me to link those videos to my students?

@@MrCracou Of course. Please share the links with your students.

Yes, that pretty much sums it up.

@@statquest You are officially a life saver.

What time point, minutes and seconds, are you asking about?

@@statquest (9.44 to 9.57) and (3.44) and (12.1 to 12.9)

Backpropagation, just like all other neural networks. However, now we have 3 terms (one for each output) we have to add together for each input value. That said, there are more elaborate ways to do this, and they are described in these videos: kzfaq.info/get/bejne/bKeihtykmtescYk.html and kzfaq.info/get/bejne/rqh1m5lnu5_LiqM.html

Noted

@@statquest As an example if you want to make the output of a dot product a specific value (say 1) for a specific input vector you can make the angle to the 'weight' vector zero. You may even get error correction in that case (reduction in variance for noise in (across) the input vector.) If you make the angle close to 90 degrees then the magnitude of the weight vector has to be large to get 1 out and the noise will be greatly magnified. The variance equation for linear combinations of random variables applies to the dot product. Understanding such things you may construct say a general associative memory out of the dot product. Eg. Vector to vector random projection, bipolar (+1,-1) binarization, then the dot product. To train find the recall error, divide by the number of dimensions, then add or subtract that to each weight to make the error zero as indicated by the +1 or -1 binarization. If you look into the matter you will see that you have added a little Gaussian noise to all the prior associations (CLT.) The RP+binarization is a locality sensitive hash. Close inputs only give a few bits different in the output. To understand the system you could consider the case of a full hash where the slightest change in the input produced a totally random change.

:)