Bias Variance Tradeoff Explained!

Рет қаралды 5,012

Жыл бұрын

What is Bias? What is the tradeoff between bias and variance? These questions and more answered today!
ABOUT ME
⭕ Subscribe: kzfaq.info...
📚 Medium Blog: / dataemporium
💻 Github: github.com/ajhalthor
👔 LinkedIn: / ajay-halthor-477974bb
RESOURCES
[1] What are estimators? en.wikipedia.org/wiki/Estimat...
[2] The importance of Mean Squared Error and it's bias-variance decomposition: en.wikipedia.org/wiki/Mean_sq...
[3] Unbiased Estimators: en.wikipedia.org/wiki/Bias_of...
[4] Bias Variance Tradeoff: en.wikipedia.org/wiki/Bias-va...
[X] More resources on bias-variance decomposition: rasbt.github.io/mlxtend/user_g...
MATH COURSES (7 day free trial)
📕 Mathematics for Machine Learning: imp.i384100.net/MathML
📕 Calculus: imp.i384100.net/Calculus
📕 Statistics for Data Science: imp.i384100.net/AdvancedStati...
📕 Bayesian Statistics: imp.i384100.net/BayesianStati...
📕 Linear Algebra: imp.i384100.net/LinearAlgebra
📕 Probability: imp.i384100.net/Probability
OTHER RELATED COURSES (7 day free trial)
📕 ⭐ Deep Learning Specialization: imp.i384100.net/Deep-Learning
📕 Python for Everybody: imp.i384100.net/python
📕 MLOps Course: imp.i384100.net/MLOps
📕 Natural Language Processing (NLP): imp.i384100.net/NLP
📕 Machine Learning in Production: imp.i384100.net/MLProduction
📕 Data Science Specialization: imp.i384100.net/DataScience
📕 Tensorflow: imp.i384100.net/Tensorflow

Пікірлер: 21

@Raj-gc2rc Жыл бұрын

Do we have any mathematical proof for when we increase the model complexity the bias of the estimator decreses ? Like expected value of the estimator tends to the population parameter as complexity increases.. Can you explain this in the context of deep learning?

@CodeEmporium Жыл бұрын

Lemme preface this by saying I am not a math expert and that this is a very good question that has gotten me thinking. To me, Statements like “increased model complexity decreases bias” stem more from intuition (strengthend by say, a simulation) than mathematical rigor. I hesitate to say “prove” since I am not sure if we can say model complexity _always_ will decrease the bias of an estimator. This statement is dependent on data after all. I would instead say “we can build intuition to support this statement for the most part”. “Complexity” can mean many things. But let’s assume it is “increasing the number of parameters in a model”. The way I would think of proving your statement (or rather building intuition) is to create some known distribution; sample data from it; train a base model like y = constant + epsilon; calculate bias of the output y by using many rounds of training and eventually testing each model on the same input X. Then do the same for a slightly more complex model like y = theta1 X + theta0 + epsilon. Keep adding parameters to add complexity. You can then just build intuition to see how the bias (and variance ) change with this complexity.

@CodeEmporium Жыл бұрын

To your last question, I wouldn’t think about bias and variance much more different for a linear regression than I do for a neural network. Internally, yes they work differently. Neural networks can have many tunable parameters (as can other regression models). But on the surface, a linear regressor and a neural network take the same inputs, try to learn some parameters to predict some output. They do so sometimes with even the same loss functions like mean squared error which decompose into a bias and variance term in similar ways. I could be wrong. But this is my intuition. Happy to hear disputes, other perspectives too. Great question!

@Raj-gc2rc Жыл бұрын

@@CodeEmporium tysm for answering ... What's the best approach to learn machine learning for jobs vs for reaserch? ... I have habit of getting into mathematical formalism for everything ...I don't know if it is the right way of learning... Can I mail you for querues regarding data science ?

@CodeEmporium Жыл бұрын

@@Raj-gc2rc Definitely no silver bullet. I think just spending time learning about the concepts, then delving into some application code, then the mathematics helps for a start. With this, you can build general intuition. I find myself always revising concepts but greater detail every time. Every time I touch a concept, I learn some things I didn’t know before For jobs, I would try working on some machine learning projects. With these projects, you can test your intuition. Doing so you might realize that some concepts your learned as “hard truths” actually are not (as you and I alluded to). In interviews, it’s common to talk about case studies. So not only would you need to know core concepts, you need to relate them to a real world problem. This is a nice next step

@kinanradaideh5479 Жыл бұрын

The best video on this topic, and ive been looking all day for vids on this topic! Never stop please, your videos are excellent!

@vitoriacoghi 2 ай бұрын

Great explanation. Thank you ☺

@mdsharaf95 Жыл бұрын

That was awesome. Please keep posting.

@CodeEmporium Жыл бұрын

That’s the plan! Thank you :)

@sneha_more Жыл бұрын

Thank you for the detailed explanation.

@CodeEmporium Жыл бұрын

You are very welcome!!

@seekerpro Жыл бұрын

Amazing work!

@CodeEmporium Жыл бұрын

Thank you so much!

@ahmedgabr8009 Жыл бұрын

Great Video

@CodeEmporium Жыл бұрын

Thank you!

@jameszhang9628 Жыл бұрын

Amazing!

@CodeEmporium Жыл бұрын

Thank you:) I’ll

@ajaytaneja111 Жыл бұрын

Hi Ajay, excellent video. Can you advise of the reference that explains about the MSE as you did? Thanks again

@CodeEmporium Жыл бұрын

Ajay! Thank you! In the description, there are 5 resources that I read to surmise my thoughts. Some of them are wiki articles that I thought were well written. About MSE specifically, I do think the Introduction to Statistical Learning text book has great insight. Let me know if you needed more specifics, but I hope this is a good start for you!