Decision and Classification Trees, Clearly Explained!!!
Рет қаралды 692,162
Күн бұрынDecision trees are part of the foundation for Machine Learning. Although they are quite simple, they are very flexible and pop up in a very wide variety of situations. This StatQuest covers all the basics and shows you how to create a new tree from scratch, one step at a time.
NOTE: This is an updated and revised version of the Decision Tree StatQuest that I made back in 2018. It is my hope that this new version does a better job answering some of the most frequently asked questions people asked about the old one.
Note, you may also want to learn about...
Regression Trees: • Regression Trees, Clea...
Bias and Variance (and over fitting): • Machine Learning Funda...
Cross Validation: • Machine Learning Funda...
Pruning Trees: • How to Prune Regressio...
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying my book, The StatQuest Illustrated Guide to Machine Learning:
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
KZfaq Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
0:18 Basic decision tree concepts
3:16 Building a tree with Gini Impurity
9:15 Numeric and continuous variables
12:35 Adding branches
13:56 Adding leaves
14:32 Defining output values
15:12 Using the tree
15:38 How to prevent overfitting
#StatQuest #decisiontree #ML
@statquest 3 жыл бұрын
NOTE: This is an updated and revised version of the Decision Tree StatQuest that I made back in 2018. It is my hope that this new version does a better job answering some of the most frequently asked questions people asked about the old one. Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/
@yulinliu850 3 жыл бұрын
Awesome work!
@tim40gabby25 3 жыл бұрын
The very model of clarity. Thanks :)
@haoyuanliu8034 Жыл бұрын
Hi Josh, great video! I have one question. When you are calculating the total Gini impurity based on the weighted average, why is it not (1 - weight)*Gini instead of (weight)*Gini? Since we want to minimize Gini, wouldn't the Gini value with the most sample size have its overall Gini reduced (as in (1 - weight)*Gini ) instead of increase (as in (weight)*Gini) ? Thanks!
@statquest Жыл бұрын
@@haoyuanliu8034 The more data we have to support something, the more trust we have that that something is correct. Likewise, if I don't have much data to support something, then I should probably have less confidence that that something is correct. And that's what we're doing here. The more data observations we have in a leaf, the more data we have to support the predictions made by that leaf. Thus, the weight amplifies the gini value for the leaf for the most data.
@Noah-xn5th 26 күн бұрын
Hello, when it comes to variables like age: How do u decide if u should use "
@XShollaj 2 жыл бұрын
The complexity of understanding the concepts and explaining them so simply show what a great teacher Josh is.
@statquest 2 жыл бұрын
Thank you! :)
@2andreseg 3 жыл бұрын
Josh, I just finished watching absolutely all your videos on this channel. Congratulations for them, you are the best!
@statquest 3 жыл бұрын
WOW!!! That's a lot of videos! Thank you very much! :)
@simplypqz 3 жыл бұрын
how much are you sponsoring? LOL
@amaarmarco530 2 жыл бұрын
why did you do that, What do you do?
@2andreseg 2 жыл бұрын
@@amaarmarco530 I am Data Scientist, and I wanted to have a really good knowledge about statistics
@taciofbarros Жыл бұрын
Haven't watched them all yet but probably will. And even that you have and will receive more compliments, it's always worth keeping on thanking you for this amazing job!
@LiquidBrain 3 жыл бұрын
Save the environment by planting some decision trees
@statquest 3 жыл бұрын
bam!
@Flylikea 10 ай бұрын
Environmentally friendly decisions
@techlogical8059 3 ай бұрын
@@statquestDouble bam 🎉
@mohmmadwaseq1603 3 жыл бұрын
I am half way through your Machine Learning playlist. It has been so helpful and resourceful, I can't put into words. Thank you Josh.
@statquest 3 жыл бұрын
Thank you very much! :)
@TheSamkerim 2 жыл бұрын
I'd never seen a youtube comment section so full of thankful, enlightened and happy people. You must have revolutionized teaching. Thank you Josh, for these excellent videos. You rock!
@statquest 2 жыл бұрын
Thank you! :)
@hantang7371 3 жыл бұрын
This channel helped me a loooot! It helps me from researching to looking for a job, from recreating myself to exploring the field of statistics and machine learning. You are the best! I can't express my gratefulness in words!
@statquest 3 жыл бұрын
Thank you very much! :)
@driftypencil0148 24 күн бұрын
Absolutely Brilliant. Such a simple explanation. Love this vid
@jithendrayenugula7137 3 жыл бұрын
I never watched Andrew NG's OG course.... i just come back to these videos if I have any doubts or if I need to refresh my knowledge. Thanks a lot josh ;)
@statquest 3 жыл бұрын
Bam! :)
@dfla5472 2 жыл бұрын
If "love at first sight" is real, then this video made me love your way of teaching!
@statquest 2 жыл бұрын
Hooray! Thank you very much! :)
@iftrejom 2 жыл бұрын
Thank you Josh, there is no channel on KZfaq (or maybe on the Internet) that explains this topics as nifty as you do.
@statquest 2 жыл бұрын
Thanks!
@Dan-wq8id 3 жыл бұрын
Brilliant, thanks Josh - exactly the slow and steady explaination I needed
@statquest 3 жыл бұрын
Thanks!
@kuikui815 2 жыл бұрын
I am so happy I found this video. Thank you for making it. It is so clear how the decision tree actually works.
@statquest 2 жыл бұрын
Glad it was helpful!
@reshmafernandez2300 2 жыл бұрын
Hi Josh,your videos are the best in understanding the working of machine learning algorithms in the simplest way!!!
@statquest 2 жыл бұрын
Thank you very much! :)
@arpanasingh4679 2 жыл бұрын
I just became fan of yours....the way you teach complicated things with humour and fun, its simply amazing....
@statquest 2 жыл бұрын
Thank you so much 😀
@rbhambriiit Жыл бұрын
BAM!
@tarunkumarbehera6226 4 ай бұрын
This is one of best videos on Decision Trees on the internet. Thanks Josh!
@statquest 4 ай бұрын
Thank you!
@ayushmishra9415 3 жыл бұрын
I will spent my first salary from the job by buying your merch and supporting your channel , you are just great prof
@statquest 3 жыл бұрын
Hooray!!! Thank you very much! :)
@subhamagrawal9894 2 жыл бұрын
hey bhai i need help from u to understand this concepts for my assignment, can u contact to me?
@silentsuicide4544 2 жыл бұрын
about two weeks ago i was trying to learn how the slit is made on numerical data for best split. I was using python for this and was always setting the split space with np.linspace, to find the best split, but the way you showed with averaging a sorted list is very intuitive. If I have only watched this video it would let me save few days of learning how to manually calculate information gain and best split to better understand how DT is working. Great video!
@statquest 2 жыл бұрын
Thank you!
@Emp3 2 жыл бұрын
Thank you SO much Josh. This has been the most helpful guide on decision trees I have come across. :)
@statquest 2 жыл бұрын
Glad it was helpful!
@maxmak7371 3 жыл бұрын
Bam! Josh, I keep watching many videos on your channel! You can explain things in a simpler way! And you videos are inspiring. You are the best teacher I have ever had! We need more great channels like you! Double Bam! Could you make some videos in different distributions series? Like topic in Gamma, chi-squared, beta, poisson distribution, etc. I could hardly find a KZfaq channel that explains them clearly. :( Triple Bam! I know you did not “officially” teach statistics and just made videos for fun, but the world needs you to create more great videos lol! Your “To-do” list would be huge! Keep it up! :D
@statquest 3 жыл бұрын
Thank you very much! :)
@norebar5848 Жыл бұрын
This guy is amazing! I also love how he reminds us of what we were doing, why we were doing it and how we were doing it. Usually, halfway through my lectures I have forgotten where we came from and why we are doing what we're doing. Can't see the forest for all those trees... B)
@statquest Жыл бұрын
Thank you so much! I'm glad you like my videos. :)
@KalyanAngara 2 жыл бұрын
The explanation is so simple and rewarding too. Thank you.
@statquest 2 жыл бұрын
bam!
@lemmaxiom 2 жыл бұрын
Thank you a ton for these Josh, these explanations are super clear. Love the humor too.
@statquest 2 жыл бұрын
Thank you very much! :)
@2NormalHuman Жыл бұрын
amazing video, 18 minutes of your video conveys more useful information than a 3 hours lecture at my uni
@statquest Жыл бұрын
Glad it helped!
@zhihaoxu8119 2 жыл бұрын
You are the REAL GOAT! The best and most intuitive textbook is ISL and your KZfaq video makes this even better. Hats off to your hard work.
@statquest 2 жыл бұрын
Wow, thanks!
@konradkawka6920 11 күн бұрын
Hi, would you mind sharing what textbook you are reffering to? I noticed there is a reference to some textbook in the video. I'm guessing it's referring to the Introduction to Statistical Learning with Applications in R, but I'm not sure about the edition, and at least in the electronic versions, I can't find the relevant information on page 321.
@chief_le0 3 ай бұрын
I'm not used to comment on youtube videos, but this one for sure deserves it. Thanks so much for the explanation, and keep up the good work!
@statquest 3 ай бұрын
Thanks, will do!
@vahiddanesh4661 7 ай бұрын
The best teacher by far I've ever seen in my life! Thank you Josh!
@statquest 7 ай бұрын
Thanks!
@llorensbf 9 ай бұрын
Your explanations are the best!! Instead of teaching the mathematical abstraction first, you teach with a small step by step example that removes the abstraction complexity, so then when reading the formal explanation I can understand it much better. That's the best teaching method, keep doing it this way :D
@statquest 9 ай бұрын
Happy to help!
@JosephLi-wr4cg Ай бұрын
Yet another fantastic stat-quest, Loved these for my class for Deep learning. Keep up the good work!
@statquest Ай бұрын
Thanks, will do!
@jamesvaughan2092 Жыл бұрын
All videos are golden. Thank you StatQuest!
@statquest Жыл бұрын
Thank you!
@beebee_0136 2 жыл бұрын
I'd like to thank you so much for making this stream cast available!
@statquest 2 жыл бұрын
Thanks!
@johng5295 2 жыл бұрын
Thanks in a million! Very well explained. This is the nth time that I am watching this again. Great content. Awesome. I couldn't find this explanation--simply put anywhere else. “Great teachers are hard to find”. Grade: A++ 💥
@statquest 2 жыл бұрын
Thank you very much! :)
@mehediazad1780 Жыл бұрын
best tutor alive on earth. thanks man. appreciate your hard work for us.
@statquest Жыл бұрын
Wow, thanks!
@michaelwangCH 3 жыл бұрын
Josh, your videos do help me as visual Lerner a lot. Thanks.
@statquest 3 жыл бұрын
Happy to help!
@ChaR1ot33r Жыл бұрын
Josh, thanks for these videos and the awesome intros. Your channel really helps me study for my bioinformatics coursework and exams. Much love 💖
@statquest Жыл бұрын
Good luck with your exams! :)
@ChaR1ot33r Жыл бұрын
Thanks 🙏🏼💖
@deprofundis3293 3 жыл бұрын
Thank you so much for this, Josh! I am still fumbling around, trying to figure out the best analysis for my research, and I am new to classification trees and trying to see if they're appropriate for my data. This was a great introduction! I'd LOVE to see an R tutorial for Classification Trees, especially for small sample sizes. And maybe suggestions for what you do afterwards...I'm guessing you have to do more than just classification to build a final model, e.g., some kind of error test or goodness-of-fit?
@statquest 3 жыл бұрын
You can use Cross Validation to get a sense of how well your model will perform: kzfaq.info/get/bejne/nLmpp9143N2mhqs.html
@arenashawn772 4 ай бұрын
I love this video ( in the same spirit of many other of your machine learning algorithm videos) because after watching it, I actually managed to code a simple classification tree on my own to just solidify the things I learned here, and after watching this video, all the parameters in scikit-learn DecisionTreeClassifier are making sense to me. Most of the ML videos and many of the classes out there only talk about very generalized, high-level ideas of these models. You don’t. You always do such a great job giving clear yet detailed explanation of the nitty gritty of these models. Between the ISLP/ISLR books and your videos I am able to gain basic understanding beyond just making api calls of caret in R or sklearn in python. It really made me feel like I am learning, instead of just typing formulas on the keyboard. Could never thank you enough ❤❤❤
@statquest 4 ай бұрын
Hooray!!! I'm so glad you enjoy my videos. :)
@akashvenugopal2489 8 ай бұрын
Thank god, i found your video. You explained it so well, that I literally couldn't control jumping in happiness.
@statquest 8 ай бұрын
Glad it helped!
@_mohitg 3 ай бұрын
your method of teaching is so simple, yet so amazing
@statquest 3 ай бұрын
Thank you very much!
@marcao121 Жыл бұрын
this is BY FAR! one of the best explanations ever!
@statquest Жыл бұрын
Thanks!
@ketangupta549 Жыл бұрын
Your videos are absolutely amazing! Thank you so much for this beautiful content!
@statquest Жыл бұрын
Thank you very much! :)
@leylayim 3 жыл бұрын
You might be the best teacher I've ever had :D
@statquest 3 жыл бұрын
Thank you very much! :)
@OgulcanYardmc-vy7im 2 ай бұрын
Oh great video. Wish the lecturers would have same knowledge about this topic as yours... Thanks man!
@statquest 2 ай бұрын
Glad it was helpful!
@reemgody5492 3 жыл бұрын
Your explanations are always the best. Thank you
@statquest 3 жыл бұрын
Thank you! :)
@easypolitics6673 Жыл бұрын
You are such an amazing teacher Josh!
@statquest Жыл бұрын
Thank you! :)
@dollaya8893 9 ай бұрын
Hey thank you so much. Your video is easy to follow. I can tell that you put efforts and heart in it!
@statquest 9 ай бұрын
Glad it was helpful!
@ericmcnally5128 8 ай бұрын
You are amazing. I really wish that University professors had the ability and drive to actually teach like this.
@statquest 8 ай бұрын
Thank you!
@ShwetaSingh-ry3gd 3 жыл бұрын
Josh...I love you man ...you really making the concepts clear n easy for us.. thanks, thanks n big thanks...Lots of Love from my side and INDIA...
@statquest 3 жыл бұрын
Thank you very much! :)
@dhanushkajayathilaka5711 7 ай бұрын
The best teacher i ever had ....i will send a gift on teachers day Mr Josh!
@statquest 7 ай бұрын
Bam! :)
@dhanushkajayathilaka5711 7 ай бұрын
@@statquest Sir can you pls tell me how should i start ML as beginner. Is this the place that should start kzfaq.info/get/bejne/fdxpkpeqsq6qmH0.html from your tutorial
@aryadubey6802 Жыл бұрын
this was the most exciting and crystal clear explanation . thanks a lot
@statquest Жыл бұрын
Thank you! :)
@rishikeshp7880 4 ай бұрын
You are the best teacher for Stats and ML!
@statquest 4 ай бұрын
Thank you!
@123XanderS 2 жыл бұрын
Another amazing video! Keep up the good work. It helps a lot with studying :)
@statquest 2 жыл бұрын
Thank you! Will do!
@prathameshdinkar2966 11 ай бұрын
Very nicely explained! I can't find better explanation than this!! Double Bam!!
@statquest 11 ай бұрын
Glad you enjoyed it!
@lolomgwtfun 2 жыл бұрын
Hey Josh, I am about to go into my last exam before I graduate and this is the last video I'm watching for a topic that was covered in a day I missed I'm sure you won't see this but thank you for all the help you've done
@statquest 2 жыл бұрын
Thanks and good luck on your exam! Let me know how it goes.
@ioannissioutas5641 2 жыл бұрын
Best channel on youtube, such a treasure!
@statquest 2 жыл бұрын
Wow, thank you!
@beyzanurercan615 2 жыл бұрын
That was very informative. I can do my homework thanks to this video. Thank you so much.
@statquest 2 жыл бұрын
Glad it was helpful!
@gibson3730 Жыл бұрын
Amazing video to learn Decision and Classification Trees from zero to hero!
@statquest Жыл бұрын
bam!
@AnaJessicaOliveiraBatista Жыл бұрын
You are amazing. Great content, pleasing visuals, and great songs.
@statquest Жыл бұрын
Thank you very much! :)
@askpioneer 2 жыл бұрын
It's innovative way of teaching. thanks for creating and uploading.
@statquest 2 жыл бұрын
Thank you!
@oguzhanpolat1216 2 жыл бұрын
I don't know who are you but man you are the best instructor ever I have ever seen. I wish my math teacher met you, she was teaching us the same way you do 😍
@statquest 2 жыл бұрын
Wow, thank you!
@beepybeepy4903 Ай бұрын
Love your explanation! Thank you so much for your videos!
@statquest Ай бұрын
Thanks!
@Chippin-luv 9 ай бұрын
your 2 vids of Knn and this explain better than my 2 hrs lecture and 1 hr lab which done by my uni teacher. thank you
@statquest 9 ай бұрын
Happy to help!
@venkilfc 2 жыл бұрын
Simply brilliant! Thank you!
@statquest 2 жыл бұрын
Thanks!
@ankushkothiyal5372 4 ай бұрын
Thank you, Josh. Based on the methods you provided I tried creating a Python function that calculates the GINI impurity for each independent variable, It really helped deepening my knowledge. thanks again.
@statquest 4 ай бұрын
bam!
@thhiep Жыл бұрын
Thank you for explaining it in a so easy way to understand.
@statquest Жыл бұрын
Thanks!
@user-nk8ry3xs5u 8 ай бұрын
Hi Josh, Many many thanks for you invaluable videos which make complex concepts / models easier to understand! Not sure if you ended up finding where Gini comes from, Wikipedia has is as being named after and Italian mathematician. "Gini impurity Gini impurity, Gini's diversity index,[23] or Gini-Simpson Index in biodiversity research, is named after Italian mathematician Corrado Gini and used by the CART (classification and regression tree) algorithm for classification trees. " (source Wikipedia)
@statquest 8 ай бұрын
Thanks!
@huyentrangvu3361 9 ай бұрын
Thank you for your explaination. It is so clear to understand.
@statquest 9 ай бұрын
Glad it was helpful!
@alikoohi8265 2 жыл бұрын
Great contents and thank you...Please look forward to making your website mobile friendly with more modern appearence.I think that would help you get much more visits on your website...you are the best👍
@shubhamgattani5357 2 ай бұрын
World's best video on Decision Tree Classifier 💚💚💚💚
@statquest 2 ай бұрын
Thank you!
@nithyapalanisamy4904 Жыл бұрын
Thanks for explaining many complex concepts in simplest way.....plz upload more data science basics
@statquest Жыл бұрын
Thanks! I'll keep that in mind! :)
@direnb3401 Жыл бұрын
wow...best explanation ever..I'm impressed. Thanks a lot
@statquest Жыл бұрын
Glad you liked it!
@NeotenicApe 2 жыл бұрын
More important than teaching people statistics and machine learning, you teach people they are capable of understanding things they would of otherwise thought themselves incapable of understanding.
@statquest 2 жыл бұрын
bam! :)
@utkarshsingh2675 Жыл бұрын
such a simple and beautiful explanation...BAM!
@statquest Жыл бұрын
Thank you!
@shraddha5588 2 жыл бұрын
Perfect explanation. Thanks a lot!
@statquest 2 жыл бұрын
Glad you enjoyed it!
@ekosetiawan_indo 2 жыл бұрын
What an awesome explanation, Thanks a lot
@statquest 2 жыл бұрын
Glad it was helpful!
@anishchhabra5313 Жыл бұрын
Splendid job!!🙌
@statquest Жыл бұрын
Thank you!
@vivianyang3130 2 жыл бұрын
Best Explanation I have ever saw!!!
@statquest 2 жыл бұрын
Thank you!
@omerdeniz7007 3 ай бұрын
You explained it great, thanks !
@statquest 3 ай бұрын
Thank you!
@rishipatel7998 Жыл бұрын
Couldn't resist to thank you a SECOND TIME!!
@statquest Жыл бұрын
Double bam!!! :)
@andresg297 2 жыл бұрын
Great video as usual. Thank you.
@statquest 2 жыл бұрын
Thank you! :)
@stukey9751 2 жыл бұрын
You're a hero Josh, thankyou 🧠
@statquest 2 жыл бұрын
Thank you!
@shreyanshvinit7189 3 жыл бұрын
Hey, you are really a cool intellect. Thanks for the cool learning tutorials!
@statquest 3 жыл бұрын
Thanks!
@xjtuwl Жыл бұрын
with this channel, go to school clase is just waste time. Great teacher Josh is.
@statquest Жыл бұрын
Thanks!
@vickylin715 Жыл бұрын
Love your videos so much!!!!!
@statquest Жыл бұрын
Thank you! :)
@m07hcn62 9 ай бұрын
I love the knowledge and the humor in this channel. Baaaam 😎
@statquest 9 ай бұрын
bam! :)
@eg12p34 Жыл бұрын
You know your content is fire when even the professor at our university used your videos in his lectures.
@statquest Жыл бұрын
bam!
@supersql8406 11 ай бұрын
I would be working for NASA by now if all my teachers/Profs were as good and concise as Mr Josh!
@statquest 11 ай бұрын
bam!
@driftypencil0148 24 күн бұрын
Absolutely fucking brilliant. Love this video. Such a simple explanation. Brilliant.
@statquest 24 күн бұрын
Thanks!
@user-lq1cs Жыл бұрын
CLEARLY EXPLAINED SIR THANK YOU!
@statquest Жыл бұрын
Thanks!
@vokoramusyuriy106 Жыл бұрын
Super tutorial. Thanks a lot!!!
@statquest Жыл бұрын
Thank you!
@evarakovska3302 Жыл бұрын
The great job! Thanks, now I have idea how to explain it to my students in easy way ;) I need to translate it, but it is good! Thanks!
@statquest Жыл бұрын
Glad it was helpful!
@nicohambauer Жыл бұрын
Wow great explanation! Veryy well done!! BAM!!
@statquest Жыл бұрын
Glad you liked it!
@surabhipurwar5 Жыл бұрын
this channel is amazing!
@statquest Жыл бұрын
Thanks! 😃
@AnilAnvesh 2 жыл бұрын
Thank you for this video. God bless you ❤️
@statquest 2 жыл бұрын
Thanks!
@ficoalvarez Ай бұрын
Great video! Thanks!
@statquest Ай бұрын
Thanks!
@aishwarya3639 2 жыл бұрын
Hi Josh please make a video on Entropy as well. You have explained Gini impurity so well!
@statquest 2 жыл бұрын
Here's my video on entropy: kzfaq.info/get/bejne/j9qVlaqH3ZOqqKs.html
@Enkumnu Жыл бұрын
Thank you very much. Very clear. Bought your Paperbook. :-)
@statquest Жыл бұрын
Awesome, thank you!
@virtualrendezvous2220 6 ай бұрын
excellent explanation!!
@statquest 6 ай бұрын
Thank you!
@BillVoisine 3 жыл бұрын
How could anyone dislike "Cool As Ice?" Vanilla Ice is the man!
@statquest 3 жыл бұрын
That is the eternal question! :)
@auzaluis 3 жыл бұрын
Hi Josh, great video!!!
@statquest 3 жыл бұрын
Thank you very much! :)
StatQuest with Josh Starmer
Рет қаралды 1,1 МЛН
edureka!
Рет қаралды 497 М.