How to implement Decision Trees from scratch with Python

Рет қаралды 58,659

Күн бұрын

In the fourth lesson of the Machine Learning from Scratch course, we will learn how to implement Decision Trees. This one is a bit longer due to all the details we need to implement, but we will go through it all in less than 40 minutes.
You can find the code here: github.com/AssemblyAI-Example...
Previous lesson: • How to implement Logis...
Next lesson: • How to implement Rando...
Welcome to the Machine Learning from Scratch course by AssemblyAI.
Thanks to libraries like Scikit-learn we can use most ML algorithms with a couple of lines of code. But knowing how these algorithms work inside is very important. Implementing them hands-on is a great way to achieve this.
And mostly, they are easier than you’d think to implement.
In this course, we will learn how to implement these 10 algorithms.
We will quickly go through how the algorithms work and then implement them in Python using the help of NumPy.
▬▬▬▬▬▬▬▬▬▬▬▬ CONNECT ▬▬▬▬▬▬▬▬▬▬▬▬
🖥️ Website: www.assemblyai.com/?...
🐦 Twitter: / assemblyai
🦾 Discord: / discord
▶️ Subscribe: kzfaq.info?...
🔥 We're hiring! Check our open roles: www.assemblyai.com/careers
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#MachineLearning #DeepLearning

Пікірлер: 67

@KitkatTrading2024 2 күн бұрын

OMG, so glad I found your channel. Your way of education is super useful. Intuition, hands-on coding , what else can I expect?

@TheWorldwideapple Жыл бұрын

by far the best set of videos on ML algorithms from scratch and I have seen so many githubs and youtube videos on the same subject. this content is 10/10

@samuellefischer9596 7 ай бұрын

Unbelievable how helpful this was! More than I learned in lecture

@sherimshin9319 Жыл бұрын

I love you...💖💖 your tutorials saved my machine learning assignments....😭💖

@Karankaran-mx3lb 5 ай бұрын

One of the best videos i watched on the subject. Great Job and Thank you for creating it. 👍🙏

@thangha9100 Жыл бұрын

thanks, i have been searching for this

@AssemblyAI Жыл бұрын

You're very welcome :)

@yusmanisleidissotolongo4433 2 ай бұрын

So nice from you sharing. Thanks so much.

@sunnybhatti9567 Жыл бұрын

Thanks for a clear presentation, it helped me a lot!

@AssemblyAI Жыл бұрын

Great to hear!

@Amir7Souri 10 ай бұрын

very nice and smooth presentation

@yusmanisleidissotolongo4433 2 ай бұрын

You are just GREAT. Thanks so very much.

@patbentolilarhythmking Жыл бұрын

Very nice presentation.

@MrThought2012 11 ай бұрын

I think there is a small error in the entropy calculation at 23:37. The formula shown uses the logarithm to the base 2, so I think np.log2() would be correct as np.log() represents the natural logarithm. Nevertheless, the Videos as well as the whole series is great! Thank you for the content!

@KleinSerg Жыл бұрын

Many thanks

@anatoliyzavdoveev4252 8 ай бұрын

Thank you 🙏

@Wizhka Жыл бұрын

Excelente! Me tomará un par de días digerir todo lo expuesto en 37 min. pero no dudo que valdrá la pena. Gracias.

@AssemblyAI Жыл бұрын

🙏

@chineduezeofor2481 Жыл бұрын

Great video!

@AssemblyAI Жыл бұрын

Thanks!

@sinethembamakoma5485 Жыл бұрын

Super!!! 10/10

@XuyenNguyen-jb9sj 6 ай бұрын

The example in the beginning shows a dataset with mixed data types (categorical and numerical), how ever it looks like the code you provided only handle numerical data points, right?

@runggp Жыл бұрын

wonderful tutorial! I found I enjoy watching ppl coding :)

@AssemblyAI Жыл бұрын

Great!

@jurnalphobia7089 Жыл бұрын

you are so cool... im still learning from the bottom

@vidumini23 10 ай бұрын

Thank you..

@kunalkhurana7822 Ай бұрын

you are awesome! it's just that you use basically a lot. maybe 245 times in a 30 minutes video!

@user-lp6qb2og3t Жыл бұрын

You give an incorrect explanation of p(X) at 4:46 - n is not the total number of nodes, but rather total number of data points in the node

@morzen5894 11 ай бұрын

what type of decision tree are you making ?

@ajithdevadiga9939 Ай бұрын

Great implementation, May I know why GINI index was not considered as It can give better result compared to entropy and information gain. the code complexity might have reduced. kindly share your thoughts 🙂

@LouisDuran Ай бұрын

How might this code change if I were to use the Gini Index instead of Entropy to decide on splits? Does that make sense?

@mahatmaalimibrahim6631 Жыл бұрын

fantastic

@AssemblyAI Жыл бұрын

Thank you!

@MiroKrotky Жыл бұрын

im learning a lot of python while copying your code

@AssemblyAI Жыл бұрын

That's great!

@pranavmadhyastha190 2 ай бұрын

How do i print the decision tree in the output

@sumanthk2781 Жыл бұрын

Same code can you do the chi square for decision tree

@slyceryk5639 11 ай бұрын

Bruh, seeing people translate human language to programming language that easy is amazing. I want that experience too

@BleachWizz Жыл бұрын

what is X and y?

@mattayres9748 Жыл бұрын

Great!

@AssemblyAI Жыл бұрын

Thanks!

@anupamvashishtha1970 Ай бұрын

Maam explain more while writing code as we are beginners

@hansmcconnell1418 Жыл бұрын

thank you sm for this video!! may i ask if this is id3 based?

@mikeoxmaul8678 Жыл бұрын

yep

@bendev6807 Жыл бұрын

Super

@marcelohuerta2727 9 ай бұрын

greeat video, is this id3 algorithm?

@rareschiuzbaian Жыл бұрын

Running this multiple times sometimes results in: File "", line 97, in _most_common_label value = counter.most_common(1)[0][0] IndexError: list index out of range I haven't figured out the reason yet, but I will update this when I find it.

@icam8784 Жыл бұрын

Same for me

@noringo46 Жыл бұрын

modify your check the stopping area. I solve with depth>self.max_depth and n_samples

@gokul.sankar29 Жыл бұрын

Make sure the entropy is a negative sum. That was what fixed it for me

@samuelsykora9246 5 ай бұрын

I think you need to check gain in stopping area, when gain is 0 you don't want to make split right ? If someone search for solution. This helped me.

@chirantanbiswas9330 7 ай бұрын

At around 4:50, I do not understand how p(x) is defined. Can some one help me out on it?

@bhavyajain4973 2 ай бұрын

p(x) is basically probability of getting a sample with category/label x if we select a sample randomly. So its calculated as (num of all samples with label x / num of all samples)

@MohammadBahiraei Жыл бұрын

💪💪💪👍

@wesinec Жыл бұрын

Amazing, but only works for numeric attributes, right?

@roonywalsh8183 Жыл бұрын

I guess classification dec trees work for non numeric attributes... regression ones for numeric

@justinlimjg3052 Жыл бұрын

What about one for GINI

@zinebtighremt2305 Жыл бұрын

Did you find any thing please ?

@saahokrish7219 5 ай бұрын

predict some model not seen data

@hadiashehwar9505 Жыл бұрын

You are using self.value, threshold and many other......what is self used for

@omerisik1032 Жыл бұрын

hello ı from turkey.can you share your decision tree code with me in mail?

@AssemblyAI Жыл бұрын

You can find it here: github.com/AssemblyAI-Examples/Machine-Learning-From-Scratch

@azizmamatov6723 9 ай бұрын

This looks like reciting the code previously learned from somewhere, like rapping in a foreign language. I would first implement the functions, explain them and only then combine them in the class. You can't just build a class and functions, without even explaining or understanding what they do.