Support Vector Machines in Python from Start to Finish.

No video

Support Vector Machines in Python from Start to Finish.

Рет қаралды 133,337

Күн бұрын

NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumro...
This webinar was recorded 20200609 at 11:00am (New York Time)
NOTE: This StatQuest assumes that you are already familiar with:
Support Vector Machines: • Support Vector Machine...
The Radial Basis Function: • Support Vector Machine...
Regularization: • Regularization Part 1:...
Cross Validation: • Machine Learning Funda...
Confusion Matrices: • Machine Learning Funda...
For a complete index of all the StatQuest videos, check out:
statquest.org/...
If you'd like to support StatQuest, please consider...
Buying my book, The StatQuest Illustrated Guide to Machine Learning:
PDF - statquest.gumr...
Paperback - www.amazon.com...
Kindle eBook - www.amazon.com...
Patreon: / statquest
...or...
KZfaq Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshi...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer....
...or just donating to StatQuest!
www.paypal.me/...
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
4:16 Import Modules
6:36 Import Data
11:27 Missing Data Part 1: Identifying
16:57 Missing Data Part 2: Dealing with it
21:04 Downsampling the data
24:35 Format Data Part 1: X and y
26:35 Format Data Part 2: One-Hot Encoding
31:25 Format Data Part 3: Centering and Scaling
32:45 Build a Preliminary SVM
34:55 Optimize Parameters with Cross Validation (GridSearchCV)
37:58 Build and Draw Final SVM
#StatQuest #ML #SVM

Пікірлер: 373

@statquest 4 жыл бұрын

NOTE: At 31:25 we should use the mean and standard deviation from the training dataset to center and scale the testing data. The updated jupyter notebook reflects this change. ALSO NOTE: You can support StatQuest by purchasing the Jupyter Notebook and Python code seen in this video here: statquest.gumroad.com/l/iulnea Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

@Dani-hh3qd 2 жыл бұрын

By scaling do you mean data normalization?

@statquest 2 жыл бұрын

@@Dani-hh3qd Normalization is a specific type of scaling.

@baomiTV 3 жыл бұрын

After eight years of employment after graduation, I got laid off in 2020. I went back to school to pursue my second master in Data Science. I was still confused after machine learning classes, but after I watched your videos which were the same topics as the ones in my classes, you led me into a totally different world. Same concepts were taught by you in much easier way. BAM!!!

@statquest 3 жыл бұрын

I'm glad my videos are helpful! :)

@aurkom 2 жыл бұрын

A 2nd master? How much has the curriculum changed in the past 8 years?

@mucahitdemirc 3 жыл бұрын

I will definitely donate to this channel as soon as I got a job! Thanks.

@statquest 3 жыл бұрын

Thank you very much! :)

@kenricktan5271 3 жыл бұрын

I'm so happy to find out that saying BAM + DOUBLE BAM comes naturally to you (and was not just for the videos). Amazing walkthrough as usual, Josh!

@statquest 3 жыл бұрын

Triple bam! :)

@t.t.cooperphd5389 3 жыл бұрын

455 likes and 0 dislikes.... that's a double BAM!

@statquest 3 жыл бұрын

Thanks!

@AntonioRodriguez-bg2mb 3 жыл бұрын

i will be the first one

@ivnesapple479 3 жыл бұрын

Really appreciate for your slow speaking speed ,which makes it possible for not a English speaker ,like me ,a Chinese,to learn.

@statquest 3 жыл бұрын

Thank you!

@randommcranderson5155 4 жыл бұрын

You're a pretty amazing nerd, I love it. This is an amazing tutorial.

@statquest 4 жыл бұрын

Thanks! 😃

@GaMiNGYT-dc2cf 2 жыл бұрын

This guy doesn't deserve the dislike button to be in his videos...what a clear explanation!!!

@statquest 2 жыл бұрын

Awesome! Thank you very much! :)

@shwetaredkar734 2 жыл бұрын

Triple BAM!! Guess What?? You are the best teacher I've ever come across. My life is saved. Good to know you play Tabla too.

@statquest 2 жыл бұрын

Thank you very much!!! :)

@forrest404 Жыл бұрын

I love this kind of webinar where you teach in real time and go through concrete examples. Just purchased the material package and can't wait to go through them with you. I hope you'll make more content like this in the future 😊(I love the short and sweet vids too but I learn by doing so this helps solidify all the theory stuff!)

@statquest Жыл бұрын

Thank you, and thank you for your support!

@khaganieynullazada2794 3 жыл бұрын

Again great work Josh, thanks so much. I actually worked at UNC-Chapel Hill, but I discovered you after moving to another University. Hope will meet you one day to thank you in person for the amazing content you are creating.

@statquest 3 жыл бұрын

Wow! Thank you very much! :)

@ashishgoyal4958 3 жыл бұрын

Thank you so much for making this amazing code-walkthrough for SVM. Looking forward for more code walkthroughs like this.

@statquest 3 жыл бұрын

You're very welcome!

@lucianotarsia9985 4 жыл бұрын

Hi from Argentina. Great video! It really was from start to finish, it covers every step with dedication. Thanks for sharing your knowledge!

@statquest 4 жыл бұрын

Thank you very much! :)

@jack.1. 3 жыл бұрын

Really amazing video, I've been in and around data science and ML for a while but this is the first time I feel like I've gone the full way from mathematical concept -> working program (using medium complexity ML methods) -> insight/ question answered.

@statquest 3 жыл бұрын

Glad you enjoyed it!

@swalehomar3753 3 жыл бұрын

This is amazing! Am in love with your approach of handling these stuff. Very clear and concise.

@statquest 3 жыл бұрын

Thank you! :)

@tolga1292 2 жыл бұрын

You Sir are an outstanding educator.

@statquest 2 жыл бұрын

Thank you!

@roni123467 4 жыл бұрын

Really detailed and nice lesson! I liked how detailed the explanations were, It is definitely DOUBLE BAM worthy! Thank you.

@statquest 4 жыл бұрын

Glad you enjoyed it!

@RahulEdvin 4 жыл бұрын

Josh, you are Phenomenal! Love and Respect from Madras !

@anjalivijay9577 4 жыл бұрын

Respect from kerala too

@sidharths9416 4 жыл бұрын

@@anjalivijay9577 adhaan💥💪

@statquest 4 жыл бұрын

Hooray!!! Thanks! :)

@anjalivijay9577 4 жыл бұрын

@@statquest 🤩🤩🤩🤩

@sidharths9416 4 жыл бұрын

@@statquest BAAAAM

@NaumRusomarov Жыл бұрын

svm are kinda my favourite thing in ML. very simple and mathematically concise yet highly usable.

@statquest Жыл бұрын

Nice!

@moona5454 4 жыл бұрын

I am not an expert but a small help for everyone here ^_^ , if you want to find the missing values very easily, you can type dataframe.isnull().sum() ; dataframe is the name of the object containing the data. And thank you Josh for the amazing webinar ♥

@statquest 4 жыл бұрын

Nice tip!

@sebastioncornejo4440 Жыл бұрын

Haha the double bam at 31:22had me dying lol. Great content! And love your channel!

@statquest Жыл бұрын

Thank you so much! :)

@konstantinlevin8651 Жыл бұрын

I've reread the "hitchhikers guide to galaxy" again (first time I read I was 12) and now it makes a lot more sense why the random state is 42 :)))

@statquest Жыл бұрын

Yes!

@nick9198 2 жыл бұрын

Your dedication is unreal, you replied to all the comments. Wow! p.s. thanks for the video

@statquest 2 жыл бұрын

bam!

@cool_sword 2 жыл бұрын

You already get a lot of love, but I have to add to it and tell you how great these are. No joke, I've had nights when I plan on watching some TV or some movies and I decide to check out some 'Quests instead!

@statquest 2 жыл бұрын

BAM! Thank you very much! :)

@liangke4276 3 жыл бұрын

you video deserves to be translated into more languages so people don't speak English can also learn from your amazing content

@statquest 3 жыл бұрын

Thanks! :)

@ionut5316 3 жыл бұрын

I purchased the notebook and I also watched the whole ad so you can make more money.

@statquest 3 жыл бұрын

Thank you so much for your support! It means a lot to me. BAM! :)

@aktasberk7 Жыл бұрын

Thank you Josh, this taught me a good lesson on both PCA and SVM. Great work!

@statquest Жыл бұрын

Bam! :)

@anikar1302 3 жыл бұрын

i always love those musical intros

@statquest 3 жыл бұрын

Bam!

@vram11 3 жыл бұрын

Precise and to the point. Luv this and I am def going to extend my support to you

@statquest 3 жыл бұрын

Thank you! :)

@annapeng88 3 жыл бұрын

I feel a bit starstruck finally seeing your face... :p Love your videos as always!

@statquest 3 жыл бұрын

😊 thank you

@pareshnavalakha7127 4 жыл бұрын

Hope to listen to the Tabla's behind you at the start of your training one day.

@statquest 4 жыл бұрын

Maybe one day!

@pareshnavalakha7127 4 жыл бұрын

@@statquest Amen to that ✌️

@steelcitysi 4 жыл бұрын

You are awesome. I hope you do something on NLP (tf idf, word2vec, etc.), for some reason your style was made for my brain

@statquest 4 жыл бұрын

Thanks! :)

@md.nazrulislamsiddique7492 Жыл бұрын

Your video is so awesome. Everything related to SVM in one video, BAM.

@statquest Жыл бұрын

Glad you liked it!

@kappa7072 3 жыл бұрын

Josh, you are wonderful! Thanks a million form Italy!

@statquest 3 жыл бұрын

Thank you very much!!!

@sahilpandita2964 2 жыл бұрын

When Josh said 'OH NO!!', I was waiting for the line 'Terminology Alert!!!'.

@statquest 2 жыл бұрын

@KomangWahyuTrisna 4 жыл бұрын

I learned a lot from your channel. I am a big fan of you. Looking forward for your Deep learning and NLP tutorial with python

@statquest 4 жыл бұрын

Awesome, thank you!

@alkapandey1008 4 жыл бұрын

You are amazing. Keep posting. Best wishes from India.

@statquest 4 жыл бұрын

Thank you very much! :)

@irmaktekin3287 3 жыл бұрын

Thanks! I really like the way you explain things: calm and simple :)

@statquest 3 жыл бұрын

Thank you! :)

@pkmath12345 4 жыл бұрын

Love python! Been using R much lately! Would love to have some of R videos

@statquest 4 жыл бұрын

Yes, I'm going to cover all of these topics (and more) in R. For example, R does a much better job with Random Forests than Python.

@pkmath12345 4 жыл бұрын

StatQuest with Josh Starmer I totally agree! Expect videos to come~

@bytesizebiotech 4 жыл бұрын

So, although the publishing company is elsevier, they are not the ones who did the research. If you ever want to read a paper, you can send an email to the primary investigator (the last author of the paper) or any of the first authors really, and they will freely give you the article to read

@statquest 4 жыл бұрын

That's a great idea! :)

@engrmuhammadumar 4 ай бұрын

Those who are facing error can update the code as follow. clf = SVC(random_state=0) clf.fit(X_train_scaled, y_train) predictions = clf.predict(X_test_scaled) cm = confusion_matrix(y_test, predictions, labels=clf.classes_) disp = ConfusionMatrixDisplay(confusion_matrix=cm,display_labels=clf.classes_) disp.plot()

@statquest 4 ай бұрын

Yep. The notebook has been updated.

@martinparidon9056 2 жыл бұрын

Thanks a bunc. Helping me a lot getting started with my SVM. Regards

@statquest 2 жыл бұрын

Happy to help!

@zahrasoltani8630 3 жыл бұрын

Can you explain why you used 'x_test_pca =pca.transform( x_train_scaled) when you wanted to transform test data with PCA?

@statquest 3 жыл бұрын

I decided it was interesting to draw two different PCA versions: 1) of the training data - so we can see the classifier with respect to the data it was trained on and 2) of the testing data - so we can see the classifier with respect to the data it was tested with. So the code has both versions, however, one of them (the latter) is commented out. However, you can swap which line is commented out and draw the latter.

@zahrasoltani8630 3 жыл бұрын

@@statquest Thank you so much

@ramakdixit8648 4 жыл бұрын

Wow. Thanks Josh . Your videos are always a go to resource

@statquest 4 жыл бұрын

Thanks! :)

@elvsrbad2 4 жыл бұрын

This video came out the same week I decided to learn this. Get out of my head!

@statquest 4 жыл бұрын

BAM! :)

@ilducedimas 2 жыл бұрын

what a lovable smart man, thanks for the great work!

@statquest 2 жыл бұрын

Thank you!

@user-tz9sr4fy1z 3 жыл бұрын

Your videos are amazing !!!! I am soo happy u clearly explain many of the the topics I need!! :) (p.s. do u receive requests? I would really love a StatQuest on AR,MA,ARIMA,SARIMA models)

@statquest 3 жыл бұрын

I'll keep those topics in mind.

@bosepukur 4 жыл бұрын

Josh u r an inspiration in teaching...Plz keep it up

@statquest 4 жыл бұрын

Thank you! :)

@omidforoqi4163 3 жыл бұрын

I love StatQuest. please continue to make video with python =)

@statquest 3 жыл бұрын

Thank you! :)

@jamesvalencia3298 4 жыл бұрын

I always ser your videos! Please continue this series of videos and surely I will purchase a notebook soon.

@statquest 4 жыл бұрын

Thank you very much! :)

@joxa6119 2 жыл бұрын

The most perfect guide for SVM in KZfaq. Will donate after I get my first job! Thank you so much. Btw, I have question, why don't you use PCA before doing the modelling part? Are PCA only been use for visualization?

@statquest 2 жыл бұрын

In this case, we only use PCA for visualization.

@joxa6119 2 жыл бұрын

@@statquest I see but, so far what I know it will reduce the accuracy, but will help to avoid multicollinearity. But because of we have done OneHotEncoder, multicollinearity will be not occur. Am I right?

@statquest 2 жыл бұрын

@@joxa6119 Using PCA first would definitely reduce multicollinearity if that was something we thought we needed to deal with. Multicollinearity usually means that we have 2 or more highly correlated features (also called variables), and thus, they are somewhat redundant. One-hot-encoding will not change the fact that those variables are redundant.

@deojeetsarkar2006 4 жыл бұрын

Good to see no haters for the saintly man.

@statquest 4 жыл бұрын

Thanks!

@anuragsharma-os3vj 3 жыл бұрын

Your videos are so informative as always. The way you explain the topics are on another level. But I see a Tabla(twin hand drums) behind you. Do you play that? I also loves to play Tabla. Double BAM!!!! :D

@statquest 3 жыл бұрын

I used to play Tabla a lot. I spent a lot of time in Chennai when I was a kid because my dad taught at the IIT there. When I was there I took lessons on tabla and veena.

@bjornlarsson1037 4 жыл бұрын

Great tutorial Josh! You must truly have one of the highest thumbs up to thumbs down ratios on youtube. Just two questions. 1) Right now you are using standarscaler on all of your variables, including the ones you have encoded. What is your reasoning for this instead of just scaling the continous variables, or maybe it doesn't affect the result? 2) What are your thoughts on onehotencoding before vs after splitting the data? Obviously right now, when your doing get_dummies your are doing it before splitting the data. From what I have understood, whether to do it before or after splitting is a pretty heated topic and I have found several questions on stack exchange where half the people say do it before and the other half say that doing it before is absolutely wrong and that it instead should be done after. In this dataset it will have an effect, because using your random states will produce a train test that on some variables have fewer categories than the test data does, which would mean that those observations should be dropped if onehotenconding is done after splitting. If I instead used onehotencoding before splitting, they would not be dropped. Would love to hear your thoughts on that topic, because I have found no real consenus on what is the right approach. Thanks again Josh!

@statquest 4 жыл бұрын

1) For support vector machines, I'm pretty sure it does not effect the result. However, I have not tried it both ways. 2) I think there is a fear that if you one-hot-encode before splitting the data, then there will be data leakage. With most transformations, this is a problem, but for one-hot-encoding this is not the case. If a value in one dataset does not occur in the other dataset, then the column representing that value will be full of zeros and not have an effect on classification. In fact, the preferred method for industrial pipelines is "ColumnTransformer()", which keeps track of the values during the initial one-hot-encoding and when a testing set has new values, it throws an error.

@bjornlarsson1037 4 жыл бұрын

@@statquest Thanks for your insights Josh! Really appreciate it

@causticmonster 9 ай бұрын

@@statquest is it the same for K-means cluster analysis also ?

@statquest 9 ай бұрын

@@causticmonster Presumably if you use ColumnTransformer().

@abir95571 3 жыл бұрын

929 likes and 0 dislikes ... that's a triple BAM !!

@statquest 3 жыл бұрын

Hooray! :)

@deepanjan1234 4 жыл бұрын

You are just awesome. I just love your videos as they are really amazing. Stay safe .

@statquest 4 жыл бұрын

Thank you! You too!

@jovanagluhovic3139 3 жыл бұрын

It helped a lot! Thank You on shared time and knowladge.

@statquest 3 жыл бұрын

Thank you! :)

@zahrasoltani8630 3 жыл бұрын

Hello Josh, Do you have any lecture about support vector data description (SVDD) as well. Actually, your way of describing problems is amazing.

@statquest 3 жыл бұрын

Not yet! :(

@stardust857 Жыл бұрын

Thank you so much, it was a wonderful video!!!

@statquest Жыл бұрын

Glad you enjoyed it!

@martinparidon9056 2 жыл бұрын

I have a request. You explain brilliantly (also with your background info in other videos) how to create and optimize your SVM. Could you also make a video about how to actually use your svm in a target system? That would make sense I think. Because I think that this would necessitate saving the scaler during creation of the SVM and loading it at runtime. Regards.

@statquest 2 жыл бұрын

Good idea!

@RaviRajput-mq2ew 2 жыл бұрын

This is really great. Thank You Sir for this great effort!!

@statquest 2 жыл бұрын

Glad you liked it!

@TD-in5qe 3 жыл бұрын

This is amazing. Thank you, Josh!

@statquest 3 жыл бұрын

Thank you!

@Jannerparejagutierrez 4 ай бұрын

Thank you very much for the video! I have a question, in SVM should the variables only be numeric or does it also support text? Thank you!

@statquest 4 ай бұрын

Only numeric

@statquest 4 ай бұрын

Hooray! :)

@iunknown563 2 жыл бұрын

Very approachable!

@statquest 2 жыл бұрын

Thanks!

@yeyuan4235 3 жыл бұрын

Josh - Thanks for the video and it is super helpful!! A couple of questions though: 1. Under "Transform the test dataset with the PCA...", should we use the code that you commented out - i.e. X_test_pca=pca.transform(X_test_scaled), instead of X_test_pca=pca.transform(X_train_scaled)? didn't get why we applied the PCA transformation on train dataset to derive testing data. 2. Noticed that 1,000 defaults and 1,000 non-defaults were selected to construct the training sample. Do the numbers of two classes have to be equal for SVM? If not, would this cause any bias as the ratio seems a lot different from the original data? Thank you!

@statquest 3 жыл бұрын

1) Because the SVM was fit to the training data, I wanted to show how it "looked" relative to the training data. However, you can also "see" how the boundary applies to the testing data. It's up to you. 2) Typically it's a good idea to have "balanced" data - data with an equal number of both classes. However, this is not a requirement for SVM - and, whether or not you need it depends on how you want the SVM to perform. For more details, see: kzfaq.info/get/bejne/n7qorbWHsdW4gWQ.html

@DatascienceConcepts 4 жыл бұрын

Awesome teaching! Very interesting lectures.

@statquest 4 жыл бұрын

Thank you! :)

@Mustistics 2 жыл бұрын

One final question (I swear!): At the final code segment, you type X_test_pca = pca.transform(X_train_scaled) Isn't that supposed to be X_test_scaled?

@statquest 2 жыл бұрын

Hmm....I'm actually on vacation right now and can't dig through this code. Can you re-ask this question in a few weeks?

@ndbweurt34485 7 ай бұрын

that tabla behind you tho!😵

@statquest 7 ай бұрын

I used to play and took lessons when I lived in Chennai.

@arjunmallick4901 4 жыл бұрын

Aahhhh....Something that I was stuck with...thanks a lot❣

@statquest 4 жыл бұрын

Hooray! :)

@dhirachatchayaporn8769 2 жыл бұрын

Thank you for great tutorial!!!

@statquest 2 жыл бұрын

Thanks!

@cmpunk3367 2 жыл бұрын

Thanks for the brilliant tutorial Josh! You are truly an inspiration. I just had two questions here :- 1) You applied a regularization technique here by finding the right value for C. What kind of regularization is this? L1, L2 or L1&L2? 2) Is it possible to apply L1, L2, and elastic net regularization on SVMs? If yes, how should I do it?

@statquest 2 жыл бұрын

C controls L2 penalty. I think that might be the only regularization you can use with scikit-learn svm.

@cmpunk3367 2 жыл бұрын

@@statquest Yes I read the documentation of scikit-learn svm and the only other penalty allowed is L1.

@alexandremondaini 4 жыл бұрын

Hi Josh, Thank you very much for your lessons ! you explain very well unlike many teachers. I just have one doubt, when you scale(X_train) and scale(X_test) you're actually scaling the encoded 'categorical' variables. Thus the sparse encoded matrix of 0 and 1 encoded by the features ['SEX','MARRIAGE',....] will be scaled as well, is that correct ? Shouldn't be only the numerical features to get scaled ? Thanks a lot for your lessons

@statquest 4 жыл бұрын

It doesn't really matter if you scale binary variables or not: stats.stackexchange.com/questions/59392/should-you-ever-standardise-binary-variables

@alexandremondaini 4 жыл бұрын

@@statquest thanks for the reply! BAM

@midhileshmomidi2434 3 жыл бұрын

The man behind the voice

@statquest 3 жыл бұрын

@leebradbury8879 2 жыл бұрын

Another great video, I wish I had found this channel years ago! I am assuming the way you have coded for the optimising of Parameters could be used as the basis code for other models like Random Forest and it will just be the parameters changing dependent on the model that is being optimised?

@statquest 2 жыл бұрын

Yes. However, the scikit-learn implementation of random forests is terrible...

@imdadood5705 3 жыл бұрын

How it started: df How it is going: df_23_without_missingdata_scaled_with_magic_powers

@statquest 3 жыл бұрын

Bam! :)

@felipenogueira2462 2 жыл бұрын

I instantly liked the video just for the Ukulele

@statquest 2 жыл бұрын

Hooray!

@toxic_roy 2 жыл бұрын

To all guys like me who tried using the data from uci respository through link, and its not working, its probably becoz pd.read_csv cannot read .xls files. You need the xlrd package( python -m pip install xlrd)[type in command prompt]. then use read_excel instead of read_csv. Enjoy

@statquest 2 жыл бұрын

Thanks!

@Mayday4u 4 жыл бұрын

@StatQuest with Josh Starmer First before all, I really like _all_ your videos especially this one! I have a comment on your comment on your scaling of the X data. You are scaling the train and test data with two sperate scalers. However this is not optimal since the mean/std of the train and test data can differ. For example, let's say the mean of your train data is 5 and the mean of your test data is 6 and that both sets have a std of 1. Then a observation x with a value of 4 is normalized differently depending on when it is observed. If it is observed in the train set then it is normalized to -2 and if it is observed in the test set then it is normalized to -1. This should not be happening since the information of the observation should not depend on the scaling. Further since the model is trained on the train set a value of -2 could have a different meaning to the model and ultimately for the prediction than a value of -1 (if the unscaled trainset contains 3 for example). So you should scale the train and test data using a scaler which is trained on the train data.

@statquest 4 жыл бұрын

That is correct. Sorry for the mistake.

@statquest 4 жыл бұрын

I'll make a note about this in a pinned comment and update the notebook asap.

@user-ib6yl4bu1u 5 ай бұрын

Hi i have a question, aren't we supposed to split the data even more, and then use the validation dataset for hyperparameter tuning, we can pass it to grid_search, e.g. grid_search(x_validation,y_validation) instead of using the training dataset again?

@statquest 5 ай бұрын

You can definitely do that.

@mainakray6452 4 жыл бұрын

gr8 experience, looking for ANN.

@statquest 4 жыл бұрын

Thanks!

@andreabvtt 3 жыл бұрын

Amazing content! How do I know when you have a webinar planned? and where do you stream it? Thanks!!

@statquest 3 жыл бұрын

If you subscribe, you can find out about webinars.

@andreabvtt 3 жыл бұрын

@@statquest excellent, will do!

@whispers191 3 жыл бұрын

Great tutorial! Thank you!

@statquest 3 жыл бұрын

Glad you enjoyed it!

@JoaoVictor-sw9go 2 жыл бұрын

Josh, this video has helped me out a lot in my studies, but I have a question. When we scale the data, we should also include the categorical variables? Shouldn't we just scale all the data excluding the categorical ones?

@statquest 2 жыл бұрын

Because the categorical variables are one-hot-encoded, we can scale them. All of the 0s will stay the same and the 1s will all turn into another constant value. In other words, when one-hot-encoding, 1 is arbitrarily chosen to begin with, so it doesn't hurt to turn it into another arbitrary number.

@JoaoVictor-sw9go 2 жыл бұрын

@@statquest Got it Josh, thanks for responding

@user-kc1hr5ug6d 3 ай бұрын

why do you use 1:1 resampling instead of stratified resampling? The dataset contains 3.5 no_default:1 default. Does this affect SVM results?

@statquest 3 ай бұрын

What time point, minutes and seconds, are you asking about?

@lprashanthi7298 3 жыл бұрын

How do we set values for C and gamma especially the penalty Parameter C .. is it only by Hit and trial?

@statquest 3 жыл бұрын

Pretty much. You just test a bunch with values with cross validation and see which is best.

@kris12326 3 жыл бұрын

Thanks a lot Josh!!

@statquest 3 жыл бұрын

Thanks! :)

@schris3587 4 жыл бұрын

Thank you Josh!!!

@statquest 4 жыл бұрын

My pleasure!!

@tanphan3970 3 жыл бұрын

Dear Josh, My understanding, n_components hyperparameter in PCA() is the number of dimensions that we want to reduce down to. Therefore, I make some confusion. 1. If we use PCA() with no reference any n_components, so what exactly is the number of components in this case? 2. In other tutorials, n_components can set in floating (0.0 to 1.0), it is not make sense if we understand as a dimension number. Thanks, have a nice week!

@statquest 3 жыл бұрын

The number of components is explained here: kzfaq.info/get/bejne/pbimmtRqm5zdips.html

@tanphan3970 3 жыл бұрын

@@statquest Thank for your recommend video. I understanding in this way. when we use PCA() with no n_components hyperparameter, the program will calculate all PCs of data. n_components in this situation is equal to all dimensions of data (pca.explained_variance_ratio_.shape[0]) and when we use PCA(n_components=2) that we only take care 2 first PCs. Sorrry if this question make inconvenience from you. I am only want to sure that my understanding is correct.

@statquest 3 жыл бұрын

@@tanphan3970 Yes, that is correct.

@leonisaacs7231 3 жыл бұрын

Hi Josh, really great content, learning a lot. Out of curiosity when doing One Hot Encoding, is there a reason why you did not say drop-first=True to avoid Multi-collinearity?

@statquest 3 жыл бұрын

Yes, this is different from a linear model.

@moodrammer8205 2 жыл бұрын

Very useful ! Thank you very much !

@statquest 2 жыл бұрын

Thanks!

@hrdyam865 4 жыл бұрын

Thank you very much.. In the radial basis function video, only hyperparameter gamma was involved.. regularization parameter C was not there in the radial kernel function.. Are we using different radial kernel function here or the same one which was shown in radial kernel video? Thanks again.. your videos are great help ..

@statquest 4 жыл бұрын

We are using the same kernel - so the only kernel parameter that we are optimizing is gamma. However, most, if not all, machine learning implementations also include regularization in one form or another. So we'll be talking about that as well.

@NicolasValderrama-pv6qt Жыл бұрын

Very helpful! thanks :)

@statquest Жыл бұрын

@EvandroSegundo 4 жыл бұрын

Great tutorial! In fact, all your videos are great. I have just on question: When looking for the best value for C, the algorithm went for the upper limit. Shouldn't we try again with higher values as suggestions? I haven't tried myself so I really don't know what would happen.

@statquest 4 жыл бұрын

Yes, we should probably try higher values.

@hareezvizard9233 2 жыл бұрын

If my df is 101, what value should i set on n_samples? is there a specific number? like in the video, you use n_samples=1000. one more thing is downsampling the same as splitting? i mean both are the same but different ways/methods or what?

@statquest 2 жыл бұрын

Down sampling randomly selects a smaller subset of the data. In this case we downsample to balance the data. So, it really depends on your own data and goals.

@tanphan3970 3 жыл бұрын

Hello Josh Starmer, Can you explain more about some hyperparameter in resample? replace=False --> we will not change any data in original data (df_default) and if True mean original df_default will be changed? random_state --> help others can get the same result with you? So how many people can get same result to you? 42??? Thanks

@statquest 3 жыл бұрын

1) Yes 2) We are setting the seed for the random generator to the number 42, this ensures that everyone will get the same results. In other words, the random number generator generates a sequence of random numbers based on a starting value. If we all set the starting value to the same number (in this case, 42) then we will all get the same sequence of random numbers.

@finderlandrs7965 4 жыл бұрын

Hey Josh, could you make a video explaining the softmax function? Thanks!

@statquest 4 жыл бұрын

Noted!

@vegaarcturus509 Жыл бұрын

Correct me if im wrong but when people think of machine learning, they think of ai self improvement but SVM is just finding correlations between data sets?

@statquest Жыл бұрын

When I think of machine learning, I think of classifying things and making predictions. SVM can be used to classify things.

@hareezvizard9233 2 жыл бұрын

33:46 how is your svc show many details like the value of C, degree and etc? when i run my code like yours, it only showed SVC(random_state=42)....

@statquest 2 жыл бұрын

I have no idea.