Why do we split data into train test and validation sets?

Рет қаралды 52,640

Жыл бұрын

To train machine learning models we need to provide the model with a training and testing set. And sometimes even a validation set. These terms tend to be used interchangeably causing confusion. So once and for all, let's learn what each of these data splits do and how they contribute to model development.
👋 Keep in touch?
==========================
🐥 Twitter - / misraturp
🔗 LinkedIn - / misraturp
📹 KZfaq - / @misraturp
🌎 Website - misraturp.com/
Courses & resources
============================
📙 Fundamentals of Deep Learning in 25 pages
misraturp.gumroad.com/l/fdl
👩‍💻 Hands-on Data Science: Complete your first portfolio project
www.misraturp.com/hods
📥 Streamlit template
misraturp.gumroad.com/l/stemp
🤖 Deep Learning 101 with Python and Keras (FREE)
• 50 Days of Deep Learning
🏃‍♀️ Data Science Kick-starter mini-course (FREE)
misraturp.gumroad.com/l/kick-...
🐼 Pandas cheat sheet (FREE)
misraturp.gumroad.com/l/pandascs
📝 NNs hyperparameters cheat sheet (FREE)
misraturp.gumroad.com/l/hcs

Пікірлер: 33

@iaboodws11 Жыл бұрын

Just what I was looking for, your video is so simple and easy to understand, and straight to the point!!!

@misraturp Жыл бұрын

Great to hear, thank you :)

@jamalnuman 4 ай бұрын

Thanks for making a distinction between testing and validation

@sapnilpatel1645 Жыл бұрын

video is very much useful. Your channel is so underrated.

@toyl6727 Жыл бұрын

Brilliant and clear!

@facundostratocaster356 3 ай бұрын

Simple and good explanation, thank you so much ☺️

@syahwiza Жыл бұрын

Thanks for your explanation, it is very useful for me.

@ArifMuhammad-qd6vf Жыл бұрын

Superb

@bay-bicerdover Жыл бұрын

Good one!

@volodyslove Жыл бұрын

You are the best, thank you!😊

@misraturp Жыл бұрын

Thank you!

@misha4915 Жыл бұрын

After finding the best hyperparameters for a model using validation data, should we retrain the model using both the training and validation data before using it on the test data?

@bay-bicerdover Жыл бұрын

@SocialAviation Жыл бұрын

I love your content. Everytime I split my data into train and valid, either using trainsplit function or manually, my val loss does not decrease below 1. The only way to get my val loss lower and lower, is to use part of my train data as validation data 😢

@misraturp Жыл бұрын

Your model might be overfitting in the first case :/

@jamesadeke9873 7 ай бұрын

Good day ma, please can you help me out? I have been trying to figure out this for a long time but i could not. I want to know the best evaluation plots for machine learning models, specifically for classification problems. How best can someone visualize performance? Unlike deep learning models, you can use train and test curves, how best can we visualize using machine learning models? Do you have any video you have done about that? been checking your playlists but i can't find such, kindly help us out. Thanks

@ozgurartok9488 10 ай бұрын

Teşekkürler.

@097_suryakantdhote9 Жыл бұрын

please make a video on logestic regression

@patiklimikrofon 2 ай бұрын

Teşekkürler!

@misraturp 2 ай бұрын

Rica ederim!

@Randomsi_10 16 күн бұрын

This is very helpful on my ongoing thesis🥹

@aowowow-no1xe 22 күн бұрын

What camera do you use?

@jamalnuman 4 ай бұрын

what is the need of testing data is the hyperparameters don't to be optimized?

@jameshopkins3541 5 ай бұрын

Can you explain something about it????? Example the meaning and useful of each one

@jsingh190 Жыл бұрын

Can you make a project use of some ML application

@bay-bicerdover Жыл бұрын

0:50'de blop efekti ödümü kopardı

@SyamKishoreNaidu Жыл бұрын

Can we expect more pandas related videos

@misraturp Жыл бұрын

For sure! more to come.

@jameelabduljalil25 Жыл бұрын

No demo?

@misraturp Жыл бұрын

Not for this one :)

@sumitranjan7858 9 ай бұрын

You are soo cute❤❤

@babaabba9348 9 ай бұрын

Ah, if you were living in France, I would have married you immediately, I would have taken you to some fancy restaurant everyday and during the night, you would have done my assignments within data science.

@jameshopkins3541 5 ай бұрын

NOLIKE for UN useful!!!!!