Python Pandas Tutorial (Part 9): Cleaning Data - Casting Datatypes and Handling Missing Values

Рет қаралды 230,772

4 жыл бұрын

In this video, we will be learning how to clean our data and cast datatypes.
This video is sponsored by Brilliant. Go to brilliant.org/cms to sign up for free. Be one of the first 200 people to sign up with this link and get 20% off your premium subscription.
In this Python Programming video, we will be learning how to clean our data. We will be learning how to handle remove missing values, fill missing values, cast datatypes, and more. This is an essential skill in Pandas because we will frequently need to modify our data to our needs. Let's get started...
The code for this video can be found at:
bit.ly/Pandas-09
StackOverflow Survey Download Page - bit.ly/SO-Survey-Download
✅ Support My Channel Through Patreon:
/ coreyms
✅ Become a Channel Member:
/ @coreyms
✅ One-Time Contribution Through PayPal:
goo.gl/649HFY
✅ Cryptocurrency Donations:
Bitcoin Wallet - 3MPH8oY2EAgbLVy7RBMinwcBntggi7qeG3
Ethereum Wallet - 0x151649418616068fB46C3598083817101d3bCD33
Litecoin Wallet - MPvEBY5fxGkmPQgocfJbxP6EmTo5UUXMot
✅ Corey's Public Amazon Wishlist
a.co/inIyro1
✅ Equipment I Use and Books I Recommend:
www.amazon.com/shop/coreyschafer
▶️ You Can Find Me On:
My Website - coreyms.com/
My Second Channel - / coreymschafer
Facebook - / coreymschafer
Twitter - / coreymschafer
Instagram - / coreymschafer
#Python #Pandas

Пікірлер: 179

@coreyms 4 жыл бұрын

Hey everyone. Hope you all had a great weekend! I will be traveling to Vancouver this week to visit a Quantum Computing company and learn more about the work they're doing, so I'm not sure when the next Pandas video will be ready for release. I will be working on it while I'm there, but I likely won't have it recorded and released until midway through next week. Let me know if anyone has any questions they would like me to ask them about Quantum Computing!

@harshvardhan1156 4 жыл бұрын

Hey, Corey. Thankyou for everything. I am not from Computer Science background, Out of curiosity I started learning to code and here I am now, has done more than 20 datascience project. Your videos are literally best, I have taken some courses for high price and I can un-undoubtedly say that your way of teaching is way more interactive, complete and easy to grab. I just want to know how you plan for any course, like in 1st or 2nd video You said that I will cover this topic in later videos. So do you make whole content, practice it? deepdive in it and make your own order and then start teaching? It would be very helpful for me if you share about how you prepare for any topic. Thank you very much Love from INDIA

@harjotsinghbaidwan2204 4 жыл бұрын

I have many times seen while using dataframe that column names are not at same level and this creates an issue during extraction of values. Do you have any idea about it?

@JiminPark-ld2xx 2 жыл бұрын

How do I download dataset after cleaning my data using Jupyter notebook online? Plzz ans..

@ahammadshawki8 4 жыл бұрын

Please make a playlist on numpy after pandas.

@ChetanAnnam 3 жыл бұрын

Yeah please do that

@malikdiallo9976 4 жыл бұрын

I like this series in pandas. thank you so much Corey.

@corben3348 4 жыл бұрын

Good teaching is an art... This playlist is so helpful ! Thank you for your work !

@benhancock1541 4 жыл бұрын

Thanks for this Corey - your tutorials are always great! I've been using pandas for almost 2 years and still learned stuff 👍

@gauravmarwaha8466 4 жыл бұрын

this series on pandas is the most complete and informative series ive found till date...!!!

@ishanpand3y 4 жыл бұрын

This is the most amazing series on Pandas ever. I just finished watching number 9th. Sir thank you so much providing such great content. 🧡🤍💚

@ashishdeora8522 3 жыл бұрын

Thank you Corey for this. My parents urged me to join your community. They are saying you are doing wonderful job. Thank you Corey for enabling us

@adamgdev 3 жыл бұрын

You never disappoint!! And I never have to speed you up because you keep a great pace with no BS! Thank you!!

@saravanannatarajan6515 4 жыл бұрын

Corey you're teaching is awesome!!! Much appreciated!!! Expecting series on Machine Learning/Deep Learning in the near future...

@sayantanchakraborty75 4 жыл бұрын

Best series on Python Pandas . Thank you so much Mate. Love from India

@andreykaok9497 4 жыл бұрын

Brilliant tutorials on Pandas! Very much looking forward to the time series lessons.

@gagansoni9665 4 жыл бұрын

i understand your pandas tutorials very clearly. this is helping me a lot. thank you so much corey. i wish to see your tutorials on machine learning using python.

@zixinlee2165 4 жыл бұрын

Thank you so much for creating these videos!! They're really valuable for self-learners like me.

@YeekyYeeky 3 жыл бұрын

can't wait for your numpy series , this channel is gold , Thank you Corey

@mapa5000 Жыл бұрын

You really care about making a video addressing many scenarios and possible issues … that’s phenomenal !! … I really appreciate it … thank you so much!!

@Ian-bb7vv 3 жыл бұрын

I had to say, thank you!! I think you guys are really helping to fill the unequal educational resources between the rich and the poor. Great job and I hope you now that what you are doing is really meaningful

@rauberhozenplotz7009 4 жыл бұрын

Great content - great style of speaking and explaining - thank you!

@rockeyvalley 4 жыл бұрын

Great stuff Corey!!! Keep up the good work!

@minxxdia1132 4 жыл бұрын

wow, this is the best playlist for python pandas. thankyou so much!

@darkmaraux 4 жыл бұрын

This video was so smooth! Right in the point! Thanks!!!

@codegeek8256 4 жыл бұрын

Hi @ Corey Schafer I am very with your teachings, these are great building blocks towards data science, i hope one day we arrive there.

@stanislawjarzynski6133 3 жыл бұрын

You're a great teacher, Corey!

@juancarcelen3437 4 жыл бұрын

Hi Corey thank you so much for posting these videos. Your tutorials have helped me transition the concepts I know into actual useful code. I would like to test my progress and would really appreciate if you can put out a link with some data analysis projects (i.e. a database to download, questions to answer using data analysis, and the code that was written to answer those questions). Thank you so much and keep the videos coming you're an amazing teacher!!

@danielflorea3001 2 жыл бұрын

Simple and clear explanations. Great job.

@VikasGuptacherie 4 жыл бұрын

Very helpful series with nice explanations !!!

@kuls43 4 жыл бұрын

11:36 we can use df.replace(['NA', 'Missing'], np.nan, inplace=True) instead

@AtlasIndustries101 3 жыл бұрын

could've used in other df.replace(...) line too. But I think he is trying to keep it simple for us to understand it easily.

@davebeckham5429 4 жыл бұрын

Many thanks for sharing excellent tutorials Corey.

@zzzorgjanbatist564 4 жыл бұрын

As usual Corey best of the best!!!

@srivathsgondi191 7 ай бұрын

Now thats a lovely explaination, i like how u showed the function can be used in different scenarios!

@manishgpt25 3 жыл бұрын

thanks a ton for this series..helped a lot in clearing concepts!!

@ahmedhosny3855 Жыл бұрын

such a great work done by you , hope you all the best man

@FakeAccount 4 жыл бұрын

You're a legend, my guy.

@gayatriwaghmare6293 3 жыл бұрын

The series is very helpful to me. Thank you sir.

@Shkkmj6868 4 жыл бұрын

It's very useful .You are great at articulating . Thank you so much .

@Al-Ahdal 4 жыл бұрын

Boss, it is requested to kindly make videos on comprehensive data analysis series, covering all aspects in much detail, and covering all possible areas for data analysis. Your channel and vdos are awesome. Great work indeed...... 👍

@haiderali2050 3 жыл бұрын

Thank you so much, i have learnt a lot and able to automize my daily Excel routine work

@andr101 4 жыл бұрын

great series, thanks!

@analyticswithothello8213 2 жыл бұрын

Corey, you are teaching the best!

@finncollins5696 Жыл бұрын

Learnt a lot so far. Thanks so much Corey,.

@TopicalAuthority 4 жыл бұрын

Great lesson!

@varunkrishnaKyathanpally 4 жыл бұрын

Thank you , excellent tutorial as always :)

@lucasbartomioli7861 3 ай бұрын

Man, i love you! Thanks a lot from Argentina!

@stressfreetrading1341 4 жыл бұрын

Love the way u teach. thanks a lot... Love from India

@saraghafelehbashi5808 2 жыл бұрын

much appreciated! could you please have more video like that? cleaning data and see the diffrent errors come with it! it would really helpful for juniors.

@dadoll1660 4 жыл бұрын

This is gold.

@NikitaSharma-bs4gg 2 жыл бұрын

That was such a good video- thank you for sharing

@002_priyanshugoswami5 4 жыл бұрын

love you coreyyyyy best channel

@kameshinipillay4587 Жыл бұрын

Thank you, learning so much :)

@robertmnganya7533 3 жыл бұрын

Excellent teaching. Thank you.

@JR-mj8ph 3 жыл бұрын

These are awesome! Thanks a million!

@mohammedkaifmirza7585 2 жыл бұрын

Amazing tutorial 😍👌

@stephanie_ong 3 жыл бұрын

Thanks again for such a helpful video.

@muntadher8087 2 жыл бұрын

Thank you so much!! You are the best

@kirannagar8295 3 жыл бұрын

Hey , truly glad for your all series . If possible , please do make a course video on Pyspark .

@litan5006 2 жыл бұрын

Good pandas video. Thank you

@alexthewebdesigner1856 Жыл бұрын

@Corey Schafer Something told me that I'hd better watch this video. Just when I thought that I'd sanitized a large data set, I realize now that there could potentially be some data (or missing data) that could crash my application. Great video. Thank you Sir!

@arkahm 4 жыл бұрын

Great video! How about a video in spitting data and passing the split into a function? That would be great!

@teetanrobotics5363 4 жыл бұрын

I love your tutorials. Could you also make tutorials for scipy and scikit learn?

@gauravmarwaha8466 4 жыл бұрын

good video again..!! thanks a lot

@mohamedikbalguetout32 3 жыл бұрын

hey bro always I fond the solutions in your videos thanks man

@shadmanmartinpiyal4057 3 ай бұрын

excellent!

@interestingstudies4422 2 жыл бұрын

Amazing video...solved my problems ☺️☺️🙏🏻

@ironpolux 2 жыл бұрын

Great vid, pls do one on multiple indexes!

@quoit99training83 4 жыл бұрын

amazing series - hi Corey, how many PARTS u think will end up in this playlist? Thank you for helping the community :)

@d_omar1468 3 жыл бұрын

great job brow

@ADNANAHMED-eo5xx 4 жыл бұрын

Thanx a lot

@md.abdullahalmasum4942 2 жыл бұрын

thank you sir .

@saqibhussain1354 4 жыл бұрын

Great video - I wonder if you can do a few on the business side like freelancing and how to get clients as python developers?

@mikkybricks 4 жыл бұрын

Thanks Corey

@jongcheulkim7284 2 жыл бұрын

Thank you^^

@ajinzrathod 3 жыл бұрын

Corey you are great.❤️ Love from India ❤️

@KevinTempelx 3 жыл бұрын

Thank you!

@user-pd8fb5xh1z 3 жыл бұрын

Thank you

@bartdziubek327 3 жыл бұрын

good job :)

@aegystierone8505 4 жыл бұрын

Please do a video about your visit to the Quantum Computing trip in Vancouver!

@RahulGupta-ke9su Жыл бұрын

thank you

@bharaths1396 3 жыл бұрын

Your content is awsome....! How do replace nan values with other values only in a particular column? Please Help Thank You

@alfredsfutterkiste7534 3 жыл бұрын

Lovely.

@adarshtiwari7395 3 жыл бұрын

That is BRILLIANT

@rybltbt 3 жыл бұрын

Thank You its really helpful

@samratsengupta8881 4 жыл бұрын

Thanks Corey, i have no words to say. As an inspiring data scientist, your pandas videos were really cool. I don't know if you will ever read this but this has helped and has put a smile on my 'confused about pandas' face. i have subscribed and will watch your videos for becoming a self taught data scientist. God Bless You

@abhishekgupta1060 4 жыл бұрын

Hi Corey, great video! Learned a lot! And have a Happy trip , and also I would like you to ask one question - "Is time travel possible with Quantum computing as depicted in the movie 'Avengers:Endgame' ".

@ADNANAHMED-eo5xx 4 жыл бұрын

Please continue the series sir

@athas12 Жыл бұрын

for the last part of the video, you can actually create two lists and use these lists in replace method to change all values at once. It is slightly easier especially if the df has multiple values to replace

@KimJennie-fl3sg 4 жыл бұрын

This also work if we want to drop a column if 0 and 1 index have NaN df.dropna(axis='columns', how='any', subset=[0, 1])

@kinjalvora256 3 жыл бұрын

Hi Corey, Thanks for the awesome series. While I have not yet finished the series, I would like to know, how we can deal with duplicates. If you have a column let's say with duplicate apps and the apps have reviews, size, installations and you want to let's say get a mean for the reviews, take the first size and sum of the installations and merge the rest of the columns for those apps as they were, like Ratings. How would one do that?

@SusanAmberBruce 4 жыл бұрын

Corey, do you happen to know what Linux distro's ship with python 3 currently?

@bobsalita3417 4 жыл бұрын

Can you use join() or merge() to do multiple replacements?

@muntadher8087 2 жыл бұрын

useing this func ( df.fillna("Unfilled", inplace = True) ) to replace the missing values is good practice I belive, for me it's easier than replace and more dynamic

@shivavijaya1537 4 жыл бұрын

Hi Corey, please post a video on python sys module

@harishrudroju1379 4 жыл бұрын

Hii corey, can u plz make a video on how to bypass captcha while scrapping a web site

@stayinawesum 4 жыл бұрын

can you make video explaining: primitive data types vs data types vs adt vs data structure

@nikhilb3880 4 жыл бұрын

I love this series man, more than you could expect. If I may ask, what state and country are you from? Because I saw snow on your 2nd channel and now I'm confused about whether you live in the USA or in a European country. Thanks again for this series

@coreyms 4 жыл бұрын

Hey there. I currently live in Greenville SC in the United States. The snow videos were likely from Boulder Colorado where I lived for several years.

@SandeepPatel-qk5jc 4 жыл бұрын

Hi Carey, In the example codingyear where you replaced less than 1 year mand more than 50 year with nan. But if data is very large number of unique values and that are not of numeric type and its not possible to manually replace them. In that case how to overcome the problem.

@md-ayaz 4 жыл бұрын

@Corey Schafer Can you make video on getting started on Open Source Contribution?

@nathanthreeleaf4534 4 жыл бұрын

What would be the proper way to handle a data set that had bad column names? Say for example, using the read_csv method and the csv being read has column headers with spaces in them?

@dhanraj112 4 жыл бұрын

is brilliant give certificates after completion of course?

@codewithluq 4 жыл бұрын

Thank you Corey again. My resume is getting more interesting everyday. Viva

@prasad1686 4 жыл бұрын

Hi Corey, your videos are "the great". I am beginner can you please tell me how to get "cheet sheet" or ".py scripts" of your video playlist "Python Tutorials" "1 to 136", to speedup learning as i am slow in typing. Thank you.