Machine Learning Tutorial Python - 4: Gradient Descent and Cost Function

Рет қаралды 639,734

Күн бұрын

In this tutorial, we are covering few important concepts in machine learning such as cost function, gradient descent, learning rate and mean squared error. We will use home price prediction use case to understand gradient descent. After going over math behind these concepts, we will write python code to implement gradient descent for linear regression in python. At the end I've an an exercise for you to practice gradient descent
#MachineLearning #PythonMachineLearning #MachineLearningTutorial #Python #PythonTutorial #PythonTraining #MachineLearningCource #CostFunction #GradientDescent
Code: github.com/codebasics/py/blob...
Exercise csv file: github.com/codebasics/py/blob...
Topics that are covered in this Video:
0:00 Overview
1:23 - What is prediction function? How can we calculate it?
4:00 - Mean squared error (ending time)
4:57 - Gradient descent algorithm and how it works?
11:00 - What is derivative?
12:30 - What is partial derivative?
16:07 - Use of python code to implement gradient descent
27:05 - Exercise is to come up with a linear function for given test results using gradient descent
Topic Highlights:
1) Theory (We will talk about MSE, cost function, global minima)
2) Coding - (Plain python code that finds out a linear equation for given sample data points using gradient descent)
3) Exercise - (Exercise is to come up with a linear function for given test results using gradient descent)
Do you want to learn technology from me? Check codebasics.io/?... for my affordable video courses.
Next Video:
Machine Learning Tutorial Python - 5: Save Model Using Joblib And Pickle: • Machine Learning Tutor...
Very Simple Explanation Of Neural Network: • Neural Network Simply ...
Populor Playlist:
Data Science Full Course: • Data Science Full Cour...
Data Science Project: • Machine Learning & Dat...
Machine learning tutorials: • Machine Learning Tutor...
Pandas: • Python Pandas Tutorial...
matplotlib: • Matplotlib Tutorial 1 ...
Python: • Why Should You Learn P...
Jupyter Notebook: • What is Jupyter Notebo...
To download csv and code for all tutorials: go to github.com/codebasics/py, click on a green button to clone or download the entire repository and then go to relevant folder to get access to that specific file.
🌎 My Website For Video Courses: codebasics.io/?...
Need help building software or data analytics and AI solutions? My company www.atliq.com/ can help. Click on the Contact button on that website.
#️⃣ Social Media #️⃣
🔗 Discord: / discord
📸 Dhaval's Personal Instagram: / dhavalsays
📸 Codebasics Instagram: / codebasicshub
🔊 Facebook: / codebasicshub
📱 Twitter: / codebasicshub
📝 Linkedin (Personal): / dhavalsays
📝 Linkedin (Codebasics): / codebasics
🔗 Patreon: www.patreon.com/codebasics?fa...

Пікірлер: 705

@codebasics 2 жыл бұрын

Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced

@honeymilongton8401 2 жыл бұрын

Sir can you please upload the slides also sir

@mukulbarai1441 3 жыл бұрын

It has become so clear that I am gonna teach it to my dog.

@codebasics 3 жыл бұрын

👍🙂

@farazaliahmad3257 3 жыл бұрын

Just Do it....

@Austin-pw2ud 3 жыл бұрын

Dont do it! He may become a threat to humanity!

@eresque7766 2 жыл бұрын

@@Austin-pw2ud he may become the cleverest but he'll remain being a good boy

@Austin-pw2ud 2 жыл бұрын

@@eresque7766 ouuuuuchh! Tht touched my ♥

@codebasics 4 жыл бұрын

Stochastic vs Batch vs Mini gradient descent: kzfaq.info/get/bejne/f7tlmdigvqi4cYE.html Step by step roadmap to learn data science in 6 months: kzfaq.info/get/bejne/fpqJltSGsb-VXYU.html Machine learning tutorials with exercises: kzfaq.info/get/bejne/ndOmqcSgx9OblYU.html

@angulimaldaku4877 4 жыл бұрын

3Blue1Brown is a great channel so is your explanation. Kudos to you! Also, it is quite appreciable how you positively promote and credit other's good work. That kind of Genuity is much needed.

@vanlindertpoffertje3032 5 жыл бұрын

Thank you so much for the detailed explanation! I have difficulties understanding these theories but most of the channels just explain without mentioning the basics. With your explanation, it is now it is soooo clear! amazing!!

@codebasics 4 жыл бұрын

How to learn coding for beginners | Learn coding for free: kzfaq.info/get/bejne/edakpc-qqZawemw.html

@alidi5616 4 жыл бұрын

This is the best tutorial i have ever seen. This is truly from scratch. Thank you so much

@officesuperhero9611 6 жыл бұрын

I’m so excited to see you uploaded a new video on machine learning. I’ve watched your other 3 a couple of times. They’re really top notch. Thank you. Please keep this series going. You’re a great teacher too.

@waytosoakshya1127 Жыл бұрын

Finally, found the best ML tutorials. Coding with mathematics combined and explained very clearly. Thank you!

@chokoprty 2 ай бұрын

Watching this at 2x, like if you are too 😂

@IVIRnathanreilly Жыл бұрын

I've been struggling with my online lectures on machine learning. Your videos are so helpful. I can't thank you enough!

@codebasics Жыл бұрын

👍👍🙏

@ayushlabh 5 жыл бұрын

It's the most helpful video I have seen till now on gradient descent . Great work . Looking forward for more videos on machine learning .

@khushidonda7168 Жыл бұрын

can you help me how to plot all values of m and b on chart?

@mamtachaudhary5281 3 жыл бұрын

I have gone through so many materials and couldn't understand a thing on these, but this video is amazing .Thanks for putting all you videos.

@codebasics 3 жыл бұрын

Glad it was helpful!

@mdlwlmdd2dwd30 3 жыл бұрын

For people who wants to know whats behind of scene: The reason we get partial derivative m t function (mse): - 2/n (summation) x_i ( y_i (mx_i+b)) is due to chain rule in calculus. We want to take m deriviative and as you see m would be gone as m^(1) and m^(1-1) = 1 and leave only x_i. with chain rule we dissect the function. so suppose we have random function F(m)= (am+b)^2, we would deal with (am+b)^2 first -> 2*(am+b) X df/dm (am+b) -> 2*(am+b) X a . likewise you'd use chain rule for same MSE above. and get - 2/n (summation) x_i ( y_i (mx_i+b)) Please don't accept as it is then you never learn why things are working completely and come up with your own solution. Easy way is never get you where you want it.

@datalearningsihan Жыл бұрын

None really understands why we have -(2/n) instead of 2/n. if you do the calculations, even with the chainrule, you will get 2/n, never will get negative values!

@awakenwithoutcoffee 2 ай бұрын

@@datalearningsihan I think it is indeed to prevent any negative values to occur.

@saltanatkhalyk3397 3 жыл бұрын

thank you for such easy explanation. was reading about gradient descent many times but this is the first time I understood the math behind that.

@ishaanverma9asn523 2 жыл бұрын

this is the best ML course I've ever came upon !

@moududhassan3026 5 жыл бұрын

The best one for Gradient Descent, Thank you,

@AlonAvramson 3 жыл бұрын

You provide this complex material in such a nice and easy way. Thank you!

@akashmishra5553 3 жыл бұрын

Hey, thanks for creating all these playlists. These are so good. I think the viewers should at least like and comment in order to show some love and support.

@hfe1833 5 жыл бұрын

Thank you, I think I found the right channel for machine learning

@codebasics 5 жыл бұрын

Great. Happy learning.

@i.t.878 2 жыл бұрын

Such an excellent tutorial, the clearest I have seen on this topic. Kudos. Thank you.

@ramsawasthi 5 жыл бұрын

Great tutorial, explained in very easy language in very less time.

@codebasics 5 жыл бұрын

Glad you liked it ram.

@shijilts4139 4 жыл бұрын

All your tutorials are amazing!! Thanks a lot.

@sharathchandrachowdary6828 Жыл бұрын

This video is just enough to describe the excellence of your explanation. Simply mind blowing.

@khushidonda7168 Жыл бұрын

can you help me how to plot all values of m and b on chart?

@SudiKrishnakum 3 жыл бұрын

I followed tonnes of tutorials on gradient descent. Nothing came close to the simplicity of your explanation. Now I have a good grasp of this concept! thanks for this sir!

@codebasics 3 жыл бұрын

👍☺️

@GlobalDee_ 3 жыл бұрын

Waoh ,waoh. Codebasics to the world.You are such a great teacher sir.Thanks for sharing these series.........

@rajanalexander4949 2 жыл бұрын

Sharp, to the point, succinct. Great stuff!

@yashagarwal3999 4 жыл бұрын

so calmly and nicely u have explained a tough topic to beginners

@Opinionman2 2 жыл бұрын

Best video on the topic I’ve seen so far! Thanks

@jhansisetty9429 5 жыл бұрын

It was a very useful video. After watching many other videos, I understood the concept in the best way after watching your video. Keep making such tutorials which are simple and easy to understand the complex topics. Thankyou.

@srishtikumari6664 3 жыл бұрын

Insightful! Deep understanding of ML is necessary. You explained it very well

@wolfisraging 6 жыл бұрын

I am glad someone gives perfect explanation

@fridayemmanueljames4873 Жыл бұрын

Waooo, for a long time I've struggled to really understand the gradient descent algorithm. I feel like a pro

@prajwal3114 3 жыл бұрын

One of the best Tutorial for Gradient Descent.

@amandaahringer7466 2 жыл бұрын

Exceptionally done! Great work and thank you!

@yangfarhana3660 3 жыл бұрын

Clearly broken down concepts, very very good video, thank you for this amazing guide!

@codebasics 3 жыл бұрын

Glad it was helpful!

@TheSocialDrone 4 жыл бұрын

This was a difficult topic for me; then I spent the time to watch your video, thank you for making my learning easier! Very nice explanation.

@codebasics 4 жыл бұрын

👍😊

@khushidonda7168 Жыл бұрын

can you help me how to plot all values of m and b on chart?

@yousufali_28 5 жыл бұрын

Thanks for taking step by step approach and making it easy. 👍

@princeekanim1804 4 жыл бұрын

This tutorial made me finally understand gradient descent and cost function ... I dont know you how u did but you did... thanks man. I really appreciate

@codebasics 4 жыл бұрын

You're very welcome Prince :) I am glad your concepts are clear now.

@princeekanim1804 4 жыл бұрын

codebasics no problem keep it up you’re a great teacher

@jenglong7826 5 жыл бұрын

This was an excellent explanation! Not too technical and explained in simple terms without losing its key elements. I used this to supplement Andrew Ng's Machine learning course on Coursera (which has gotten technical real quick) and it's been really helpful thanks

@codebasics 5 жыл бұрын

Glad you found it useful Chia Jeng.

@mohamedarif3464 5 жыл бұрын

Thanks for teaching in this approach...great!!!

@vijaydas2962 5 жыл бұрын

Perfect explanation. Thanks for your effort

@aayush135 2 жыл бұрын

Superb!! Your lectures are very good and make complicated things very easy. May you keep growing in your life.

@RAJESHMANDALGAU-C- 6 жыл бұрын

Here is the video I found. Great to watch!

@kasahunabdisa6022 Жыл бұрын

great and simple approach to learning gradient descent . Thank you for your effort

@fernandoriosleon 4 жыл бұрын

Finalmente aprendí Gradient descent, Finally I leaned Gradient Descent, thank you so much 🙏

@justforfun7855 3 жыл бұрын

BONNE!!!

@VijaykumarS7 9 ай бұрын

You explained in the simplest way this complex concept. Best teacher in the world 🎉🎉

@codebasics 9 ай бұрын

Glad you liked it ! 😊

@ishitasadhukhan1 2 жыл бұрын

The best tutorial on Gradient Descent !

@mayankjain24in 4 жыл бұрын

awesome explanation. plz keep it up ..... also appreciate how you credit others for their work, that's very rare

@sararamadan1907 3 жыл бұрын

I wanted to thank you before ending watching the video just to tell you that you make my day by implying this lesson

@codebasics 3 жыл бұрын

sara i am glad you liked it and thanks for leaving a comment :)

@yourlifeonpower 4 ай бұрын

Very clear, concise and helpful! Thank you !

@sarikamishra7051 3 жыл бұрын

Sir u r the best teacher I ever got for Machine Learning.

@codebasics 3 жыл бұрын

Glad it was helpful!

@usmanriaz8396 2 жыл бұрын

best video on gradient descent and cost function. understood the match pretty well., excellent,. love from pakistan

@ireshaweerasinghe8705 5 жыл бұрын

Thanks... i am new to ML and your tutorial is very useful to me :)

@jc-co1ck 2 жыл бұрын

Thanks for your explanation and it is really clear and easy to understand. They are really awesome, thank you.

@mahmoudnady4388 2 жыл бұрын

Thank you, teacher. Your explanation is clear, interesting and useful ❤👌

@krystianprogress4521 2 жыл бұрын

Thanks to you I finally understood what the gradient descent is

@ajaykushwaha-je6mw 2 жыл бұрын

Best ever video on Gradient Descent.

@MondayMotivations 4 жыл бұрын

I don't know why you are so underrated. Only 73K SUBSCRIBERS. You deserve way more than that, I mean the way you clear the concepts. You're simply awesome man.

@codebasics 4 жыл бұрын

I am happy this was helpful to you

@geekyprogrammer4831 3 жыл бұрын

now he got 281K and in future I expect it to be more :D

@vishnusagubandi8274 4 жыл бұрын

I think this is best gradient descent tutorial even better than andrew ng sir I got stuck with andrew sir tutorial and later came up here Finally got it...Thanks a lot bro🙏🙏

@halyan2033 4 жыл бұрын

thank you so much. You saved my life!

@akhileshchauhan7422 5 жыл бұрын

upcoming few days I will see your whole channel

@shubhamkanwal8977 4 жыл бұрын

This is pure gold!

@boubacaramaiga4408 5 жыл бұрын

Excellent tutorial. Many thanks.

@wasirizvi2437 4 жыл бұрын

Explained well in easy language ! Thanks bro.

@daychow4659 Жыл бұрын

Omg!!! This is my first time seeing people to calculate how gradients decent works!!!!

@sukumarroychowdhury4122 3 жыл бұрын

Hey: you are absolutely excellent. I have seen many guys offering machine learning tutorials. None is as simple, as clear and as educative as you are. Best regards, Sukumar Roy Chowdhury - ex Kolkata, Portland, OR, USA

@codebasics 3 жыл бұрын

Sukumar, I am glad this video helped 👍🙏

@alokpratap2094 5 жыл бұрын

sir ur videos are really awesome, sir please try to complete these series as soon as possible, cover all the topics of machine learning like cluster analysis, principal component analysis, etc

@ecgisamal 2 жыл бұрын

If there will be an award of best teacher of theworld so the the award would go to this person, programming hero and brackeys

@codebasics 2 жыл бұрын

🙏 thanks for your kind words ayuro

@ecgisamal 2 жыл бұрын

@@codebasics Please can you make a playlist on tutorial of opencv python?

@clarizalook2396 4 жыл бұрын

i'm confused. This is something very new to me despite I've studied calculus in my undergrad years. I did not get it fully but the code worked from my end. Perhaps soon, the more I get into different models, I'd slowly understand this. Thanks for sharing all these.

@codebasics 4 жыл бұрын

Yup clarie. The tip here is to go slowly without getting overwhelmed. Don't give up and slowly you will start understanding it 😊👍

@ajinkyap4246 4 жыл бұрын

Ty for this wonderful tutorials. Literally the best channel I found on youtube for Data Science. Have followed your guidelines since the beginning and it has helped me very much. Exercise Output: Learning Rate = 0.0002, m = 1.0177381667350405, b = 1.9150826165722297, Cost = 31.604511334602297, Previous Cost = 31.604511334602297, Iteration = 415533 PS: Try it in pycharm jupyter get's stuck after 91915 iterations.

@ankitnayal6091 2 жыл бұрын

how did you come to this learning rate to be your input ?

@antimuggle_ridhi2565 9 ай бұрын

how did you tweek the number of iterations ?

@TY-zl1vw 7 ай бұрын

Thanks for sharing your result, my most recent tweak only reduced Cost to 31.60451133489039. With the build-in LinearRegression(the sklearn Module approach), the Cost end up as 31.60451133352958, so, use that as the benchmark I suppose.

@HARSHRAJ-2023 6 жыл бұрын

Hope sir you are more regular in uploading the video. It will help us a lot. Eagerly waiting for new upload.

@nice2meetyou631 4 жыл бұрын

This is so amazing. Thank you

@How_About_This428 2 жыл бұрын

Indians as always, so smart and brilliant people Thank you for the video it helped me a lot

@stephenadjei8413 3 жыл бұрын

BEst I have watched...totally clear now

@nirmalyamisra 5 жыл бұрын

this was such a great video..many thanks !

@codebasics 5 жыл бұрын

Nirmalya thanks for leaving a comment :)

@user-ys1qg4cl9g 10 ай бұрын

Best explanation even seen. Love from Bangladesh

@AYUSHKUMAR-dm1xg 4 жыл бұрын

who are the people disliking these videos. These people work hard and make these videos for us. Please if you don't like it, don't watch it but don't dislike it. It is misleading to the people who come to watch these videos. I know many of us have studied some of these concepts before, but he is making videos for everyone and not for a few section of people. I feel that this channel's videos are amazing and doesn't deserve any dislikes.

@codebasics 4 жыл бұрын

Thanks ayush. I am moved by your comment and kind words. I indeed put lot of effort in making these videos. Dislikes are fine but at the same time if these people put a reason on why they disliked, it will help me a lot in terms of feedback and future improvements 😊

@dhairyashiil 3 жыл бұрын

Thank you... You saved my Marks! ❤

@ashishsinha8893 6 жыл бұрын

Great to see you again bro😂

@satyanarayanaguggilapu3735 Жыл бұрын

Very effective teaching. Thanks for the videos.

@deshabhaktg6530 3 жыл бұрын

You're Legend🔥🔥 Thank you so much for such an amazing explanatory video.

@codebasics 3 жыл бұрын

Glad it was helpful!

@vinodkinoni4863 6 жыл бұрын

happy to sea u ,great work sir

@imtiyazshaik9950 5 жыл бұрын

big bow sir please continue making videos please !!!!!!!

@ou8xa1vkk64 3 жыл бұрын

Little hard for me! I cant do the exercise myself. But 100% sure no one will teach easier than this in the world. Keep doing it love you lot!!!!!!!!!

@mdrafikbk 5 жыл бұрын

Excellent and simple explanation for Gradient descent. Thank you br.

@codebasics 5 жыл бұрын

Mohamed Rafik, I am glad you liked it

@arnav707 3 жыл бұрын

this deserves a sub

@naderasaleh1222 4 жыл бұрын

You gave a good explanation. Great video

@rishabkumar9578 3 жыл бұрын

You are the best teacher for data science... thanks

@codebasics 3 жыл бұрын

Glad you enjoyed it

@lindseyfeng9610 3 жыл бұрын

thank you for your video! your explanation is really clear and easy to follow ;)

@codebasics 3 жыл бұрын

Glad it was helpful!

@OpeLeke 2 жыл бұрын

well done man! Great Expllanation!

@valijoneshniyazov 3 жыл бұрын

when you calculate partail derivatives, dont assume x or y zero, assume them constants instead. for example f(x,y)= x*y your partial derivatives will be 0 but it should be x and y

@rajdipdas1329 2 жыл бұрын

no why partial deriative will be zero ,we have to analyze it as a constant df(x,y)/dx=x.dy/dx+y this will be the derivative with respect to x and df(x,y)/dy=x+y.dx/dy.

@allann9046 3 жыл бұрын

Great video, thank you! Super helpful.

@codebasics 3 жыл бұрын

Glad it was helpful!

@hrithvikreddy6643 2 ай бұрын

The Exercise that you have shared is taking many no of iterations to get to the correct intercepts and coefficients... my laptop hung many times doing that problem 😵‍💫😵‍💫😵‍💫😵‍💫

@Haven_Hue Ай бұрын

code (try this): import pandas as pd import numpy as np import math data = pd.read_csv("D:\\Machine_learning\\Grad_des\\test_scores.csv") x = data['math'].to_numpy() y = data['cs'].to_numpy() def gradient_descent(x,y): m_curr = b_curr = 0 iterations = 10000 n = len(x) learning_rate = 0.001 prev = 0 for i in range(iterations): y_predicted = m_curr * x + b_curr md = -(2/n)*sum(x*(y-y_predicted)) bd = -(2/n)*sum(y-y_predicted) m_curr = m_curr - learning_rate * md b_curr = b_curr - learning_rate * bd cost = (1/n) + sum([val**2 for val in (y-y_predicted)]) if math.isclose(cost,prev,rel_tol=1e-09): break print("m {}, b {}, cost {}, iteration {}".format(m_curr,b_curr,cost,i)) prev = cost gradient_descent(x,y)

@moumitaadhikary5635 2 жыл бұрын

Very helpful. Thank you!

@krishkonnect814 4 жыл бұрын

Thankyou very much Sir ji. Thankyou for all the important and essential info in improving my learning curve.

@codebasics 4 жыл бұрын

Thanks for your kind words of appreciation

@harryinatube 6 жыл бұрын

Great video and teaching method. You have an art of keeping things simple but still teach advanced concepts. I get a very good and quick overview and understanding from your videos. Thanks a lot.