Loss or Cost Function | Deep Learning Tutorial 11 (Tensorflow Tutorial, Keras & Python)

Рет қаралды 141,939

4 жыл бұрын

Loss or a cost function is an important concept we need to understand if you want to grasp how a neural network trains itself. We will go over various loss functions in this video such as mean absolute error (a.k.a MAE), mean squared error (a.k.a MSE), log loss or binary cross entropy. After going through theory we will implement these loss functions in python. It is important to go through this implementation as it might be useful during your interviews (if you are targeting a role of a data scientist or a machine learning engineer)
Code: github.com/codebasics/deep-le...
Exercise: Go at the end of the above notebook to see the exercise
Do you want to learn technology from me? Check codebasics.io/ for my affordable video courses.
🔖 Hashtags 🔖
#lossfunction #costfunction #costfunctionneuralnetwork #lossfunctionneuralnetwork #costfunctiondeeplearning #lossfunctiondeeplearning
Why not MSE for logistic regression:
towardsdatascience.com/why-no...
Next video: • Gradient Descent For N...
Previous video: • Matrix Basics | Deep L...
Deep learning playlist: • Deep Learning With Ten...
Machine learning playlist : kzfaq.info?list...
Prerequisites for this series:
1: Python tutorials (first 16 videos): kzfaq.info?list...
2: Pandas tutorials(first 8 videos): • Pandas Tutorial (Data ...
3: Machine learning playlist (first 16 videos): kzfaq.info?list...
Website: codebasics.io/
Facebook: / codebasicshub
Twitter: / codebasicshub

Пікірлер: 104

@codebasics 2 жыл бұрын

Check out our premium machine learning course with 2 Industry projects: codebasics.io/courses/machine-learning-for-data-science-beginners-to-advanced

@ashwin4991 6 ай бұрын

Loving this series. You're so good at looking at everything from the learners perspective and clarifying everything! Much better than all the other tutorials I've tried

@user-zy8sf7tv2f 3 жыл бұрын

Man, I love this series.

@codebasics 3 жыл бұрын

Glad you enjoyed it

@GamerBoy-ii4jc 2 жыл бұрын

it's very easy exercise i just replace the absolute with square and it done...thanks for this playlist

@ukanivikas2694 3 жыл бұрын

Beautiful explaination ❤️

@viniciushedler3132 2 жыл бұрын

When changing y_predicted to avoid errors with log(0) it is possible to do [ abs( y - epsilon ) for y in y_predicted ], which will work for both y=1 and y=0 cases

@satyarthpandey2941 2 жыл бұрын

if earlier, some y was epsilon(which is fine), then now it will become 0 from your function(which is a problem).

@mohitupadhayay1439 2 жыл бұрын

That log loss vs MSE article was a BOOM!

@abhishekfnu7455 2 жыл бұрын

Thank you so much for this course, sir

@izharkhankhattak 3 жыл бұрын

Great man does a great job. Thank you so much.

@izharkhankhattak 3 жыл бұрын

@Devin Preston ???

@priyabratapanda1216 3 жыл бұрын

Sir in binary cross entropy the loss function should be c=-- 1/n sum (y*log(p) + (1-y)log(1-p)) Where p is the probability of the sample for the positive class. By the way nice intuitive lectures ,I love you way of teaching 🙏

@rajdipdas1329 2 жыл бұрын

yes it comes from bernoullie's theorem P(x)=p^x*(i-p)^n-x; after that if we take log on both side.

@mohdsyukur1699 4 ай бұрын

These series are the best

@jntb3000 3 жыл бұрын

Thanks for the presentation. Small question: whereas y-hat is actual predicted result (fractional values) y seems to be from Step Function (ie, 0 or 1, not fractional value). Is this correct ?

@maheshsingh2205 2 жыл бұрын

Am just stuck with the series.. It's super interesting

@BahauddinAziz 3 жыл бұрын

Feedback: You can create chapter of the tutorials, will be helpful to navigate

@abikrsen3205 2 жыл бұрын

This video cannot have chapters since he's teaching the concept as a whole and not discrete things in the video that could be bookmarked like in other videos. 🙄

@umarzia1985 2 жыл бұрын

The tutorials are very helpful. Can we get ppt slides of the tutorials plz?

@ronyjoseph7868 3 жыл бұрын

very benificial videos, can u upload videos of some online or offline job oppurtunities of machine, deep learning

@nilupulperera Жыл бұрын

Great.Thank you very much

@wildannasrullah61 3 жыл бұрын

Do you have tutorial like this but for NCE Loss function? Thank you

@YoutuberEnjoy Жыл бұрын

Simply great💌

@vishaljaiswar6441 2 жыл бұрын

Amazing tutorial :)

@udipfeffer9859 Жыл бұрын

Thank you for this very clear lesson. Is it possible that the "y_predicted" and "y_true" are confused? I would expect y_true to be in {0,1}, and y_predicted represent a probability in [0,1]

@Brian5094 Жыл бұрын

Have a question, in 7:50 you mentioned that MAE is the cost function, but why in 1:05, you said it is loss function? Besides, in TF's parameter, should not the parameter "loss" should be "cost"? Because it is cost function not loss function?

@asamadawais 2 жыл бұрын

You are excellent boss. Allah bless you!

@dsran6358 2 жыл бұрын

hi sir,when we are doing log loss function in that we are taking -np.mean(log function) but we didn't take n value hear -1/n ? at 22:37

@leelavathigarigipati3887 3 жыл бұрын

Thank you so for the detailed explanation. Could you please explain about SGD, ADAM optimizations.

@codebasics 3 жыл бұрын

I have a video coming up on SGD, batch and mini batch. It will be live today.i will add video on Adam too in future

@leelavathigarigipati3887 3 жыл бұрын

@@codebasics Thank you so Much 🙇

@anjmuk79 3 жыл бұрын

Thank you so much Sir.

@codebasics 3 жыл бұрын

Most welcome

@EarlWallaceNYC 2 жыл бұрын

thanks for the video

@harshkartiksingh5356 Ай бұрын

i was frightened from that corona virus mate, it freaked me out😂😂,thanks for this awesome video.

@pardeepsangruri 3 жыл бұрын

hi , i have a problem understanding the example in the sklearn documentation from sklearn.metrics import log_loss >>> log_loss(["spam", "ham", "ham", "spam"], ... [[.1, .9], [.9, .1], [.8, .2], [.35, .65]]) 0.21616... can you please explain this and extend the log_loss idea to multiclass classification. Where they take np.argmax of true_values and subtract the predicted. can you please make a video and expain this concept. Thanks a lot for making this great video

@karanveersingh1658 Жыл бұрын

min(i, 1 - epsilon) puts all values below even if it is above 1

@bhamrags 2 жыл бұрын

Dhaval ji it would be better if you also explain if one get 'nan' error due to log(1-y_pred) part in log loss equation when (1-y_pred) value becomes 'negative' which is invalid value encountered in log. And How to resolve this issue?

@abhisheknagar9000 4 жыл бұрын

Nice Explanation. Do we need to use Binary crossentropy in case of Binary classification only or can be used in Multi class classification? Also kindly explain about the sparse categorical entropy.

@codebasics 3 жыл бұрын

In multi class classification we need to use categorical cross entropy or sparse categorical entropy. Yes I will cover difference among them in future videos

@vinaychitturi5183 3 жыл бұрын

@@codebasics Hi Sir, This is a nice explanation about MSE and MAE. Have you made video on Categorical cross entropy? Looking for this video. Thanks.

@SwarnaliMollickA 2 жыл бұрын

You are rock🙏

@nirmesh44 4 жыл бұрын

very nicely explained

@codebasics 4 жыл бұрын

👍😊

@manujarora5062 Жыл бұрын

at 17:32 the formula needs a parentheses to begin before yi and end at the end of expression.

@manujarora5062 Жыл бұрын

Here is my intuition for the scary looking Log loss formula Case 1: We predict perfectly As per formula if y is actually 0 and you predict 0 then the loss is 0. if y is actually 1 and you predict 1 then loss is 0 Case 2: We predict with some errors If y was actually 0 then the first term is 0. The Second term is putting a very high penalty (exponentially higher) if you predict value closer to 1. The penalty would be infinite if your algorithm predicted a value of 1 when the actual was value was 0 Vice versa also holds

@sakalagamingyt3563 Ай бұрын

Instead of doing manually we can avoid log(0) using numpy: epsilon = 1e-15 y_predicted_new = np.clip(y_predicted,epsilon,1-epsilon)

@isaakimaliev5584 2 жыл бұрын

RuntimeWarning: divide by zero encountered in log

@meenakshichippa260 Жыл бұрын

U should not hesitate to mention Andrew NG 's name. We all learn from somebody. Andrew NG has his own expertise and u have ur own. It is always perfectly okay to share from sources we learn.

@codebasics Жыл бұрын

Of course. I think pretty much all the people who know ML have learnt from Andrew NG at some point. I have a great admiration for him 👍🏼

@dhirajkumarsahu999 3 жыл бұрын

Thank you for the video

@codebasics 3 жыл бұрын

👍😊

@bilkisuismail6096 21 күн бұрын

I was wondering why we have 0.5 as a result for MAE with Python code and 0.26 with Numpy code

@pratikbhansali4086 3 жыл бұрын

Pls add more videos of this Playlist sir

@abdallahlakkis449 8 ай бұрын

Awesome content thanks a lot

@codebasics 8 ай бұрын

Glad you enjoyed it.

@deepakhg2963 2 жыл бұрын

When i use tf.keras.metrics.binary_crossentropy, the result is different. please comment.

@abhaydadhwal1521 2 жыл бұрын

Sir, in Binary cross entropy ... won't the true value (y) create a problem if it is 1 ?.... because it makes the equation 0 as 1-1 = 0

@esraahamada4784 9 ай бұрын

Yeah, I think y_true will make problem because if it's zero all the left portion will be ignored as it's doesn't exist and if it's 1 all the right portion will be ignored . Will that cause a problem I don't know but I think we lost a valuable information.

@useryaya-r5h 6 ай бұрын

We need to upgrade to read the full article that you are referring to in the video...

@sanooosai 6 ай бұрын

thank you

@testcode9589 2 жыл бұрын

why both np.sum(np.abs(y_predicted-y_true)) and np.mean(np.sum(np.abs(y_predicted-y_true))) have same result when i calculate on my own

@basitarif9128 3 жыл бұрын

why don't you change to value of y_true with epsilon when doing log loss function ?

@universe3603 3 жыл бұрын

look at equation 16:42 there's no log for y_true. so no need to change it with epsilon

@pickase Жыл бұрын

for real i really performed all those code without using your notebook sir because of the virus warning.😅😅

@user-cz6ht4ot7o 2 жыл бұрын

Sir please make a course on deep learning and AI🙏🙏.please reply

@user-do5pn6hb2i Жыл бұрын

Where can I get this PPT for my revision

@user-em5ep9zw4z 10 ай бұрын

If log loss outputs the range from 0 to 1, why is your model giving values outside this range?

@sourabhmaharajpet 3 жыл бұрын

Guessing, is chance so on bases on chances the error should have been calculated

@faezeabdolinejad731 2 жыл бұрын

😅😄you are cool, thank you

@udaybhaskargajavalli7684 3 жыл бұрын

Any tutorials on multiclass-multioutput regression or classification

@universe3603 3 жыл бұрын

you mean classification?

@aodhrinfo510 3 жыл бұрын

you are such a great man 👨

@Anonymous-md4ql 3 жыл бұрын

Naively explained

@duminda-kodagoda 2 жыл бұрын

Very nice video tutorial.. keep on doing this!! if predicted value is always 0 and 1 then following is easy right? y_predicted_new = [ abs(i-1e-15) for i in y_predicted ]

@sarveshamrute2959 3 жыл бұрын

Hello sir.....I'm a beginner machine learning learner...I started learning it from your channel...but one of my friend suggested me to do the course of Machine Learning by andrew ng from coursera...but in that course..after linear and logistic regression, he has started directly teaching neural networks while in your playlist first all ml concepts are suggested as an prequisite. So what should I do...hope you clear my doubts asap....thnx

@codebasics 3 жыл бұрын

Yes please follow my entire machine learning playlist. You need to have good understanding on basic machine learning models, train test split, k fold, GridsearchCV etc before you start deep learning. And I have covered all of that in my ML playlist

@sarveshamrute2959 3 жыл бұрын

Also he teaches in very deep...like mathematics and all as compared to you...so are your videos sufficient to do various projects?

@universe3603 3 жыл бұрын

@@sarveshamrute2959 without ML u can't move smoothly in DL. If you dont have enough knowledge in ML algorithms you probably stuck everywhere in DL. Codebasics gives you an amazing grip in ML with awesome explanation as this video so you can implement any project smoothly. go to ml playlist now. i'm saying this with experience. and also he has discord server u can join there and get instant help from experts in ml and dl whenever u stuck. good luck

@shaikansarbasha4169 3 жыл бұрын

sir what is the difference between (math.e)**(-15) and 1e-15

@mikeminotakis9092 3 жыл бұрын

By math.e you call the Euler's number from math module in python so your example is (2.718281)**-15 whereas by 1e-15 you mean the scientific notation so it's 1 * 10**(-15).

@shaikansarbasha4169 3 жыл бұрын

@@mikeminotakis9092 thank you anna

@deeptimaanbanerjee6330 3 жыл бұрын

Reflecting back, seems like many people clicked on the solution link without trying to solve for the answer themselves.

@1bitmultiverse 2 жыл бұрын

I have accidentally opened the solution file ... Now my computer is feeling fever and headache 😂 I think you were right... Now I need to get a vaccination for my computer, kindly make a tutorial for that too.... 😂 😂 😂

@codebasics 2 жыл бұрын

Pfizer is still inventing the vaccination for the computer :)

@1bitmultiverse 2 жыл бұрын

@codebasics but I want you to make a vaccine for my computer.... 😂😂 because you send this coronavirus in my computer or send me NVIDIA GeForce RTX 3070 ( Vaccine for my pc ) 😂😂 😂😂😂😂😂😂

@gamesniper98 Жыл бұрын

@adiflorense1477 3 жыл бұрын

6:29 Great content. sir, how can be like that y=0?

@vishwajeetpanda1637 3 жыл бұрын

it was the actual value, have_insurance=y=0

@hamzakhurshed2650 10 ай бұрын

Your Virus came to my computer

@daljeetsinghranawat6359 11 ай бұрын

i am implimenting mini batch graddesc but y_predicted is very far from y_true..no matter how many epochs i try.....

@mahmoudgadelrab5133 3 ай бұрын

Nobel prize

@shakib_04 3 жыл бұрын

poor corona virus :(

@SwarnaliMollickA 2 жыл бұрын

MSE is 0.366

@raj345to 2 жыл бұрын

sir please register a company and give us internship in a project. ready to pay you anything!!It will be an honour and privilege to work with you.

@24-7Pain 3 жыл бұрын

Lmfao coronavirus solution! Just a quick question -> On the mean square error, what does n represent? is it the number of output nodes or the number of training samples in the dataset?

@qwantisheku7452 3 жыл бұрын

The second one. If your dataset has 30 records, so it will be 1/30*Loss

@MohamedAshraf-zs6nv 3 жыл бұрын

point nineninenineninenineninenine

@thinhnguyen8158 6 ай бұрын

dcmm

@ashvinikumardongre9330 2 жыл бұрын

Sir, can we have more clarification on the terminiologies we use?initially you said individual error is called loss and cumulative error is called cost function. which means MSE and MEA should be called as cost function and not loss but later in the model you put loss = 'mean-squared-error'. Shouldn't it be costfunction = 'mean-squared-error' Obviously 'loss' must be a keyword used by tensorflow, so should we assume that every thing(absolute error, MAE, MSE, log loss) can be called as 'loss' or 'cost' or 'loss function' or 'cost function' interchangeably?? if so then why should we differentiate between loss and cost function at all?? @codebasics

@_RMSG_ 2 жыл бұрын

Loss and cost are used interchangeably in machine learning