pdf of exponential is (lambda)*e^(-lambda x)

This video just transformed my life

That was a great intuitive explanation of inverse Transform Sampling. It seems so easy to me after watching this video,. Thanks a lot.

Thank you so much for doing these videos.

Great video. It made the underlying concept crystal clear. Thanks a lot, Ritvik.

Just saw two 20 minutes long videos before this, none made me understand this at all. Then, I saw this 10 minutes long video of yours and it made this subject so much clearer than before. Amazing professor, congratulations!

You solved a big curiosity I had. I learned about the power of MonteCarlo analysis and how easy it is to get a uniform distribution from Excel, but knew I would always need more specific distributions. So the question was how to get any distribution from a set of randomly generated numbers from the usual Excel Rand() generator. Thanks for the brilliant and easy demonstration! Congrats for your terrific work!

I really want to thank you because your clear explanation helped me get an A in my statistical programming exam. You are a hero.

omg you saved my stats degree, much thanks

Love how calm you are. I'm shitting myself when I have to explain topics like these to someone.

You have brilliantly and simply explained a topic that I have been struggling with for a whole semester. Thank you so much! :)

I can't thank you enough. You have been of help in many subjects from time series analysis to this. I would like to see EM algorithm, latent class models, and hidden Markov models in the future.

He is going to be a fabulous professor

hi, is there an error with the PDF function? f(x) = lambda * exp^(-lambda)(x)? thank you for this video!

u r best teacher ever

Brilliant teacher! I guess it is a sort of gift.

Great video! Your exponential density is missing it's normalizing constant, though. Since your CDF is correct, no harm , no foul, but it might confuse some people.

Brilliant!!!!

I watched a few videos on this and yours was by far the clearest. Thank you.

Thanks for the video! I'm still struggling with this, but your explanation definitely helped!

Lets say u = 0.25. Then 1 - u = 0.75, right? Could someone explain how 1- u = u in the uniform distribution?

Super helpful, thank you very much!

thanks for good explanation about Inverse Transform sampling !

Trying to wrap my head around this in class but to no avail. Thank you so much for your amazing explanation

Thanks for the video! I was struggling to understand the motivation behind it, but your explanation has made it much easier for me :)

This was super pedagogical, thank you very much.

Great explanation! Thanks.

you saved my life

Thanks a lot! your videos on DS and Stats is the best!

Thank you. Our instructor did not explain it and just gave the theorem. I was confused like I have three heads.

Just fantastic ! keep it up man great videos and great explanation

Thank you for this

very clear explanation, thanks for sharing!

Well explained .. thanks. One minor suggestion .. if there is a way you can make the video screen capture friendly or leave a screen capture slides to the video, that would be super helpful. Thanks for the clear presentation.

I love your explanation always produce the best please don’t stop

nice! Thanks for the work. Like the way you explained concepts in a straightforward and smooth way. Please keep it up ! :)

Greatly explained! Thanks!

this guy is epic!!!

great explanation

Excellent explanation! Keep ut the good work!

Excellent. !

Yo Ritvik not sure if you still remember me we talked during orientation (I was the guy work with Tasty). We had a class last week about MCMC and I was confused about certain parts and KZfaq directed me to this video lol. Great job man keep it up. Hope we can catch up when things get back to normal after the pandemic

Quite an amazing explanation. Well done!!

Great video, it really helps me out a lot. One thing I still dont really understand is why we might do this. As in, why would we use the inverse transformation method to find the exponential random variable instead of just using the exponential PDF directly if we have lamda?

Thanks for sharing. Just one small comment....pdf of the exponential is lambda*e^(-lambda*x).

Thanks a lot bro, so helpful

Thank you 🙏

Great Explaination! Thanks so much :)

Then, why is important the uniform pdf?. I mean you could sample directly from one distribution to another just by putting the value returned from the CDF of the first pdf as input to the inverse CDF of pdf you want to arrive at. Am I wrong?

this is great thank you so much :)

very helpful, thanks for this video!

Can you please do a video about Copulas? For example in a (credit) risk management context

Thanks!

That was awesome. Thank you !!!!!

Thank you so much. You help me a lot with a homework I have.

Thank you, this helped a ton! :)

Nice lecture!

Hey - great video! I think you might have forgotten the lambda in front of the exponential for the exponential PDF. If you calculate the CDF from what you have written you will get a 1/lambda factor.

That subtle pen flip at 5:49.. Damnn

Gem level bhau

perfect!

Brilliant, thank you so much.

Hi! I loved the video, but I've got a question. What are the cases when the CDF is not invertible? And what are the strategies then? Should we try to make the CDF invertable by interpolating it or should we use another random variate generation technique? Thank you in advance! Happy New Year.

could you make more advance time series tutorial? really like your videos and i'm struggling in grad level time series course

Thank you!!!!!!!!!!!

What is the role of lambda? I've seen other videos that don't include it, so now I'm curious. Amazing explanation btw!

excellent

Great video. Quick question: at the end of the video, you said we could swap 1- u for u. That means 1 - u = u, which translates into u = 1/2. Yet u is a random variable, it is not necessarily 1/2, right? What am I missing?

Thank you for this, it has helped me a lot! :)

Very nice video! thank you! The uniform should be (0,1] without 0 right? so the ln will be defined.

thank you so much sir, I would like to know which probability distributions commonly used that we use inverse method with.

Hi Ritvik, great videos! would be interested to have a set of videos explaining variational bayes, ELBO etc. in order to perform bayesian optimisation on hyper-parameters

great explanation, thanks a lot!

Are there any resources I can look at to understand why it's valid to assume that p(T(U)

great video! thanks!

thanks, this is a great video!

Well explained. Thanks a mil

TLDR: The distribution of the CDF of _any_ PDF is uniform, so if you want to sample from a PDF that has an invertible CDF, you can sample from the uniform distribution and convert it to the desired distribution with the inverse of the CDF.

this is super clear and I can go to bed very happy

Question : Hi, i have rainfall data as a 2d matix/frame of the UK every 5 minutes so the data is spatially and temporarily correlated. The data has severely positive skewness. Around 90% of pixels or points are less than 10 and 10% between 10-128. When i train a cnn, it is only predict rainfall of low values because of the data imbalance. I would like to transform to uniform distribution. I tried log transformation which compressed the data but still there is imbalance. Do you know how to convert to a uniform distribution so all of the values have the same chance to be predicted? It is a regressio task to predict the next 12 frames of rainfall. The data is represented by only one continuous variable, rainfall intensity. Many thanks

I don't understand how you reached the final conclusion that P(u

I don't get it why the exponential distribution is called memoryless? Yes, I know that that lambda or hazard rate is constant but isn't that just the speed or rate of the probability (not the actual probability because the lambda can be more than 1). From the exponential PDF, you can clearly see that the chances in the early phase are bigger than in the later phases so why is it called memoryless? If I sampled time to failures, should I get more numbers early than later because of that decreasing curve?

Thank you man

I have question about the math, on how to derive other inverse transformations especially for datasets that predict number of clicks on a web page for example. Some of them are tricky and might even need estimation by iterative numerical methods or ML, because the Poisson is simple to find the inverse function for. And then how do you put the inverse transform into an sklearn pipeline exactly? Here's why I ask this: Sometimes I am using a Generalized Linear Model which provides a convenient link function already built-in, but we are not always going to just use a linear model as we might need to use for example the large feature vectors that an NLM model is producing to describe some text. GLM is not necessarily the only tool to consider. Besides for random sampling, Transforms are also good for ML preprocessing and postprocessing pipelines to help your model learn easier. The log(Y) and e(Y) are the Poisson distributions transformations when your response Y is a count. Quasipoisson and Negative Binomial are good for count data when the mean and variance are not staying equal as the Poisson requires, but instead are showing some overdispersion or underdispersion. There's also zero inflation model which combines a logistic model and a Poisson model together in sort of an ensemble to help pre-predict the count = 0 case when 0 appears a lot more often than plain old Poisson can account for alone.

I probably missed this moment: why transformation to CDF actually gives you desired distribution?

Thx body u the best

In data science can we transform weibull distribution into Gamma or poison distribution?

in order to find the inverse of CDF, we just find the value of x..why? in other word, how come x is the inverse of CDF?

there is an error for the pdf of the exponential distribution, the lambda is missing.

If I understand correctly, the reason why uniform distribution is used because its output range from 0 to 1. Just out of curiosity, can we use beta distribution to replace uniform distribution?

Great video. Could you do one on copulas, building on this one?

beauty

Not very proficient in statistics, but in sum, if I do the transformation and have the final function, given a number u that is randomly generated from a uniform distribution, I will get an equivalent randomly generated number that falls under an exponential distribution? great video, I will subscribe and continue to watch them!

So if I had some other distributions apart from exponential one, I just need to derive its inverse, and set the number of simulations I will like to do with a U that is unif from 0 to 1? I just need clarification in that part.

Was hoping to hear more about motivations for WHY i need to know this method for DS. "that's how computer gives you random samples from a distribution" is not enough to care about it. What about cases where maybe I don't have a pdf or it cannot be integrated or I get only proportionality of pdf (like in Bayesian model) so I can't just plug in the variable into the proportional pdf and get accurate samples..... maybe that's when I need to use this method.....

I have a question if we graph the inverse function of that exponential function how it will looks like? whether it looks similar to graph of uniform distribution? otherwise how this can be equal?

if the distribution we want is not the exponential distribution, are the steps are still the same？

its not a ''datascience'' method (which sounds like it comes from modern era). it is known as smirnov method who lived around 1900 and likely known before

What if CDF is not invertible?

Having a hard time understanding EWMA and GARCH model ,can you make some videos introducing them?thx