Bootstrapping Main Ideas!!!

Рет қаралды 430,433

Күн бұрын

Bootstrapping is one of the simplest, yet most powerful methods in all of statistics. It provides us an easy way to get a sense of what might happen if we could repeat an experiment a bunch of times. It turns point estimates into distributions that can be used to calculate all kinds of stuff, including standard errors, confidence intervals and even p-values. In this video, we show how it can be used to calculate standard errors and confidence intervals. In Part 2, we'll see how to calculate p-values.
For a complete index of all the StatQuest videos, check out:
statquest.org/video-index/
If you'd like to support StatQuest, please consider...
Buying my book, The StatQuest Illustrated Guide to Machine Learning:
PDF - statquest.gumroad.com/l/wvtmc
Paperback - www.amazon.com/dp/B09ZCKR4H6
Kindle eBook - www.amazon.com/dp/B09ZG79HXC
Patreon: / statquest
...or...
KZfaq Membership: / @statquest
...a cool StatQuest t-shirt or sweatshirt:
shop.spreadshirt.com/statques...
...buying one or two of my songs (or go large and get a whole album!)
joshuastarmer.bandcamp.com/
...or just donating to StatQuest!
www.paypal.me/statquest
Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
/ joshuastarmer
0:00 Awesome song and introduction
2:18 Bootstrapping in action!
4:23 Bootstrapping defined
6:40 Calculating standard errors and confidence intervals with bootstrapping
8:13 What makes bootstrapping so awesome
Correction:
5:55 8^8 combinations of observed values and possible means assumes that order matters, and it doesn't. So 8^8 over counts the total number of useful combinations and the true number is 15 choose 8, which is 6435
#StatQuest #Bootstrap #Statistics

Пікірлер: 642

@statquest 2 жыл бұрын

NOTE: A lot of people ask "What happens when the original collection of measurements is not representative of the underlying distribution?" It's important to remember that a confidence interval is not guaranteed to overlap the true, population mean. A 95% CI means that if we make a ton of CIs using the same method, 95% of them will overlap the true mean. This tells us that 5% of the time we'll be off. So yes, a sample that is totally bonkers is possible, but rare. Understanding this risk of making the wrong decision, and managing it, is what statistics is all about. Also, at 5:55 I say there are up to 8^8 combinations of observed values and possible means, but this assumes that order matters, and it doesn't. So 8^8 over counts the total number of useful combinations and the true number is 15 choose 8, which is 6435 (for details on this math, see: en.wikipedia.org/wiki/Multiset#Counting_multisets ) Support StatQuest by buying my book The StatQuest Illustrated Guide to Machine Learning or a Study Guide or Merch!!! statquest.org/statquest-store/

@natasgestel6873 2 жыл бұрын

We take for granted all that went behind that idea of 95% CI that you stated - it was Jerzy Neyman's who came up with that definition. Have you read "The Lady Tasting Tea"? A bit of a history of some incredible mathematicians, including Ronald Fisher and Jerzy Neyman. The 95% comes up on page 123. Thanks for all your valuable statistics videos!

@statquest 2 жыл бұрын

@@natasgestel6873 Yes, I've read the book. Those dues were pretty smart.

@PastryDonut Жыл бұрын

Thank you for explaining that order doesn't matter. I was looking for the clarification on this everywhere.

@alexandersmith6140 7 ай бұрын

So, if we take our sample of 8 observations, and we calculate a 95% confidence interval around the sample mean by bootstrapping, and then a genie appears and tells us that the true population mean lies outside of that confidence interval, that's the same as saying that our original 8-observation sample's mean actually wouldn't appear 95% of the time if we repeated the experiment infinitely many times, each experiment being an 8-observation sampling of the population?

@statquest 7 ай бұрын

@@alexandersmith6140 The definition is of a 95% CI is that if we repeated the process of creating the 95% CI a ton of times, 95% of the CIs created that way would overlap the true mean. Thus, if collected 8 measurements and used Bootstrapping to calculate a 95%, then that if we repeated that process of creating the 95% CI a ton of times (collected 8 measurements, then calculated the CI with bootstrapping), then 95% of those CIs will overlap the true mean. In other words, it doesn't matter if we use bootstrapping, or some formula to calculate the CI, in both cases we have to collect 8 measurements a ton of times.

@yannivdp1600 Жыл бұрын

I have done a master's in stats and a course in data analysis, and the only reason I've passed these things is that after a long and confusing lecture I can just come and watch you explain it in simple terms. Bam! Thank you so much!!

@statquest Жыл бұрын

Thanks! I'm glad my videos are helpful! :)

@BheezHandle 7 ай бұрын

I am presently in your shoes, taking a Data Science Course but thanks to @statquest. giving him Double Bam!!

@PunmasterSTP 2 ай бұрын

That's cool! How are your studies/career going?

@haloforgeguy453 2 жыл бұрын

Still floored that this works as a method

@statquest 2 жыл бұрын

I know - it's so easy, yet so effective.

@patrickjdarrow 2 жыл бұрын

@@statquest is it really so effective? We really can only be as confident -- that bootstrapping produces characteristic data -- as we are that the sample is representative of the distribution -- right? Unconfident extrapolation seems like a good way to pollute datasets.

@statquest 2 жыл бұрын

@@patrickjdarrow Just like any statistical method, you have to have a reasonable sample size. n = 8 as a minimum is a good starting point.

@NeuralNine Жыл бұрын

There is nobody on KZfaq that explains statistics better or in a more entertaining way than you! Keep it up!

@statquest Жыл бұрын

Wow, thanks!

@NTesla00 2 жыл бұрын

Passed all my stats courses already (thanks to your videos for a major part), but I'm still watching these as they come out, lol. Keep it up Josh, this channel is so good.

@statquest 2 жыл бұрын

Thank you very much! :)

@gamuchiraindawana2827 5 ай бұрын

What I love about you is that you explain the big picture first. You help me understand why we should care in the first place, or the motivation behind the concept. Then you dive into the details afterwards, you make the information more accessible without compromising the technical integrity of the information. A very rare skill indeed, I'm reading Introduction To Statistical Learning in R ( ISLR ) and some chapters aren't intuitive, whenever I read a chapter that doesn't make sense I just watch your videos. That's how I know you're not compromising the technical integrity of the information, because what you say doesn't contradict what I read in academic papers, it's just easier to understand than what I read in academic papers. You are one of a kind!

@statquest 5 ай бұрын

Thank you very much!

@gamuchiraindawana2827 5 ай бұрын

@@statquest No, thank YOU Josh!

@CaptainFeatherSwordz 2 ай бұрын

I think you summed up the value of these videos really well. Starting with the big picture and then zooming into the details is so much more beneficial for learning and I think this is one of the things Josh nails!

@gamuchiraindawana2827 2 ай бұрын

@@CaptainFeatherSwordzIt's worlds apart from what the education system has conditioned us to right?

@monoarul_islam_3 2 жыл бұрын

You're probably the best guy for this job. Even though I don't know where I'm gonna apply all these. I just keep going through all of your videos. After finishing up this playlist I'll watch the ML playlist. Keep amazing us. Thank you JOSH

@statquest 2 жыл бұрын

Thanks!

@Hemzees 2 жыл бұрын

I can never get over how your videos make me love statistics when all my professors and recommended texts made me run away from it. Super grateful!! Also, I think I asked when this video was coming about a year ago.

@statquest 2 жыл бұрын

Glad it finally came out! :) Sorry it takes me so long to make videos.

@aishah8398 2 жыл бұрын

I wonder n feel so much regard for the institution and teachers, who taught you... no doubt, you are doing an incredible job...stay blessed always

@statquest 2 жыл бұрын

Thank you! :)

@abhishekpawar921 2 жыл бұрын

Thanks for uploading these videos. It takes a lot of time and efforts to make such quality content. Thank you, Sir.

@statquest 2 жыл бұрын

Glad you like them!

@KillerFridge 2 жыл бұрын

I love SQ, because I finally "get" bootstrapping, despite having used it for years!

@statquest 2 жыл бұрын

BAM! :)

@KillerFridge 2 жыл бұрын

@@Synthanicmusic I do as a data scientist. Honestly, If you know that you don't know what you're doing then you are going to be better positioned than most; it means you will be questioning why you are applying certain tests/methods, rather than just doing so blindly. Especially in the workforce you will see a lot of badly reasoned statistics!

@rhexieleelafuente4271 2 жыл бұрын

What a comprehensive and fun discussion! I really had trouble understanding the concept of bootstrapping by myself but your lecture helped me a great deal :> Kudos!

@statquest 2 жыл бұрын

Glad it was helpful!

@Esmeralda_Art 3 ай бұрын

My man sounds sounds excited and bored at the same time and Im here for it 😂 Great explanation, something my Prof couldn’t manage. Elite university my ass lol

@statquest 3 ай бұрын

bam!

@qi2li2n_prance 7 ай бұрын

You made this concept so much easier to understand than what I was supposed to be learning it from. Thank you so much!!

@statquest 7 ай бұрын

Glad it was helpful!

@aneishajay7949 Жыл бұрын

I read a section on bootstrapping countless times and only understood it finally after watching your video! All I have to say to that is: BAM! (and thanks a bunch)

@statquest Жыл бұрын

Hooray!!! :)

@diegobuenovillafane869 Жыл бұрын

BEST explanation EVER of bootstrap. Thanks for your dedication!

@statquest Жыл бұрын

Glad it was helpful!

@yuvrajkhanna2819 Ай бұрын

Watched the Stanford's and other lectures on similar topics, but you made it really simple and easier to understand. You teach good!! BIG BOOM BAMM !! thanks man

@statquest Ай бұрын

Thank you! :)

@nautical1078 2 жыл бұрын

Thanks for the videos, embarrassingly I'm relearning a lot of these concepts even though I graduated with a Statistics major. It's coming a lot easier now.

@statquest 2 жыл бұрын

Glad to help!

@vladimirbriceno6051 2 жыл бұрын

Don't be embarrassed, it's not your fault, but the education system's

@nancygonzalez3193 2 жыл бұрын

Happens a lot more often than you think. I graduated with a physics major not long ago and I can say I still cannot consider myself a physicist. I constantly keep finding myself learning things from awesome channels like Josh's that I'm supposed to know by now.

@ankiwikman1218 2 жыл бұрын

I've always grasped well enough to get a good grade but not well enough to embed, so I have to go back a lot.

@quorrexnoway9127 2 жыл бұрын

This kinda feels illegal xD Really nice explained!

@statquest 2 жыл бұрын

Thank you! :)

@luko141 2 жыл бұрын

Mr. Josh - u are amazing. World needs more ppl like u. Its like education on another level. Thank you

@statquest 2 жыл бұрын

Thanks! :)

@travelthetropics6190 2 жыл бұрын

Wow! this is the first time I learned this. awesome!

@statquest 2 жыл бұрын

BAM! :)

@user-ww2qs3cj9n Ай бұрын

You sir are an absolute legend. Really helping me getting through my course, because my professor explains the same concept in a method that is 100 times harder to understand

@statquest Ай бұрын

Happy to help!

@davusieonus 2 жыл бұрын

i loved your bams and the illustrations for the steps and your explanation helped a lot

@statquest 2 жыл бұрын

Thank you!

@meanya4687 Жыл бұрын

All semester long I have been floundering through my statistics class, no thanks to my professors' boring and quite difficult-to-follow lectures on the materials. I've felt so dumb all semester, so when the next section called for "bootstrapping" I finally decided to throw her lecture videos aside and see if someone could explain the concepts better on KZfaq. Boy am I glad I stumbled upon this. The visuals are straight to the point and the way you talk through everything very slowly and clearly is SOOO helpful. The enthusiasm and goofiness helps me keep my attention, which is a pain for me with ADHD. I could rewatch my prof's videos 5 times and retain nothing. Makes me wanna just burst into tears from frustration. But I felt like I could actually keep up with this video and _understand_ it! TL;DR thank you for making this, it was a HUGE improvement over my professor's teaching style and I will DEFINITELY be consulting you for future topics. You're a peach

@statquest Жыл бұрын

Hooray! Thank you very much. Just for reference, here's a list of all of my videos: statquest.org/video-index/

@meanya4687 Жыл бұрын

@@statquest thank you very much

@AvatardSwag 2 жыл бұрын

This is the most amazing video I've seen on bootstrapping, thank you! Quadruple Bam!

@statquest 2 жыл бұрын

Wow, thanks!

@alessandrobucci951 Жыл бұрын

One of the most useful video on this topic on youtube, thanks!

@statquest Жыл бұрын

Wow, thanks!

@hugoviotti9795 2 жыл бұрын

Always glad to see a new statquest! BAM!

@statquest 2 жыл бұрын

BAM! :)

@punditgi 2 жыл бұрын

Very clear explanation. Well done!

@statquest 2 жыл бұрын

Thank you! :)

@adityarazpokhrel7626 2 жыл бұрын

So smoothly explained. Thank you sir.

@statquest 2 жыл бұрын

Thank you!

@oscarrosalescorzo 9 ай бұрын

The work you do is awesome!! Love it.

@statquest 9 ай бұрын

Thank you!

@EmirTT 11 ай бұрын

Excellent explanation as always by StatQuest!!! Thx a lot!!!

@statquest 11 ай бұрын

BAM! :)

@lahaale5840 2 жыл бұрын

Thank you so much, your videos are always so helpful to me

@statquest 2 жыл бұрын

Glad you like them!

@witchaponkitthaworn5998 10 ай бұрын

I am learning machine learning and came to this term , this videos explain it very clear, thank you.

@statquest 10 ай бұрын

Thanks!

@yohanessatria2220 Жыл бұрын

bruh... this explanation is simply awesome!

@statquest Жыл бұрын

Glad you liked it

@gitaecookiemonster2892 2 жыл бұрын

Love all of your videos!! Thanks a lot!

@statquest 2 жыл бұрын

Glad you like them!

@darnelb1912 2 жыл бұрын

Great video! I learn so much with you!

@statquest 2 жыл бұрын

Awesome! Thank you!

@vtphan2012 2 жыл бұрын

Another great video. This video explains how to do bootstrap, which is the easy part. The more difficult part is to understand why bootstrap works. The conceptual challenge is that bootstrapping assumes that if we were to repeat an experiment, it would produce one of the outcomes we had observed. This could be a huge assumption, depending on the applications. Boot strapping does not add any new information to what has been observed.

@statquest 2 жыл бұрын

Noted

@sgpleasure Жыл бұрын

"The reason why this works is because the histogram of the sample tends to look very similar to the histogram of the population. That's really the key idea behind the bootstrap, and we will see how this idea can be used in all kinds of complicated situations. " Taking an online course on bootstrap regression and came here to try to understand why bootstrap works when it does not generate any new information.

@vtphan2012 Жыл бұрын

@@sgpleasure When you sample from a population, it’s unsurprising that the distribution of the sample resembles the distribution of the population. So, you’re not really obtaining any new information. In essence, we’re only pretending it’s new information, when in fact, it’s just reconfirming existing information.

@hermesavurquijo9961 Жыл бұрын

Thanks a lot, your video helps me and hopefully it will help my paper too

@statquest Жыл бұрын

Thanks!

@namitaratnakar_6591 Жыл бұрын

Love the explanation ..... Thank uh soo much❣️

@statquest Жыл бұрын

Thanks!

@utkarshsingh2675 Жыл бұрын

easy to understand....thanks josh!

@statquest Жыл бұрын

Thank you!

@arifmemovic3383 2 жыл бұрын

You rock Josh. Thanks for making this video!

@statquest 2 жыл бұрын

Thanks!

@dejanspasovic377 Жыл бұрын

Amazing videos, simple and well explained.

@statquest Жыл бұрын

Many thanks!

@ignasialemany7738 2 жыл бұрын

The triple BAM was amazing, thank you!

@statquest 2 жыл бұрын

Thank you!

@williamss4277 Ай бұрын

Thank you for the teaching 🎉

@statquest Ай бұрын

Any time!

@Artificial_Historian Жыл бұрын

Awesome explanation!

@statquest Жыл бұрын

Thank you!

@thomasdelossantosverrijp8117 Жыл бұрын

thank you, this was very helpful

@statquest Жыл бұрын

Glad it was helpful!

@alexyang2002 Жыл бұрын

really clear, thanks

@statquest Жыл бұрын

Thank you!

@svenheiter1608 2 жыл бұрын

Not the information I was looking for but i couldn't stop myself from watching it to the end. It was quite entertaining :)

@svenheiter1608 2 жыл бұрын

*BAM

@statquest 2 жыл бұрын

That's awesome! BAM! :)

@ayenisedaghat4667 3 ай бұрын

You are a legend my friend! A legend. I am doing my masters in Data Science this fall and this is amazing

@statquest 3 ай бұрын

You can do it!

@yingyanhuang1688 Жыл бұрын

You are amazing! Thank you!

@statquest Жыл бұрын

Thanks!

@user-ex1fj5rn2o 2 жыл бұрын

Love it!!! It's really helpful

@statquest 2 жыл бұрын

Thanks!

@estevaodias6404 2 жыл бұрын

Awesome! The proofs about it seems to be nice

@statquest 2 жыл бұрын

Thanks!

@TwilightPhantom1 2 жыл бұрын

You're the best man, thank you!

@statquest 2 жыл бұрын

Thanks!

@tsuchiko_7155 Жыл бұрын

OMG, it was a truly easy-to-understand video! Both the animation, narration, and explanation!!!! I wanna give a billion likes!!!

@statquest Жыл бұрын

Wow, thanks!

@musefaredi5540 Жыл бұрын

Nice way of explanation!! BAM!!!

@statquest Жыл бұрын

Thanks!

@ElSamyto 8 ай бұрын

Okay you are the best thank you for doing this video !

@statquest 8 ай бұрын

Thank you!

@mahdiranjbar3569 2 жыл бұрын

Ur videos are just so cool, tnx a lot

@statquest 2 жыл бұрын

Glad you like them!

@everyhinghere1377 Жыл бұрын

this is better than college-level advanced course !!! thank you

@statquest Жыл бұрын

Wow, thanks!

@annaspitczok9471 2 жыл бұрын

Big BAM for so much statistic knowledge in such little time

@statquest 2 жыл бұрын

Hooray!

@JosephElfassi 2 жыл бұрын

Super clear, thanks!

@statquest 2 жыл бұрын

Thank you!

@TheShaha26 2 жыл бұрын

this is amazing. thank you !

@statquest 2 жыл бұрын

Thanks!

@user-gc5ih9fm5y 9 ай бұрын

The intro was gold 🔥

@statquest 9 ай бұрын

bam!

@jasonvoreseeas6578 2 жыл бұрын

I'm going to recommend this channel to a bunch of my machine Learning nerds. This guy deserves every hype possible!

@statquest 2 жыл бұрын

Thank you! :)

@andreyele3464 Жыл бұрын

Josh, you're sent to us from heaven, thanks

@statquest Жыл бұрын

@DragonDragon-qr6mq 2 жыл бұрын

bam. so clear explaination. thx

@statquest 2 жыл бұрын

Thanks!

@RobertaBraga_ Жыл бұрын

Thanks Josh, you are the one!

@statquest Жыл бұрын

Thank you and congratulations again. I'm so glad I was helpful. BAM! :)

@dutonic Жыл бұрын

Wow this is so good. The intro made me laugh so hard, it wasn't even that funny I just didn't expect it.

@statquest Жыл бұрын

Thanks!

@Pedritox0953 2 жыл бұрын

Love your videos !!

@statquest 2 жыл бұрын

Thanks!

@Cuddpo 6 ай бұрын

Thank you 🙏

@statquest 6 ай бұрын

You’re welcome 😊!

@AnalyticsAlchemy-nq3kq 3 ай бұрын

Nice explanation...awesome

@statquest 3 ай бұрын

Thank you!

@olivier306 2 жыл бұрын

absolutely perfect god bless you

@statquest 2 жыл бұрын

Thank you!

@marlocosta5531 2 жыл бұрын

Thank you JOSH!

@statquest 2 жыл бұрын

bam!

@tripp8833 Жыл бұрын

Thanks!

@statquest Жыл бұрын

Thank you so much for supporting StatQuest!!! BAM! :)

@thomasporritt8323 2 жыл бұрын

Great way to break bootstrapping into common language.

@statquest 2 жыл бұрын

Thanks!

@dylanspodcast 8 ай бұрын

This guy is great!

@statquest 8 ай бұрын

Thanks!

@datascience_azamat 2 жыл бұрын

wow, great content!

@statquest 2 жыл бұрын

Thanks!

@leonelemiliolereboursnadal6966 Жыл бұрын

¡Gracias!

@statquest Жыл бұрын

Hooray!!! Muchas Grasias for supporting StatQuest!!! BAM! :)

@Acampandoconfrikis 2 жыл бұрын

Best intro ever Pixar would envy you

@statquest 2 жыл бұрын

bam!

@user-xp7ye3fr5l 3 ай бұрын

blud just dropped one of the best explanatory videos out there and thought we wouldnt notice☠☠☠

@statquest 3 ай бұрын

bam! :)

@thomasdelossantosverrijp8117 Жыл бұрын

thank youuuu

@statquest Жыл бұрын

@sevenindigo5945 Жыл бұрын

I can only say one thing: BAM!!! you are the best teacher BAM!!!

@statquest Жыл бұрын

Thank you!

@leonelemiliolereboursnadal6966 Жыл бұрын

you are the best man!!!, tirple bamm!

@statquest Жыл бұрын

Thank you!

@ninavanleeuwen1183 Жыл бұрын

thank you sir

@statquest Жыл бұрын

Thanks! :)

@maurobruno6954 2 жыл бұрын

Dear Josh I bought a few study guides :) Thanks so much for your videos

@statquest 2 жыл бұрын

Awesome! Thank you so much for your support!!

@theburtmacklin9615 2 жыл бұрын

Nice - “Part 1”, gonna have another nice solid multi-part set.

@statquest 2 жыл бұрын

At least 2 parts. Part 2 will be out soon.

@PunmasterSTP 2 ай бұрын

Bootstrapping? More like "Bro, it's awesome knowledge you're dropping!" 👍

@statquest 2 ай бұрын

Bam! :)

@PunmasterSTP 2 ай бұрын

@@statquest Boot! 🥾

@suheyla655 Жыл бұрын

You are the best!

@statquest Жыл бұрын

Thanks!

@nikhilravichandran4302 Жыл бұрын

The best stats teachings out there! Kudos!!!! Question : Do we need to know/estimate the distribution(normal/gamma/exponential/etc) of the bootstrapping histogram to determine the 95% confidence interval in cases where central limit theorem doesn’t apply( such as median)?

@statquest Жыл бұрын

@nikhilravichandran4302 Жыл бұрын

@@statquest Thanks! :)

@dsanjoy 2 жыл бұрын

Instantly subscribed!!

@statquest 2 жыл бұрын

Thanks!

@PythonArms Жыл бұрын

great video

@statquest Жыл бұрын

Thanks!

@ezequielzion 2 жыл бұрын

Amazing, I love when you say Bam 😂

@statquest 2 жыл бұрын

Bam! :)

@petercourt 2 жыл бұрын

Awesome work! Thanks Josh! It seems weird to me that this creates useful/meaningful results. You're just reusing the same data again and again, so even though we can generate new means by bootstrapping, are those means actually correct? In other words, has anyone compared bootstrapping (and calculating the mean) to repeating the experiment a bunch of times (and calculating the mean)?

@statquest 2 жыл бұрын

Yes. Bootstrapping has been around for about 40 years and has tons of theoretical justification etc. It's the real deal.

@jeanf6295 2 жыл бұрын

You can simulate bootstrapping techniques easily. The bootstrapped distribution has the same mean as the statistical mean of the sample, and the more points you have in your original sample, the closer its width will be to the spread of the means you would get if you redid the experiment a bunch of time. Bootstrapping is not the same however as doing the experiment a bunch of times, as the latter would give a bigger dataset with a narrower confidence interval. This is a confidence interval estimation method that does not make assumptions about the underlying data distribution.

@jaundersousa3530 2 жыл бұрын

A professor at the university I studied at was apparently a key contributor to Bootstrapping. Excellent job at explaining it in such an easy-to-understand way!

@aditya4974 2 жыл бұрын

What's the University and Prof's name?

@statquest 2 жыл бұрын

bam!

@anakagung7613 2 жыл бұрын

Awesome song!

@statquest 2 жыл бұрын

Thanks!

@GeerathBhat-vm1vp Жыл бұрын

Thank you so much for your wonderful videos. I have a small request to provide a lecture on FLDA, GMM, EM Algorithm, MLE estimation, MAP estimation. Also, there are some lectures which are not in the book, please also include those lectures too. Thank you so much again!!!. I want to learn more and more from your lectures.

@statquest Жыл бұрын

Thanks! I'll keep those topics in mind.