Correction: Thanks Yidan Shang -- At ​9:12, the Spool calculated should be 1.06 instead of 1.099. Thanks Ruby Jiang -- At 7:38, the mean of treatment is 1.7 instead of 2. The subsequent calculations should be changed accordingly.

What I love about your channel is that you don't charge $300 to unemployed job seekers for this information.

我觉得这个系列真的非常不错。每一个视频虽然看起来短，但是把要点提炼的非常好，详略得当，非常goal oriented，就是为了interview来的。谢谢！

I recommended you to all my classmates. Excellent work and presentation of what is needed!

My hiring manager actually recommended this series of videos. Super helpful for someone who doesn't have much business experience. Thank you!

quality of content is top notch! thanks for making these videos .looking forward to learn more from you.

All your videos are gold mine. Keep up the good work

Great content!!! The best explanation of z-test and T-test on KZfaq! Great examples!!! Feel very lucky to find you here🙏! Thank you!!!

so so so helpful! Thank you Emma

Really great video thanks 😋 I appreciate the effort 🙏

Thanks a lot for your channel! It helped me to prepare and get a job offer! :)

Thank you sooooo much Emma! I have trouble with combining the hypothesis and A/B testing knowledge together, your video saved me!!!

Amazing tutorial !!!

Super helpful!

Great explanation Emma! Nice work! For the second case, can you show in formula and calculation of how did you come up with the lower bound is more than Dmin 0.05?

Hi Emma, really appreciate that you made all the great videos, which is very helpful. I was wondering if you can make some videos about how to handle the take home challenge such as Lyft and Airbnb. Any information will be highly appreciated! Thanks

Thank you Emma, I am learning a lot, God bless you! I finally feel I understand this topic.

Hi Emma, thanks for making these great videos. I have a quick question about the term you use at 9:00. Should it be pooled standard deviation instead of pooled standard error here?

Hi Emma - Thank you so much for the sharing! I have learnt a lot. While I would like to clarify the formula you used to calculate the SE in the Confidence Interval of two sample proportions (4:12). The formula you used is the SE in the test statistic of two proportion Z-test, but the SE for the CI should be different (sqrt(p1*(1-p1)/n1 + p2*(1-p2)*n2) . Please correct me if I am wrong here. Thanks

Hey Emma, great video and love the decision flow chart. One quick question - at 9:12, why the Spool =1.099? I calculated the pooled variance =1.13, and the square root of that would be 1.06 instead of 1.099.

Pretty good explanations with example. Thank you. Do all companies use t-test and z-test or do they use boostrap test by running simulations?

excellent video! i had one question though: what do you do if you are dealing with unequal variances in your control and treatment group? I'm assuming you can't calculate SE using the pooled formula you have shown, right? would appreciate some advice on this!

Thank you for sharing this! I have a question - in example 2, although we don't know the population variance, but the sample size is large. Can we use the sample standard deviation to estimate the population standard deviation (large sample size) so that we can use Z-test? Thanks!

Hi Emma, thank you for your valuable video. In the video at 5:57, dmin = 0.05, but at dmin = 0.01. I am a little confused the value of dmin. why it was changed from 0.05 to 0.01 and the value is determined randomly or it could be calculated by some formula? Thanks

This is a BLESSING. Thnk u so much.

Hey Emma great video, really helping me understand the process of AB testing! From watching all of your series on this topic, one thing I'm having trouble understanding is the relationship between MDE and practical significance. I understand MDE is used to calculate sample size such that the sample size calculated is the sample size needed to detect statistical significance at the magnitude of our MDE. In my mind I thought the point of the MDE is such that we can have our null hypothesis be "variant is not larger than control by given MDE (if MDE is positive)"; however, the case seems to be null hypothesis is just control metric != treatment metric. Is the MDE used later on then as the practical significance boundary you mentioned in this video then and it doesn't have anything to do with determining statistical significance beyond helping estimate our initial sample size?

Super helpful!!!

One question Emma. at 7:43, the mean of the treatment is 2. However, by calculating directly from the data array you gave, the mean in treatment I got is 1.7. This can be validated by the sum of squares since if you use 2, the SS of the treatment group at 9:10 will actually be 37 instead of 34.3. Could you double-check? And also could let us know that why the difference of mean at 9:22 is 0.633? Since it would be 0.6 if you use 2 - 1.4, and 0.3 if you use (1.7 - 1.4). Thanks!

非常感谢！讲的比学校老师好太多了！！

Great video! Coding rounds are also big part of the interview process. Can you please make some videos on that too? That will be great. Thank You.

Hey Emma @Data Interview Pro, Thanks for the great video. But I am quite confuse regarding the CI for second case. As the practical interval is 5%, but the CI you calculated is the actual number (number of post interval). Just wondering how they can be compared.

Hi Emma, I'm analysing a conversion A/B test result. I wonder how to account for the issue that a change of conversion is due to the different directions of numerator (checkout sessions) and denominator (homepage sessions) ? E.g. the homepage sessions reduced while the checkout only increased slightly or no change at all? Can we conclude that the treatment group actually performs better? Thanks.

Thanks 😊

Hey Emma at 5:52, when calculating the margin of error, the Z you used is 1.96 which is from statistical significance level instead of the practical significance boundary. Could you please confirm if this what's supposed to be used? Thank you!

One question - do we also look at p value to decide if there is a statistical significance, or we only need to compare test statistics and z-score?

In your example, what is the timeframe for the average number of posts? If I just want my average posts per user to go up, how do I decide the timeframe to run the test? Also, say I wanted average posts per day to go up and I have data for the same users across multiple days, how do I check my hypothesis then? Hope this makes sense.

Great video as always Emma! I have a question, is it alright to schedule a Technical phone screen 3 weeks out and the on-site interview a whole month after the technical phone screen? Is it possible to get rejected by the recruiter if I ask for such far out interview dates?

Thanks Emma for the video! Very helpful! is it a typo on 9:10, should it be “pooled” standard deviation? Cause it is formula to calculate sd instead of se, thanks！

What is the difference between using the practical significance boundary and minimum detectable effect?

Can you do a video on Chi - squared test, like here the click through probability would be characteristic type.

Hi Ding, How can we find the most precise flow chart of statistical test, there are versions of them on google.

Hi @emma_ding thanks for creating such great and informative videos on A/B testing. I did not understand how did you calculate SS(sum of squares for control and treatment groups)?

Why does the difference in sample proportions (d) follow a normal distribution? is it because the sample size is large enough?

Hi Emma, I have a question about the sample size. Do the control and treatment group have to have the exact same sample size? For example it’s 1000 users here for both. Can I have control group 1023 users, and my treatment group have 1048 users? Will this affect our result? Thank you! Thanks you for all the great contents!!! It significantly helped me understand better about hypothesis testing! ❤

Thanks for going through these problems step by step. In interviews, are we supposed to use built-in libraries (for instance scipy) for these questions or implementing everything from scratch? First one shows we are familiar with libraries, latter one shows we know the math :) I am not sure which way is the best one to go.

I can't thank you enough

CI in the second case is coming inside the practical boundary for me. Am is missing something? CI for d = 0.633(+ or -) 2.2018 (Multiply 2.002*1.0998). so the range is -1.56 and 2.83. Really helpful video. Thanks Emma!

why do we use the same z value used for p value to calculate the confidence interval. Should we not choose the z value for calculating confidence interval based in 0.01 practical significance boundary (6:50)?

Hello miss, In some books, they make use of permutation test to carry on hypothesis testing to analyse A/B experiment’s results ? Can you tell us when to use Permutation test or the test you have presented in your video (binomial… t and z test) ? Thanks

Hi Emma, how did you calculate the CI in the second example exactly when assuming similar variance for both control and treatment? I got the margin of error = t-score*SEpool = 2.002*1.0999 = 2.202. Then with \hat{d} = 0.6, which would then give a very big CI that includes the entire [-dmin, dmin] = [-0.05, 0.05]. But it seems like in your slides the CI is strictly on the right side of [-dmin, dmin]. I'm very confused and would appreciate some help! Otherwise, your videos have been super super helpful!! Thanks.

Hello Emma, Thanks for all your helpful content :) I attempted the steps you gave in the video but was unable to solve this question. In Loan application analysis task. What is the best way to solve this Hypothesis testing? What's the effect of owning a car on the likelihood of a loan application being accepted? Own_car attribute is Yes(1) or No(0) loan_application_accepted attribute is True(1) or False(0) The dataset has 1000records own_car value count: No 598 yes 405 loan_application_accepted count: False 703 True 300 own_car value count where loan_application_accepted is True: No 187 yes 113 Null Hypothesis: Owning a car doesn’t affect loan application acceptance Alternate Hypothesis: Owning a car does affect loan application acceptance

Hi How did you find critical Z score in the first part as 1.96?

Hey Emma, just a quick question, have you heard of Strata Scratch? Is it a good platform to practice interview-style SQL and Python coding questions? Are the questions actually asked in those companies which are tagged? Would love to hear your thoughts.

Hi Emma, thanks for the video! How do we know that we have a large sample size?

At 5:05, should you not be using the Z-score for alpha = 0.025 instead of 0.05 since you're using a two-tailed test?

Thanks for showing all pooling formulas and concepts in one place. At 1:16 and 7:50 you mentioned practical significant boundary of 0.01 and 0.05, how does the experimenter come up with these values? Is it from calculating business costs and revenues? You talked about checking CI, which reminds me of something confusing i read from point 4 on hookedondata.org/guidelines-for-ab-testing/. Could you comment on why the cases she cites are possible? (CI that is wide and close to 0 vs CI that is tiny and far away.) What i don't understand is assuming the same p-value (not sure if this assumption is required for this discussion), how can a CI be tiny (think she means narrow )AND far away simultaneously? CI width depends on standard error, so a narrow CI means a narrower sampling distribution of whatever statistic, and the centre of CI should be closer to the null hypothesis sampling distribution centre compared to the CI with a larger width. Is my reasoning correct that a narrower CI would have a centre that is closer to the null centre? How to understand what Emily is saying there then? She seems to say the narrower CI can have a centre of CI that is further.

How did we get practical significance boundry?

I am very confused by the practical significant boundary, why in example 1, dmin =0.01, and in example 2, its 0.05?

Did anyone else try to reproduce the results of the two-sample test of means? I get that the mean of the treatment is 1.7 (not 2.0 like in the video). This changes the conclusion, the result is not statistically significant. I think it's a mistake since my calculation for the pooled standard error (which uses the standard deviation of the treatment) matches perfectly

5:22 How do we know that the center of the CI is 0.012?

Hey Emma, Thanks for this amazing video.. Just wanted to know ... why did we picked 0.01 as practical level of significance... we could have picked any other value as well like 0.02 or 0.03 or 0.04 (any thing < 0.05 ( α - statistical level of significance)). Does practical level of significance less need to be = 0.01 ?

Hi Emma, around 2:28, where you say "Bernoulli" population, I think, to be clear, you want to make a designation. Bernoulli deals with the data which only has ONE trial and two possible outcomes, but the Binomial is the collection of Bernoulli trials for the same event (multiple trials of the same event). So since we're doing a hypothesis test on the number of successes in multiple trials, the assumption is that successes follow a binomial and not a Bernoulli, right?

Do interviewers ever ask to calculate test stats or confidence intervals? In that case can we look up or ask for the formulas?

Hi , l have 2 sets of the supermarket customer bring their own plastic bag sample. One sample is the supermarket provide free plastic bag, the other sample is the supermarket charging a fee for plastic bag. What kind of statistics test should I use ? Can I use Two -sample test of proportions ? Thanks

8:00 Case 2 T - test

Where did the practical sig boundary come from？ Rule of thumb？

Just to confirm, in the 2-sample test of means, were you doing a 2-sided test? because I saw that you looked up the t-score under the 0.975 column for alpha = 0.05.

Mean of treatment is 1.7 not 2. Then I think the second example is not significant...

did you put on a programmer plaid shirt😂

Coming from a statistics background, this is some weird notation. That aside, I think she's doing well, but the notation just throws me off. Data science is a strange mess