A/B Testing Metrics: What You Need to Know About Success, Driver, and Guardrail Metrics!

No video

A/B Testing Metrics: What You Need to Know About Success, Driver, and Guardrail Metrics!

Рет қаралды 37,213

Emma Ding

Күн бұрын

Пікірлер: 33

@emma_ding 3 жыл бұрын

Typo correction: Thanks Squiderify, at 10:38 "metics" should be "metrics".

@jackyzhang6848 3 жыл бұрын

Hello Emma, thanks for your sharing, may I ask for a question, for weighted combination of, say 3 metric, how do you decide the weight among 3 metrics? For example, the engagement of FB newsfeed, the number of likes, comments and shares, there are 3 main metrics to measuring the FB newsfeed engagement, compare to like, share and comment are more meaningful to measure the interaction on FB, but how would you decide the weight among those 3 metrics in a scientific way? Really thanks for your answer.

@abhishekchandrashukla3814 Ай бұрын

I want to cry, I understand now, how much effort goes into videos like these! I in love with you!!

@SM-gf7ux 11 ай бұрын

Emma - watching your videos helped me land a data scientist position at a FAANG company. So grateful for your knowledge and your ability to share it with a wide audience :)

@tungthanhle6797 3 жыл бұрын

Thanks, Emma for your contributions to our DS community! I have been enjoying your content so much so I even clicked on the like button before I watch any of your new videos :D. Thank you very much!

@emma_ding 3 жыл бұрын

Glad my videos are helpful!

@TrinaMoitra 6 ай бұрын

We actually use a 3rd category of guardrail metrics. Program metrics. To ensure overall quality of the experimentation program. One of the guardrails we have is ratio of iterative to exploratory tests.

@saeidsamizade6257 3 жыл бұрын

This channel is gonna be a great technical resource for professionals. Thanks 👍

@duchen1844 3 жыл бұрын

这节说的真好，微软AnE说的也不过如此。口才好，写书的Rony若是听了也要点赞。

@aymanz83 2 жыл бұрын

Great source of information, and clear explanation for tough topics

@where_is_georgio 3 жыл бұрын

You explain things so well, very helpful for those of us breaking into the data science world. Thank you Emma

@qingchen7835 2 жыл бұрын

给Emma点赞！！最近面试Product Analytics 多亏看了你的视频！！讲的又好又清晰！！因为自己没有Product经验，所以面试还挺费劲儿哈哈~ 希望自己可以早日上岸！再次谢谢Emma的视频

@yashnikhare2210 Жыл бұрын

Amazing content and often asked in technical interview. Keep it up Emma 🙌

@tianyifang6601 3 жыл бұрын

so useful and helpful!!! I learned a lot from your video! thanks so much

@omkarkhedkar1885 3 жыл бұрын

Really intuitive videos, Thanks! Keep up with the good content!!

@technicallyeasy 2 жыл бұрын

i love your channel! you should create amazon referral links for the book you mentioned in your video. that way you can get referral revenue when your followers buy that book based on your recommendation. i want to read that book now

@ranpan2861 2 жыл бұрын

Hi Emma, could you dive deeper into how to answer the question “If one metric goes up, and the other goes down, what should we do?”

@robinhood3203 2 жыл бұрын

good explanation, please keep up the good work!

@vaibhavsingh39 8 ай бұрын

Best videos

@saishastech8023 3 жыл бұрын

Good explanation of metrics 😊 thanks for sharing 👌 keep going 👍

@tinawang1291 Жыл бұрын

6:13 for the sample ratio mismatch check, I think we should only apply Chi-square test, which is to compare a proportion with an expected value. Can you help clarify how to use a T-test for checking SRM? Thanks

@asyvli1226 2 жыл бұрын

hi Emma, thanks for your sharing, could you please share more detail about “If one metric goes up, and the other goes down, what should we do?” thanks

@emma_ding 2 жыл бұрын

Fantastic observation! When metrics go a bit wild we can experience strange outcomes for all kinds of reasons worth talking about. This has given me a great idea for more content about how to avoid analysis paralysis! Thanks for sharing your comment with me!

@DataKnight 3 жыл бұрын

Thank you for sharing

@doreenshen3690 3 жыл бұрын

Hi Emma, thanks for the super clear and structured content! I am still a bit confused about what an "attributable" metric means. Could you add more clarity here? Thanks!

@DataKnight 3 жыл бұрын

Thanks for sharing

@rubaldahat4813 3 жыл бұрын

Can u shows its implementation so that it easy to remember.

@sparklingamber792 3 жыл бұрын

Hi Emma, for the sample ratio mismatch, shouldn't we use z-test rather than t-test? I read that the variance p(1-p) and mean p are not independent for proportion comparison, so it will follow a z-distribution.