How Hotstar Application Scaled 25 Million Concurrent Users | Performance Testing

How Hotstar Application Scaled 25 Million Concurrent Users | Performance Testing | Load Testing

Рет қаралды 173,673

2 жыл бұрын

Credit : ‪@HasGeekTV‬
The video is uploaded here for educational purpose
👍 Like us on Facebook : / perfology
👍 Add us on Instagram : / perfologys
👍 Follow us on Linkedin: / perfology
Speaker bio
Gaurav is a Cloud Architect at Hotstar, India’s leading OTT provider that holds a global concurrency record for live streaming to 25Mn+ viewers. At Hotstar, he loves building cost-efficient infrastructure that can scale to millions in minutes. He is also passionate about chaos engineering and cloud security.
The talk will focus on why traditional autoscaling doesn’t work for Hotstar (Disney’s OTT streaming service) who recently created a global record for live streaming to 25.3 million concurrent viewers. It will talk about challenges in scaling infrastructure for millions and how to overcome it, how we run gamedays before actual games, how we use load testing monster called Hulk to prepare for peak traffic and also how we add chaos (engineering) in between all these epic scale madness.
1. Outline
2. About Hotstar
3. Why traditional autoscaling doesn’t work for Hotstar
4. Challenges in scaling
5. Gamedays and load testing for millions
6. How we prepare for peak traffic (50M)
7. Chaos engineering at high level
Slides
docs.google.com/presentation/...

Пікірлер: 127

@reventon357 8 ай бұрын

Here after 54 million in IND v NZ Semi-Final of 2023, crazy to think that the new record is DOUBLE than this!

@vamsivishalallu1931 8 ай бұрын

54 Million

@akshaysingla951 8 ай бұрын

58 million IND VS AUS world cup 2023 final

@sourabbhowmik5706 8 ай бұрын

@@akshaysingla951touched 59M

@lfe360-sagarsvlogs3 8 ай бұрын

Thats 54 million

@reventon357 8 ай бұрын

Yup, my bad@@lfe360-sagarsvlogs3

@theoutlet9300 Ай бұрын

wow those numbers are insane. I would love to see a cost graph during the match as well

@sahiljain5979 Жыл бұрын

streaming services: oh no.. , spikes are nasty,, cable: hold my beer...

@SomebodyYouUsedToSee Ай бұрын

Technically cable loads are predictable and it’s broadcasting .So, thing is a lot different

@electrolyteorb Ай бұрын

Cable technology is truly impressive... He is showing spikes in the video... For cables it's just another Tuesday

@nikhilgv9 Ай бұрын

Cable technology is a broadcast even without any acknowledgement from the receiver. That is exactly is what is preventing anyone from rewinding 5 minutes to see a part they missed. For streaming services it is like every user is having their own channel.

@vpstateofmind Ай бұрын

@@nikhilgv9 so is it a possibility that the system will crash if half of the viewers suddenly decides to rewind?

@Z3U5.0g 12 күн бұрын

@@nikhilgv9You are talking about the analogue era, nowadays due to digitisation cable has become IPTV.

@vikasrai9244 2 жыл бұрын

Beautifully explanaed Gaurav, Thanks

@panditvenkatesh 2 жыл бұрын

Good info for Perf test - enthusiasts , well explained

@vijayrk7329 Ай бұрын

Amazing Man. Hats off Gourav and team!

@abhishekjain5150 Ай бұрын

And now jio hired all hotstart employee to build juocinema 😅 and now jio is also able to handle that load. Good video❤

@rajeswaril3931 Ай бұрын

and handled of about 500M views during CSK vs RCB playoff selection 2024

@sarcaastech Ай бұрын

Jio cinema shows cumulative views not concurrent views

@priyasingh-qv5pk 2 жыл бұрын

Well done Hotstar Team👏

@kapilpatel9379 Ай бұрын

Fantastic talk, would love to learn more

@sakshisingh-ue4nr 2 жыл бұрын

Amazing Work

@victarpankchu8969 2 жыл бұрын

Thank you for this video!!!!

@eswarkada6173 8 ай бұрын

Assembled all AWS Cloud & DevOps Engineers 😂😂

@JohnSnow-gi7iv 8 ай бұрын

Watching this after 58M concurrent users in IND vs AUS final.

@JZL003 Ай бұрын

pretty fascinating that if service is degraded they'll just allow anyone to watch the video without logging in or paying. Very nice of them and makes sense, for live sports matches, better to let everyone in than deal with all the headlines of "match went down 5 minutes before game ended"

@abridgetool 2 жыл бұрын

After watching this video, I am like "Asli kaam to ye log kar rahe hain, hum to bus apni... "

@harshjain8345 11 ай бұрын

True 😂

@ExploreSciFi Ай бұрын

Today, Pak vs India match in NewYork was Streaming on Tamasha App and there were 69+ millions viewers 😅

@Rastogi-yt Ай бұрын

Too helpful🎉

@Indian-JaiHindusthan Жыл бұрын

Awesome.. what is underlined load testing tool used to orchestrate?

@rubanantony9028 2 жыл бұрын

Superb

@kuldeeppilaji Ай бұрын

Excellent engineering❤🎉🎉

@sathish43 2 ай бұрын

Very useful ❤

@ksb4899 Ай бұрын

I selected this video very eagerly on how to learn, after knowing it is the same game that gave me traumatic vibes

@eyalsaadon3996 2 жыл бұрын

Kudos

@lakshminarayanans5261 Жыл бұрын

Awesome

@Prince-qe1do 15 күн бұрын

This is real Engineering.

@shraddhasharma7278 Жыл бұрын

@perfology how push notifications get to know about upcoming information?

@immaculateness 21 күн бұрын

Great talk 🗣️

@ankitbhatti667 Ай бұрын

Thanks for sharing

@indianmemeswaala7653 Жыл бұрын

Jio cinema handled 2.2 cr (22 million) concurrent traffic for a normal ipl 2023 league match of CSK vs RR Wonder what will be the live numbers in IPL 2023 final

@karinweinbach5030 2 жыл бұрын

Project HULK !!

@ashishchoksi8501 8 ай бұрын

How you guys handle the fanout problem of push notification to 120M users?

@Aloktiwari-sf2im Ай бұрын

People directly close the tab most of the time in such cases.

@bhagyalakshmi1053 Жыл бұрын

How to payment system in the login in background in the concreting details

@abhishekvanenooru4959 10 күн бұрын

super anna

@darkwhite2247 8 ай бұрын

why hotstar didn't use private cloud in aws? why use public cloud?

@nandagopal.05 3 ай бұрын

Just some thoughts, For a minimum base scale i guess its best to use bare metal server without any other things like virtualizattion, logging etc to get raw performance and to use private cloud instead of public to have higher bandwidth. Never worked on this much high scale but just my thoughts. Also one thing i have noticed till date is hotstars marketing notifications are way delayed then the actual event. Not sure if this is decision making delay or tech delay. Nevertheless its insane engineering ❤

@v.rajasekar5044 Ай бұрын

actually, this is the perfect use case for public cloud. because, this kinda peak is rare and doesn't make sense to acquire CPUs just for this one occasion and let it be idle for almost 99% of the year and even then you never know what will be the actual peak. To give a comparison this year csk vs rcb had 500 million active users at one point and it was almost 10x than previous record viewers in IPL. If you go to public cloud, you can use multiple regions and multiple availability zones and multiple instance types and even multiple clouds too. this is why even Netflix still now uses AWS.

@devbiswarupz Ай бұрын

i am a beginner, but i have a doubt, if we build any system in micro service arc, how we set a value to start a new service to avoid the booting time gap?

@omkarachary 10 ай бұрын

Great job hotstar team . Kudos from a Cleartax Architect (60million scale).

@krutikpatel906 Ай бұрын

how is it a big deal to spin off more nodes on cloud?

@bhagyalakshmi1053 Жыл бұрын

What this cabinets?

@joyoshish 8 ай бұрын

Insane work Hotstar!

@bhagyalakshmi1053 Жыл бұрын

I want ops hotstar suppliers. What this can correct.

@mighty_rudhra5603 8 ай бұрын

Watching again after the Ind vs Nz world cup-23

@sudhanshuprasad2666 Ай бұрын

37:12 "bypass the payment"

@ShardulPrabhu Ай бұрын

Just suggestion stream your data 15s later so you can take advantage of auto scaling 😛😛 because you know now there will be spike

@neels6314 2 жыл бұрын

👍

@abuzardev1568 Ай бұрын

Dhoni for a reason ❤

@nishwalshetty3432 Жыл бұрын

wonder if this team of hotstar has moved to jio cimena

@nandagopal.05 3 ай бұрын

At least their CTO has moved to jio cinema 😅

@python3.x Ай бұрын

There is simple, add hpa, set threshold, spin more instances, spend thousands of dollars

@y0geshpatil 11 ай бұрын

What is C59 class machine ?

@darkwhite2247 9 ай бұрын

did you find the answer?

@coding3438 8 ай бұрын

C59x. It’s an instance type of EC2. EC2 is amazon web service’s virtual machine offering.

@WholeBits 8 ай бұрын

I expect 70-100 Million Simultaneous Viewers at Peak for India-Australia 2023 Cricket World Cup Final. All The Best To The Entire Disney Hotstar Team. ✌🏽😊

@darkwhite2247 8 ай бұрын

its 40 to 50 million i think

@LakshmiNarayanaVelayudam Ай бұрын

thala for a reason

@navajyothmp1871 9 ай бұрын

This world cup india newzealand match 43 million 🔥🔥

@navajyothmp1871 8 ай бұрын

53 million india newzealand semi final 🔥🔥

@nani3209 Ай бұрын

I was always wonder, how apolo 11 launch was watched by 650 million people.

@vinodt.h4914 Ай бұрын

I have one doubt. In this video they mentioned that they are not using auto scaling of AWS. But they wrote their own autoscaling tool to achieve autoscaling. Am i correct? and if it is correct understanding means what is the difference between both?

@rohanbhatia8406 Ай бұрын

For autoscaling you can use default autoscaling feature provided by AWS but as he mentioned, it is not reliable hence they had to come up with their own autoscaling, Hope that makes sense

@SunnyRoy-iq5oy Ай бұрын

What about 49.9 cr (499 million) during CSK vs RCB in 2024 IPL on Jio chinema

@Hector_don Ай бұрын

Views and concurrent viewers me difference hota ha. He is talking about handling 20+ mill users at a time.

@jaiho4237 Ай бұрын

In this main player are cloud and devops engineer

@darkcoder7007 Ай бұрын

bro is challanging straight to AWS... i mean you need to redefine & rethink everything with such a huge scale.

@newbie8051 Ай бұрын

Woah, 216TB RAM

@akashagarwal6390 Жыл бұрын

RPS vs TPS?

@kunalverma9325 10 ай бұрын

RPS - Requests per second TPS - Transactions per second

@digitalshankar007 Жыл бұрын

Do it can handle my website which has 100 visitors per month?

@freindimania11 9 ай бұрын

Only if they can anticipate the traffic and have ec2 instances warmed up😂

@javaremastered1530 Жыл бұрын

but now next ipl to jio

@re1konn 2 ай бұрын

4:14

@AgrimGupta Ай бұрын

Which match was this?

@satyasaineelapala570 Ай бұрын

2019 sf

@AgrimGupta Ай бұрын

@@satyasaineelapala570 ah ok, got it.

@abhisatpute-bb1wx Ай бұрын

we want to know about jio cinemo Application Scaled 2000 Million Concurrent Users | Performance Testing | Load Testing

@ShubhamPatil-di1nx Ай бұрын

Thanks to Harkirat found this video.

@B-Billy Ай бұрын

plot twist: Hotstar and IPL francise giving alot of money to Dhoni to not retire. 😆

@digitalshankar007 Жыл бұрын

Do hostingraja server can handle hotstar

@electrolyteorb Ай бұрын

Depends™️

@uvnok Ай бұрын

Bhai ke pass macbook why techies prefer why not gaming laptop rbg se aag laga dunga Just asking

@harshitpant07 2 ай бұрын

In IPL 2024 Jio Cinema recorded a huge 590 million viewership on opening day which is just mind boggling when compared here with 25 million.

@mdrehanulhaque8604 2 ай бұрын

Views and live views, both are different

@mohitranka9840 2 ай бұрын

That is not concurrent users. Its just views (eg. page load), which is meaningless.

@aryanruia3491 Ай бұрын

Jio hired most of their engineers from hotstar only

@bhagyalakshmi1053 Жыл бұрын

Edureka how much in the days is a continue my coaching gal I will friend for you I send in for your so money timing you have in receiving is a never to explaining in the total videos so simple tips for you cheap to take up for you my education my systems is my mind to attending after again reject I can do for you lot of tenses of only demo section them of election today more section 211 to change if are you demo section in the head my head is coming your head is pal so many times also here coming up

@BigDataLogin Ай бұрын

C00L

@bhagyalakshmi1053 Жыл бұрын

Buffer fail in the for buffer using for introduction essay in storage information in buffer file and concrete information in the handling this video?edu

@freindimania11 9 ай бұрын

Yes

@gauravchauhan1645 Ай бұрын

Nothing technical...

@rizvanhawaldar Ай бұрын

1. How great we are 2. How great we became by ignoring issues 3. I don't care about people feedback 4. You guys just watch theory in PPT which you can read from any books 5. Read from books 6. Tum logo se nahi hoga, only we can do 7. Please clap 👏 and share this video for marketing purposes.

@electrolyteorb Ай бұрын

Real... He doesn't even go into the technical details

@user-kk5cv1rs5r Ай бұрын

what accent is this

@rajandeepsingh8277 2 ай бұрын

jio cinema handling load of 250+ million concurrent users and not showing off.

@mystica7284 2 ай бұрын

I see it as more educational than a brag. I wish more companies were transparent and share their learnings.

@blasttrash 2 ай бұрын

ipl never had 250M concurrent users. They are total views or something. No way ipl gets that kind of traffic. Only world cups or ind vs pak gets it

@HelmutZemo1978 Ай бұрын

Wrong

@ashishira6095 Ай бұрын

have you served 1 people let alone 15 million people?

@humanthkumar Ай бұрын

jio cinema is too slow

@stockbro1997 2 ай бұрын

Just having ipl and cricket rights was the reason for these metrics. Whatever bullshit this guy said is all irrelevant. At the end of the day if you have the rights for cricket in India all other things are stupidly irrelevant because ur ad revenue is constant for the entire 2 innings. I don’t understand why hotstar invested and burnt money in all these bullshit metrics.