Twitter System Design on AWS | Microsoft, Google, Facebook Whiteboarding Interview Question

Рет қаралды 21,106

Күн бұрын

Designing Twitter is a common system design interview question. In this video, we will cover features like tweeting, retweeting, timeline, trending. This design needs to be scalable, highly available, and secure.
In addition to the design, this whiteboard architecture is also implemented using AWS services.
VP of Engineering at Twitter Design Talk: www.infoq.com/presentations/T...
Please Subscribe
▶ / @cloudwithraj
*Connect with me*
🤳 Instagram: / cloudwithraj
🏢 LinkedIN: / rajdeep-sa-at-aws
🐦Twitter: / cloudwithraj
🖼 Facebook: / cloudwithraj
👩🎓**My Courses - Comment in video for discount coupons**
Discounted Link for my Kubernetes on EKS course: www.udemy.com/course/rocking-...
Discounted Link for my Serverless Course: www.udemy.com/course/aws-serv...
Discounted Link for my CloudFormation, CDK with Devops, Interview Guide Course: www.udemy.com/course/rocking-...
Timestamps:
0:00 Requirements and features
2:06 Table design
5:36 Twitter timeline design
8:53 Tweet, retweet design
12:34 Database selection
17:21 Edge case and design challenge
19:24 Trending feature design
21:34 Security
#SystemDesignInterview #WhiteboardInterview #AWSArchitecture

Пікірлер: 64

@cloudwithraj 2 жыл бұрын

My Rocking Systems Design course is out now in Udemy! Limited time discounted link to get the course for $9.99 - www.udemy.com/course/rocking-system-design/?couponCode=LEARNSYSTEMDESIGN

@muratcosar6303 2 жыл бұрын

I cant watch it in corporate side of the udemy. is it not available to companies?

@TheElan5 2 жыл бұрын

Raj, your videos are incredibly helpful. You are masterful at breaking down complex technical requirements into readily understood concepts. I’ve learned a ton from just a few videos. Thank you for all of your help!

@mattborisov 2 жыл бұрын

Amazing episode! Thank you.

@samhunk1227 3 жыл бұрын

Thanks ! Excellent understanding .

@ronnmv 3 жыл бұрын

Thanks Raj!! Let it flow....it's amazing how you turn this complex topics in to simple & understandable

@cloudwithraj 3 жыл бұрын

Thanks Rony for the kind words 🙏

@kashmorakash4396 3 жыл бұрын

Explanation is precise and simple. Please continue with such videos. Thanks for your effor

@cloudwithraj 3 жыл бұрын

Thank you Kashmora

@nilaindeethin1803 Жыл бұрын

Raj, this is super simple. Have also bought your course.

@goodnightmr5892 2 жыл бұрын

Excellent presentation!

@AliTwaij Жыл бұрын

Excellent thankyou

@ejbjms Жыл бұрын

Great videos from you ! I am learning a lot of AWS !! Thanks a lot

@cantbefooled8044 Жыл бұрын

youre like the only channel that makes information so digestible i really appreciate it

@cloudwithraj Жыл бұрын

I appreciate that!

@amitparks 3 жыл бұрын

Your biggest fan..keep sharing the knowledge 👍 ...More system design please 😁

@cloudwithraj 3 жыл бұрын

More to come!

@babusivaprakasam9846 2 жыл бұрын

Brilliant Video. Many Thanks Raj

@cloudwithraj 2 жыл бұрын

Glad you liked it!

@machireddyshyamsunder987 3 жыл бұрын

Nicely explained Raj , Thank you so much

@cloudwithraj 3 жыл бұрын

Thanks Shyam for your support as always 🙏

@zubyranks80 2 жыл бұрын

Perfect as usual!

@cloudwithraj 2 жыл бұрын

Thank you! Cheers!

@idabagusdiazagasatya9900 11 ай бұрын

AMAZING

@ambikaprasad9464 2 жыл бұрын

Appreciate the great knowledge sharing course.

@cloudwithraj 2 жыл бұрын

Glad it was helpful!

@nah0221 3 жыл бұрын

brilliant as usual .. Thank You!

@cloudwithraj 3 жыл бұрын

Thanks again!

@agmos01 2 жыл бұрын

Great video! TY!

@cloudwithraj 2 жыл бұрын

Glad you enjoyed it!

@Neelu2023 3 жыл бұрын

Thank you. This is very informative. Please do create more such videos. I am great fan of your videos

@cloudwithraj 3 жыл бұрын

Thank you, I will Mahesh

@Moccar Жыл бұрын

Pretty funny you chose Elon Musk as the famous person way prior to him buying the company. Great video btw!

@cloudwithraj Жыл бұрын

Thanks Moccar!

@totsubo2000 2 жыл бұрын

Great video! When explaining when to generate timelines: "It will only create the timelines for the users who have been active in the last 10 hours". I'm assuming that this reduces the compute strain? But what happens when everyone on the US East Coast wakes up, everyone starts logging and looking at their Twitter feed at roughly the same time? The system is busy creating timelines for millions of users at the same time. How does this solution cope with such a situation? Some content suggestion/request for future videos - Compare/contrasts different way to implement a problems using different AWS technology. So for example so how you would implement something using EC2 vs. using Fargate vs. Lambda and contrast the trade-offs. Same for using Redis vs. Elasticache vs DAX, or using SNS vs. SQS, etc...

@abhi1ertu 6 ай бұрын

Nice video

@sharathsachin001 3 жыл бұрын

Hey Raj Thanks for the awesome video... Smashed the like button as you said😃😃👍

@cloudwithraj 3 жыл бұрын

Thanks Sharath for your support as always 🙏

@tindisommers3313 2 жыл бұрын

Hello Raj, thank you for the video. It was very informative. I am curious though. Is Twitter really using a kind of table as you described for the relationships; where a user's name appears in multiple rows to link that user to his or her followers? I had always assumed some kind of graph database based on what I read on data structures and algorithms.

@indelible.M. 2 жыл бұрын

You are amazing

@cloudwithraj 2 жыл бұрын

Ty for the kind words Mesanga

@lguevarae 3 жыл бұрын

Hi Raj! excellent video! I have a question about the EC2 and ASG part, what if instead of EC2 we use Step Functions with Lambda and EventBridge? could be an option? I mean having a Step Functions workflow that inserts the tweet on Redis and then to Kinesis. And also, could Redis be replaced with Dynamodb?

@totsubo2000 2 жыл бұрын

Hi Luis, Raj mentions in the video that Redis can be replaced by DynamoDB and if the system needs even better performance than plain DDB, you can use DynamoDB Accelerator (DAX)

@resham1729 2 жыл бұрын

Like ur shirt bru👊

@cloudwithraj 2 жыл бұрын

Ty bro!

@megabenman 2 жыл бұрын

How would the schema look like if you were to use DynamoDB? The keys need to be unique, so you could not have three rows of Tina unless you had some other partition key. Would you have some random-ish hashkey as the partition key and then have the Tina column be a GSI?

@cloudwithraj 2 жыл бұрын

Another great comment megabenman. Yes, the table needs to have both partition and sort key to make each record unique.

@mangeshxjoshi 3 жыл бұрын

Thank you , excellent video , very well explained , in simple language . really appreciated your video can we have some video on AWS Glue ETL tool ?

@cloudwithraj 3 жыл бұрын

Thanks Mangesh, yes planning on doing an AWS Analytics video in near future!

@ahsanali-gm4ci Жыл бұрын

Raj thanks a lot for this video. As far as group messaging is concerned, we will use notification service with one queue. As for grouping message is same for every member so our message will pass by SQS and after SQS we will use a notification to every single group member. Is it right please suggest? Rest of the design would be the same.

@techsolutions123 3 жыл бұрын

For mobile app deployment will need separate two builds ? One for apple other android app store right?

@mahmoudsakr3136 3 жыл бұрын

Hi Raj, great videos, please continue , I like your content, but question: is iy sql or nosql DB?

@cloudwithraj 3 жыл бұрын

Hey Mahmoud, Redis is considered a NoSQL database, however it has limited capabilities to create keys and query using the keys. The primary reason to use Redis is to have really fast read/write ability.

@hemanthgadarla86 10 ай бұрын

We can use any graph db to store followers relationship if not why?

@megabenman 2 жыл бұрын

Would API Gateway scale enough to handle twitter-esque scale, or would you need ALB/NLB? The AWS Quotas docs state that AWS Gateway is limited by default to 10,000 RPS, but it can be increased upon request. Still, even at a 10x increase, I'm not sure 100k RPS would be enough.

@cloudwithraj 2 жыл бұрын

Great question megabenman! The limits are based on account/region. Beyond raising the limits, you also segregate the APIs on different region/API based on user location. Also CloudFront comes into play to cache certain contents without hitting the backend API. Generally it's a mix of ALB and API Gateway for different APIs. ALBs you can prewarm to handle high spokes for certain events (New Year), but it does cost you more.

@codewithmubin8866 2 жыл бұрын

Can you please make a video on Bandwidth pricing of AWS and other cloud providers?

@Udayanverma 2 жыл бұрын

Question: kms gets overloaded with high number of api calls, they cached it with S3 but how does it happen with other services. What is the throughput and how kms cops up with api calls

@amolwadekar2554 2 жыл бұрын

Why do we do not use EKS or Kubernetes in EC2 before Database rather than use Autoscaling and EC2? It will reduce Cost of Autoscaling and invocation of EC2?