Instagram System Design | Meta | Facebook

  Рет қаралды 31,770

ByteMonk

ByteMonk

Жыл бұрын

Interviewers frequently ask candidates to design the architecture of a social media platform like Instagram. Gain valuable knowledge about system scalability, handling high concurrent uploads, optimizing news feed generation, and implementing features like likes, comments, and user following.
Learn how Instagram's microservices architecture, distributed queues, and caching mechanisms come together to create a seamless user experience. Explore the components such as the image service, feed service, fanout service, and more that power Instagram's core functionalities. Gain insights into the scalability and performance optimizations implemented by Instagram to handle millions of users and their interactions. Whether you're a software engineer, aspiring developer, or simply curious about the inner workings of popular social media platforms, this video is a must-watch!"
System Design Interview Basics Playlist:
► • System Design Intervie...
AWS Certification:
►AWS Certified Cloud Practioner: • How to Pass AWS Certif...
►AWS Certified Solution Architect Associate: • How to Pass AWS Certif...
►AWS Certified Solution Architect Professional: • How to Pass AWS Certif...

Пікірлер: 51
@chelschubb5363
@chelschubb5363 3 ай бұрын
okay not only is this a great explanation but your presentation voice is spectacular, its so easy to listen to you
@abhishekgowlikar
@abhishekgowlikar 10 ай бұрын
Benefit to every techie. Kudos
@nomamkhan174
@nomamkhan174 11 ай бұрын
Thank you, you did a great job. It is beneficial.
@SanjayKumar-rw2gj
@SanjayKumar-rw2gj 9 ай бұрын
Excellent video with detailed information.
@nachiketkanore
@nachiketkanore 6 ай бұрын
Very well explained the high level overview!
@divyeshmali
@divyeshmali 3 ай бұрын
Really very awesome video. Thankyou so much for great explanation 🔥🔥
@randiadhityassaputra1681
@randiadhityassaputra1681 11 ай бұрын
keep gooing bro, thanks you
@saravanasai2391
@saravanasai2391 2 ай бұрын
That is a great design. How to handle a use case of user scrolling a feed. We have to track that user viewed the post/feed. So , we won't show the feed once again on his timeline. How to handle this use case.
@anilitagi3340
@anilitagi3340 8 ай бұрын
Quality content bro😊
@deepanshusuneja7399
@deepanshusuneja7399 8 ай бұрын
For new feed generation, why is the request not using the news feed cache that was asynchronously generated by the fanout service by the user's followers? Or is it an hybrid approach that you mentioned here. For users who have a lot followers the feed service queries the metadata and image service to get the feed in real time, but for the users who have few followers it, the fanout service creates a newsfeed cache?
@ClocianClokes
@ClocianClokes 8 ай бұрын
Can somebody/@ByteMonk explain the calculations? According to me Storage Capacity per month = Avg Image size * Image uploaded per month by 1 user * MAU (10% of 10^9 registered users ) = 0.005 GB * 500 * 10^8 = 250 PB Network Capacity = Total data transferred/Time taken = Concurrent Uploads * Avg Image size/avg image upload time = 100000 * 0.005GB/5 = 100Gbps
@aworden
@aworden 8 ай бұрын
Also, 1 GBps == 8 Gbps
@evalyly9313
@evalyly9313 20 күн бұрын
My question is do we really. need so many microservices for like, comment, feed, etc? Or one service with multiple endpoints/API? What are the pros and cons?
@abdulhye
@abdulhye 4 ай бұрын
Kindly explain your capacity planning, how did you reach those numbers, you just say the numbers and move on. Explain the formula and how you reached the number, as it is not clear, our Capacity planning numbers is not matching yours
@ankitpersie1
@ankitpersie1 Ай бұрын
In the Schema design represented as relational, the design is not normalised. We already have a relation between user and post in post table. You could have used the id of this table as foreign key in comments and like table.
@uzmaahmed9031
@uzmaahmed9031 Ай бұрын
for upload, I think endpoint should be done through user, something like this: POST user/{userId}/Image
@Giullianosiviero
@Giullianosiviero Ай бұрын
Instead of creating the Feed on the fly, wouldn´t it be less "stressful" to the DB and Servers to use a NoSQL DB to store a "pre-bulit" Feed? Meaning when a post/coment/like is created a worker updates a feed object in a NoSQL db? This way, fetching the feed would be just accessing one object instead of "Joining" massive tables, what do you think of this approach?
@sabyasachimohapatra1340
@sabyasachimohapatra1340 3 ай бұрын
Sorry. This may be a question out-of-context. Just curious to know what tools you use for designs and diagrams
@Techie-pk2yd
@Techie-pk2yd 4 ай бұрын
there is a problem with user feed design. You mentioned that service first fetches the followers list from metadata service and then gets posts corresponding to each follower. Imagine if a social media addict following 100K users, do u fetch 100K users and all their posts, feed service will take forever to load.
@XdoughboyxX
@XdoughboyxX 2 ай бұрын
Great video. I’m delivering a mobile app product to the Austin, Texas market that’ll be supported by a similar framework, with an additional layer of ML to encourage in person exploration based on relational account interests
@ByteMonk
@ByteMonk 2 ай бұрын
That is awesome!
@XdoughboyxX
@XdoughboyxX 2 ай бұрын
@@ByteMonk where are you located?
@XdoughboyxX
@XdoughboyxX 2 ай бұрын
Any startup experience? Funding, technical oversight?
@mz7640
@mz7640 9 ай бұрын
The API box is a web server ?
@ByteMonk
@ByteMonk 9 ай бұрын
Web servers are a common platform for hosting APIs, but there are other options available. For example, APIs can be hosted in cloud platforms, such as AWS or Microsoft Azure.
@mz7640
@mz7640 9 ай бұрын
@@ByteMonk this means API gateway will forward request to web server (Apache running on ec2). Which will forward request to microservice running on Lambda?
@tommyls4357
@tommyls4357 8 ай бұрын
@@mz7640 I think you would typically have the API gateway forward the request to the microservice directly. Each microservice will be running an HTTP server to handle incoming HTTP requests.
@amarshukla
@amarshukla 7 ай бұрын
@@mz7640 your assumptions are as per the monolithic architecture but Instagram example here is built using Micro Service Architecture which does not follow traditional webserver-Appserver interaction approach.
@SAURABHGUPTA_DINOSAUR
@SAURABHGUPTA_DINOSAUR 9 ай бұрын
tbh, I thought other than meta employees or big tech people who're into this stuff, no common people know these things. Because this is legitimately complex. Obv they know stuffs we don't hence they're the leaders of the world and we're the followers.
@ByteMonk
@ByteMonk 9 ай бұрын
Every expert was once a beginner. Knowledge is accessible to anyone willing to learn and grow, so don't underestimate your potential - with dedication and effort, We all have the ability to be leaders of our own journey :)
@RelaxingMusic-iz1bl
@RelaxingMusic-iz1bl 6 ай бұрын
​@@ByteMonkdamn thanks for this❤
@dontdoit6986
@dontdoit6986 2 ай бұрын
Tbh, a lot of tech people treat things as far more complicated than they need to be and other things as far simpler than they are. This video is detailed, but it is also high level in some places. He is showing you a method of how things COULD be implemented. It’s a jumping off point, an inspiration.
@ThatsJoshTyler
@ThatsJoshTyler 28 күн бұрын
The actual stuff here is simple, but its not easy to complete. I think thats where the confusion comes. Simple does not equal easy. Ex. Push ups are simple, but doing 200 push ups is hard. Take it a step at a time and as you go, youll realize you know more than you thought
@SAURABHGUPTA_DINOSAUR
@SAURABHGUPTA_DINOSAUR 28 күн бұрын
@@ThatsJoshTyler Thankyou for your brief insight and pov on the subject matter being discussed here. I gotcha
@4uT3p
@4uT3p 10 ай бұрын
loading feed in realtime? I think it is a bottleneck.
@ByteMonk
@ByteMonk 10 ай бұрын
Excellent observation. This is going to be a long answer and a good conversation topic during the interview. While real-time loading offers a dynamic and engaging user experience, it's important to carefully consider the trade-offs and design choices to ensure optimal performance and scalability. Advantages of Real-time Loading: Dynamic Content: Real-time loading allows users to see the latest updates and activities as they occur, creating a more engaging and interactive experience. Instant Feedback: Users receive immediate feedback on their actions, such as likes and comments, enhancing their engagement with the platform. User Engagement: Real-time feeds can increase user engagement and retention, as users are more likely to stay active and interact with the app. Challenges and Bottlenecks: Scalability: Real-time systems need to handle a high volume of concurrent users and activities, which can strain the infrastructure and impact performance. Latency: Ensuring low-latency delivery of real-time updates can be challenging, especially as the user base grows. Network Traffic: Frequent updates can generate significant network traffic, affecting the app's responsiveness and consuming bandwidth. Mitigating Challenges: Caching: Implement caching mechanisms to reduce the load on the backend and minimize redundant requests. Distributed Architecture: Employ a distributed architecture to distribute the load and scale horizontally as the user base grows. Batch Processing: Aggregate and process updates in batches to minimize the number of real-time requests. Content Prioritization: Prioritize important updates and ensure that less critical updates are delivered in a less resource-intensive manner. Ultimately, the decision to implement real-time loading depends on the specific requirements of your application, the user expectations, and the technical capabilities of your infrastructure. It's essential to conduct thorough performance testing, monitor system health, and continuously optimize the real-time feed delivery to strike the right balance between engagement and performance
@frankcallahan3631
@frankcallahan3631 9 ай бұрын
This looks like a chat GPT reply - I’m not saying it’s wrong to use GPT to reply but it’s also less genuine than hearing the authors original thoughts.
@ByteMonk
@ByteMonk 9 ай бұрын
@@frankcallahan3631 In our experience chatGPT and other tools hallucinate a lot, and produces too much information. However, when needed, we prompt them with our thoughts to get feedback and use some of the text it generates, but always proof read and refine them. Thanks for your feedback, will strive to be as genuine and raw as possible in future.
@frankcallahan3631
@frankcallahan3631 9 ай бұрын
@@ByteMonk I loved the video by the way.
@frankcallahan3631
@frankcallahan3631 9 ай бұрын
And I subscribed
@AbhiKumar-kq2kx
@AbhiKumar-kq2kx 8 ай бұрын
you read very fast. Very difficult to understand your voice and fit into mind. Please take some more time and try to teach slow
@ByteMonk
@ByteMonk 8 ай бұрын
Thanks for the feedback, I try to balance speed for each video. For this video, can you also check out playback speed KZfaq provides and try playing it at 0.75%
@worldwide6626
@worldwide6626 8 ай бұрын
It’s fine for me.
@electrolyteorb
@electrolyteorb Жыл бұрын
These are very very "Interview"-heavy videos... In a way you cannot really clone it with something like Node.js + Postgres + AWS (any hosting).... And HOW ON EARTH DO YOU IMPLEMENT THE BLOCKING SYSTEM?
@ByteMonk
@ByteMonk Жыл бұрын
Yeah, these videos are made from interview perspective. I imagine Implementing a blocking system will involve handling database operations and building the necessary API endpoints to enable users to block and unblock others.
@aulin_ahsan
@aulin_ahsan Жыл бұрын
@@ByteMonk I think block is more complicated than just blocking and unblocking.. Suppose you have blocked an user and that user commented in one of your friends post. How do you prepare custom post metadata that doesn’t include the blocked user activities. ?
@ByteMonk
@ByteMonk Жыл бұрын
Great point, this could be a follow up question/discussion during an interview or a separate system design question altogether. To prepare custom post metadata that doesn't include the activities of blocked users, you can follow these steps: 1. Retrieve Post Metadata: Fetch the metadata of the post that you want to display, including the comments and activities associated with it. 2. Filter Blocked Users: Iterate through the comments or activities and identify any users who have been blocked by the viewer. Check against the blocked user list to determine if the comment or activity was generated by a blocked user. 3. Exclude Blocked User Activities: Remove any comments or activities generated by blocked users from the post metadata. This can be done by either excluding them from the response or flagging them as blocked. 4. Prepare Custom Metadata: Create a modified version of the post metadata that excludes the activities of blocked users. This may involve creating a new object or modifying the existing one to remove the blocked user information. 5. Return Custom Metadata: Return the custom post metadata to the client application for display. Ensure that the modified metadata only contains the activities of non-blocked users, providing a customized view that respects the blocking preferences of the viewer. By filtering out the activities of blocked users during the preparation of post metadata, you can create a custom view that doesn't include any content or actions from blocked users. This ensures that the viewer's experience is tailored to their preferences, providing a more personalized and relevant feed without unwanted interactions from blocked individuals.
@J3R3MI6
@J3R3MI6 8 ай бұрын
@@ByteMonkso helpful! Thank you!
Distributed Caching for System Design Interviews
7:04
ByteMonk
Рет қаралды 8 М.
Каха ограбил банк
01:00
К-Media
Рет қаралды 10 МЛН
Khóa ly biệt
01:00
Đào Nguyễn Ánh - Hữu Hưng
Рет қаралды 21 МЛН
DO YOU HAVE FRIENDS LIKE THIS?
00:17
dednahype
Рет қаралды 36 МЛН
Master Youtube System Design
17:46
ByteMonk
Рет қаралды 11 М.
Meta system design interview: Design Instagram (with ex-Meta data engineer)
1:01:48
IGotAnOffer: Engineering
Рет қаралды 5 М.
Twitter system design | twitter Software architecture | twitter interview questions
36:56
System Design: TINDER as a microservice architecture
36:41
Gaurav Sen
Рет қаралды 1,2 МЛН
Web Crawler System Design Concepts Nobody Talks About
21:42
Pratiksha Bakrola
Рет қаралды 4,4 М.
How Discord Stores TRILLIONS of Messages
13:06
Coding with Lewis
Рет қаралды 654 М.
How WebHook works | System Design
7:43
ByteMonk
Рет қаралды 71 М.