For anyone confused by the pie chart, the explanation he does makes sense only when you watch the whole video. In a nutshell, when you at first have 4 servers, each server handles 25% of the users. The hashing function takes users' user id or some other information that somehow encapsulates the user data (and is consistent) so any time you want to, for example, fetch a user profile you do it via the same server over and over again since user id never changes (therefore hash of a user id never changes and will always point to the same server). The server remembers that and it creates a local cache for that information for that user so that it doesn't have to execute the (expensive) action of calculating user profile data, but instead just fetches it from the local cache quickly instead. Once your userbase becomes big enough and you require more processing power you will have to add more servers. Once you add more servers to the mix, the user distribution among server will change. Like in the example from the video, he added one server (from 4 servers to 5 servers). Each server needs to now handle 20% of the users. So here is where the explanation for the pie chart comes from. Since the first server s0 handles 25% of the users, you need to take that 5% and assign it to 2nd server s1. The first server s0 no longer serves the 5% of the users it used to, so the local cache for those users becomes invalidated (i.e. useless, so we need to fetch that information again and re-cache it on a different server that is now responsible for those users). Second server s1 now handles 25%+5%=30% of the traffic, but it needs to handle 20%. We take 10% of its users and assign it to the third server s2. Again like before, the second server s1 lost 10% of its users and with it the local cache for those users' information becomes useless. Those 10% of users become third server's users, so the third server s2 handles 25%+10%=35% of the traffic. We take third server's 15% (remember, it needs to handle only 20%) and give it to the fourth server s3. Fourth server now handles 25%+15%=40% of the traffic. Like before, fourth server lost 20% of its users (if we're unlucky and careless with re-assignment of numbers it lost ALL of its previous users and got all other servers' users instead) and therefore those 20% of users' local cache becomes useless adding to the workload of other servers. Since fourth server handles 40% of the traffic, we take 20% of its users and give it to the new fifth server s4. Now all servers handle users uniformly but the way we assigned those users is inefficient. So to remedy that, we need to look at how to perform our hashing and mapping of users better when expanding the system.

I'm a UX Designer, irrelevant for me to know this but just watching your video and sitting here willing to complete the whole series, thats how brilliant you are explaining chap.

I love how you've taken the time and effort to teach complex topics in a simple manner with real world examples. You also stress on words that are important and make analogies. This helps us students remember these topics for life! Thank you and really appreciate the effort!

This channel is a total gem. I don't think i've seen anything similar on youtube in regards to quality. Really appreciated it !

Notes to self: * Load balancing distributes requests across servers. * You can use `hash(r_id) % n_servers` to get the server index for a request `r_id`. -> Drawback: if you add an extra server `n_servers` changes and `r_id` will end up on a different server. This is bad because often we want to map requests with the same ids consistently to the same servers (there could e.g. be cached data there that we want to reuse). * "Consistent hashing" hashes with a constant denominator `M`, e.g. `hash(r_id) % M`, and then maps the resulting integer onto a server index. Each server has a range of integers that map to their index. * The pie example demonstrates, that if an extra server is added, the hashing function stays the same, and one can then change the range-to-server-index mapping slightly so that it remains likely that an `r_id` gets mapped to the same server as before the server addition.

First of all, this is a perfect lesson and I have absorbed 100% of it as a school student. the pie chart was little confusion at first cus with 4 servers it's like 25 Buckets and then you added 1 server it's pretty much 20 - 5 buckets. so divide pie to 5 and mark each one 20 buckets is the easiest way

Your way of explanation with real-life examples is really effective. I can visualize everything and remember it easily. Thanks for this.

That's an eye-opener. Have been working in industry for a few years now but never realised how small changes like this can affect the system. Thank you so much for the content!

If anyone is confused by the pie diagram, We need to reduce the distribution from 25 each to 20 each. So we take 5 from the first server and merge it with the second one. Then we take 10(5 from the first one and 5 from the second one) and merge it with third. So now, both one and two have 20 each. Then we go on taking 15 from third and merging it with fourth and finally, taking 20 from four to create the fifth server's space. Please correct me if I'm wrong. This is just a simple breakdown which I think is what he intended

You give great explanations !! And little video editing efforts make the video so interesting. Going to watch all videos uploaded by you.

seems like a 4th-year senior teaching the juniors...your videos are Beast :)

Hey man, cool explanation for all the advanced system design learners... nice! keep it coming!!

I subscribed/bookmarked your channel I don't know when I knew that I'll need it at some point in time and that time is now. Thank you for the series...❤️❤️

This was really insightful, I thought load balancing was simple but the bit about not losing your cached data was something I didn't know before.

Man, thanks! You made this easy to 100% understand. Your teaching style is excellent!

Notification Squad :-) The problem with having awesome teachers like Gaurav Sir, is that you want the same ones to teach you in college too :-) Thanks Gaurav sir for System Design series.

you made it really simple and easy to understand. I don't bother searching any other video on system design, I can simply look into your channel. Thank you very much, appreciate your efforts .

You seem very passionate about the subject. It makes it 10x better to learn that way. Thank you.

Quite a smart way of explaining the concept. Keep up the good work. Subscribed!

amazing video with extremely lucid explanations. wishing you the best, keep growing your channel. from a Civil Engineer just randomly crazy about Comp Science.

This method is ok when accessing a cache and the problems that arise are somewhat mitigated by consistent hashing. However there are two thing i want to point out 1. Caching is typically done using a distributed cache like memcached or redis and the instances should not cache too muchinformation. 2. If you want to divert requests from a particular request id then you should configure your load balancer to use sticky sesssions . The mapping between the request id and the ec2 instance can be stored in a git repo or cookies can be used etc.

Hey Gaurav, You have a knack for explaining things in a very simple manner (ELI5). There is one part of this discussion which I feel conveys some incorrect information (or I might have understood it incorrectly). You mention that 100% of requests will be impacted on addition of a new server. However, I believe that only 50% of the requests should be impacted (server 1 retains 20% of its original requests, server 2 15%, server 3 10%, and server 4 5%). In fact, it's always exactly 50% of the requests that are impacted on addition of 1 new server irrespective of the number of original servers. This turned out to be a pretty fun math problem to solve (boils down to a simple arithmetic progression problem at the end). The reason your calculation results in a value of 100% is because of double calculation: Each request is accounted for twice, once when it is removed from the original server, and then again when it is added to the new server.

I really like how you explained why we need consistent hashing.

you are amazing teacher, teaching such a complicated topic in such a efficient manner,I didn,t get even a first job , and i am able to understand

Just stumbled on your channel, and in few minutes, i have learned a lot

Done thanks Traditional hashing will make requests go to different servers if new servers are added, and ideally we want requests from the same user to hit the same server to make use of local caching

Thanks bro for sharing your knowledge.Your style of explaining is Great.

Hi Gaurav, you explained the concepts properly. keep it up. Also focus on the tools you use. use a better and darker maker. manage light reflection from the board. in this video these 2 were bit annoying. Your way of teaching is good and to the point. Good luck.

This is like the closest thing to an ELI5 on KZfaq !! Greta stuff man 😀🙌

Thanks for the concept lesson !

Thank you Gaurav, very cool! Seriously though, thanks, your videos are really helping me through college. Very epic.

Thanks for sharing. The concept of changing the cache on every server is cool.

Gaurav, your explanation is awesome . i addicted to your way of teaching . God bless you with more power :-)

Nice Explanation. Was finding a perfect playlist on YT to teach me SD. Now, its solved. Thank you for such a valuable content.

indian accent is one of accent that can make my learning process easier. It sounds clear on my ears rather native english spoken by most western tutors. Great job man

This is informative and thanks once again :-)

Thanks Gaurav for this interesting series Keep it up... : )

Helped me a lot! Thank u👏

Hey Gaurav , in the first part of the video you mention and by taking the mod operation you distribute the requests uniformly . What kind of assumptions do you make ( and why) on the hash function which insures that the requests are uniformly distributed . I could come up with a hash function which would send all the requests to say server 1

10:26 - my expression to this video. Amazing content!

Superb bro .. thanks a lot for your efforts to make videos...

Really amazing lecture. Waiting for your B+ Trees video as promised :P

Simple & clean explanation

You got a new subscriber. You are very very great

Nice video. I feel that you are meant to be a teacher :) ! You have the flair for it. Not everyone with knowledge can teach it. Do consider when you get time in life ;) ! Cheers, all the best!

Thank you for putting this out!

I'm late here but the videos are amazing. I'm going to watch the whole playlist

Love this!

Damn man, you are really good at instruction... sub’d!

Very well explained!!

On the entire KZfaq this is best❤️

You are a great teacher!

Hi Gaurav, the concept of load balance design is very good and understandable. Thanks for your effort. I have small doubt that if same number of old servers removed and added new servers to the distributed system, will this load balance have effect? or will this consistent hashing mechanism be same with no cost effective?

Gaurav your videos are amazing ! Its being a great help . Thank you so much :-)

Brilliant. thank you!

Great videos, thank you!

amazing info..thank you for sharing

Hi Gaurav great video, can you quickly elaborate on the Pie Chart you made and did the 5+5+10... maths you kinda lost me there and I am trying to figure out intuitively what you tried to show using the Pie Chart example when a new server is added. Thank you!

awesome! consider blocking the background light with something to reduce light reflection!

nice explanation of the concepts

Awesome gk. I would suggest you to more focus on system design videos as it's resources are less

You got a subscriber!!!

Awesome explanation... Waiting for middle ware admin related issues... Was tomcat weblogic

awesome video brother. keep it up!

@Gaurav Sen: I want to summarize what I understood from PI chart part, just rectify me if I understood it wrong. Directly adding another server will definitely reduce the load on each server but if not implemented properly then it could impact on the cache part. If major of the request got re-directed to different servers than our cache part will be of not that help to us. Taking small % of req load will not impact the cache part much.

The only tutorial with an Indian accent which I enjoy watching :) Thanks dude! :)

Gaurav, I just love the way you deliver the lectures. I have a query, You spoke about spending people to specific servers and have their relevant information will be stored in the cache. But won't it be a good idea if we have an independent cache server running master-slave?

That moment at 10:25.... 😂😂 I actually felt the server engineer cry for help

I wished i could have watched this video earlier so i could answer in an interview. Thanks gourav for teaching. It is quite similar to how hashmap work.. isn't it?

Dear Mr Gaurav, I am a civil engineer and currently working on one project where we are trying to add some technology in construction activity, I was struggling to understand what is system design, which is the best combination of front end and back end language, which system design should I adopt and many more things like this as I am not from IT field but I must say you made me understood so many technical thing in very layman language. Thank you so much for that

Hi Gaurav. Thanks for the informative video . Quick question. How did u got the values h(10) , h(20) and h(35) ? are these the random numbers generated from the server for each request ? Also did how did each of these request generated the hash value of 3, 15 and 12 respectively?

Hello @Gaurav Sen. Started watching this awesome series because I wanted to get some good knowledge about building large scale applications. You teach extremely well (as usual). However, I can't really wrap my head around what the pie chart at 8:16 exactly means and how it changes when we add the fifth server. It would be really awesome if you could kindly explain that here or share a link that discusses it instead. Thanks a lot! :) Edit: I finally understood it! Yusss! The issue was in the positioning of the numbers in the pie chart. I think, the initial numbering was supposed to be 0, 25, 50 and 100 instead of all 25's. Thanks again. :)

For the 2nd optimal pie chart drawn, the change is 40 right? 5 lost by each existing server, and 20 gained by new server.

X/N, where 1/N is the load factor. What exactly does this load factor represent ?

i couldnot understand the pie chart thing well, what should i read to understand this?

Hey @Gaurav! Amazing set of videos on system design for newbies like me. I would just like to know if there is any suggested order in which we've to grasp material? Like after the 8th video in this playlist, we have whatsapp system design and then what is an API, NoSQL databases and so on. So, if at all any order exists which allows to learn/grasp concepts in a better way, what order/approach would you suggest ? Thanks :)

Hi Gaurav, Awesome video . Can you please do a video on Distributed Transaction Managment

Awesome, I subscribed ...thanks

i did not understand the pie chart, breakdown of request. How the request numbers breakdown like that.

Thank you sensei !

Awesome explanation

Hey, thanks for this video. I have a question. If request IDs are randomly generated (with equal probability) why don't we just have them in a range 0..n which maps directly to servers?

Thanks. what if we associate user id to server but at a time say 80% of that server are offline then our server is underutilized

How adding server to the normal hashing technique is a big change? Are we storing these mapping somewhere of yes then what's the need to that?

Very well explained Sir. Can you please suggest any books for the same?

i didn't get the pie chart thing. What he mean by this has to take this 5 bucket at 8:40? and bucket means numbers so here that number is the number of requests that comes to the server? Can somebody please explain me ?

Hey, I had a doubt when u say that hash function is universally random, then how can one be sure that it would balance load equally?

Very well explained

Dude... great vids, you can't expect uniformity out of randomness [ it wouldn't random anymore if it was uniform).... apache, ngnix etc all look at the number of requests that each machine is processing and send figures out the one with the lowest and sends it to that

Great content! Why wouldn't the first approach work for a newly added server? Are we worried more about the cache miss? I believe if the cache has a timeout, and the frequency of adding a new server is not often. The system will be in a good performance after the old cache timeout.

@Gaurav Sen sir could you please explain that how does the server goes up to 40 buckets .......I did not understand at (8:46)

Nice topic, thank you

Amazing!

Can anyone help me explaining the first pie chart?

This is what I understand about the pie chart explanation. The objective is to reduce the measure of randomness in the distribution of request among the servers because of the caching done on each server and the fact that user request are mapped to specific servers. So you have 100% request been served to 4 servers which means the load is distributed by 25% to each server. When you scaled up and add a new server, you reduce the load on each server by 5% to make up the extra 20% needed by the new server, that way reducing the random user distribution to each server and ensuring that user data stored in the caches is relatively consistent. Correct me if I'm wrong about this, thank you.

More system design Videos

I don't understand the pie, why there is a 5 change off when a request come? can anyone explain it for me?

You are certainly an awesome teacher. It was crystal clear. Thanks.

Hey one suggestion before making new videos: you could list in the description what are the prerequisites before watching this video. Like in this video I didn't know about hashing. So I had to know about that first. So if you could list down the prerequisites we could go and get to know those terms before hand for understanding the video better.