In this video we solve the thundering herd problem. This problem occurs when there are a huge number of requests on the server, and this results in the server crashing due to overloading. One of the solutions to this problem is rate limiting. In a distributed environment, the rate limiting problem is a complex problem to solve.
When designing systems on the server side, we often need to predict the capacity of a server and apply limits on the number of requests it can receive per second. This is mentioned in the service QPS document. QPS stands for queries per second.
The system design discussion helps us understand how to deal with requests based on priorities, how to deal with cascading failures, how to handle a large number of requests, viral posts or videos and their requirements.
Some other scenarios like job scheduling or cron jobs being fired in batches is also discussed. We use batch processing and approximations to reduce server load.
The last few approaches include gradual deployments and using caches to store common requests information. This helps us improve performance. Improving QPS and performance helps us handle more requests, which means more users and more money for the product.
Caching and coupling systems can also help improve performance. However, they must have time outs and appropriate cache eviction policies set while designing the system.
Request throttling is one of the many approaches we discuss, but it is the most important. Dropping requests helps the server recover and operate at the right requirement level.
We often use message queues for rate limiting requests.
References:
highscalability.com/blog/2012/...
blog.ably.io/how-adopting-a-d...
www.researchgate.net/post/How...
Social Links:
Facebook: / gkcs0
Quora: www.quora.com/profile/Gaurav-...
LinkedIn: / gaurav-sen-56b6a941