What are Distributed CACHES and how do they manage DATA CONSISTENCY?

Рет қаралды 978,839

Күн бұрын

Caching in distributed systems is an important aspect for designing scalable systems. We first discuss what is a cache and why we use it. We then talk about what are the key features of a cache in a distributed system.
The cache management policies of LRU and Sliding Window are mentioned here. For high performance, the cache eviction policy must be chosen carefully. To keep data consistent and memory footprint low, we must choose a write-through or write-back consistency policy.
Cache management is important because of its relation to cache hit ratios and performance. We talk about various scenarios in a distributed environment.
System Design Video Course:
interviewready.io
00:00 Who should watch this video?
00:18 What is a cache?
02:14 Why not store everything in a cache?
03:00 Cache Policies
04:49 Cache Evictions and Thrashing
05:52 Consistency Problems
06:32 Local Caches
07:49 Global Caches
08:56 Where should you place a cache?
09:35 Cache Write Policies
11:38 Hybrid Write Policy?
13:10 Thank you!
A complete course on how systems are designed. Along with video lectures, the course has architecture diagrams, capacity planning, API contracts, and evaluation tests.
System Design Playlist: • System Design Playlist
Code: github.com/coding-parrot/Low-...
You can follow me on:
Facebook: / gkcs0
Quora: www.quora.com/profile/Gaurav-...
LinkedIn: / gaurav-sen-56b6a941
Twitter: / gkcs_
References:
Guava Cache - github.com/google/guava/wiki/...
LRU - www.mathcs.emory.edu/~cheung/C...
en.wikipedia.org/wiki/Cache_r...
Implementation of Sliding Window Cache policies (Caffeine) - github.com/ben-manes/caffeine
highscalability.com/blog/2016/...
docs.microsoft.com/en-us/prev...)
#SystemDesign #Caching #DistributedSystems

Пікірлер: 529

@VrajaJivan 5 жыл бұрын

Gaurav nice video. One comment. Writeback cache refers to writing to cache first and then the update gets propagated to db asynchronously from cache. What you're describing as writeback is actually write-through, since in write through, order of writing (to db or cache first) doesn't matter.

@gkcs 5 жыл бұрын

Ah, thanks for the clarification!

@KumarAbhishek123 5 жыл бұрын

Yes, would be great if you can add a comment saying correction about the 'Write back cache'. Thanks for the great video!

@gururajsridhar7314 5 жыл бұрын

I agree.. a comment in the video correcting this would be good update to this.

@mrityunjoynath7673 4 жыл бұрын

So Gaurav was also wrong in saying "write-back" is a good policy for distributed systems?

@jyotipandey9218 4 жыл бұрын

@Gaurav Yes that would be great. That part was confusing, had to read about that separately.

@waterislife9 4 жыл бұрын

Write-through: data is written in cache & DB; I/O completion is confirmed only when data is written in both places Write-around: data is written in DB only; I/O completion is confirmed when data is written in DB Write-back: data is written in cache first; I/O completion is confirmed when data is written in cache; data is written to DB asynchronously (background job) and does not block the request from being processed

@rajee120 Жыл бұрын

@GK-rl5du 5 жыл бұрын

Other variants 1. There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors. 2. There are only two hard problems in distributed systems: 2. Exactly-once delivery 1. Guaranteed order of messages 2. Exactly-once delivery

@gkcs 5 жыл бұрын

Hahahaha!

@GK-rl5du 5 жыл бұрын

@@gkcs A humble suggestion, I think you should have a sub-reddit for the channel, because these are such critical topics [not just for cracking interviews], I'm sure they'd definitely encourage healthy discussions. I think YT's comment system is not really ideal to have/track conversations with fellow channel members.

@RAJATTHEPAGAL 4 жыл бұрын

This is an underrated comment .... 😂😂😂

@kumarakantirava429 4 жыл бұрын

@@gkcs Can you please give some hints on WHY "out of order Delivery" is a problem in distributed systems, if the application is running on TCP ..................PLease Kindly reply.

@kumarakantirava429 4 жыл бұрын

@goutham Kolluru , Can you please give an hint on WHY "out of order Delivery" is a problem in distributed systems, if the application is running on TCP ..................PLease Kindly reply.

@mengyonglee7057 Жыл бұрын

Notes: In Memory Caching - Save memory cost - For commonly accessed data - Avoid Re-computation - For frequent computation like finding average age - Reduce DB Load - Hit cache before querying DB Drawbacks of Cache - Hardware (SSD) much more expensive than DB - As we store more data on cache, search time increases (counter productive) Design - Database (Infinite information) vs Cache (Relevant information) Cache Policy - Least Recently Used (LRU) - Top entires are recent entries, remove least recently used entries in cache Issue with caches - Extra calls - When we couldn’t find entry in cache, we query from database. - Threshing - Input and output cache without ever using results - Consistency - When update DB, we must maintain consistency between cache and DB Where to place the cache - Close to server (in memory) - Benefit - Fast - Issue - Maintaining consistency between memory of different servers, especially for sensitive data such as password - Close to DB (global cache, i.e. Redis) - Benefit - Accurate, Able to scale independently Write-through vs Write-back - Write-through - Update cache, before updating DB - Not possible for multiple servers - Write-back - Update DB, before updating cache - Issue: Performance - When we update the DB, and we keep updating the cache based on that, much of the data in the cache will be fine and invalidating them will be expensive - Hybrid - Any update first write to cache - After a while, persist entries in bulk to database

@pushp3593 7 ай бұрын

nice, but write through and write back notes part is wrong, pls correct it. you can check other comments. thanks

@cheerladinnemouli2864 4 ай бұрын

Nice notes

@mannion1985 4 жыл бұрын

I can already hear the interviewer asking "with the hybrid solution: what happens when the cache node dies before it flushes to the concrete storage?" You said youd avoid using that strategy for sensitive writes but you'd still stand to lose upto the size of the buffer you defined on the cache in the e entire of failure. You'd have to factor that risk into your trade off. Great video, as always. Thank you!

@jsf17 3 жыл бұрын

The world needs more people like you. Thank you!

@SatyadeepRoat 3 жыл бұрын

I am actually using write back redis in our system but this video actually helped me to understand what's happening overall. GReat video

@bhavyeshvyas2990 5 жыл бұрын

Dude you are the reason for my system design interest Thanks and never stop making system design videos

@Sound_.-Safari 3 жыл бұрын

Cache doesn’t stop network calls but does stop slow costly database queries. This is still explained well and I’m being a little pedantic. Good video, great excitement and energy.

@user-oy4kf5wr8l 4 жыл бұрын

each of ur videos, i watched ay least twice lol, thank you!! WE ALL LOVE U! U R THE BEST!

@rishiraj9131 3 жыл бұрын

I also watch his videos mamy times. At least 4 times to be precise.

@legozxx6655 5 жыл бұрын

Great explanation. You are making my revision so much easier. Thanks!!

@rajeevkulkarni2888 2 жыл бұрын

Thank you so much for these videos!. Using this I was able to pass my system design interview.

@mayankvora8329 3 жыл бұрын

I don't know how people can dislike your video Gaurav, you are a master at explaining the concepts.

@devinsills1281 2 жыл бұрын

A few other reasons not to store completely everything in cache (and thereby ditching DBs altogether) are (1) durability since some caches are in-memory only; (2) range lookups, which would require searching the whole cache vs a DB which could at least leverage an index to help with a range query. Once a DB responds to a range query, of course that response could be cached.

@zehrasubas9768 4 жыл бұрын

Hi Guarav, I really like your videos thank you for sharing! I need to point out something about this video. Writing directly do DB and updating cache after, is called write around not write back. The last option you have provided, writing to cache and updating DB after a while if necessary, is called write back

@gkcs 4 жыл бұрын

Thanks Zehra 😁

@anjurawat9274 4 жыл бұрын

I watched this video 3 times because of confusion but ur pinned comment saved my mind thank you sir

@kabooby0 3 жыл бұрын

Great content. Would love to hear more about how to solve cached data inconsistencies in distributed systems.

@AnonyoX 4 жыл бұрын

Great video. But I wanted to point out that, I think what you are referring to as 'write-back' is termed as 'write-around', as it comes "around" to the cache after writing to the database. Both 'write-around' and 'write-through' are "eager writes" and done synchronously. In contrast, "write-back" is a "lazy write" policy done asynchronously - data is written to the cache and updated to the database in a non-blocking manner. We may choose to be even lazier and play around with the timing however and batch the writes to save network round-trips. This reduces latency, at the cost of temporary inconsistency (or permanent if the cache server crashes - to avoid which we replicate the caches)

@jajasaria 5 жыл бұрын

always watching your videos. topic straight to the point. keep uploading man. thanks always.

@enfieldli9296 2 жыл бұрын

I just can't find a better content on YT than this, thanks man!

@vakul121 4 жыл бұрын

It is a really great video.Finally found a detailed video.Thank you for sharing your knowledge!!

@rahuljain5642 2 жыл бұрын

If someone explains any concept with confidence & clarity like you in the interview, he/she can rock it seriously. Heavily inspired by you & love your content of system design. Thanks for the effort @Gaurav Sen

@an_R_key 4 жыл бұрын

You articulate these concepts very well. Thanks for the upload.

@harisridhar6698 3 жыл бұрын

Hi Gaurav - good video on distributed caching! This expands a bit more on what I learned in my computer architecture class - I didn't recall thrashing the cache too well, or what distinguished write-through vs. write-back. I think learning caching in the context of networks is more interesting, since it was initially introduced as a way to avoid hitting disk ( on a single machine ), but is also a way to reduce network calls invoked from server to databases.

@kfqfguoqf 4 жыл бұрын

Your System Design videos are very good and helpful, thanks!

@akash.vekariya 3 жыл бұрын

This man is literally insane in explanation 🔥

@Mysterious_debris_1111 4 жыл бұрын

Awesome explanation gaurav. You're cool man. We want a lottt more from you. We admire your ability to explain topics with great simplicity.

@semperfiArs 5 жыл бұрын

Extremely good video series bro. Just subscribed yesterday and loving it so far. Suggest you to start a interview series where you can answer a few important questions. Will be helpful.

@prakharpanwaria 2 жыл бұрын

Good video around basic caching concepts. I was hoping to learn more about Redis (given your video title)!

@neeraj91mathur 3 жыл бұрын

Nice video Gaurav, really like your way of explaining. Also, the fast forward when you write on board is great editing, keeps the viewer hooked.

@muraliboddu4007 2 жыл бұрын

nice quick video to get an overview. thanks Gaurav. you are helping a lot of people.

@mostinho7 4 жыл бұрын

Once we have updated an entry in the database, how does of say S2 or S_n (any other server) know that this entry has been updated in the db and that it needs to invalidate it?

@happilysmpl 2 жыл бұрын

Excellent! Great video with tremendous info and design considerations

@muhammadanas11 4 жыл бұрын

The way you explained concepts is AWSOME. Can you please create a video that decribes DOCKER and Containers in your style.

@Not0rious7 3 жыл бұрын

You continue to offer great content. thank you !

@manasbudam7192 4 жыл бұрын

What you explained as write-back cache is actually a write-around cache. In write-back cache...you update only the cache during the write call and update the db later (either while eviction or periodically in the background).

@Satu0King 5 жыл бұрын

Description for write back cache is incorrect. Write-back cache: Under this scheme, data is written to cache alone and completion is immediately confirmed to the client. The write to the permanent storage is done after specified intervals or under certain conditions. This results in low latency and high throughput for write-intensive applications, however, this speed comes with the risk of data loss in case of a crash or other adverse event because the only copy of the written data is in the cache.

@gkcs 5 жыл бұрын

Thanks for pointing this out Satvik 😁👍

@justinmancherje6168 4 жыл бұрын

I believe the description in the video given for write-back cache is actually a write-around cache (according to grokking system design)

@mostinho7 4 жыл бұрын

What if the cache itself is replicated? Will write-back still has risk of data loss

@arpansen964 2 жыл бұрын

Yes, as per my understanding, write-through cache : when data is written on the cache it is modified in the main memory, write back cache: when dirty data (data changed) is evicted from the cache , it is written on the main memory, so write back cache will be faster. The whole explanation around there two concepts given in this video seems fuzzy.

@pat2715 5 ай бұрын

amazing clarity, intuitive explanations

@JinkProject 4 жыл бұрын

this video was gold. studying for my facebook on-site and i need to understand a bit more how backend works. cheers @gaurav sen

@jayantsogani8389 5 жыл бұрын

Thanks Gaurav, your lecture helped me to crack MS. Keep posting video's

@gkcs 5 жыл бұрын

Congrats!

@shubham.1172 4 жыл бұрын

Are you in the Hyd campus?

@rahulchawla6696 2 жыл бұрын

wonderfully explained. thanks

@chenwang7194 3 жыл бұрын

Nice video, thanks! For the hybrid mode, when S1 persists to DB in bulk, the S2 is still having the old data, right? How do we update S2?

@TheHalude 4 жыл бұрын

Thanks this was a good video for a high level overview of cache, easy to follow,

@1970mcgraw 3 жыл бұрын

Excellent info and presentation - thanks!

@sandeepk9640 3 жыл бұрын

Nicely packed lot of information for glimpse.. Great work

@NohandleReqd 2 жыл бұрын

Teaching and learning are processes. Gaurav makes it fun to learn about stuff, then let it be systems or the egg dropping problem. I might just take the InterviewReady course to participate in the interactive sessions. Take a bow!

@GustavoRodrigues-le3zw 2 жыл бұрын

Amazing Explanation!! Thanks!!

@andreigatej6704 5 жыл бұрын

Very well explained! Thank you

@asankaherath1744 5 ай бұрын

Thank you so much..! your videos are really valuable. Really appreciate your effort, sir.!!

@RiteshKumar-qk6uy 5 жыл бұрын

Hi @Gaurav, In write through policy can this will be also issue , let's suppose someone did some update transaction it updated the cache and went through to db to update but there is some check is enabled in db where that transaction failed , so we will having incorrect data in cache? same goes with hybrid model out of 10 transaction let's say 2 failed while updating in db , in this way we will be having incorrect response from the cache .

@nishantmahajan55 5 жыл бұрын

Very helpful and to the point. Thanks.

@renon3359 5 жыл бұрын

Keep making these videos brother, great job as always. :)

@psrajput09 5 жыл бұрын

Superb, now I started using my free time to learn something ! Thanks!!!

@souradiptachoudhuri4724 5 жыл бұрын

This video is just awesome. Thank you

@i2chang 4 жыл бұрын

Thank you, this video is very good!

@RpraneelK 3 жыл бұрын

Very informative and concepts explained clearly. Thanks

@osheennayak9477 4 жыл бұрын

Hi Gaurav, there is a slight confusion, as per the "grokking the system design" write-back cache invalidation means the writing to cache only and then later on writing to DB, something that you mentioned as the hybrid approach. Can you please clarify

@oscarjesusresendiz100 4 жыл бұрын

Great explanation Dude! You killed it!

@ZALOP123 3 жыл бұрын

@Gaurav Sen Thanks for the video. I have a question for you: I have a system where I need to cache a small amount of data (user IDs to usernames - in a map). This is so that on the front end I can populate the data with the auto-complete feature on the UI. For this scenario, would you recommend a cache infrastructure like Redis? Or could we just use a Concurrent HashMap in a programming language like Java to do this? thanks.

@CloudXpat 3 жыл бұрын

Great explanation for caching. I believe you'll go far.

@mana5473 2 жыл бұрын

Great video, thank you!

@codingart7736 5 жыл бұрын

Loved your sharing Thanks a lot

@sakshichawla3946 3 жыл бұрын

Very well explained !!

@silentknight2851 4 жыл бұрын

hey Gaurav, for holidays I'll watch your videos day in and day out... So please teach new topics asap. I love to listen you

@kevinz1991 2 жыл бұрын

learned a ton in this video thanks so much

@djanupamdas 5 жыл бұрын

I think simply telling THANK YOU will be very less for this help !!! Superb video.

@gkcs 5 жыл бұрын

Glad to help :)

@jagatsastry 4 жыл бұрын

I mean you can always do more by becoming a channel member 😄

@akashnag3879 5 жыл бұрын

Loved it.. Thank you for such amazing video.. keep coming up with more.

@ashwinasokan Жыл бұрын

Bhai. u r a life saver! Brilliant tutoring. Thank you!

@OwenValentine 5 жыл бұрын

Gaurav, what you initially described as write-back at around 10:30 I have seen described as write-around. Write-back is where you write to the cache and get confirmation that the update was made, then the system copies from the cache to the database (or whatever authoritative data store you have) later... be it milliseconds or minutes later. Write through is reliable for things that have to be ACID but it is slower than write back. You later describe what I have always heard as write-back at around 12 and a half minutes

@gkcs 5 жыл бұрын

Yes, I messed up with the names. Thanks for pointing it out 😁

@durden0 19 күн бұрын

@@gkcs so does this mean mean that write-through is good for critical data (financial/passwords) and write-back/write-around is not?

@jatinchugh373 5 жыл бұрын

dude your content is amazing

@CodeSbyAniz 3 жыл бұрын

You have explained it very nicely. Thanks.

@ivandrofly 5 жыл бұрын

My boy look very energized... keep it up!

@gkcs 5 жыл бұрын

😁

@jask00 5 жыл бұрын

Great job. Really well explained

@GK-rl5du 5 жыл бұрын

Hi Gaurav, for write-back update mechanism for in-memory caches how would you propagate the DB update event to other processes? Do we implement some kind of observer pattern through message queues. or in this case DB_UPDATE_TOPIC which other application processes are subscribed to?

@gkcs 5 жыл бұрын

Yes we need some sort of pub sub model. Active publishing, meaning sending messages have their advantages as in it's real time and hence consistency is high. But the fanout and db load goes up. Something like kafka or postgres' abilities should do the job.

@grijeshmnit 4 жыл бұрын

Your explanations are improved a lot now... correct sequence of information was missing in your earlier vdos... May you can improve your flow chart, fig further for less confusing...

@ShaliniNegi24 4 жыл бұрын

Nice Explanation! Thanks :)

@raghavabharati 4 жыл бұрын

Pls refer to the caching topic page in Educative(That you've recommended). Is the explanation in your video and that page aligned?

@michaelscheppert3664 2 жыл бұрын

thanks for this quick tutorial :) your English is really good

@harshdubey9951 4 жыл бұрын

For in-memory cache across the servers, can we use Hazelcast as an option for caching?

@ratneshchouhan6 5 жыл бұрын

Hey @Gaurav Sen , So u mentioned bat global cache but dnt u thnk wht if it fails??

@sharifulhaque6809 3 жыл бұрын

Very easy understanding Gaurav. Thanks a lot !!!

@zainsyed9811 5 жыл бұрын

Awesome overview thanks. One other possible issue with write-through - it's possible to make the update to the cache then the DB update itself fails. Now your cache and db will be inconsistent.

@gkcs 5 жыл бұрын

True 😁

@ss53210 5 жыл бұрын

You explain so well! :) Thank you

@gkcs 5 жыл бұрын

Thanks!

@ananava254 3 жыл бұрын

Thank you Gaurav, it was a really good explanation

@shreyasns1 2 жыл бұрын

Thank you for the video. You could have gone a little deeper about how the cache is implemented? What’s the underlying data structure of the cache?

@sirunworld 4 жыл бұрын

Great going, Gaurav. You have a great future!

@SuperAzizx 3 жыл бұрын

awesome Gaurav thanks

@fakhruddintahery1561 2 жыл бұрын

Great explanation

@chriszeng5406 4 жыл бұрын

Good video. Thank you. From Canada.

@meletisflevarakis40 5 жыл бұрын

Your explanation is awesome. Keep it up!

@gkcs 5 жыл бұрын

Thanks!

@VikramKumar-qo3rg 3 жыл бұрын

Fun part. I was going through 'Grokking The System Design Interview' course, found the term 'Redis', started searching for more on it on youtube, landed here, finished the video and Gaurav is now asking me to go back to the course. Was going to anyway! :)

@gkcs 3 жыл бұрын

Hahaha!

@user-jk1qi2gj2p 3 жыл бұрын

Thanks for this video

@chakreshtiwari 4 жыл бұрын

Hi I have a scenario where i am adding data to cache whatever data present to db .and then i added some records to db but that records are not coming to cache. Cache have only old eecords why this scenario. If i weite function to check and match recirds in cache and db then again it is a headache. Then how to handle this type of problem i am using spring boot for developing application. Thanks Suggestions please.