CAP Theorem - From the First Principles

No video

CAP Theorem - From the First Principles

Рет қаралды 28,083

Күн бұрын

System Design for SDE-2 and above: arpitbhayani.m...
System Design for Beginners: arpitbhayani.m...
Redis Internals: arpitbhayani.m...
Build Your Own Redis / DNS / BitTorrent / SQLite - with CodeCrafters.
Sign up and get 40% off - app.codecrafte...
In the video, I delved into the concepts of CAP theorem and its practical implications using examples from Google's Spanner database blog. CAP theorem, introduced by Eric Brewer, states that distributed data stores can provide only two out of three guarantees: Consistency, Availability, and Partition Tolerance. I explained the differences between consistency in ACID and CAP theorem, highlighting the trade-offs between consistency and availability during network partitions. Google Spanner aims to achieve both consistency and availability through globally distributed data management and specialized hardware like the TrueTime algorithm, challenging traditional CAP theorem assumptions.
Recommended videos and playlists
If you liked this video, you will find the following videos and playlists helpful
System Design: • PostgreSQL connection ...
Designing Microservices: • Advantages of adopting...
Database Engineering: • How nested loop, hash,...
Concurrency In-depth: • How to write efficient...
Research paper dissections: • The Google File System...
Outage Dissections: • Dissecting GitHub Outa...
Hash Table Internals: • Internal Structure of ...
Bittorrent Internals: • Introduction to BitTor...
Things you will find amusing
Knowledge Base: arpitbhayani.m...
Bookshelf: arpitbhayani.m...
Papershelf: arpitbhayani.m...
Other socials
I keep writing and sharing my practical experience and learnings every day, so if you resonate then follow along. I keep it no fluff.
LinkedIn: / arpitbhayani
Twitter: / arpit_bhayani
Weekly Newsletter: arpit.substack...
Thank you for watching and supporting! it means a ton.
I am on a mission to bring out the best engineering stories from around the world and make you all fall in
love with engineering. If you resonate with this then follow along, I always keep it no-fluff.

Пікірлер: 37

@aayushgupta1186 10 ай бұрын

This is legit “Asli Engineering” with top notch content quality in terms of explanation and breaking down the things right from the published paper. You’re actually raising the expectation bar high, and this is what the audience expects from a Google staff SE. Thanks for being a great educator. I hope this reaches out to you, and you keep making such videos.

@rocksh1234 7 ай бұрын

"@Asli Engineering" What a great explanation on CAP. I request you upload videos on SOLID principles, their actual implementation and understanding required for a production-ready code. Many people just explain definition but never get in depth, on how these principles help in building a strong design of a large base code. Request you to create an extensive video on SOLID principles.

@elizakarki9446 6 ай бұрын

Great explanation. I was asked in one of the Apple interviews, "are you sure CAP theorem is true in today's scenario?" I wasn't aware of any other theorems so I told the traditional definition of CAP theorem (either AP or CP ). But the interviewer was trying to point that it is possible to achieve all three what exactly is availability or consistency or partition tolerance. He told me to read papers where it was possible to achieve all three. So, I guess I came to the right video. Had I known about spanner then, I could have mentioned about it. Nevertheless, its a learning curve for me.

@AsliEngineering 6 ай бұрын

Thanks for resonating 🙌 we all learn through experiences and glad you took it in a positive way. All the very best ✨

@mohd.tahauddin9001 13 күн бұрын

I think you misunderstood. Achieving all 3 is not possible. It'd not CP vs AP, rather its C vs A given P always happens. If you did find a paper that accomplishes all 3, please link it

@chandan07talreja 10 ай бұрын

Very informative thank you Arpit.

@prithvirajpatil7388 16 күн бұрын

Very informative video

@chinmaypawar 10 ай бұрын

Thank you for considering my suggestion of taking CAP theorem as a topic

@AsliEngineering 10 ай бұрын

Thank you for suggesting. Hope this clears up CAP once and for all 😅 Do suggest other topics that you want me to touch upon. Happy to accomodate.

@hrishii.02 10 ай бұрын

@@AsliEngineeringYou could have a session where you analyse and decode some interesting research papers

@AsliEngineering 10 ай бұрын

@@hrishii.02 this has been in the back of my mind for a long time. Working on something cool around this. Thanks for the nudge.

@chinmaypawar 10 ай бұрын

apart from your vast knowledge one thing that impresses most is your humility , keep it up dude only sky is the limit for you , blessings for your little one 😄

@AtharvaRao0104 10 ай бұрын

at 6:42 - I think it is not necessary to take a downtime of entire system. We are not getting the availability as per the definition of CAP ( because some non-failing nodes are choosing not to respond to read/write requests to maintain consistency as there is a partition) the system as whole need not have a downtime(outage). We can still have 99.9% kind of high availability by redirecting all reads to the other partition where writes are happening.

@AsliEngineering 10 ай бұрын

I agree. But while explaining it is easy for people to understand and comprehend the word downtime vs availability. In the context, I meant exactly what you wrote but just to simplify the understanding and help people build intuition I used that term. But yes. Thank you for pointing it out. This comment will definitely help people understand that you can still serve reads making your system partially available.

@dheerajkrgoyal 5 ай бұрын

I dont understand one thing. In case we choose consistency in case of partition. We are expecting a downtime. (i.e. missing availability) but then since the system is down now how it is partition tolerant?

@cyriacgc Ай бұрын

You said due to the use of specialized hardware Spanner gives almost 100% CAP. So, what happens when the 0.0001% happens? I presume it causes a split partition. If so, what does Spanner choose, C or A?

@harishreddy3310 2 ай бұрын

You should also discuss PACELC theorem which is more relevant to the new systems

@aashishkumarjha5738 10 ай бұрын

Correct me Arpit if am wrong but I think that in reaity AP systems are more present and used than any other combinations of C, A, P.

@AsliEngineering 10 ай бұрын

This is a generalization which might or might not be true. To be honest, this is not how you should look at databases, as in, which is large in numbers. 1. There are large number of databases out there 2. Nobody has gone and analysed all of them But, if you are referring to a sea of NoSQL databases that have been built in the last few years, then yes. They do prioritise Availability over Consistency. But in reality, most of them deployed within one data center so the chances of Partitioning happening is surprisingly low making them CA (for practical reasons). But theoretically and as per their code they would have to choose C or A and a lot of new DBs chose A over C.

@aashishkumarjha5738 10 ай бұрын

@@AsliEngineering thanks Arpit! Got that fact of one data centre aspect.

@wadhwa_neeraj 10 ай бұрын

Hi Arpit, how about CQRS and Event Sourcing as a topic for this series?

@pallabsonowal7110 10 ай бұрын

Interested as well

@sanjarcode 9 күн бұрын

if (P) choose C or A.

@pradyumnarahul 10 ай бұрын

Really good video sess! - I was reading about CAP theorem today too xD

@adithyabhat4770 7 ай бұрын

Great video!

@Neilblaze 10 ай бұрын

#AsliEngineering FTW

@utkershgahoi7101 10 ай бұрын

Hi Arpit just a quick question may be i am totally wrong,In the context of the CAP theorem, when a system prioritizes consistency and availability (CA), it sacrifices partition tolerance. This means that in the event of a network partition, the system might not be able to guarantee the consistency of the data or the availability of the services,so this means every time system will not be consistent and available ,so it means there might be a chance that we will not be able to achieve any of this (CAP)

@FargoWarrenbot 9 ай бұрын

there is no network if you have ca. you can only have pc and pa if you have it over multiple networks

@shashankb1271 10 ай бұрын

Bhaiyya use OBS if you are just using it to record data.This looks like apne koi Video conference application mai ye khola hai, boht blurry hai

@decadewgame9802 10 ай бұрын

How do you add subtitles to your video ? is there any easy and fast way ?

@AsliEngineering 10 ай бұрын

I recorded this through Reverside - www.riverside.fm/?via=arpit It is a pretty dope tool for recording and editing podcasts (although I did a single guest video) and added subtitles using that.

@decadewgame9802 10 ай бұрын

@@AsliEngineering do you use the paid version?

@AsliEngineering 10 ай бұрын

@@decadewgame9802 yes. Recently moved to paid tier. This was my first video using the paid tier. Subtitles and text based video editing are pretty dope. The subtitles you see are part of the paid feature.

@irounik 7 ай бұрын

"Software hai, fatega."

@user-kq5om9bi3i 4 ай бұрын

SET GLOBAL sync_binlog = 1; Until you set this in MySQL it stays a CA system. Your video is confusing. i understood better before going through your video.

@AsliEngineering 4 ай бұрын

Async replicated databases are always prone to high replication lag due to partitioning. if master goes down and we switch this replica to be the master then the data is now not consistent. Also, it takes some time for replica to become the master so the system is not available during the failover. A highly available system has zero downtime during failover and it is typically dome with dual writes across nodes.

@user-kq5om9bi3i 4 ай бұрын

@@AsliEngineering Thanks for replying. I was super confused yesterday. It will definitely not be a CA system.