This is the single best 2PC and 3PC video I've come across! Really helpful for my exam tomorrow

I need more videos that talk about distributed systems like this. Thank you so much. 🙏

Got best gift on my birthday. Thank u bhaiya a ton. 🎉🎉🎉

consistency in ACID essentially means that the data should be a in a consistent state before and after the transaction i.e., it should satisfy all DB constraints.

Finally the wait is over.. GOAT of HLD is back,, thankyou for such content, u rock !!

Hi, great video overall, though I find a gap in the explanation. In the traditional Two-Phase Commit (2PC) protocol, participants do not communicate directly with each other; they only communicate with the Transaction Coordinator (TC). This is why, 3 PC introduces additional complexity (coordinating between participants). If participant start communicating between each other in both, then 2PC and 3 PC are not much different.

Great explanation like always. Thanks!

Hi Shreyansh , just a small doubt . How does pre-commit solve the problem in 3 phase , it can be done the same way in 2 phase only . If participants have information of each other and in 2 phase algorithm only they didn't got any COMMIT message ( coordinator went down before 2nd phase only ) , they can abort there only right ?

What if pre-commit msg sent from co-ordinator fails? Isn't this same as commit msg from co-ordinator fails in 2PC and participant is waiting for what to do ? If participants can talk to each other whether other participants received commit msg or not in both 2PC and 3PC, how 3 PC solves the problem of 2PC(commit msg lost)?

Thank you for the informative videos. I have a doubt at 35:00. You said that all participants know about each other. So in 2 PC, suppose TC sends commit msg to one participant and then fails, meaning only 1 out of two participants got the commit msg. Then in this case, 2PC should not be blocking the transaction as the Participants are aware of each other's status. Thanks

very well explained!

Much needed...thanks

I have a small doubt regarding the Three-Phase Commit (3PC) protocol: If the coordinator fails after sending the 'CanCommit' requests in 3PC, how is this scenario effectively managed differently from the Two-Phase Commit (2PC) protocol, where a similar failure leads to a blocking issue?

Shreyansh one thing like can u please pin some of the common doubts asked so that u also need not to answer every time and we also need not to see every chat. Just a suggestion. Also good content 🎉

Hi Shreyansh, Great video. One doubt, in 2pc if the commit msg is lost, since services can talk to each other it can ask other services if they have received the commit msgs if yes than continue with commit or abort. Why go for 3PC?

Waiting for long fine!....Thanks

5:36 One transaction should not be locking another transaction. Ideally, only row-level locks should be used to maximize concurrency. If both transactions are accessing the same row, an Exclusive lock should be applied to ensure data integrity. However, if the isolation level is set to Serializable, transactions might still lock each other to maintain consistency and prevent issues like phantom reads.

if participants can communicate with each other, then why is there a blocking phase in 2PC? Since a participant can check with others regarding the final decision and act accordingly

Thankyou for the nice explanation. Could you please elaborate more on how the participating MSs enquire each other about decision taken by TC? Furthermore, if participating MSs can enquire each other if they have received decision from TC then isn't it a good idea in 2PC that the MSs can abort the transaction and release the lock on the corresponding rows, thus reducing the blocking behaviour? And when the TC comes up again it can check the incompleted transactions (like the above scenario where even before taking decision the TC went down) and discard them.

Very well explained

Hi Shreyansh, You have explained it very well. I have few doubts here. I browsed these doubts all over the internet and I did not get proper answer in GPT as well. 1. What if the paricipants have multiple replicas ? How co-ordinator will make sure the prepare and commit request will always hit the same replica ? 2. I am thinking out this doubt in Java/Spring Implementation. I know transactions can be created using Transaction Annotation and a same thread will work until the transaction is committed. Once participant creates local transaction, it must be in a new transaction thread. So how this transaction thread is stopped from committing and how it commits after receiving the external request(http thread) from co-ordinator since both threads are different. In simple terms, how the two different threads are communicated ? one is new http thread(prepare or commit) and already created transaction thread ? I hope the understanding of doubt is clear. Please let me know if there is a problem. I will elaborate maybe. I am badly searching for the doubts.

Suggestion: please try laser pen and more pictorial reprresentation and will be great if used tab to write down things properly or just keep a slide of it

Can u pls make video on Kafka partitions working?

Consider this scenario -> Coordinator sent a commit message to all the participants but didn't get commit success from 1 participant. Now it tries to send the abort to all the participants and just before sending the, it goes down. 1. Failed participant will keep the lock on that row or number of rows until the coordinator goes up and send the abort message to release the lock. Right? 2. Are the rest of the participants going to wait? (a) -> If yes, then why? (b)-> If not, say a new transaction starts on the same row in any of the participants and the co-ordinator sends an abort message now. What sort of handling would be there now for the ongoing transaction and the previous transaction?

great stuff , Can you make a video to do it practically if it is possible, thanks

2 phase commit is a blocking protocol because the 'commit' phase keeps waiting forever for some signal from co-originator right ? instead of introducing three phase commit to solve this problem, one simple work around was to introduce 'timeout' for participants in 'commit' phase as well (just like 'timeout' in 'prepare' phase). correct ?

amazing video , some doubt though :- 1. Why cant we have the timeout at the decision phase during the 2 phase commit (like we have in the voting phase, where if the participant doesn't receive the prepared message form coordinator within the given amount of time, it goes for timeout, and thus abort) if the same would had been applied to the decision phase of 2PhaseLocking, then there was no need of 3 Phase commit procedure) 2. In SAGA pattern, we are committing at every service, and not waiting for all other dependent services to complete, what if the child transaction fails, then it will publish the failure message in queue, but we had already committed, then how will the microservices which comiited long time back , will rollback now?......wouldn't it create an issue ?

In saga pattern let's say we have 2 microservices and we have committed on the 1st microservice and now the commit on 2nd microservice fails. But since they are asynchronous what happens if we get a read call on the 1st microservice before the transaction is reverted?

Can you make a video on optimistic and pessimistic locking

Can you please complete this series as per the roadmap in the first videos

there's unclear about the explaination of "pre-commit phase" of 3PC.

Hey Shrayansh, can you please also make a video explaining how we handle thread safety while using saga for transactions on a multicore machine?

Hi Shreyansh, great videos. Can you tell me what further videos you plan to bring in the hld playlist, as these 21 videos seem to be more setting up basics. Would you have subsequent videos with real hld questions?

I didnt understand the need of 3PC over 2PC and what its actually solving when in case of 2PC also they can communicate with each other and eventually they can abort the process just as in 3PC case

One doubt in SAGA desing pattern @Shrayansh Since each local transaction does commit and move on next service and then complete its transaction and do commit. Suppose last microservice get failed, you said it generate some sort of event which let all the previous transaction to be rolled back. But how come it happen since for all the previous transaction we have done commit. so those are already persisted to the db memory.

Hi @Shreyansh , can you please explain about the kafka resiliency and is it same as Saga Pattern ?

when are the other videos coming?

Could you please also help on how to secure Microservices.

In Springboot all these can be done with the help of only 1 annotation - @Transactional on a class or method , right ? Very nice video, crystal clear !!

i dont undestand the difference between 2pc and 3pc ?

Hi Sir, first of all, thank you for the nice content it really helps me to learn something new. I have one doubt may you or anyone can help to understand it? As in Phase 1 if any message ('Prepare', 'OK', 'NO') is lost then both coordinator and participant have a timeout mechanism. please correct me if my understanding is wrong? Then 2nd phase when the commit message is lost, why was it blocked, and why is Transaction Coordinator not timeout as it did in phase 1? Please help to clarify this doubt.

Hey why some of the old LLD videos which were free for everyone are now members only?

That participants inter communication can be done in 2 phase itself ?

Cover JUnit and architecture patterns if possible

Sir, what happens in case of SAGA pattern, if service 1 commited transaction then service 2(Async call) Transaction got failed? we have to write custom rollback logic for that or manually delete/update?

Hi shreyansh sir Completing your hld series one suggestion can you please put your onenotes so that it will be easier to revise this thing again in a lot faster ways. Thanks for great series 🙏

can u please share notes of this this and previous lecture

Better to have blogs to be written

Sir, How to implement transaction coordinator?

supppose in 2pc a commit msg has been sen to one microservice and Transaction coordinator fails to send the msg to another microservice as the transaction is already performed at one microservice what would happend or how can he revert back

Sorry for the noob question, which video do we have cache eviction in? I cant find it.

Hi Which tool are you using to record and type the session?

Thanks Sir, Is it the second part of 19th video ?

If there is a coordinator for 2 microservice, will it not be orchestrator pattern, which is type of SAGA pattern?

Hi Shreyansh, Thanks for the great video. I have a query in 3 phase commit, Totally understood, it helps to tackle the blocking issue likewise in 2 phase commit, what if blocking happens before the pre-commit(or right after the prepare TC get fails) in 3 phase commit, that will be same condition as blocking in 2 phase commit. Could you please help me with this confusion. Thanks in advance!!

👍👍

In 2 PC .. Suppose, in phase 2, commit done is replied by 1 participant, but another participant failed to commit, then ?

Wondering if you could throw some light on modules of a cloud native apps. When can you say, now my app is fully cloud native.

Thanks for this video.. But to be honest didn't get much clarity when any of the MS or Coordinator goes down in 2 phase commit.. How it read from logs file considering logs file had tons of records in real world application... and if any of these MS let say fail to responded.. how to get a response from Coordinator log files which Coordinator already send a abort request to another instance of that same MS.

❤

Please provide the pdf

what if done message failed?

Your explanation is not precise...mixed up and confused to me with 2ph and 3ph

can you please provide notes also of these topics??