Redo, Undo and WAL logs | The Backend Engineering Show

  Рет қаралды 8,348

Hussein Nasser

Hussein Nasser

Күн бұрын

Database logging is a critical feature to maintain durability, in this show I discuss them in details
0:00 intro
1:00 ad
3:30 Start of the Show
6:00 What is commit
9:30 What if a commit fails half way
11:00 WAL
23:00 Checkpointing
27:00 fsync
33:00 Undo logs
Fundamentals of Networking for Effective Backends udemy course (link redirects to udemy with coupon)
network.husseinnasser.com
Fundamentals of Database Engineering udemy course (link redirects to udemy with coupon)
database.husseinnasser.com
Introduction to NGINX (link redirects to udemy with coupon)
nginx.husseinnasser.com
Python on the Backend (link redirects to udemy with coupon)
python.husseinnasser.com
Become a Member on KZfaq
/ @hnasr
Arabic Software Engineering Channel
/ @husseinnasser
🔥 Members Only Content
• Members-only videos
🏭 Backend Engineering Videos in Order
backend.husseinnasser.com
💾 Database Engineering Videos
• Database Engineering
🎙️Listen to the Backend Engineering Podcast
husseinnasser.com/podcast
Gears and tools used on the Channel (affiliates)
🖼️ Slides and Thumbnail Design
Canva
partner.canva.com/c/2766475/6...
Stay Awesome,
Hussein

Пікірлер: 19
@hnasr
@hnasr Жыл бұрын
Get my fundamentals of database engineering udemy course database.husseinnasser.com/
@luisgomes1814
@luisgomes1814 2 жыл бұрын
Another absolute masterpiece Hussain, loved it! Would love if you could make a follow up video showing WALs in action in a distributed DB (replica sets, etc).
@shyampramanik780
@shyampramanik780 2 жыл бұрын
Awesome explanation as always !! Hussein , can you please make a video on databases having non-relational data model and how do they store and retrieve data efficiently from disk (data pages) like document DB, key value stores, graph DB , time series DB and each of their uses cases ?
@vikingthedude
@vikingthedude 2 жыл бұрын
Another banger episode Hussein
@vinitsunita
@vinitsunita Жыл бұрын
Writing to wal means we are persisting changes in Disk which could be slow operation also. What is the way to make it faster?
@user-hv3xm2ev2g
@user-hv3xm2ev2g 6 ай бұрын
I wish I could like it more than once, Thank you, Hussain!)
@chessMan786
@chessMan786 16 күн бұрын
I have a question. Let's say WAL is almost full and we are trying to flush the data from page to disk, but now the flushing is not successful as it crashes but simultaneously more logs are getting written in WAL, and once WAL will reach to it's maximum limit and still the first flush is not successful to disk does this means that the old logs from the WAL will be removed to have the new logs and eventually we will loose the changes does with that old change?
@chandkeshav001
@chandkeshav001 2 жыл бұрын
"Computer scientists build the wal and the dbas paid for it" - Hussain Nasser, 2022 🤣🤣
@nonya69
@nonya69 Жыл бұрын
I loved that one
@PiyushChauhan2011
@PiyushChauhan2011 2 жыл бұрын
Nice, it would nice to learn about nodejs code which can read from write ahead logs from postgres
@akashagarwal6390
@akashagarwal6390 Жыл бұрын
How would it work in case of a distributed db or anything related of that sort?
@et4493
@et4493 10 ай бұрын
26:30 please make this a movie lmao
@vivekatbitm
@vivekatbitm 2 жыл бұрын
What happens if it crashes before transaction is committed, will the changes written to WAL be reverted. does wal has some kind of dirty flag for uncommitted changes? Also does each table has a separate wal file, or one wal per database?
@nitishreddy55
@nitishreddy55 2 жыл бұрын
That is where you have the the undo segments. So every time you start a transaction a undo segment is assigned to the Txn and it has the snapshot that guarantees read consistency. This is called MVCC in databases. This is how read statements get the correct fat when your transactions have an intent to write. Also as the transaction progresses and you make data changes (dmls) a time snapshot (SCN or CSN) is also maintained in undo and also at times this is written to redo ahead of time about every 3 seconds or other scenarios. Basically to put all these together redo or wal and undo are used for crash recovery. When instance starts it reads the redo and undo to see which statements were commuted but not written to disk and write them to disk. Which are the statements that are not committed that are in redo and sometimes even on the disk. You undo those changes and update the checkpoint. You can go deep. Oracle does this best. That is the reason oracle is what oracle is! When isolation, consistency and durability is the concern there is a reason banking systems and critical systems use oracle or mssql databases and pay for them rather than open source ones.
@Prakashsharma-eh8uq
@Prakashsharma-eh8uq 2 жыл бұрын
why we need to Apply Undo coz WAL is source of truth and if we apply WAL we are sure we have applied commited Trxns we need to apply Trxs which were not committed
@autohmae
@autohmae 2 жыл бұрын
To bad I don't drink coffee so I could help you with your sponsor.
@rohitsharma-rp2jh
@rohitsharma-rp2jh 2 жыл бұрын
Noice
@pratik_balpande
@pratik_balpande 2 жыл бұрын
It possible to get free modile data internet using vpn /sock5 please make video about
Sorting in Postgres 15 is 371% faster in certain cases
9:38
Hussein Nasser
Рет қаралды 11 М.
Best KFC Homemade For My Son #cooking #shorts
00:58
BANKII
Рет қаралды 73 МЛН
Gym belt !! 😂😂  @kauermotta
00:10
Tibo InShape
Рет қаралды 18 МЛН
Index Fill Factor | The Backend Engineering Show
33:48
Hussein Nasser
Рет қаралды 7 М.
WAL: Everything you want to know
49:00
Postgres Open
Рет қаралды 10 М.
The Cost of Memory Fragmentation
38:19
Hussein Nasser
Рет қаралды 7 М.
What are write-ahead logs and what are the gotchas?
6:44
Why this query is fast
17:02
Hussein Nasser
Рет қаралды 19 М.
How Discord Stores Trillions of Messages | Deep Dive
1:08:33
Hussein Nasser
Рет қаралды 174 М.
Rate This Smartphone Cooler Set-up ⭐
0:10
Shakeuptech
Рет қаралды 6 МЛН
تجربة أغرب توصيلة شحن ضد القطع تماما
0:56
صدام العزي
Рет қаралды 64 МЛН
Samsung laughing on iPhone #techbyakram
0:12
Tech by Akram
Рет қаралды 7 МЛН