System Design: Why is Kafka fast?

  Рет қаралды 1,084,778

ByteByteGo

2 жыл бұрын

Weekly system design newsletter: bit.ly/3tfAlYD
Checkout our bestselling System Design Interview books:
Volume 1: amzn.to/3Ou7gkd
Volume 2: amzn.to/3HqGozy
Other things we made:
Digital version of System Design Interview books: bit.ly/3mlDSk9
Twitter: bit.ly/3HqEz5G
LinkedIn: bit.ly/39h22JK
ABOUT US:
Covering topics and trends in large-scale system design, from the authors of the best-selling System Design Interview series.

Пікірлер: 463
@ByteByteGo
@ByteByteGo Жыл бұрын
Subscribe and Kafka will say thank you :)
@tubenzr
@tubenzr Жыл бұрын
ok, it's done Sir
@DrRishabhGarg
@DrRishabhGarg Жыл бұрын
What software do you use to create this awesome motion graphics?
@rpidugu99
@rpidugu99 Жыл бұрын
May I know what tool you guys use to make these animated videos? Just curious..!!
@ropro9817
@ropro9817 Жыл бұрын
I just discovered this video in my feed. _Sometimes_ the KZfaq algorithm actually works! 🤠Great video! I just subscribed to your channel!
@colossus95
@colossus95 Жыл бұрын
I wish you were my professor in college.
@jay6645
@jay6645 2 жыл бұрын
The absence of any background music makes this video great.
@crowlsyong
@crowlsyong 2 жыл бұрын
This comment. Yes.
@donvercety
@donvercety 2 жыл бұрын
fully agree!
@kevalan1042
@kevalan1042 2 жыл бұрын
amen!
@vmarzein
@vmarzein 2 жыл бұрын
i agree
@alirezazarafshani
@alirezazarafshani 2 жыл бұрын
Exactly
@kurtmueller2089
@kurtmueller2089 2 жыл бұрын
What an amazing tutorial: Just the necessities, no annoying background music, no annoying calls to "subscribe and like". If all youtube channels were like that, we could heal the world. Also, I checked your channel page and was shocked to find that this was only your 3rd video. Keep being awesome!
@martinmusli3044
@martinmusli3044 2 жыл бұрын
This Tutorial is insanly "Zen" but he said "please subscribe" right at the end :P
@ridealongreactions2601
@ridealongreactions2601 Жыл бұрын
I 100% believe you should make a whole series on Kafka, your way of simplifying the subject is legendary.
@nemeziz_prime
@nemeziz_prime 2 жыл бұрын
These videos are amazingly simple and clear. The animations are spot on!! Too good xD I wish this channel never stops uploading new content
@Spiritualleace
@Spiritualleace 2 жыл бұрын
How can one keep things so deep and yet stunningly simple. Hats off!
@dishantchauhan4775
@dishantchauhan4775 2 жыл бұрын
Seriously, thanks a lot Alex for all the stuff you convey through your LinkedIn network and KZfaq videos. Just love the way you distil the topics and make them understand beautifully.
@ervamate
@ervamate Жыл бұрын
Mentioned a lot in the comments, but I have to say as well: what a great explanation, straight to the point, no bs and gives enough info without overwhelming with details. Thank you!
@MrRunchSlam
@MrRunchSlam 2 жыл бұрын
You guys are doing amazing work here. I love the aesthetics, pace, explanations, topics, and cadence of it all. Kudos!
@nishantparmar
@nishantparmar 2 жыл бұрын
Short, high quality, clean and extremely precise content...Many Thanks!
@severtone263
@severtone263 Жыл бұрын
No frills and thrills, just pure nuggets of value. Exactly what I needed. Thank you. You earned my sub.
@pranavamazon5937
@pranavamazon5937 2 жыл бұрын
this guy is so sweet. man! i was struggling on this system design, all his books and posts are too easy to follow and helped me become more confident
@SaitamaTheLegend
@SaitamaTheLegend 2 жыл бұрын
In 5 minutes I learned a lot! Amazing video! You are a good teacher! Thank you and I hope to see more videos from you!
@Athmarr
@Athmarr Жыл бұрын
I have used kafka before but never had to think about why it is actually fast. This was very informative. I like the format of the video as well
@fripickbot4043
@fripickbot4043 Жыл бұрын
Man this is gold. Saying thank you does not feel enough. Please keep it up.
@ducquang980
@ducquang980 2 жыл бұрын
Short, concise and concrete. Very easy to understand. Thanks a lot
@sami9323
@sami9323 Жыл бұрын
Absolutely fantastic video - went over a lot of concepts like minimizing disk io, engineering constraints of kafka, different memory access patterns, with very good diagrams! Thank you :)
@Drdemiurge
@Drdemiurge Жыл бұрын
So glad the algorithm found this channel for me, the content is so clear and digestible, thank you please keep up the fantastic work
@mwaikul
@mwaikul Жыл бұрын
Amazing! Love the quality and getting straight to the point. Not a second wasted.
@DanteS-119
@DanteS-119 Жыл бұрын
Lol, I heard you talk for about 10 seconds and subscribed. The tone of voice, the kind of explanation, the details, the video content, all of that quality conveyed in just a few seconds. Excellent content. Great stuff.
@aryanrahman3212
@aryanrahman3212 2 жыл бұрын
Really great presentation! I was scared when I saw Kafka but you explained it really well.
@fahuwayne8067
@fahuwayne8067 Жыл бұрын
You have a extremely clear and nice way to talk and explain! Please make more videos like that. Awesome work!
@jigneshnakhva1546
@jigneshnakhva1546 Жыл бұрын
I love all the System-design Content posted by you! Thanks for sharing your knowledge! 🙏
@riadhgharbi7985
@riadhgharbi7985 2 жыл бұрын
Very simple and efficient execution, talking about both the video and Kafka. Really good material mate, keep up the good work
@_Documentation
@_Documentation 2 жыл бұрын
Succinct. Precise. Educative. Excellent animation. Simply the best 💯
@toukaK
@toukaK Жыл бұрын
excited to see Sahn on youtube! this is by far the best tech video I've watched. concise without losing any depth! looking forward to more videos like this. I've had the fortune to (indirectly) work with Sahn and review his code. one of the few top talents that any company is lucky to have. this video is as high quality as other production of his. 2 questions for Sahn: 1. there's a small disconnection between "sequential IO throughput vs random IO throughput" and "HDD vs SSD". is there any perf number difference on sequential IO throughput on HDD vs SSD? 2. is there any perf number difference(ops per sec or latency) for zero-copy vs traditional buffer copies?
@jiajunc-yw3rn
@jiajunc-yw3rn 4 ай бұрын
You made me realize the importance of expressing thought in a clear and concise way. Thank you
@ChandraShekhar-by3cd
@ChandraShekhar-by3cd 2 жыл бұрын
Loved the animation and explanation. Keep enlightening us all!
@sherhy3689
@sherhy3689 2 жыл бұрын
i wanted to comment that i appreciate the level of detail in the explanations in the video. looking forward to more useful content!
@GiacomoPetronio
@GiacomoPetronio 2 жыл бұрын
5 minutes of high quality content, thanks!
@tomislavkristianoliveirabi9873
@tomislavkristianoliveirabi9873 2 жыл бұрын
Exactly my kind of content. Interesting, insightful and to the point.
@143Support
@143Support 8 ай бұрын
This is not the same Kafka I was expecting, but happy to learn. thanks for sharing!
@gopalkrushnapattanaik3232
@gopalkrushnapattanaik3232 2 жыл бұрын
Short ,Crisp and To the point contents , Great work !!
@John-jd2tu
@John-jd2tu 2 жыл бұрын
Very simple and clear! Thank you!
@RunOfTheTrill
@RunOfTheTrill Жыл бұрын
A truly educational and concise video. Thank you.
@parthsarthisharma4163
@parthsarthisharma4163 2 жыл бұрын
Crisp yet complete info. Good content. Thank You.
@codygaurav6384
@codygaurav6384 2 жыл бұрын
concise and crisp clear... Thanks for making such amazing and valuable videos.
@dowlathbashag65
@dowlathbashag65 2 жыл бұрын
Awesome Explanation about Kafka is amazing...Thank you, Alex
@Youvko
@Youvko Жыл бұрын
Wow, this one is super cool. No background music, cool minimalistic diagrams, calm voice!
@vikingthedude
@vikingthedude Жыл бұрын
I love the format of these videos. Looking forward to more and to the newsletters too!
@AnkitMalhotra
@AnkitMalhotra Жыл бұрын
Nice, I definitely learned something new about the Kafka internals today!
@andyserrato
@andyserrato 2 жыл бұрын
So simple yet so powerful explanation, thanks
@playniuniu
@playniuniu 2 жыл бұрын
Great video, explain kafka design so clearly. Thanks very much
@dansokolsky3963
@dansokolsky3963 Жыл бұрын
We need so much more of this.
@adamyatripathi2743
@adamyatripathi2743 2 жыл бұрын
My head exploded with the DMA. I had not idea! Great learning! :)
@lifessummerleaves
@lifessummerleaves 2 жыл бұрын
Very deep insight! Looking forward to your next videos, please keep going
@TBadalov
@TBadalov 10 ай бұрын
Thank you! Such a great delivery and explanation. Particularly, great choice of aspects to share.
@JisKriker
@JisKriker 2 жыл бұрын
wow. No BS, only content! Thank you!
@NuncNuncNuncNunc
@NuncNuncNuncNunc 2 жыл бұрын
Very clear explanation. Thank You!
@akbarsha03
@akbarsha03 2 жыл бұрын
Great work! Easy to understand the concept. Thank you
@vikram_saha7
@vikram_saha7 2 жыл бұрын
wow!! this channel is a goldmine for backend engineer
@Metruzanca
@Metruzanca 2 жыл бұрын
This is explained so well. I've love to hear you speak more about kafka. EDIT: 100% ådding that newsletter to my rss.
@amaelftah
@amaelftah 2 жыл бұрын
really this is high quality videos and lovely animations ... thanks a lot for simplifying why kafka is fast
@DevNarayan
@DevNarayan Жыл бұрын
Amazing details about frequently used software. Lucky to bump into this page. Thanks
@thalathotitharunprabhakar3390
@thalathotitharunprabhakar3390 Жыл бұрын
Thank you for the wonderful explanation of Kafkas abilities.
@sakthikumar4721
@sakthikumar4721 2 жыл бұрын
I really appreciate your work. Excellent video. Superbly Articulated. Easy to grab the concepts. Great work. 😍
@lcch12
@lcch12 Жыл бұрын
Amazing work guys! I'm subscribed to any newsletter and video you make, and it's worth it. Congratulations team 👏👏👏
@gui1221000
@gui1221000 Жыл бұрын
This is so amazing! Straight to the point!
@weiguo6805
@weiguo6805 Жыл бұрын
Greatest video series with fluenent + clear + intuiative illustration ( master-quality ##) , can not thanku enough!
@siruitao
@siruitao 2 жыл бұрын
Thanks for the useful instruction!
@NaqushabNeyazee
@NaqushabNeyazee 2 жыл бұрын
Short and Sweet! Excellent video.
@smoideen
@smoideen 10 ай бұрын
This was a clear and concise presentation. Thank you so much 👍
@nicklaspillay7923
@nicklaspillay7923 Жыл бұрын
This is an amazing video. Actually putting it out there - I LIKED AND SUBBED! Well deserved for great content 💯
@tubenzr
@tubenzr Жыл бұрын
your video is very clear and on-point Sir, thanks a lot 👍👍
@fahmidamiah
@fahmidamiah 2 жыл бұрын
Really loved this. Thank you.
@constantfear
@constantfear Жыл бұрын
Thanks, brilliant tutorial. My company are currently gearing up to adopt a data mesh architecture and It's gonna be fun moving from batch to this CDC stream methodology.
@aayushgupta1186
@aayushgupta1186 2 жыл бұрын
Amazing content! Keep posting such videos, its a great help!!!
@yaramvenkateswarluchowdary1020
@yaramvenkateswarluchowdary1020 Жыл бұрын
content is simple and crisp... thank for bringing this to us...
@mirzasohailhussain
@mirzasohailhussain Жыл бұрын
Thank u so much!!! I had this question in my mind and got explained by your in a very easy way!!!
@_rd_kocaman
@_rd_kocaman Жыл бұрын
those minimalistic graphics makes complicated topics easy to ingest. Subscribed!
@prathibavijayasekaran4173
@prathibavijayasekaran4173 Жыл бұрын
Very simple with good animation to explain things clearly. Keep publishing these kinds of useful videos.
@suman14san
@suman14san 2 жыл бұрын
Stunning. It's not abt any topic related to computer science or tech, if anyone teach me anything like this, i will skip everything and learn. Thank you for changing lives of people.
@tomok284
@tomok284 Жыл бұрын
Such a good content in just 5 minutes!
@betims
@betims 2 жыл бұрын
Amazing explanation. Thank you sir.
@amigochan
@amigochan Жыл бұрын
影片中說明兩個為什麼 Apache Kafka 能夠提供高流量傳輸大量紀錄的特性: 1. 循序 I/O 以 C 來說,當使用 fopen() 需要開啟一個檔案為 append 模式,file pointer 會直接在檔案尾端準備以新增方式繼續加入新資料,會比每次加入資料需要移動 Pointer 到特定位置再寫入來的快速。如果用硬碟的循序讀寫與隨機讀寫,會更容易理解。 在 File-based Database,例如 dBASE, COBOL + ISAM, Paradox,也是直接將新紀錄寫在檔案後方。可以用 PC-Tools 打開檔案觀察 HEX Code 確認。風險在於如果來不及寫入 EOL,沒有順利關閉檔案,就會造成檔案損毀與資料遺失。 刪除紀錄也只是在記錄上做個標記,並不會真正刪除,需要等到執行 compact database 才會真正刪除。因此我在設計需要確實刪除客戶個人資料時,會以無意義的字串覆蓋,直接刪除其實只是標記,資料還在。 2. [Zero Copy](en.wikipedia.org/wiki/Zero-copy) 避開將相同資料在不同記憶體區塊再次複製後移動,縮短傳送路徑。例如在提供 DMA 模式情況下,讓系統函數直接將讀取已經被讀入記憶體緩衝區的資料放入網卡 NIC 緩衝區開始傳送,省略 Socket Buffer 路徑。
@abdulelahaljeffery6234
@abdulelahaljeffery6234 Жыл бұрын
WOW, amazing stuff
@SurenderSingh-pj7xk
@SurenderSingh-pj7xk 2 жыл бұрын
Thanks Alex for such a great video. You are amazing !! The way you explain the stuff is pretty awesome and simple. Thank you once again for such great work.
@patrickdee7365
@patrickdee7365 Жыл бұрын
Very cool channel you keep the most important stuff compact, not everyone can do that.
@DotDager
@DotDager 10 ай бұрын
First time I actually WANT to subscribe to a newsletter.
@AungBaw
@AungBaw 2 жыл бұрын
Short & sweet. Thank you.
@fokerfakerfuker
@fokerfakerfuker 2 жыл бұрын
wow the comments are right. simple and clear... subscribed
@ayusharora8141
@ayusharora8141 28 күн бұрын
Brilliantly explained!! 👏
@kirankapadia5551
@kirankapadia5551 5 ай бұрын
Amazing! Thank you for making this video, appreciate it esp diagram explaining zero copy principle
@antirus5481
@antirus5481 2 жыл бұрын
Simple and very insightful, I like the lack of music and the use of motion graphics, helps me focus.
@safiuzkhan5463
@safiuzkhan5463 2 жыл бұрын
Very beautifully explained 👌
@ansitun
@ansitun Жыл бұрын
After going through the video and your explanation, I am decided to take a paid subscription in byte byte go! Your explanations are to the point and succinct to understand a topic ! Thank you for the video.
@TricoliciSerghei
@TricoliciSerghei 2 жыл бұрын
Very informative video, thank you so much!!
@joross8
@joross8 2 жыл бұрын
Awesome video. Looking forward to the next one.
@gopalsv5230
@gopalsv5230 Жыл бұрын
Nice intro about Kafka, learned quickly, now you got a new subscriber 👍
@DominikRoszkowski
@DominikRoszkowski 2 жыл бұрын
Nice, that was really clear explanation, thanks a lot!
@vishnu-mk
@vishnu-mk Жыл бұрын
Nice stuff, great content with simple words
@dcn4lyf
@dcn4lyf Жыл бұрын
Very insightful. The diagrams made me understand the concepts
2 жыл бұрын
Great technical explanation. I just want to add that Kafka can be used for much more than just data ingestion sending data from a data source to a data sink. The Apache Kafka open source project also includes Kafka Connect for data integration and Kafka Streams for data processing. Therefore, you can leverage the characteristics explained in this video to build a modern data flow with a single (scalable and reliable) real-time infrastructure instead of combining several different components (like Apache Kafka for ingestion, Apache Camel for data integration, and another stream processing framework like Apache Flink for real-time analytics).
@EverydayRoadster
@EverydayRoadster Жыл бұрын
Reliability of Kafka has yet to be proven. Ever so often it does not meet data integration core requirements on reliability, especially in the area of disruption and recovery, where it quickly says GoodBy to “At-most-once” semantics. Don’t get me wrong, Kafka is really great for what it is designed for: efficient streaming in BigData architecture, but that architecture will tolerate a certain fuzziness of data, which pure data integration architecture would not allow for.
@karthikarg
@karthikarg Жыл бұрын
Short and Sweet, and Deeeep.... Awesome explanation..!🔥
@jamess5330
@jamess5330 Жыл бұрын
Thank you for putting up this tutorial! Study vidoes like this and then practice at Meetapro with mock interviews will help you land multiple offers.
@TheAceEditor
@TheAceEditor Жыл бұрын
Essential collection of videos in this channel for a software developer
@kailashkolluru2398
@kailashkolluru2398 Жыл бұрын
Love this explanation!
@klausdupont6335
@klausdupont6335 2 жыл бұрын
The animation is amazing!
@GughaGSrinivasan
@GughaGSrinivasan Жыл бұрын
ASMR experience :) i have subscribed... Neat explanations... I am not curious about Kafka, but curious about the optimization techniques and strategies they have accomplished which I would like to learn... Please do more!
@geehaf
@geehaf Жыл бұрын
This is excellent. Thank you. Loved the Redis video too.
1 or 2?🐄
00:12
Kan Andrey
Рет қаралды 28 МЛН
Just try to use a cool gadget 😍
00:33
123 GO! SHORTS
Рет қаралды 85 МЛН
Пробую самое сладкое вещество во Вселенной
00:41
когда повзрослела // EVA mash
00:40
EVA mash
Рет қаралды 3,3 МЛН
1 or 2?🐄
00:12
Kan Andrey
Рет қаралды 28 МЛН