Apache Kafka in 6 minutes

  Рет қаралды 928,777

James Cutajar

5 жыл бұрын

A quick introduction to how Apache Kafka works and differs from other messaging systems using an example application. In this video I explain partitioning, consumer offsets, replication and many other concepts found in Kafka.
Please support me through my Udemy courses:
Pass your coding interview in
Java : www.udemy.com/course/beat-the-codility-coding-interview-in-java/?referralCode=FF8ECC71208BD571CE81
Python: www.udemy.com/course/beat-the-codility-coding-interview-in-python/?referralCode=0F7DC2EE6CDED3193CF2
Ruby: www.udemy.com/course/beat-the-codility-coding-interview-in-ruby/?referralCode=516125F3BD830B1500FC
JavaScript: www.udemy.com/course/beat-the-codility-coding-interview-in-javascript/?referralCode=D3D4690B37439960F58E
Learn Dynamic Programming in,
Java: www.udemy.com/course/dynamic-programming-java/?referralCode=2BB4EA66048238BB9F12
Python: www.udemy.com/course/dynamic-programming-python/?referralCode=4C2450B7471149BE7BD2
Ruby: www.udemy.com/course/dynamic-programming-ruby/?referralCode=ECC1ED1041DDD5E32F90
Multithreading in,
Go Lang: www.udemy.com/course/multithreading-in-go-lang/?referralCode=D20A3CBD00E90DB2ABF8
Python: www.udemy.com/course/parallel-computing-in-python/?referralCode=6FEC1503E429D7510154
Java: www.udemy.com/course/master-parallel-computing-and-multithreading-in-java/?referralCode=06D0A16CF517339CD181
Book on Kafka (Manning publications):
www.manning.com/books/kafka-in-action?a_aid=cutajarj&a_bid=b93db6bc
Blog: www.cutajarjames.com

Пікірлер: 255
@georgelza
@georgelza 4 жыл бұрын
Would love to see more Kafka subjects communicated this way, awesome to give to a team to get basic concepts down.
@renatoiwashima3890
@renatoiwashima3890 3 жыл бұрын
Finally a tutorial using examples instead of abstract concepts. Thanks!
@evisionwithjps602
@evisionwithjps602 2 жыл бұрын
New kafka practical tutorial ..kzfaq.info/get/bejne/o9J4Y9xoq7XZm4k.html
@ravisemwal5363
@ravisemwal5363 4 жыл бұрын
This is exactly how an introduction to a tool should be made. Thanks James, liked and subscribed! I've seen a lot of these videos about user organisations and purpose, but not the exact problem they solved using this.
@bstancel12
@bstancel12 4 жыл бұрын
Wow! You created a great video - informative and concise. It gives enough technical details, but not too many in a short amount of time. Rarely do you see technical videos this well made on KZfaq. Thank you.
@chrise202
@chrise202 2 жыл бұрын
What on earth was that? The most useful introduction I encountered to watch. Not only it addresses a lot of technical Q&As but also does it in a visual, perceptive and intuitive manner!
@giorgi23
@giorgi23 2 жыл бұрын
What an explanation. In 6 minutes so many details and so clear. This requires real talent
@asawari520
@asawari520 2 жыл бұрын
I was so struggling to understand why Kafka and this is it! to the point and crisp clear explanation . Thankyou !
@swarupdas5147
@swarupdas5147 2 жыл бұрын
Hours of Kafka technology within 6 mins. And that too crystal clear. Hats off.
@Sunshine-sv6lw
@Sunshine-sv6lw 6 ай бұрын
Only a person who has deep knowledge on a subject can explain anything with clarity in a short video. Looking forward to more videos from you!
@SaurabhP-gm3bt
@SaurabhP-gm3bt 4 жыл бұрын
James, this is a great video. It goes to enough technical details and a perfect first video on Kafka. It should have a million views, really. Please keep posting and sharing your knowledge. Thank you.
@evisionwithjps602
@evisionwithjps602 2 жыл бұрын
New kafka practical tutorial ..kzfaq.info/get/bejne/o9J4Y9xoq7XZm4k.html
@python2448
@python2448 2 жыл бұрын
You are right 👍
@zhengqingpei7136
@zhengqingpei7136 8 ай бұрын
I love how you don't just jump into the current implementation, but rather, goes through the thought process of starting out with single server setup, addressing issues, and then leading up to how Kafka handles it. Nice video. The videos help, the explanations are crystal clear and detailed too.
@vinayreddy1103
@vinayreddy1103 9 ай бұрын
This 6 minutes made my day! What a crisp explanation. No bullshit straight and on point.
@surbhiagarwal2322
@surbhiagarwal2322 Жыл бұрын
This is the best video hands down! I can see why Kafka is widely preferred because it's amazing distributed and fault tolerant architecture
@moveonvillain1080
@moveonvillain1080 2 күн бұрын
Idk how many videos I went over but not a single one would say that partition in Kafka is JUST A QUEUE. They would unnecessarily make it complicated with more technical jargon. Thank You good Sir🧔‍♂👋🎩
@pankajpundir3410
@pankajpundir3410 2 жыл бұрын
Hours of knowledge, juiced up in 6 minutes. Amazing video
@dhruva1221
@dhruva1221 4 жыл бұрын
extremely helpful to cover the base terms & workflow!!! brilliant piece! :D Thanks James
@coolplayer2512
@coolplayer2512 3 жыл бұрын
Very Helpful, explained in a very cleaned way. Thanks James for sharing this.
@shlomokon
@shlomokon Жыл бұрын
What a great video! Perfect balance between an illustrating example and technical depth. Easy to follow along. I tried another video before, with which started out talking about "source systems" and "target systems", never explaining what they mean by those, without any illustrating example, and that video lost me right away. In contrast, yours easily made sense from the first second and was easy to follow along as you kept adding depth incrementally.
@jayeshborgaonkar9166
@jayeshborgaonkar9166 4 жыл бұрын
Hands down the best video introduction to understanding kafka in 6 mins, liked and subscribed
@essskkk
@essskkk 2 жыл бұрын
One of the best video to learn about Kafka basics … Simple and very clear , Thanks a lot
@ImranChaudhry
@ImranChaudhry 9 ай бұрын
Straightforward explanation in six minutes, thank you.
@zacharyohare2118
@zacharyohare2118 8 ай бұрын
This is THE way to teach concepts as an introduction... I know because I already have questions about limitations and applications, and I'm thinking of a lot of other services that use something similar or actually are using K somewhere underneath; services I didn't REALLY understand, and now make much more sense. Thank you.
@pratikshapanwar1552
@pratikshapanwar1552 3 жыл бұрын
Please make more such videos. Your content is good. Thank you for bringing this. The video is to the point.
@alimomeni4777
@alimomeni4777 Жыл бұрын
James, really liked this tutorial! Concise with fantastic illustrations. Bravo
@shanemay-gunlogson9575
@shanemay-gunlogson9575 4 ай бұрын
Best intro to Kafka video I've seen. Thank you.
@TheDQR
@TheDQR Жыл бұрын
Excellent video, great explanation and visual representation. Amazing job!
@prakharlondhe3876
@prakharlondhe3876 2 жыл бұрын
Simple, clear and concise.. That's how teaching should be.. Kudos!
@fieryscorpion
@fieryscorpion Жыл бұрын
It started off clear, but got very confusing after half way.
@27sosite73
@27sosite73 8 ай бұрын
@@fieryscorpion same feeling but it seems like I am too stupid for now to understand the second part. this might be the reason
@mosalman5174
@mosalman5174 6 ай бұрын
Simple clear and concise. Thank you James
@ravitejapenki
@ravitejapenki 3 жыл бұрын
Very well done man, simplified demonstration
@1921Designs
@1921Designs 10 ай бұрын
Thorough Notes Apache Kafka is a solution to the problems faced by the distribution and scaling of messaging systems. Kafka is used to design a system that listens to various basketball game updates from various sources and displays the game's status on various channels such as mobile devices and computer browsers. In the architecture, a process called a producer reads these updates and writes them in a queue. Downstream processes called consumers consume these updates to display them on the various channels. As the system expands to follow more games, the servers struggle to handle the load due to the queue being hosted on one server which is running out of memory and processing capacity. To add more computing power, the architecture needs to be distributed. However, distributing a queue data structure can lead to consuming updates in the wrong order, resulting in inconsistencies. Kafka allows the application to specify the way to distribute the items in the queue. In the example, items are distributed using the match name, ensuring that updates from the same match are on the same queue, maintaining an ordering per basketball match. Each queue in Kafka is called a partition, and the total number of partitions is called a partition count. Each server holding one or more of these partitions is called a broker. Each item in a partition is called a record. The field used to decide which partition the record should be stored in is called the partition key. If no key is specified, Kafka assigns a random partition. A grouping of partitions handling the same type of data is called a topic. Each record in a topic is identified by a partition number and an offset. Kafka allows for parallelizing consumer applications, with one consumer per partition guaranteeing ordering per game. Consumers can live on one machine or distributed amongst multiple ones. Consumers in Kafka are lightweight and many can be created without affecting performance. Kafka only needs to maintain the latest offsets read by each consumer. Consumers belonging to the same consumer group do not share partitions, meaning each consumer reads different records from the other consumers. Kafka provides various policies that allow it to do a record cleanup. For example, using a retention policy, you can provide a record age limit after which the records are automatically deleted. Kafka stores records in a fault-tolerant and durable way. Each record is stored on persistent storage so that if a broker goes down, it can recover when it comes back up. Kafka replicates partitions so that when a broker goes down, a backup partition takes over and processing can resume. This replication is configured using a replication factor.
@sephyshen2510
@sephyshen2510 2 жыл бұрын
Great video. Although I never use Kafka before, it is very clear for me to understand the its mechanism under the hood. Thank you.
@suvamgupta2914
@suvamgupta2914 11 ай бұрын
Your way of explanation is Outstanding!! Thanks and keep the good work flowing
@user-uw4ib6jt3m
@user-uw4ib6jt3m 7 ай бұрын
Thank you for the detailed example with the matches and the visualization, it really helps to understand the topic intuitively. Keep it up, James!
@RamakrishnanParameswaran
@RamakrishnanParameswaran 2 жыл бұрын
This is the fantastic video I ever watched on Kafka understanding.
@contactsrividya
@contactsrividya 2 жыл бұрын
Wow this is a great video ! I am about to take couple of virtual onsite interviews for data engineer role and looking for a short video to explain Kafa.. This is the best video !! Thanks
@Shruti_starlight
@Shruti_starlight 3 жыл бұрын
Watched many videos on Kafka but this is what i was looking for. Thanks for the simple yet effective tutorial for beginners. Loved it.
@George-mk7lp
@George-mk7lp 2 жыл бұрын
Shruti
@sriramakrishnabattu3408
@sriramakrishnabattu3408 2 жыл бұрын
This is a perfect video explaining the core concepts of Kafka
@shibu9389
@shibu9389 2 жыл бұрын
Awesome video! Great intro with just the right amount of information.
@rajaramarumugam3878
@rajaramarumugam3878 Жыл бұрын
Excellent Video, in just 6 mins you have covered core concepts of the Apache kafka.
@edytarciopereira3461
@edytarciopereira3461 4 жыл бұрын
Excellent explanation! I just felt frustrated for not finding a Kafka course on your list!
@nenapse7750
@nenapse7750 3 жыл бұрын
Really very simple, and complete, description of Kafka. Thx!. Subscribed
@rittikaghatak7318
@rittikaghatak7318 2 жыл бұрын
Thanks, James... Please post more videos. Precise and clear
@itgurukul
@itgurukul 2 жыл бұрын
One of the best explanation of KAFKA to understand simply
@prathibavijayasekaran4173
@prathibavijayasekaran4173 Жыл бұрын
Best and easy summary on Kafka. Pls keep posting useful videos like this.
@rajeevmahajan9227
@rajeevmahajan9227 2 жыл бұрын
This is the best explanation I have found on internet on the topic many thanks :)
@creaturerohit
@creaturerohit Жыл бұрын
loved it ! simply explained , to the point .Thankyou
@deepusun
@deepusun 4 жыл бұрын
James very simple and easy to understand
@travelchimps6637
@travelchimps6637 Жыл бұрын
Amazing example, great graphics and very neat explanation. Thank u for the video!
@dishaponkiya1836
@dishaponkiya1836 3 жыл бұрын
the basic terms explained very very well.. Thank you.
@bikashkarmokar
@bikashkarmokar 2 жыл бұрын
Best kafka intro so far. great video. thanks a lot
@amrutnandedkar2840
@amrutnandedkar2840 2 жыл бұрын
Truly Exceptional ! Thanks for making this.
@sachinnaik4849
@sachinnaik4849 2 жыл бұрын
Awesome video. Simple and super clear!!
@yogeshkurane123
@yogeshkurane123 9 ай бұрын
❤ Wohoooooo! Simplicity at its peak. Thanks for explaining Kafka in a simplistic way.
@BryanChance
@BryanChance Жыл бұрын
Really concise overview of Kafka. Thank you.
@glui2001
@glui2001 Жыл бұрын
Straightforward and intuitive video. Thx!!!
@pranavbhat92
@pranavbhat92 Жыл бұрын
Thank you for explaining this with an example! 😊
@manju4ever222
@manju4ever222 Жыл бұрын
Amazing. You just reduced 50 hours of research for me ❤
@Vinod_Kumar827
@Vinod_Kumar827 2 жыл бұрын
You just nailed it very well. Thanks for the great explanation. Indeed, it was useful for me :)
@deepakrai5175
@deepakrai5175 Жыл бұрын
you explained it so nicely and in a simple way.
@kshow666
@kshow666 2 ай бұрын
Great intro to the fundamentals! Great use of example too
@AvinashThombrethe_leo
@AvinashThombrethe_leo Жыл бұрын
So greatly explained! Really helpful.
@abhinavtripathi970
@abhinavtripathi970 4 жыл бұрын
I loved this video....nice presentation..........super simple to understand
@franziskakuesters1091
@franziskakuesters1091 4 жыл бұрын
great video. your animations really supported what you said.
@yulia0g
@yulia0g 11 ай бұрын
such a great video with great explanations and examples. learned a lot!! thanks!!!!
@alishiram
@alishiram 2 жыл бұрын
Great introductory video! Thanks!
@mandeepbhogal7502
@mandeepbhogal7502 Жыл бұрын
Clear explanation in 6 minutes...this is the best video to understand how Kafka works and what it solves...thanks...Can you also make a video comparing Kafka with JMS queues?
@DeepakSingh-ji3zo
@DeepakSingh-ji3zo Жыл бұрын
Best explanation under 7 minutes!!
@tabliqatchi6696
@tabliqatchi6696 Жыл бұрын
Really amazing video, I've subscribed without thinking about it.
@HarpreetSingh-xr6em
@HarpreetSingh-xr6em Жыл бұрын
Great and very good summary of Kaffka in small amount of time
@mikahoy
@mikahoy 6 ай бұрын
view a lot of videos about kafka, and you 're video is the best one
@GeorgeZoto
@GeorgeZoto Жыл бұрын
Very well designed and created content, thank you for sharing :)
@ketanjoshi4568
@ketanjoshi4568 2 ай бұрын
Love the brevity of the video !
@nagasai6668
@nagasai6668 11 ай бұрын
Great explanation Sir 🙏from India 🇮🇳..will follow your classes from now❤
@A.n.a.n.d.
@A.n.a.n.d. 2 жыл бұрын
Amazing "nutshell" intro to kafka. thanks.
@pepper856
@pepper856 14 күн бұрын
extremely helpful! Thanks James!
@brothermalcolm
@brothermalcolm Жыл бұрын
this is way better than the other kafka in 5mins video on youtube
@user-nx1jc7wq8n
@user-nx1jc7wq8n 7 ай бұрын
Looking for more content like this. Thanks!
@vitalyv301
@vitalyv301 9 ай бұрын
Thanks a lot for the succinct introduction!
@Raadha_Ji
@Raadha_Ji Жыл бұрын
James what a Content with everything ..Thanks !! keep uploading
@griglog1309
@griglog1309 6 ай бұрын
Its crazy how good this video is
@udit2114
@udit2114 Жыл бұрын
Amazing video man, very helpful !! Thanks for this .
@rameshsoundarrajan3831
@rameshsoundarrajan3831 Жыл бұрын
What a fantastic video... Made very very simple. 4:25 was little tough to understand. all others were made very very simple and easy to understand. You are amazing
@shimesami
@shimesami 2 жыл бұрын
This is an amazing recap! Thanks
@jayjaayjaaay94
@jayjaayjaaay94 Жыл бұрын
the most clear explanation of kafka I ve seen
@arona2211
@arona2211 Жыл бұрын
AGREE,
@andydataguy
@andydataguy Жыл бұрын
This video was amazing. New subscriber!
@smoideen
@smoideen 8 ай бұрын
Thank you so much for the this video. The diagrams really helped me understand this better.
@insmallportions
@insmallportions 3 жыл бұрын
Excellent video, don't know how you did it, but you included all the core concepts in under 7 minutes. Great introduction or review for someone like me that is a bit rusty on the topic.
@evisionwithjps602
@evisionwithjps602 2 жыл бұрын
New kafka practical tutorial ..kzfaq.info/get/bejne/o9J4Y9xoq7XZm4k.html
@ashwinmarathe9844
@ashwinmarathe9844 9 ай бұрын
Very well explained. keep up the good work. Your videos will help a lot.
@saurabhsharma6209
@saurabhsharma6209 3 жыл бұрын
As Einstein said - 'If you can't explain it simply, you don't understand it well enough.' Perfect explaination for a beginner. Thanks much.
@evisionwithjps602
@evisionwithjps602 2 жыл бұрын
New kafka practical tutorial ..kzfaq.info/get/bejne/o9J4Y9xoq7XZm4k.html
@sairavib2865
@sairavib2865 2 жыл бұрын
Wonderful video !! Thank you for detailed info
@davidguardado8282
@davidguardado8282 Жыл бұрын
The concepts are explained briefly but clearly. You have done in 6 minutes what most other "gurus" on yt are not able to do in 30 minutes. They go on talking and talking without really being able to hit the keys, probably because they don't themselves understand it.
@vijaygavandar1956
@vijaygavandar1956 2 жыл бұрын
Thank you so much James. Very well Delivered .....!!!!
@lsmeteor4652
@lsmeteor4652 4 жыл бұрын
Great video. Thank you for the clear explanation. This is very similar to RabbitMQ. How does it compare?
@NikhilThota-ry5pn
@NikhilThota-ry5pn 7 ай бұрын
100% Perfect introduction to Kafka
@tungle6068
@tungle6068 2 жыл бұрын
Sir, this video is just awesome !!
@mhadamannavi595
@mhadamannavi595 4 жыл бұрын
@James - great video. Thank you!
@sruthiravi8464
@sruthiravi8464 Жыл бұрын
Wow.. thanks for the clear explanation ❤
@charlesopuoro5295
@charlesopuoro5295 Жыл бұрын
James, thank you so much for this video. I love Basketball too and so it particularly appealed to me.
@isabellamendozacortes2186
@isabellamendozacortes2186 3 жыл бұрын
that was an amazing sum up!
@jamesgrant5088
@jamesgrant5088 3 жыл бұрын
Simple, clear and concise.
@evisionwithjps602
@evisionwithjps602 2 жыл бұрын
New kafka practical tutorial ..kzfaq.info/get/bejne/o9J4Y9xoq7XZm4k.html
路飞关冰箱怎么关不上#海贼王 #路飞
00:12
路飞与唐舞桐
Рет қаралды 5 МЛН
INO IS A KIND ALIEN😂
00:45
INO
Рет қаралды 23 МЛН
когда одна дома // EVA mash
00:51
EVA mash
Рет қаралды 11 МЛН
路飞关冰箱怎么关不上#海贼王 #路飞
00:12
路飞与唐舞桐
Рет қаралды 5 МЛН