Why do we need Kafka?

Рет қаралды 40,557

11 ай бұрын

In this video, we are going to discuss why we need Kafka and why Kafka has high throughput and what is the difference between Kafka and Databases.
Kafka Crash Course: • Apache Kafka Crash Cou...
Course Links
Full Stack Twitter Clone: learn.piyushgarg.dev/learn/tw...
Docker Master Course: learn.piyushgarg.dev/learn/do...
Hashtags
#Kafka #RealTimeDataStreaming #nodejs #EventStreaming #DistributedSystems #BigData #DataStreaming #Nodejs #CodingTutorial #Developer #SoftwareEngineering #TechnologyExplained #DataProcessing #ScalableApplications #ConsumerGroups #MessagePartitions #DataDrivenApplications #OpenSource #Programming

Пікірлер: 84

@vipulsharma9094 11 ай бұрын

Such an amazing video, I should really thank yt algorithm that I came across your channel. PS:Your Full Stack Twitter Clone course is really amazing.

@goodrat7 11 ай бұрын

Your teaching skill is excellent. I have seen someone teaching and me absorbing so well after a long while. You are awesome brother

@vasutke1187 6 ай бұрын

Ji, Importance of Kafka is explained with high clarity. Similarly I considered your Video is highly important to learn Kafka. I feel Great efforts taken to bring this video is highly appreciated. Thanks & Regards

@mma-dost 11 ай бұрын

Thanks a lot bhaiya keep making these type of videos like about general software engineering scaling or anything about software development. Thanks a lot bhaiya.

@varunpandey5819 11 ай бұрын

Sir your teaching skill is so good, thanks for providing such amazing skills 👍

@anandpawar3 10 ай бұрын

Awesome video and clearly stated the purpose of Kafka and databases …keep uploading more and more informative videos 🎉 thanks😊

@Anshucodes 5 ай бұрын

Kafka, by default, uses disk storage for message persistence rather than storing messages primarily in RAM. However, Kafka does leverage RAM for caching frequently accessed data to improve performance. It maintains an in-memory cache called the page cache, which holds recently read or written data from disk. This caching mechanism helps reduce disk I/O and improves overall throughput and latency. So, while Kafka does utilize RAM for caching purposes, it primarily relies on disk storage for message persistence.

@prashlovessamosa 11 ай бұрын

Thanks for making this.

@ankit1153 11 ай бұрын

Please continue your AWS 🙏😭 series it is really helpful.. please please i have watched AWS series video and i have learnt alot not just how to use but also about serverless, linux , ssh .. please continue it.

@AdityaGupta-im9lu Ай бұрын

dil jeet liya bro

@warisshaikh9474 Ай бұрын

dher saara love from KHI, PK

@ojasaklecha 8 ай бұрын

I have a very weird chronology to understand this. Jab hamare ghar ke kitchen me apan bartan dhote hai to unko directly unki jagah pe nahi rakhte hai na, use ek cage ya tub type me rakhte hai, taki vo sukh sake and then ham unhe poch ke unki jagah pe rakh sake. So, similarly, kafka or redis is like vo tab where you just throw or put the data, which is then processed and then stored in the DB.

@sanjeevkaushal5969 3 ай бұрын

not weird, it is a perfect example in layman's term..

@sarwajeetbharti2856 Ай бұрын

Acha example h . Basically producers are producing in fast rate n consumer usse aaram se consume kar sakte h .. Retry ka facility bhi hain kafka mein for the consumers in case they were down for some reason.

@Anshucodes 5 ай бұрын

@sumantamandal7616 3 ай бұрын

well said bro

@perrythepalteypus2185 11 ай бұрын

Excellent video 👏🏻👏🏻 please do same type of explanatory videos on prisma and docker 🚀🚀

@susanoo9958 11 ай бұрын

First. Piyush sir,Big fan of your videos.Thankyou for the great work.

@dipikeshkumar1538 9 ай бұрын

Really great video❤️

@adityak5193 11 ай бұрын

Well addressed Piyush.☺

@Digitalbyfaiyaz 11 ай бұрын

Excellent explanation 😀

@gopalsavaliya3205 11 ай бұрын

sir plz , make playlist on postgresql database

@sanketgawande3667 11 ай бұрын

@DevOps_Zone 10 ай бұрын

@pieceofcode_ 9 ай бұрын

kzfaq.info/sun/PLuQ7pgFOWEA-6RmU3pgW-K6Yyu3ZlOXbC&si=pRy7iA_kMgsNwjuy

@gaganyt7861 9 ай бұрын

@saksham_1612 7 ай бұрын

@FirstnameLastname-cl4op 11 ай бұрын

Hi Piyush, I like the quality of your videos very much. Can you list all the technologies required to learn for a full-stack developer to land a remote job?

@ShivaniSinghYadav-sm3ee 4 ай бұрын

Great explanation

@baljitsinghchahal4058 11 ай бұрын

We like your videos 👍 .... From Ludhiana

@rizwansworld 2 ай бұрын

Thanks man!

@naruto5437 9 ай бұрын

excellent video

@shailendravishwakarma8367 11 ай бұрын

Well explained

@jignalpatel2143 Ай бұрын

Good explanation ❤

@FUNNEDUCATION 9 ай бұрын

It's an amazing video I enjoyed this video and the last video on Kafka. Please one crash course on RabbitMQ. Pls

@saikatdas6126 3 ай бұрын

@piyush I think there is some mistake in your statement. Kafka is not relying in memory storage. It stores the data in the disk. Kafka works faster mainly due to sequential read write and optimised disk reading capability.

@Proxy-oh3ic Ай бұрын

Great🎉😊

@zaid_khan1 11 ай бұрын

Piyush bhai, Please, can you make a detailed video on Docker Deep Dive and Kubernetes?

@FaizSultan430 11 ай бұрын

Hy piyoush Hope u r doing well please make a crash course on prisma js

@shivamsingh7219 11 ай бұрын

It's like cache,ram and physical memory we are using all features by balancing cost and efficiency.

@DeepakSharma-st6dc 8 күн бұрын

noice!!

@propubggamer2222 11 ай бұрын

Recenty got recommend to your channel. And i want to suggest you should compile the videos in one shot as i assume you will get more views, i think you should try to re-upload node playlist to one video without deleting playlist.

@narayanmaity5881 10 ай бұрын

what is the diffference between kafka and redis? Do we have to use both to make a scalable application?

@ashraf7706 11 ай бұрын

please make a video for iron-session in next js authentication

@shivamdashore6864 3 ай бұрын

Hi Piyush! Such a great explaination can you please make a video on LLD Design Patterns

@animesh414 Ай бұрын

are we you use database and Database management system interchangeably here.

@RohitSingh-se6ch 2 ай бұрын

Can async await resolve this issue of throughput in node?

@raiyanshaikh4327 11 ай бұрын

Hello sir, can you please make a video on Apollo federation and subgraph ?

@sarvesh_k 11 ай бұрын

how you record your vedio, i am on mac

@sujan8353 8 ай бұрын

Can u give a brief idea on notification service. How Kafka is useful here. With real time scenarios. Also if possible can create one demo service of notification. How will be the data flow.

@PramodSharma-dq3pi 2 ай бұрын

Please make a video/playlist on Flutter open source framework development.

@theibad 2 ай бұрын

Make video on Kafka vs Redis ,why we need them, Is Kafka alternative of redis?

@ieltsmaverickcracktheielts3384 5 ай бұрын

Hi Piyush, Could you make one detail video on SnowFlake?

@Error-cf2mj 11 ай бұрын

sir please aap ek MERN based project pe video laiye....it would be be really helpful...you explain even hard concepts very easily

@CoderBB 11 ай бұрын

Hey piyush, What if kafka server re-start? Then will the data persisted ? As you told that kafka gets High throughput because it uses primary memory for the operations. But if the system restarts then what happens to the unprocessed data which was stored in kafka before re-start ? Btw nice video 😊

@akash-kumar737 10 ай бұрын

I think he is wrong when you say Kafka is fast because it uses RAM but according to ByteByteGo video. It is due to sequential I/O and zero copy technique. kzfaq.info/get/bejne/i7SFrZRe0tnGf4E.htmlfeature=shared

@agcolony123 7 ай бұрын

Hi Piyush , what happens for in memory databases??? Is Kafka valid for them

@_sanskriti903 14 күн бұрын

It's like view concept used in database table

@shishiragrahari1855 11 ай бұрын

Mann can you make tutorial on kafka connect, having a lot of doubts.

@chandanmohapatra6782 11 ай бұрын

Bhaiya plz complete the AWS playlist ..we all are waiting bhaiya..

@noumanmunir3166 5 күн бұрын

You said that while inserting data, db can overwhelm and go down,then the data will be lost,that's why we keep our data in Kafka(or to say the Piplines I think), But Kafka can loss the data as well if the server goes down...

@shubhamgupta-bl1tr 11 ай бұрын

Sir please come with node js project with Kafca and serverless aws

@shaqeeb.shaikh2400 2 ай бұрын

got subscriber

@dreamhunder341 8 ай бұрын

Do i need to use kafka in every api request? Because my application has lots of users. Pls explain

@pranavwani 11 ай бұрын

@piyushgargdev should we learn kafka?, as alternatives available (still it is good for large content and stability but just want to know your poin of views)

@Support-Phalestine 11 ай бұрын

Bro plzz next video ma ya btay k aws skill builder website ma sa koi ek esa course btady Jo deployment k lia ho usma buhat sary course ha samagh ni a Raha kunsa kro

@asynchronous.1 11 ай бұрын

Hii Piyush Bhaiya, you have said that you will prepare a playlist on aws lambda using serverless. So, please make a playlist in which you cover a project using express.js and mongoDB and its deployment on lambda and S3. Please bhaiya, make a playlist on serverless. It will be very beneficial.

@akash-kumar737 10 ай бұрын