Kafka Implementation with Producer and Consumer example in Spring boot | tutorial

Kafka Implementation with Producer and Consumer example in Spring boot | tutorial | Code Decode

Рет қаралды 93,306

Жыл бұрын

In this video of kafka implementation with producer and consumer example by code decode we have explained
Udemy Course of Code Decode on Microservice k8s AWS CICD link:
openinapp.co/udemycourse
Course Description Video :
yt.openinapp.co/dmjvd
Introduction to kafka : • Kafka tutorial for Beg...
Kafka Architecture : • Apache Kafka Key Termi...
Github link : github.com/codedecode25/Kafka
Link to download Kafka : kafka.apache.org/downloads.html
Commands to start Zookeeper and Kafka :
.\bin\windows\zookeeper-server-start.bat .\config\zookeeper.properties
.\bin\windows\kafka-server-start.bat .\config\server.properties
Kafka implementation with producer and consumer is very important topic with interview point of view.
What is Kafka
Apache Kafka is publish-subscribe based fault tolerant messaging system. It is fast, scalable and distributed by design.
It was initially thought of as a message queue and open-sourced by LinkedIn in 2011. Its community evolved Kafka to provide key capabilities:
Publish and Subscribe to streams of records, like a message queue.
Storage system so messages can be consumed asynchronously. Kafka writes data to a scalable disk structure and replicates for fault-tolerance. Producers can wait for write acknowledgments.
Stream processing with Kafka Streams API, enables complex aggregations or joins of input streams onto an output stream of processed data.
Traditional messaging models are queue and publish-subscribe. In a queue, each record goes to one consumer. In publish-subscribe, the record is received by all consumers.
Pros of Kafka
Loose coupling - Neither service knows about each other regarding data update matters.
Durability - Guarantees that the message will be delivered even if the consumer service is down. Whenever the consumer gets up again, all messages will be there.
Scalability - Since the messages get stored in a bucket, there is no need to wait for responses. We create asynchronous communication between all services.
Flexibility - The sender of a message has no idea who is going to consume it. Meaning you can easily add new consumers (new functionality) with less work.
Cons of Kafka
Semantics - The developer needs to have a deep understanding of the message flow as its strict requirements. Complex fallback approaches may take place.
Message Visibility - You must track all those messages to allow you to debug whenever a problem occurs. Correlation IDs may be an option.
Most Asked Core Java Interview Questions and Answers : • Core Java frequently a...
Advance Java Interview Questions and Answers : • Advance Java Interview...
Java 8 Interview Questions and Answers : • Java 8 Interview Quest...
Hibernate Interview Questions and Answers : • Hibernate Interview Qu...
Spring Boot Interview Questions and Answers : • Advance Java Interview...
Angular Playlist : • Angular Course Introdu...
SQL Playlist : • SQL Interview Question...
GIT : • GIT
Subscriber and Follow Code Decode
Subscriber Code Decode : kzfaq.info?...
LinkedIn : / codedecodeyoutube
Instagram : / codedecode25
#kafka #codedecode #kakfainterviewquestions

Пікірлер: 175

@kanishkkumar8272 Жыл бұрын

Thankyou so so so much for this. I saw this video just before my interview and cracked the interview. Kafka was asked for 20 minutes. Really, thanks a lot. Got a hike of 160 percent . Interviewer was so impressed by the answers. Thank you once again. Really appreciate it. Please never stop these.

@CodeDecode Жыл бұрын

Many congratulations Kanishk. Well deserved Man 👍👍🎊🎊🎊 keep learning keep shining👏👏

@syedaqib2912 Жыл бұрын

Congrats man !!

@prerna3404 Жыл бұрын

Which company?

@joshinim.n9863 Жыл бұрын

Thank you so much for this series of videos. I feel so grateful for it. This is so useful. You're really imparting great knowledge in people!

@CodeDecode Жыл бұрын

Thanks joshini for the nice words

@jayjoshi1366 Жыл бұрын

It is always worth learning from your videos and this Kafka implementation was really nice one which is even more than MQ. Kindly create next part as well where we can learn about asynchronous communication between multiple micro services. Thanks!

@GauravSharma-up9gs Жыл бұрын

Thanks for simplifying everything at one place. One day you must go famous for your videos in your playlist of your channel.

@CodeDecode Жыл бұрын

🙏🙏thanks a lot for these beautiful words Gaurav. These comments means a lot to us. It keeps us motivated to upload more content 🙏🙏🙂

@fakruu Жыл бұрын

The best video beginner friendly video on kafka + spring boot !

@CodeDecode Жыл бұрын

Thanks

@ankitgiri6874 11 ай бұрын

Thanks you so much for all of your videos, The way you explain, every concept becomes very easy to understand.

@CodeDecode 11 ай бұрын

Thanks Ankit 😊👍

@user-sapdev 11 ай бұрын

Thank you so much for for your great efforts and time for getting this content in a simple and clear way. It would be great and useful if you can continue the series..advanced topics like kafka stream API as well.. Thanks once again for your continuous efforts

@pranjalsingh1099 Жыл бұрын

Thanks for up to the point explanation. Please continue this series to for explanation of individual components.

@CodeDecode Жыл бұрын

Thanks Pranjal. Sure 👍🙂

@PankhuriSinghChauhan-vr9se 8 ай бұрын

Loved the explanantion.. clear and crisp. Thank you :)

@CodeDecode 8 ай бұрын

You're welcome

@aishurajas2507 2 ай бұрын

Thanks alot to you and your fabulous content for java developers! God bless you

@johnnybreathe Жыл бұрын

Thanks for the implementation. I have done synchronous microservices till now. It's totally new concept to learn asynchronous also. 🧑‍💻 Thank you so much. And please continue what ever is important these days and in coming future as a java developer. 🤗 TQ for the videos

@CodeDecode Жыл бұрын

Thanks a lot 🙂🙂 sure we will cover grpc next soon

@geethufrancis6988 Жыл бұрын

@@CodeDecode thank uu

@sourabhjinde8191 Жыл бұрын

Thanks for the detailed video 🙏. Please continue with the architecture video

@CodeDecode Жыл бұрын

Sure thanks Sourabh 🙂👍

@PranitKothari Жыл бұрын

Wow! Awesome. This can't be explained any better, this can't be explained easier.

@CodeDecode Жыл бұрын

Thanks a lot Pranit 🙂🙂

@sheiksahil7209 Жыл бұрын

Great video...loved the way you explained .... Please make a series on Event-Driven Microservices concepts also. Thank you in Advance.

@CodeDecode Жыл бұрын

Sure Sahil. 👍🙂 thanks for the nice suggestion

@aman_deep21 11 ай бұрын

Very nice demo for getting started with Kafka, thank you

@CodeDecode 10 ай бұрын

you're welcome aman deep

@AshishYadav-se4db Жыл бұрын

Thank you so much for creating Kafka video 😊,and explanation is amazing 🔥.

@CodeDecode Жыл бұрын

Thanks Ashish 🙂👍

@user-ez2jc4uz2f 3 ай бұрын

This was very helping in completing my assignments. Looking forward for more basic video. Thanks much.

@CodeDecode 3 ай бұрын

Sure we will create more soon

@tanveersyed1049 Жыл бұрын

Thank u so much for uploading one more kafka video

@CodeDecode Жыл бұрын

🙂🙂👍👍

@nitinvnk3975 Ай бұрын

Simple, Crisp and to the point thank you for this content.

@CodeDecode Ай бұрын

You’re welcome

@dipaliyadav-koli1060 Жыл бұрын

Thanks for your hard work on this.. As usual best explanation.. ❤️❤️❤️

@CodeDecode Жыл бұрын

Thanks Dipali ❤

@ithisrinu9593 7 ай бұрын

you are best among all you tuber channels tq madam

@CodeDecode 7 ай бұрын

😊❤️🙏🙏

@aayanksinghai8242 Жыл бұрын

Very Well Explained! Thank you

@CodeDecode Жыл бұрын

🙂👍

@kasperamkumar5489 Жыл бұрын

I didn't went through this video bcoz your videos always rockzz I will go through all your video in a few days. All the best 👍

@CodeDecode Жыл бұрын

Hehe sure 👍👍 keep learning keep shining Man👍👍👍👍

@shrutik1863 Жыл бұрын

Thanks for the video. Please continue series

@CodeDecode Жыл бұрын

Sure Thanks Shruti👍🙂

@rishisharma3569 Жыл бұрын

Please please please continue this series. I have subscribed and liked and shared.

@CodeDecode Жыл бұрын

Thanks Rishi. 🙂sure we will continue this series

@surpriser2292 3 ай бұрын

Can you please continue the series and provide more information regarding the Kafka Architecture. Kudos for all the efforts in compiling such great video.

@sagarsonawane3888 Жыл бұрын

Thanks, very helpful video to understand kafka messaging

@CodeDecode Жыл бұрын

You're welcome Sagar

@vengateshm2122 Жыл бұрын

Very informative hands on tutorial.

@CodeDecode Жыл бұрын

Thanx 🙂👍

@umamaheswarreddy9204 Жыл бұрын

i loved your energy , TQ for the video. 😊

@CodeDecode Жыл бұрын

Thanks Uma 🙂👍

@sreeramsarath5188 Жыл бұрын

It's very good and way of telling so .. easy to understand.

@CodeDecode Жыл бұрын

Thanks Sreeram🙂

@akashsaha9366 Жыл бұрын

Thanks maam, for the great explanation

@CodeDecode Жыл бұрын

Thanks Akash 🙂🙂👍👍

@lohir446 Жыл бұрын

Very useful concept 👌 keep doing

@CodeDecode Жыл бұрын

Thanks a lot ☺ And thanks for the previous comment too 😊

@puneetgahlot3556 Жыл бұрын

Superb , Wooww amazing.

@CodeDecode Жыл бұрын

Thanks 🙂🙂

@bpani Ай бұрын

Great session mam, thank you!

@CodeDecode Ай бұрын

Most welcome!

@charanm4871 11 ай бұрын

Thank you so much, this video helps a lot.

@CodeDecode 11 ай бұрын

Thanks 😊😊👍

@bhaumikpatel5932 Жыл бұрын

Thanks for explaining apache Kafka. Please upload more videos on kafka streams and interview questions

@CodeDecode Жыл бұрын

sure bhaumik we will create it soon

@ajitpeshane8311 3 ай бұрын

very simplified and working poc on kafka, Thank you so much. In my case i struggled while executing the command @ 12.06 minutes error: The input line is too long. The syntax of the command is incorrect. Solution: I have renamed the folder to kafka and moved that to D drive directly. worked for me. Might help others.

@cheshvikaraj7769 Жыл бұрын

nice Explanation. Thank you

@CodeDecode Жыл бұрын

😊👍

@Sedhu_17 Жыл бұрын

Yes mam we need kafka streams tutorial

@siddharthpandey835 Жыл бұрын

Super! continue more videos on Kafka.

@CodeDecode Жыл бұрын

Sure Siddharth 👍👍

@ajithkumar-xo5gj Жыл бұрын

Great video !

@CodeDecode Жыл бұрын

Thanks!

@RishavKumar28 Жыл бұрын

Very well explained!! Please make videos on streams too. Thanks.

@CodeDecode Жыл бұрын

sure we will create it soon

@siddhilalabobo3597 Жыл бұрын

Excellent video for beginners,

@CodeDecode Жыл бұрын

thanks siddhi

@gudiya1491 Жыл бұрын

You are a very good teacher i would say 😊, really..very well explained i was struggling with this kafka topic from so long thanks for providing this video.. earlier i watched exception handler ,i understood everything in one go , didn't watch any other videos to understand it better ...can you please provide videos on bamboo and veracode...also aws🙏

@CodeDecode Жыл бұрын

Sure Thanks Gudiya 👍👍

@user-cu5pf4yo8c 5 ай бұрын

Great content covered and Im ready to work my Jira story

@CodeDecode 4 ай бұрын

Thanks

@mukulmangla632 Жыл бұрын

nice video.. please continue this series. Also can you talk about offset and streams and taking a more complex and practical example

@CodeDecode Жыл бұрын

True. Real beauty of kafka is in streams 👍

@mukulmangla632 Жыл бұрын

yes but can also explain offset please

@unemployedcse3514 7 ай бұрын

Awesome 😍

@CodeDecode 7 ай бұрын

Thanks

@chetankhandave1072 Жыл бұрын

It's my humble request, please make a playlist for Apache Kafka, so that we can go through each video one by one. It will be very easy for every one. And again you have done a fantastic job. Thank you so much.

@CodeDecode Жыл бұрын

We have one Chetan : kzfaq.info/sun/PLyHJZXNdCXset6tBB1aY5aEY77YhbxWLo

@CodeDecode Жыл бұрын

We will add more videos to this playlist 👍

@chetankhandave1072 Жыл бұрын

@@CodeDecode Thank you so much for your reply. It will be very useful for every one of us.

@chetankhandave1072 Жыл бұрын

@@CodeDecode In this example, consumer and producer are in this same application, but what if Consumer is in one application and producer in another application ? How can we handle it ?

@karamjitsinghsehdev3451 Жыл бұрын

Nice and Simple. Please try to make real scenario based videos also for experienced. That will help crack interviews. Thankyou.

@CodeDecode Жыл бұрын

Next is too implement kafka in microservices application

@lukemagnotta6823 Жыл бұрын

How Eloquently was the explanation!

@CodeDecode Жыл бұрын

Thanks a lot 👍🙃🙃

@karthikeyanrm3446 Жыл бұрын

Please continue this series. If possible make a series on Event driven Microservices. Not sure if you remember - but I have been asking for so long to cover centralized logging, API gateway, Distributed tracing, ELK, docker-kubernetes also (all companies expect this) and then atlast deploy into cloud - AWS/Azure.

@CodeDecode Жыл бұрын

Yeah all in pipeline Karthikeyan 🙂

@karthikeyanrm3446 Жыл бұрын

@@CodeDecode thank you so much. Please do. You guys are awesome 😊

@rahultripathi1602 8 ай бұрын

please make Kafka Streams video too and if possible Interview questions video too. Thanks!

@kalyanimvn Жыл бұрын

U r rocking mam.asm

@CodeDecode Жыл бұрын

Thanks a lot Kalyani 🙂👍

@ourworldofknowledge5542 Жыл бұрын

Excellent video

@CodeDecode Жыл бұрын

Thanks

@bhavyar4983 Жыл бұрын

Hi mam plz do cover the interview questions thanks a lot for ur time

@AnuragTripathiShiva Жыл бұрын

I have little doubt when you say - “broker is a topic” and “consumer will subscribe to broker”. It conflicts with my understanding that broker is not a topic instead it is a kind of container which have multiple partitions from different topics. And consumer subscribes to topic rather than the broker. Please correct me if I am wrong.

@sayanbiswas8847 Жыл бұрын

Helpful

@CodeDecode Жыл бұрын

Thanks 👍🙂

@nikhilpawar7876 11 ай бұрын

awesome

@CodeDecode 11 ай бұрын

Thanks

@akshayvishwanath2591 5 ай бұрын

Crazy❤

@CodeDecode 5 ай бұрын

😊😊👍👍

@priyangapinki5715 2 ай бұрын

Thank you so much. Can you please implement kafka with your vaccination center and citizen project. It will be helpful for understanding real time projects

@start1learn-n171 3 ай бұрын

@geethufrancis6988 Жыл бұрын

Ma'm Thank you soooo much for this playlist. Can you please create a video on setting the kafka messages to Redis db...

@CodeDecode Жыл бұрын

Sure 👍👍

@chetankhandave1072 Жыл бұрын

In this example, consumer and producer are in this same application, but what if Consumer is in one application and producer in another application ? How can we handle it ?

@DhruvSharmax Жыл бұрын

Can you please let us know how to handle objects instead of string as message. Also how to use kafka streams.

@vidhiagarwal9370 2 ай бұрын

But how to get kafka communicate in different microservices? Would this work? because I see you have done the communication in the same service

@prateetisingha6729 Жыл бұрын

Hi , thank you for your videos it's realy helps a lots. Is it possible to share some videos regarding cicd pipeline.

@CodeDecode Жыл бұрын

sure we will upload it soon

@atulpatil7472 Жыл бұрын

Thanks for this wonderful video..I was practicing this code and able to consume my message..after all message consumed. I stop kafka and zookeeper server .once again I have started both server and I am again getting same messages in my consumer. How we can avoid that , as I have already read it before and don't want to read it again as it's duplicate for me

@abhinavgoel4358 Жыл бұрын

Can you please explain serialisation and deserialisation in Java with practical example

@giridharkumar8176 11 ай бұрын

I know this is a basic demo, but I have a question, here you did not create topic inside kafka cluster, and no where you added your topic name in any properties file that is being used in two commands u ran in cmd, u added in ur application's properties file but not in kafka application or zookeeper aplication, so then in enterprise application, any one can create as many topic as they want as it does not required kafka broker permission, so how to restrict it then?

@UmmakajJagadish Жыл бұрын

madam please do more videos on microservices

@CodeDecode Жыл бұрын

Sure Jagadish 🙂👍

@doctorag1135 Жыл бұрын

I am getting error group id is not found in exceptation, SO group id can be custom any thing hardcode right?

@abhishekchaudhari2344 Жыл бұрын

Can plz you clear why consumer not require deserializer there? when we use deserializer in Consumer?

@kanchankatkar5226 3 ай бұрын

When will be next video of which you said in last in video, please do keep posting

@MrGoodLuck 9 ай бұрын

Mam can you show using docker also? 🤔I tried searching for it but playlist doesn’t have that and also can you please make a video nd explain if I am using okta then how to authenticate only once for all the services 😅or any other method is also fine like how will we apply security for it I have seen the okta video but wanted to know how it will be done in microservices architecture

@abdulrehaman4462 5 ай бұрын

am facing problem in running zookeeper server , input is too long ,syntax error .... what should i do...

@rishiraj2548 Жыл бұрын

🙏👍👍

@deepsarandeepu Жыл бұрын

Installing kafka and Zookeeper means, Just downloading it from the URL mentioned ?, Some one can explain ?

@dipeekadeshmukh263 4 ай бұрын

zookeper and kafka both are not running, giving an error as The input line is too long and The syntax of the command is incorrect.

@sureshkamble1526 Жыл бұрын

Initially title was "KAKA" suddenly wonderd ☺

@CodeDecode Жыл бұрын

🙊our bad. Sorry that was typo 🙊

@nareshk329 Жыл бұрын

Please attach PPT as well for information purpose or notes to remember.

@Ammupopproductions Жыл бұрын

Please create kafka streams video also

@CodeDecode Жыл бұрын

Sure Anupam 👍👍

@souvikasus2903 Жыл бұрын

I am unable to Install Kafka binary. tgz file in my windows 11. Please help me out.

@priyanktiwary1067 Жыл бұрын

Please post video on rabbit mq implementation also

@CodeDecode Жыл бұрын

Sure Priyank 🙂

@user-zg8wv5qb7m 5 ай бұрын

MA'm, i a getting logs in infinite loop in console

@satishsali5571 Жыл бұрын

Can you give the link of Kafka with Stream video?

@smehta344 Жыл бұрын

Hi, I have one question In one interview they asked me What is the Kafka consumer group name? Please can you help me with this question.

@CodeDecode Жыл бұрын

Already gave an example in demo Sanjeev. What I can think of is consumer id here. We can have multiple consumer insame group by giving common group id/ name

@smehta344 Жыл бұрын

@@CodeDecode thanks

@CodeDecode Жыл бұрын

🙂👍

@mash6066 Жыл бұрын

Hi sister, I have one doubt . In springboot In service class for saving books we will use @Autowired Bookrepository br ; We will use br.save(); Bookrepository is an interface Can u explain ,how it will work . @Autowired we will use for object creation But here Bookrepository is an interface

@mash6066 Жыл бұрын

I have already commented , regarding this sister

@CodeDecode Жыл бұрын

This interface must extend some existing interface right? It calls internal api to save your data to db

@hamedhamed-ox9ty Жыл бұрын

How to get job as Kafka admin can any one tell me only learning Kafka administration will give job?

@venki1437 Жыл бұрын

Hai medam, Could please create one video on how to work with jsonb type data in postgress and how to integrate that data with spring boot jpa like how to insert jsonb data in Db using jpa create entity class store data in Db retrieve data from db. Note : there is no videos on KZfaq related this concept ,could you Create ASAP. it is helpful to programers

@sivareddy50 Жыл бұрын

Nice question, please do a video on jpa concept

@agusp1951 10 ай бұрын

What about JsonSerializer or CustomSerializer

@CodeDecode 9 ай бұрын

We will add them in playlist soon 👍

@resikas.b.5511 27 күн бұрын

you have to create a topic in the terminal. That is not shown in the video

@CodeDecode 24 күн бұрын

Not needed

@TummalapalliTejaswini Жыл бұрын

I am getting error can anyone help me

@letsthink1490 Жыл бұрын

Dear please change the title of this video.i really thought KAKA IS a new concept in java😅😅😅😅

@CodeDecode Жыл бұрын

Ohh really thanks for letting us know. 😆it was a typo🙊

@padmalankipalli593 Жыл бұрын

Thank you so much @codedecode

@CodeDecode Жыл бұрын

Thanks Padma🙂👍

@22ankitajaiswal Жыл бұрын

Hello codeDecode team, I am getting error when I am doing the implementation by following the video. The error is: Unexpected error in InitProducerIdResponse; The request is timed out. or.apache.kafja.common.KafkaException