Azure Event Hubs for Apache Kafka

No video

Azure Event Hubs for Apache Kafka | Azure Friday

Рет қаралды 36,107

Күн бұрын

Shubha Vijayasarathy joins Scott Hanselman to discuss Azure Event Hubs, which makes data ingestion simple, secure, and scalable. As a distributed streaming platform, Event Hubs enables you to stream your data from any source-storing and processing millions of events per second- so you can build dynamic data pipelines and respond to business challenges in real time.
With Azure Event Hubs for Apache Kafka, we're bringing together two powerful distributed streaming platforms, so you can access the breadth of Kafka ecosystem applications without having to manage servers or networks. Event Hubs for Kafka provides a Kafka endpoint so that any Kafka client running Kafka 1.0 or newer protocols can publish/subscribe events to/from Event Hubs with a simple configuration change.
[06:45] Demo Start
Azure Event Hubs docs
aka.ms/azfr/495/01
Azure Event Hubs samples (GitHub)
aka.ms/azfr/495/02
Azure Event Hubs overview
aka.ms/azfr/495/03
Azure Event Hubs pricing
aka.ms/azfr/495/04
Announcing the general availability of Azure Event Hubs for Apache Kafka® (blog post)
aka.ms/azfr/495/05
Create a free account (Azure)
aka.ms/azfr/495/free

Пікірлер: 20

@VOGTLANDOUTDOORS 4 жыл бұрын

A very succinct comparison of these 2 easily-confused distributed streaming/messaging systems. Your example with AzureEventHub using a "Namespace" to provide a "Kafka Abstracion Layer" to Java-coded Producers & Consumers was PRECISELY the explanation I was seeking for explaining AzureEventHubs to a customer! You just made my day!

@lackshubalasubramaniam7311 5 жыл бұрын

Well explained. Thanks folks.

@berndeckenfels 5 жыл бұрын

What’s the persistence properties of Eventhub, can I also store like 60 days and replay form there? How would a consumer manage its offsets?

@lorapora 5 жыл бұрын

The big question is it as good as kafka as far as performance? i am not sure folks who use kafka today will switch and give away all the producer and consumer configuration fine tuning options and replace it with a magic box in the cloud.

@thebachelorpad1110 3 жыл бұрын

Such an amazing discussion! love it! Thank you for sharing!

@abhiramikaushikaaduru2084 4 жыл бұрын

Good one and nice explanation on kafka data streaming platforms

@harishs4499 4 ай бұрын

Fantastic

@PiyushLaad1987 4 жыл бұрын

Thank you.

@sdbhattacharya 3 жыл бұрын

Why Microsoft is not positioning eventhub for service to service communication? Why its always depicted as event streaming?

@onofreantoniojuvenciojunio1499 3 жыл бұрын

Thanks for this video!

@vchandm23 2 жыл бұрын

@4:32 that sarcastic smile from Scott, says it all. Just scales smile (with cost :))

@ArvindNaikSOS 4 жыл бұрын

Do we absolutely need Event Hub to consume this Kafka data? Or can we connect to Kafka using the Azure Data Factory or Databricks?

@harishreddy2405 2 жыл бұрын

Can we read data from blob storage using Kafka?

@mykhailokozlov6641 9 ай бұрын

That's incredible! Although funny to see java in demo when we have C# :)

@snehanjanaray8795 5 жыл бұрын

Hi, I am trying same implementation in my Maven Spring Boot + Kafka application for Producer after sending data it is hitting Azure Matrix but mentioned error I am getting , Error: 2019-07-04 12:27:31.729 INFO 92607 --- [nio-8050-exec-2] c.g.k.service.KafkaPublisherService : sending message='t' 2019-07-04 12:28:31.730 ERROR 92607 --- [nio-8050-exec-2] o.s.k.support.LoggingProducerListener : Exception thrown when sending a message with key='null' and payload='Hi' to topic testhub: org.apache.kafka.common.errors.TimeoutException: Failed to update metadata after 60000 ms. And for consumer while clean build I am getting below mentioned error: 2019-07-04 12:48:26.837 WARN 94752 --- [ main] o.s.w.c.s.GenericWebApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is org.apache.kafka.common.errors.TimeoutException: Timeout expired while fetching topic metadata

@snehanjanaray8795 5 жыл бұрын

Please let me know if there is any solution for this.

@BasilsCastle 4 жыл бұрын

@@snehanjanaray8795 The EventHubs team doesn't monitor this video for issues. Open a github issue or file a support ticket with Microsoft!