Is Flink the answer to the ETL problem? (with Robert Metzger)

Рет қаралды 5,037

6 ай бұрын

Integration is probably the last, hardest, and least well thought-out part of any large software project. So anything that makes the data-streaming job easier is worth knowing about. So this week we turn our attention to Apache Flink, a flexible system for grabbing, transforming and shipping data between systems using Java, Python or good ol’ SQL.
So this week Robert Metzger-Apache Flink expert and PMC member-joins us to explain what problems Flink solves and how it solves them reliably. We cover the range from simple use cases to realtime aggregations & joins to its high availability strategy.
If you’re working on systems that include more than one database, then you’re definitely going to face the kinds of problems that Flink tackles.
-
Apache Flink: flink.apache.org/
Robert on Twitter: / rmetzger_
Robert on LinkedIn: / metzgerrobert
Kris on Twitter: / krisajenkins
Kris on LinkedIn: / krisjenkins
-
#software #programming #podcast #flink #apacheflink #dataintegration

Пікірлер: 16

@jomoho3919 6 ай бұрын

Great content. Just discovered the channel and started binge-watching!

@rallokkcaz 6 ай бұрын

Same listening while I work today!

@RaviTomar7 5 ай бұрын

Amazing talk, Thankyou :)

@dataecstasy 6 ай бұрын

Nice Great talk. More and more developers should be allowed to talk. he didn't pitched decodable, his company for managed flink.

@cj-ip3zh 6 ай бұрын

It would be nice to have savepoint triggering available from the web ui.

@nxx210 5 ай бұрын

With Ververica platform you can

@cj-ip3zh 5 ай бұрын

@@nxx210 I might attempt to make my first open source contribution to OS flink with that feature...

@IvanGeorgiev 5 ай бұрын

Can you please cover apache spark streaming?

@DeveloperVoices 5 ай бұрын

Yes! Good idea. I'll add that to my list of guests to research. Any suggestions for who would make a good interviewee?

@IvanGeorgiev 5 ай бұрын

@@DeveloperVoices I'd say somebody from databricks that could put more color on the internal workings and point out some pros and cons of using it. I'm sure they will be glad to send some of their experts to attend the podcast.

@simonmassey8850 3 ай бұрын

a vendor wil point out the cons…? 😂

@IvanGeorgiev 3 ай бұрын

@@simonmassey8850 at least will point out all the cons for Apache Flink :)

@AndreCarneiro666 3 ай бұрын

Apache Flink brings the worst in terms of maintainability. It's basically the essence of Spark RDD. A trully throwback, in my opinion.

@poojanpatel2437 Ай бұрын

Asking a 12 year veteran working in single tool whether tool support reporting like pdf, csv. Holy please stop doing any content at all. This is just embarrassing for anyone even coming to your channel for speaking.

@DeveloperVoices Ай бұрын

I think what you've missed is how many people doing data engineering are stuck with unclean data in crappy data sources. I was asking if Flink would be any help in that ugly but all-too-common reality.