Apache Kafka® Transactions: Message Delivery and Exactly-Once Semantics

Рет қаралды 21,967

Күн бұрын

cnfl.io/kafka-internals-101-m... | What does exactly-once semantics guarantee, and how do Apache Kafka Transactions deliver upon this?
In this tutorial, learn how transactional support in Apache Kafka® allows for atomic-level updates from Jun Rao (Kafka Committer, PMC Member, VP of Kafka, and Co-Founder, Confluent). You will learn how transactions enable exactly-once semantics (EOS), which form the basis for this functionality in clients, like the streaming database ksqlDB and the Kafka Streams application library. You'll learn how Kafka handles system failure with transactional functionality and a couple of caveats to using transactions that you should be aware of.
Use the promo code INTERNALS101 to get $25 of free Confluent Cloud usage: www.confluent.io/confluent-cl...
Promo code details: www.confluent.io/confluent-cl...
LEARN MORE
► Exactly-Once Semantics Are Possible: Here’s How Kafka Does It: www.confluent.io/blog/exactly...
► Transactions in Apache Kafka: www.confluent.io/blog/transac...
► Improved Robustness and Usability of Exactly-Once Semantics in Apache Kafka: www.confluent.io/blog/simplif...
► Building Systems Using Transactions in Apache Kafka: developer.confluent.io/learn/...
► KIP-98 - Exactly Once Delivery and Transactional Messaging: cwiki.apache.org/confluence/d...
ABOUT CONFLUENT
Confluent is pioneering a fundamentally new category of data infrastructure focused on data in motion. Confluent’s cloud-native offering is the foundational platform for data in motion - designed to be the intelligent connective tissue enabling real-time data, from multiple sources, to constantly stream across the organization. With Confluent, organizations can meet the new business imperative of delivering rich, digital front-end customer experiences and transitioning to sophisticated, real-time, software-driven backend operations. To learn more, please visit www.confluent.io.
#streamprocessing #confluentcloud #apachekafka #kafka #confluent

Пікірлер: 7

@rajsaraogi 7 ай бұрын

Abort marker, fence off, honestly so much jargans to make understand things, kafka teams should come with simple explainations

@user-qk3hj1gf7p 3 ай бұрын

Clear introduction, thank you !

@mehrdadk.6816 Жыл бұрын

If one using low-level one way to avoid to disable auto-commit feature in topic, and once write to debit balance topic is done we manually increase the offset. This way we ensure of app crashed, we didn't commit log, so we can restart. Nevertheless this feature of having transaction in stream is really cool.

@davidebelvedere2564 2 ай бұрын

how can it guarantee that the committed marker is pushed on coordinator and on all destination partitions? it doesn't seem much atomic as an operation

@kovalevvvvv 8 ай бұрын

Lol, alot of words about "modern mq", but kafka doesn't support xa-protocol - you can't use kafka transactions with database transactions.

@paulpasquel Жыл бұрын

New "revolutionary application" using standards mechanisms, that we have saw in Database and MQ for more than 40 years ... but not 2pc Surely Kafka will implement that in future ...