AWS re:Invent 2020: Top 5 best practices for data streaming with Amazon Kinesis

Рет қаралды 9,288

3 жыл бұрын

Latency and throughput are key considerations in stream processing workloads. In this session, you learn tips for optimizing throughput and latency in stream processing architectures. You also learn about scenarios when it’s necessary to find trade-offs between latency and throughput and about scalability. Services discussed in this session include Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, and Amazon Kinesis Data Analytics for Apache Flink.
Learn more about re:Invent 2020 at bit.ly/3c4NSdY
Subscribe:
More AWS videos bit.ly/2O3zS75
More AWS events videos bit.ly/316g9t4
#AWS #AWSEvents

Пікірлер: 9

@SergiiStarodubtsev 9 ай бұрын

09:30 Ecs,also supports many languages, Java python and other languages. To the point to comparing to Lambda by that characteristic . Along with the kinesis library, ksl, that takes cares shard consumption to the Ecs tasks. 16:05 is where is chosen to explain it. Lambda lose to many parameters if follow the matrix. To KSL, for example.

@awssupport 9 ай бұрын

Hey there! 👋 While we're unable to address these questions directly, we understand where you're coming from. I've sent off your concerns to our service team for review. 🔍 ^KS

@SergiiStarodubtsev 9 ай бұрын

@@awssupport that's is not a concern, that's friendly feedback )

@SergiiStarodubtsev 9 ай бұрын

14:00 maybe to explain right away why in single shard we have 2 partition keys, A and B. While one would think that one shard may have msgs with single partition key in it. Otherwise why not use 2 shards. To mention that's one of configurations, depending on the data spread strategy.

@ziadrida3119 3 жыл бұрын

Lambda has 1ms or better latency?! 16:38

@SergiiStarodubtsev 9 ай бұрын

I think what could be cleared out is how come we got 1 mb per second on input and 2 mb per second on output, where that +1 mb of data comes from? Like I put 1 apple 🍎 in 1 second and got 2 🍎 🍏 in another second as result. That wasn't clear in the beginning, and the question was poped out immediately.

@subhamoyguha7861 10 ай бұрын

I like to use Amazon Data Kinesis to get the access log of Amazon Api Gateway. How to do it ?

@awssupport 10 ай бұрын

Great question! To obtain access logs for API Gateway, take a look at the doc here: go.aws/3P8ZMGM. If you need additional assistance, these options may be helpful: go.aws/get-help. ⬅️ ^LG