Data Lakehouse Deep Dive: Hudi, Iceberg, and Delta Lake

Рет қаралды 3,056

Күн бұрын

On Tuesday, August 22nd, Onehouse presented a LinkedIn Live webinar on Apache Hudi, Apache Iceberg, and Delta Lake. You can see the video here or on LinkedIn:
/ 7095484265877950465
The video was based on Kyle Weller's popular blog post on the three major lakehouse projects:
www.onehouse.ai/blog/apache-h...
You can access Kyle's slides here:
uploads-ssl.webflow.com/64d1d...

Пікірлер: 8

@padam_discussion Ай бұрын

Interesting video... great

@HoorayforOranges 4 ай бұрын

Thank you so much for this. This is the only video I could find that takes a real deep dive into the data without propaganda towards any one candidate.

@JG-zu6nq 10 ай бұрын

mistake at 22:41, there's no limitation that you 'cant cross over the boundary' in a query when you do partition evolution in Iceberg

@kjweller 10 ай бұрын

You can cross the boundary, but the query predicates need to be right to get the same performance across both partition schemes.

@JG-zu6nq 10 ай бұрын

@@kjweller what exactly does that mean, one just has to write select * from table where ts > timestamp '2023-08-21 00:00:00' and even if the partitioning was evolved from say daily to hourly on 08/25 that will work and prune the partitions

@kjweller 10 ай бұрын

@@JG-zu6nq take an example if you were partitioning by date daily, and you want to evolve this to partition by userId or vice-versa. A query with only one of the predicates will be efficient just for that section of the partitioned data. It works great for evolving partitioning within different aggregate levels of same value, but struggles across different values.

@paulfunigga 10 ай бұрын

@@kjweller what about schema evolution, in your article it says that hudi's schema evolution is good only on spark sql. What if I use hudi with trino? Is schema evolution going to be bad? Also, is hudi good with trino at all? In trino's slack channel they said that they prioritize iceberg.

@paulfunigga 10 ай бұрын

@@kjweller also, in your "which format to choose" why didn't you add another point: hudi's table services are managed, compared to iceberg and delta lake, I think it's a big thing.