No video

Advancing Spark - Tracking Lineage with Unity Catalog

  Рет қаралды 5,282

Advancing Analytics

Advancing Analytics

Күн бұрын

Tracking dependencies and lineage across the lake has been a long term objective for many teams, tools and vendors, but it's hard to get right! Unity Catalog recently pushed their Lineage functionality into public preview, so it's time to take a look and see how well it works!
In this video, Simon opens up his Unity Catalog-enabled workspace and creates some inter-dependent tables, tracking the lineage objects that are created in Unity Catalog as a result!
To get started with Lineage tracking, check out the docs here: docs.databrick...
And as always, if you're starting out on your Lakehouse journey, get in touch with Advancing Analytics to give your platform a much-needed boost!

Пікірлер: 10
@paulroome3928
@paulroome3928 Жыл бұрын
Great video as always Simon! Files within lineage is coming soon 🙂
@drummerboi4eva
@drummerboi4eva Жыл бұрын
Amazing stuff SImon. I told my colleagues when Spark 3 first came out, that we need a lineage tool, and here it is. Fantastic, even though with limitations
@alexischicoine2072
@alexischicoine2072 Жыл бұрын
Super interesting Simon. I know it’s just an example to explain, but might want to make sure others don’t have permissions on these poc tables ;)
@swapnilsaxena9577
@swapnilsaxena9577 Жыл бұрын
Nice vid, Simon. With regards to extracting lineage programmatically, same doc covers REST API to extract table/column lineage too.
@arunr2265
@arunr2265 Жыл бұрын
Great Video Simin. Waiting for the video on unity catalog set up ;)
@briancuster7355
@briancuster7355 Жыл бұрын
Unity catalog is a great start but I can see a load of potential from this service.
@EvgenAnufriev
@EvgenAnufriev Жыл бұрын
Thanks for the video. Could you clarify if GA means that Unity catalog is free? If the company has Premium subscription does it requires to pay for the feature Unitiy catalog provides?
@JLRocco43
@JLRocco43 Жыл бұрын
Can answer for you. I deployed UC for my company. Just need to have premium and you can deploy it for free
@EvgenAnufriev
@EvgenAnufriev Жыл бұрын
@@JLRocco43 Thank you!
@alexischicoine2072
@alexischicoine2072 Жыл бұрын
Databricks pretty much only charges you for compute and things like the meta store or notebooks are free. You have to pay a bunch of related stuff in your cloud provider though.
Advancing Spark - Row-Level Security and Dynamic Masking with Unity Catalog
20:43
Advancing Spark - External Tables with Unity Catalog
17:25
Advancing Analytics
Рет қаралды 15 М.
Magic trick 🪄😁
00:13
Andrey Grechka
Рет қаралды 58 МЛН
Пройди игру и получи 5 чупа-чупсов (2024)
00:49
Екатерина Ковалева
Рет қаралды 4,4 МЛН
Glow Stick Secret Pt.4 😱 #shorts
00:35
Mr DegrEE
Рет қаралды 14 МЛН
Whoa
01:00
Justin Flom
Рет қаралды 51 МЛН
Cross-Platform Data Lineage with OpenLineage
34:18
Databricks
Рет қаралды 6 М.
What’s New in Unity Catalog-with Live Demos
38:08
Databricks
Рет қаралды 1,4 М.
Advancing Spark - Crazy Performance with Spark 3 Adaptive Query Execution
18:48
Core Databricks: Understand the Hive Metastore
22:12
Bryan Cafferky
Рет қаралды 15 М.
05- Unity Catalog Capture and view data lineage
13:58
NextGen Learning
Рет қаралды 7 М.
Advancing Spark - Lakehouse Observability with Unity Catalog System Tables
19:34
What is Data Lineage?
5:38
IBM Technology
Рет қаралды 14 М.
What is Unity Catalog in Databricks ?
12:05
GeekCoders
Рет қаралды 2,5 М.
Advancing Spark - Provisioning Databricks Users through SCIM
14:23
Advancing Analytics
Рет қаралды 8 М.
Advancing Spark - Setting up Databricks Unity Catalog Environments
21:21
Advancing Analytics
Рет қаралды 17 М.
Magic trick 🪄😁
00:13
Andrey Grechka
Рет қаралды 58 МЛН