No video

Advancing Spark - Provisioning Databricks Users through SCIM

  Рет қаралды 8,212

Advancing Analytics

Advancing Analytics

Күн бұрын

One of the biggest pains in managing Databricks has been user management, and with people starting to split their work across several workspaces this becomes even trickier to manage. We've previously built out integrations through the API, automatically pushing hard-coded lists of users into Databricks... but there has to be a better way... and there is!
In this video Simon walks through the SCIM connector, an enterprise application we can create inside Azure Active Directory that automatically provisions users and groups within a destination Databricks workspace. This is also used when provisioning the Databricks Account Console, which is a vital part of Unity Catalog, so it's worth getting things set up in advance!
For more info on Databricks SCIM - see the docs here: docs.microsoft...
As always, if Advancing Analytics can help you on your lakehouse journey, get in touch!

Пікірлер: 19
@zubair489
@zubair489 Жыл бұрын
Loved it.thanks for the video
@alfredoestevestorresgarave7250
@alfredoestevestorresgarave7250 Жыл бұрын
Thanks for sharing
@swapnilsaxena9577
@swapnilsaxena9577 2 жыл бұрын
Great video! Just one thing, though, post Unity catalog setup, provisioning must be done through Databricks admin account rather than at workspace level. Hence, token generation is done at account level. After provisioning is enabled, AD users/groups get synced in as Databricks account users/groups which can then be further assigned workspace level access by workspace admins or account admins.
@AdvancingAnalytics
@AdvancingAnalytics 2 жыл бұрын
Yeah, this vid was before identity federation went live! Probably another vid I need to do!
@marcocaviezel2672
@marcocaviezel2672 2 жыл бұрын
Hi Simon! Great video again! What a fantastic feature. Finally it’s here. Can you provide details in the SCIM connector what permission a person or group should have in Databricks. (E.g. only Databricks SQL)?
@briancuster7355
@briancuster7355 2 жыл бұрын
i haven't had a chance to use SCIM as most of my work involves working with an environment build by devops professionals but it sounds like a great way to sync AAD users with Databricks.
@joskessels5194
@joskessels5194 2 жыл бұрын
Anybody else experiencing that SPN’s inside an SCIM synced Azure AD group are not provisioned to the Databricks workspace? Also I expect the same issue for managed identities.
@leoafurlongiv
@leoafurlongiv 2 жыл бұрын
That is unfortunately correct. The AAD Enterprise App doesn't SCIM over SPs or MIs. :( You can use the Databricks Terraform Provider to do this though.
@kaurivneet1
@kaurivneet1 2 жыл бұрын
Thank you Simon for the great video! love the scim pun :P One question, if the list of users and groups are not known upfront and are created later in Azure AD, how can they be added to the SCIM connector afterwards? Is the SCIM api meant for that?
@AdvancingAnalytics
@AdvancingAnalytics 2 жыл бұрын
Hi! You can amend the users & groups in the SCIM connector at any time, there are settings as to how often it will sync with the destination. Also, it's worth looking at the Identity Federation news from this month, as this is an alternative approach for managing databricks users in Azure! docs.microsoft.com/en-gb/azure/databricks/administration-guide/users-groups/#enable-identity-federation
@film-masti-777
@film-masti-777 Жыл бұрын
Hi Simon, 1 question. if I have to implement Fine-grained access control at databricks at unity catalog level where we rely on databricks groups for access control, can it be achieved using Azure AD group synced with Databricks using SCIM provisioning? OR do I have to create databricks groups separately for this access control?
@michaelleuer7660
@michaelleuer7660 Жыл бұрын
Can you have multiple databricks workspaces? Or would you go about doing this for multiple workspaces?
@saurabh12521
@saurabh12521 Жыл бұрын
What if I have more then one workspace. Do I need to add all in provisioning? If I did how will my user and group have access? for eg : I have added 5 workspaces in provisioning which is used by different work groups. But we have created a single enterprise application that will have a common user/group option to add. If I add a user, will that user will have access to all workspaces?
@MrMikereeve89
@MrMikereeve89 2 жыл бұрын
Any inkling of when this might go GA? I’ve had my eye on it for a while but I’m reluctant to use it whilst it’s still in Public Preview.
@AdvancingAnalytics
@AdvancingAnalytics 2 жыл бұрын
No idea! Although I expect a raft of announcements next week, not sure if SCIM is one of them!
@MrMikereeve89
@MrMikereeve89 2 жыл бұрын
I seriously can’t wait! I’m so fed up of manually maintaining my users and groups across dev/sit/pre-prod/prod - it’s soooo tedious! 🤣
@matthiasmueller9340
@matthiasmueller9340 2 жыл бұрын
Hi Simon, would you mind showing how to do this using AWS?
@akhilannan
@akhilannan 2 жыл бұрын
Using this, can we sync users in an Azure AD group over to a Databricks Group?
@AdvancingAnalytics
@AdvancingAnalytics 2 жыл бұрын
Yep, that's exactly what this is for!
Advancing Spark - Delta Sharing
26:12
Advancing Analytics
Рет қаралды 9 М.
Advancing Spark - Understanding Terraform
23:00
Advancing Analytics
Рет қаралды 4,2 М.
He bought this so I can drive too🥹😭 #tiktok #elsarca
00:22
Elsa Arca
Рет қаралды 13 МЛН
Kind Waiter's Gesture to Homeless Boy #shorts
00:32
I migliori trucchetti di Fabiosa
Рет қаралды 16 МЛН
艾莎撒娇得到王子的原谅#艾莎
00:24
在逃的公主
Рет қаралды 54 МЛН
Oh No! My Doll Fell In The Dirt🤧💩
00:17
ToolTastic
Рет қаралды 12 МЛН
Advancing Spark - Tracking Lineage with Unity Catalog
15:00
Advancing Analytics
Рет қаралды 5 М.
How to Sync Snowflake Users w/ Azure Active Directory
15:04
Kahan Data Solutions
Рет қаралды 6 М.
Advancing Spark - Getting Started with Ganglia in Databricks
24:49
Advancing Analytics
Рет қаралды 11 М.
A Developer's Guide to SCIM
43:13
OktaDev
Рет қаралды 28 М.
A Practitioner's Guide to Unity Catalog-A Technical Deep Dive
39:56
Enabling Unity Catalog on Azure Databricks: A Step-by-Step Guide
13:04
Pathfinder Analytics
Рет қаралды 15 М.
Advancing Spark - Rethinking ETL with Databricks Autoloader
21:09
Advancing Analytics
Рет қаралды 26 М.
Databricks Networking Deep Dive - Part1
16:11
Revolution Data Platforms
Рет қаралды 4,6 М.
Advancing Spark - External Tables with Unity Catalog
17:25
Advancing Analytics
Рет қаралды 15 М.
He bought this so I can drive too🥹😭 #tiktok #elsarca
00:22
Elsa Arca
Рет қаралды 13 МЛН