Migrate 34m Records from SQL Server to Warehouse using Fabric Data Pipeline

No video

Migrate 34m Records from SQL Server to Warehouse using Fabric Data Pipeline

Рет қаралды 274

Күн бұрын

In this end-to-end Data Engineering and Analytics project, I walked you through how to migrate 34m of records from on-prems SQL Server to Fabric Data Warehouse using Fabric Data Pipeline. If you've learnt from this video, give it a thumps up, share with your connections and hit the subscribe button for more
#DataPipeline #MicrosoftFabric #MicrosoftAzure #PowerBI

Пікірлер: 9

@bloom6874 Ай бұрын

This is mind blowing explanation and need more number of view and likes.

@Abioladavid01 Ай бұрын

Thanks very much

@777bashir 2 ай бұрын

I wonder how the data refresh occurs every 5 minutes without specifying a delta incremental plan !!?

@Nalaka-Wanniarachchi 3 ай бұрын

Nice video. What are the use cases for using a intermediate staging layer ,cos we can directly import the data from on prem SQL server to WH/LHin fabric? Is there any advantage?

@Abioladavid01 2 ай бұрын

Thanks for your comment. Fabric Data Pipeline do not support ingesting data from On prems SQL Server to Fabric Warehouse directly without specify the staging. Though, Fabric support direct ingestion of data from SQL server to Lakehouse seamlessly.

@777bashir 2 ай бұрын

Another question 😊 Is it feasible to follow the same steps for migrating data to a Lakehouse instead of a warehouse, given that I don't have staging storage? I attempted this but encountered an error stating that the Lakehouse does not support SQL code!

@Abioladavid01 Ай бұрын

Yes of course. You can migrate to Fabric Lakehouse directly without the need to specify staging in ADLS

@bloom6874 Ай бұрын

Can you share any documentation from where I can learn more about this?

@Abioladavid01 Ай бұрын

Hi, you can use Microsoft Learn to learn more. Above all, it is important you keep on practicing to broaden uour knowledge