Fabric Data Factory Metadata Driven Pipelines
Рет қаралды 3,338
7 ай бұрынLearn more about metadata driven data pipelines in #MicrosoftFabric #DataFactory!
In this video, Mark will show you how to create and use a common pipeline pattern to create a metadata driven data pipelines with parameters.
Resources:
☆ More metadata driven pipelines samples in Fabric Data Factory Pipelines:
techcommunity.microsoft.com/t...
☆ Code from these demos:
github.com/kromerm/DataFactor...
#DataFactory #FabricTutorial #DataPipeline #MSFT #MicrosoftFabric
@bigglesharrumpher4139 7 ай бұрын
We are implementing this in our Company at the moment. Seems to be working ok.
@SilvanoPaxia 7 ай бұрын
We are using Dataverse, so we can use a Model Driven App Application to maintain the Metadata
@nsulliivan 6 ай бұрын
This framework looks fantastic! How are you handling the unknown source metadata (Consider if this were the FIRST run you're doing) mapping tab/component in the parameterized Copy activities that you have at the end of your loop component?
@Patches3670 7 ай бұрын
Will Azure SQL ever be part of Fabric? 🤔 And is there any explicit reason was a Fabric DWH was not used as the control DB in the demo?
@chsrkn 6 ай бұрын
Can we build metadata driven dataflows in Fabric.? for example: In the synapse Dataflows we have a Schema Drift/Map Drift in Derived activity and Auto column map on the Sink, with these features we can make the dataflow as Metadata driven, do we have the same in Fabric.?
@GuillaumeBerthier 7 ай бұрын
Any ETA to have parameterise connections in Fabric ? (like you could with old ADF linked services) Without that ability, a metadata driven pipeline in Fabric becomes useless in real world …. Otherwise you'd need a separate connection for EVERY database (even if they're all SQL server for example) which is totally daft, thus defeating the point of using a meta data table driven pipeline like shown in this video 😢
@MSDataFactory 7 ай бұрын
Yes, parameterized connection support is in the works!
@NeumsFor9 7 ай бұрын
And to think..... all this time I thought you were just revelling in 2022 and 2023 Phillies baseball for the last few years.....
@kromerm 7 ай бұрын
Phillies, Sixers, Eagles ... It's a weird feeling when they're all winning!
LearnITEveryDay - Azure ETL Solution made Easy
Рет қаралды 2,3 М.
Immersed
Рет қаралды 109 МЛН
SQLBits
Рет қаралды 2,7 М.
Chris In Tech (CiT)
Рет қаралды 240
mrpaulandrew
Рет қаралды 15 М.
Fabric February
Рет қаралды 735
Data Platform Central
Рет қаралды 5 М.
Снежана Мясникова
Рет қаралды 2,8 МЛН