19 сағат бұрын
21 күн бұрын
Ай бұрын
5 ай бұрын
7 ай бұрын
8 ай бұрын
Пікірлер
@keen8five Күн бұрын
Any chance you will bring Spark 3.5 to Azure Synapse? We are at the brink of migrating from Synapse Spark Runtime 3.3 to 3.4; however, if you will bring Spark 3.5 to Azure Synapse soonish, we can skip 3.4 all together. 😁
@quiosaevaristo7746 3 күн бұрын
Good explanation, You have showed just bronze layer I would like to ask you to make another video it it is possible so that you show silver and gold part of medallion architecture in practice
@samanthatseng9692 6 күн бұрын
Is there really no way to just pull changes from the collaboration branch into a feature branch? That would make this all so much easier.
@ItsNotAboutTheCell 7 күн бұрын
Haha! Love seeing John with that massive coffee mug! He's been awesome to work with and to collaborate with on data ingestion projects.
@sgfgdsfae 7 күн бұрын
Using the Mirror feature to sync data into Delta tables is fantastic ... however, we are missing the ability to track changes in order to perform downstream incremental updates. The delta tables in the Mirror database allow time-travel using versions but I can't find a way to enable Change Data Feed, which would be a game changer for incremental workflows. Is this in the roadmap or is there a different solution?
@MarkPryceMaher Күн бұрын
In theory, yes its possible to do what you want. As each change is a delta change, so you can work out the delta using sparkSQL today (I have done it)- but you might not get the results you expect. The issue is this is not what mirroring was designed for, its eventual consistency. The delta table on Onelake should represent the table is SQL. We may move incremental changes, but if you change the majority of the table (updating 1 billion rows changing the schema), we could take 1 billion changes or we could take a new snapshot and replace the entire thing. So if you are using the incremental updates, it would be the whole table. In the future, we may be able to support this but not right now.
@timepassbenu3853 7 күн бұрын
Great video. Could you please also make some video on what is tempdb and it is used during the workloads.
@keen8five 8 күн бұрын
I'd love to ingest data from on-prem using Notebooks (code-based)
@natehinnen9170 8 күн бұрын
How do I get my audit history table available to query?
@tariqbilal191 8 күн бұрын
Perfect explanation. :)
@pspointssara8472 12 күн бұрын
The content seems good but very poor audio and unable to hear some of the info.
@noahhadro8213 14 күн бұрын
Does this work when you run Dax queries against a semantic model?
@EsteraKot 14 күн бұрын
Great question. Autotune works when you run PySpark, Spark, or Spark SQL queries against data in a lakehouse.
@karthikeyan7521 17 күн бұрын
Awesome video and excellent content! But I'm curious to know why Native engine doesn't support writing into delta partitioned table and ANSI mode?
@shauncurtin4140 21 күн бұрын
Does this work for columns where multiple options are selected?
@tomengland1866 22 күн бұрын
We need to be able to execute pipelines via REST / service principal application authentication. Surely this should've been high on the list to implement early
@Randyminder 25 күн бұрын
Does Mirroring currently support Azure SQL Managed Instance?
@MarkPryceMaher Күн бұрын
Not yet. But it is planned. Feel free to upvote this. ideas.fabric.microsoft.com/ideas/idea/?ideaid=0e5e148e-c206-ef11-a73c-6045bdb60d0e
@sikandervempalli2035 26 күн бұрын
Why did you add "_default" to the ARM template files?
@panostanilian 26 күн бұрын
Cool feature!!Any plans to support further database engines like mySQL or Postgres in the near future?
@MarkPryceMaher Күн бұрын
We are always looking at which sources to support next, feel free to upvote this on fabric.ideas ideas.fabric.microsoft.com/ideas/idea/?ideaid=485f86e8-1ded-ee11-a73d-000d3ae360cc
@aslakjonhaugen3004 27 күн бұрын
Hi, I'm trying to set up a mirrored SqlServer database, but I'm getting this error: Unable to retrieve SQL Server managed identities. You can not mirror SQL Database with the managed identity across tenants. Please switch to a different SQL Database. Is it not possible to mirror a sqlServer database from another Azure tenant?
@MarkPryceMaher Күн бұрын
It is not supported today - its documented in the limitations. learn.microsoft.com/en-us/fabric/database/mirrored-database/azure-sql-database-limitations#network-and-connectivity-security
@MarkPryceMaher Күн бұрын
Mirroring across Microsoft Entra tenants is not supported where an Azure SQL Database and the Fabric workspace are in separate tenants.
@aslakjonhaugen3004 Күн бұрын
@@MarkPryceMaher Do you know if this will change in the near future? (I cross my fingers! :- )
@Mahmoudalgindy 28 күн бұрын
Thanks so much Scott, is there any way to select these tables programmatically using PowerShell or API or other tools? Selecting it manually, specially from D365 FinOps is a heavy task and takes a lot of time, there was a bug few months ago that Unlink the FinOps as I reported to Microsoft and I had to repeat a big process again, Thanks is advance
@azizquazi 29 күн бұрын
It is always good to provide the source file locations and the ipynb files to follow the videos. I appreciate your thoughts on this. Thanks.
@VasuN-bc4wc Ай бұрын
This looks nice. Wait for it to become available in all tenants to test it out. We need shared cluster/concurrent spark session in pipelines There are scenarios where we need to do a lot of very lightweight operations like pulling data from APIs, Databases, etc which barely need 2 core and 4GB memory but they need to run every 15 mins Right now, all we can do is run it in a 8core 64GB node, of course small node is there but it takes 3mins to start. Hope to see a solution to this problems.
@sayanipaul7240 Ай бұрын
Can I use python inside Fabric to build a ML model ?
@girishsuch Ай бұрын
Grerat presentation, Abhishek. Loved the way you compiled all the info.
@Idle5 Ай бұрын
Good job Microsoft team having your own Databricks product 🤭
@flavioleccese Ай бұрын
please add support to sparkR... this extension is the only thing allowing to actually use fabric notebooks (how would you expect to write code without an autocompleter on the platform?) and R is an important piece for many
@keen8five Ай бұрын
any chance you will bring this feature also to Synapse? 🙂
@EsteraKot Ай бұрын
The Native Execution Engine is embedded as part of Microsoft Fabric www.microsoft.com/en-us/microsoft-fabric and there are no plans to add it to Synapse.
@jenilchristo8775 Ай бұрын
Does gluten works for spark scala dataframe apis?
@EsteraKot Ай бұрын
Yes
@ManikumarAryas Ай бұрын
Great video and fantastic job! It's truly insightful.
@jayopachecoea Ай бұрын
Me encantan los vídeos de este canal, cortos y precisos (y) !
@milad987 Ай бұрын
Do the database templates in synapse analytics follow the star schema ?
@gauravchaturvedi3615 Ай бұрын
how does data governance works with Domains defined in One Lake for different departments?
@lighteningrod36 Ай бұрын
So, data sovereignty rules will limit the use if Co-Pilot in Australia, if my data is processed in the US.
@ruixinxu0130 Ай бұрын
Thank you for your feedback. Copilot in Fabric is powered by LLMs that are currently only deployed to US and EU. We are aware of the data sovereignty concern and actively expanding our deployment to more Geo regions. You can always check this link to get the latest info. learn.microsoft.com/en-us/fabric/get-started/copilot-fabric-overview#available-regions
@user-lj9fk8dg9h Ай бұрын
Hello sir, Thank you so much providing these productive videos. Today, I faced a challenge, and the solution I couldn't find elsewhere. That is How to Extract data from SAP Hana Cloud to Microsoft Fabric (cloud to cloud connectivity). Could you please help me here?
@user-ph1km5vk9l Ай бұрын
whta is name of the episode mentioned with short cuts explained ? thank you
@keen8five 2 ай бұрын
status "running" just says that the vCore "did something", right? But there is no way to tell if the cores were running at 1% or at 100% load, correct?
@jennyjiang6301 2 ай бұрын
Yes, you are right. The resource utilization chart currently only indicates that the vCore is running and does not indicate CPU or memory utilization. What kind of information are you specifically looking for?
@RSCHAB 2 ай бұрын
Hi How to add a table into the lakehouse? I dont have one.. br. R.
@ashanw 2 ай бұрын
Great explanation and good content. Can you kindly share the yml file with me? Thanks
@naimuddinsiddiqui9249 2 ай бұрын
Great explanation , If we will do any changes in dataset which is in our local pc how would it reflect the data changes in kql do we have to establish any bridge like Integration run time/Virtual/Cloud gateway ?
@olegkazanskyi9752 2 ай бұрын
I get this error when I'm trying to clone a table. Any hints on how to resolve it? Feature 'DISCOVERED TABLE' is not supported by table clone.
@vishwanathvt7701 2 ай бұрын
I have created the Synapse workspace what is the username and password? How to set that?
@MrLee1334 2 ай бұрын
Hey while working with parquet files ive noticed depending on sql query complexity it may occur that running the exact same SQL query multiple times for the exact same parquet file, it may result in different results - has anyone ever noticed that same behavior before?
@juanm555 2 ай бұрын
Excellent video, Abhishek explains everything in a wonderful way. Eagerly expecting more videos with him!
@BUY_YOUTUB_VIEWS_378 2 ай бұрын
🎉
@moeeljawad5361 2 ай бұрын
Thanks for this video, i am currently using the notebook activity in Fabric Pipelines. My notebook is mature now and it runs very well. I was thinking of gathering the code in the notebook into a job definition, for the sake of saving execution time in the notebook. Would replacing a notebook by a job definition makes the code execution faster? Another question would be about job descriptions themselves, if you have defined some helper functions in the notebook, can i move them to a side job definition that is being called from the main job definition? if yes then how? Thanks
@keen8five 2 ай бұрын
I'd love to see the Capacity Unit consumption of a Notebook execution in the Monitoring Hub
@Elizabeth-st4yk 2 ай бұрын
Noted. This request is in our backlog.
@rankena 2 ай бұрын
Is there a way to generate Bearer token programmatically?
@SumitArora-zf3of 2 ай бұрын
What are the options to build a Power BI report on a large dataset if it contains lets say 500millions of records?
@user-dy8xu7uj8k 2 ай бұрын
Hi, Good Morning!, I have to convert the existing SQL server stored procedure into fabric environment, In my stored procedures there are CURSOR commands but fabric doesnt support CURSOR commands, in this case how do I proceede, is there any alternative.
@Get_YT_Views_579 2 ай бұрын
Thanks for the positivity!
@peterlapic6761 2 ай бұрын
Is there a way to perform Lifecycle Management policy on the Dataverse using Synapse Link? Want to pull all data from Dataverse to the Datalake the way Synapse Link does but delete old data in the Dataverse but still retain it in the Data lake. I want the data in the data lake to run through the Azure Lifecycle management policy so that it ends up in the cooler tiers to save cost but still be reportable for PowerBi using serverless sql.