No video

Azure Synapse Analytics & Power BI for BANANAS SCALE!

  Рет қаралды 58,041

Guy in a Cube

Guy in a Cube

Күн бұрын

Scale your data in Power BI with Azure Synapse Analytics! Azure Synapse can take your data to the next level with great performance.
Connect with Josh Caplan: / joshcaplan1984
📢 Become a member: guyinacu.be/me...
*******************
Want to take your Power BI skills to the next level? We have training courses available to help you with your journey.
🎓 Guy in a Cube courses: guyinacu.be/co...
*******************
LET'S CONNECT!
*******************
-- / guyinacube
-- / awsaxton
-- / patrickdba
-- / guyinacube
-- / guyinacube
-- guyinacube.com
**Gear**
🛠 Check out my Tools page - guyinacube.com...
#PowerBI #Synapse #GuyInACube

Пікірлер: 78
@youngyingyang
@youngyingyang 4 жыл бұрын
I didn't know Ashton Kutcher was doing work on Synapse!!
@GuyInACube
@GuyInACube 4 жыл бұрын
lol :)
@hrishihatolkar1944
@hrishihatolkar1944 4 жыл бұрын
😂
@agasti2007
@agasti2007 4 жыл бұрын
Ashton Kutcher with Will Smith 😝
@3CloudSolutions
@3CloudSolutions 4 жыл бұрын
Sad to see Josh leave the Power BI team, but AWESOME to see the work he's doing now on Synapse Analytics. This product has really come together seamlessly like nothing we've seen before in Azure.
@apocalance
@apocalance 4 жыл бұрын
I like that you had a guest on that was outside the scope of PBI, but was definitely in scope with getting data to PBI; which is a regular battle.
@robsonads1
@robsonads1 4 жыл бұрын
Congratulations! More videos like this Patrick. Please.
@Ferruccio_Guicciardi
@Ferruccio_Guicciardi 3 жыл бұрын
Awesome ! Thank you so much to finally fully integrated Data warehouse and Machine Learning together in a nice unified Azure work space experience. True unlimited scale ! Great enterprise analytics solution ! Beautiful Data Orchestration ! Thanks for sharing ! ETLing data from all those different clouds keep me hallucinating with all those endless possibilities ! Love that Data Lake + Data warehouse integration ! AI + ML Python code generator ! Wow ! Thank you for bringing Predicting Model (Machine Learning) combined with SQL ! Awesome orchestration ! Thanks for creating Azure Synapse Analytics and sharing this explanation. Handy to have an end-to-end, single work space environment. Very, very handy. Thanks for making ETL, BI, AI and ML much more simple. Bringing the power SQL to directly query serverless the DataLake is very interesting ! Any SQL Developer can bring a rich set of analytics by just writing SQL statements !
@micahdail
@micahdail 4 жыл бұрын
Bananas scenario is bananas. ;-) Thanks for the video. It's super timely!
@GuyInACube
@GuyInACube 4 жыл бұрын
hahaha thought you would appreciate that 👊
@ced4298
@ced4298 3 жыл бұрын
Love the role play! Non techie People really want and need to be able to hear the “message” in their own language.
@dineshsadhwani3717
@dineshsadhwani3717 2 жыл бұрын
2:17 Man, I lost it at "Little baby data"
@vxmine
@vxmine 4 жыл бұрын
Nice tools guys! I love to see this maturity across the products.
@GuyInACube
@GuyInACube 4 жыл бұрын
Agreed. Synapse is a great evolution of Azure SQL Datawarehouse.
@shinellebovell4251
@shinellebovell4251 4 жыл бұрын
I was worried that you could only create the PowerBi dashboard from the Azure Synapse portal but I am sooo happy that you can connect to your data warehouse directly from PowerBI desktop app.
@jw715
@jw715 3 жыл бұрын
When we create external table or view pointing to the data file with delta format in the data lake, it will return all the versions of each record, which means synapse does not support delta format files.
@johnnyp3161
@johnnyp3161 3 жыл бұрын
Hi @Guy, What are main reasons for using synapse database/view like this VS connecting directly to DataLake from Power BI?
@swerick
@swerick 4 жыл бұрын
An amazing feature would be to cache the on-demand SQL as well as the pool so no formal DW is necessary. Leverage views on top of raw data in blob or a data lake and do all translations with that ANSI SQL. I get that SQL Pool (Legacy SQL DW Gen 2) has this capability, but I don't want all of the heavy lifting. Let's tie into ADLS directly, do transformations with ANSI SQL and cache those results so we only hit that expensive workload once. Kimball methodology at the highest value. That's a major game changer....
@JCap383
@JCap383 4 жыл бұрын
There is some caching of the files there today. You'll notice your queries will get even faster if several different queries are accessing the same files. Everything I showed in the video was from a cold cache. Remember that the product is still new, and it will get even better from here as we incorporate customer feedback.
@crax2000
@crax2000 4 жыл бұрын
Just jumping here to say something about that YOOOO!!! from @Josh ! ;) Great video! Awesome intro to the whole new modern dw world! Keep going!
@heeigi9450
@heeigi9450 Жыл бұрын
Thanks for the video ❤
@DylanvanRiel
@DylanvanRiel 4 жыл бұрын
Any recommendations on when to use Synapse over SQL Database (thinking about data size/data formats/frequency of loads/etc.)?
@AHMEDALDAFAAE1
@AHMEDALDAFAAE1 3 жыл бұрын
Good question! From my experience getting data from SQL server DB to Power BI is very slow, and most SQL DB Admins freaked out when someone is connecting directly to SQL DB, rather it is better to use SSAS, Cube, or Azure DB as a bridge to Power BI as they offer Four's V: Value, Velocity, Variety, and .. I forgot the last one... Please correct me if I am wrong
@coilinboylanjeritslev3656
@coilinboylanjeritslev3656 3 жыл бұрын
...volume
@florianvandillen
@florianvandillen 4 жыл бұрын
Great stuff!
@yahorsinkevich4451
@yahorsinkevich4451 3 жыл бұрын
Please, please, please, more videos on Synapse Serverless & Power BI
@karout94
@karout94 4 жыл бұрын
Resultset Cache layer in Synapse will also be a layer before materialized view !! isnt it ?
@sau002
@sau002 3 жыл бұрын
Nice video. Please help me understand if this demo was about Azure Synapse Analytics OR Azyure Synapse Workspace? Thank you
@HorologicRannygazoo
@HorologicRannygazoo 4 жыл бұрын
Cautiously optimistic, but prior versions of this have been too pricey for companies I've worked with either with slot/concurrency issues on the DW or egress charges. Very interested to see the relationship of the performance to the pricing and the differences between the Spark-y stuff in Synapse over Data Lake, vs. Azure Databricks on Delta as well as the details on how "serverless" works and is billed, including when Power BI embedded is added to the mix. The integrated interface is definitely slick, though.
@saudmeethal644
@saudmeethal644 4 жыл бұрын
Yeah, the price becomes a major roadblock for smaller clients with huge data sets.
@zycbrasil2618
@zycbrasil2618 4 жыл бұрын
I agree. In my option reading huge datasets from DLGen2 as data source is not a best practice. Technically is ok but I would never recommend. I could list many reasons to avoid that.
@DarkOri3nt
@DarkOri3nt 4 жыл бұрын
Shhhh🤫 they don't like when you tell the truth. There are truths to the power of synapse. But its all using the same sort of engines that databricks use. And dropping files with no etl, great if the data is in the right format, it never is.
@HorologicRannygazoo
@HorologicRannygazoo 4 жыл бұрын
Yeah, I'm not expecting "no ETL". However, sometimes it would be nice to simply ingest cleansed source data and park it somewhere read only in partitioned parquet files and figure out if/what we want to do with it later.
@zxccxz164
@zxccxz164 2 жыл бұрын
Omg we r so look ok over budget on our project.
@ankitaahuja1809
@ankitaahuja1809 2 жыл бұрын
Can we connect Azure Synapse SQL Pool with Power automate? I am getting a bad gateway error when i try to connect.
@kyasoubi7407
@kyasoubi7407 3 жыл бұрын
Very interesting topic. could you make a video on azur analysis service and power BI!
@venugopal-jt6jm
@venugopal-jt6jm Жыл бұрын
What if i would like to connect with power bi report builder. How to connect synapse with power bi report builder
@bassemamer4893
@bassemamer4893 3 жыл бұрын
How to build a tabular model for power bi in synapse, or we have to use Azure analysis services integrated with synapse. and how power bi use Synapse with data modeling?
@zxccxz164
@zxccxz164 2 жыл бұрын
I can't find anyone to answer that
@elizabethtachjian3082
@elizabethtachjian3082 3 жыл бұрын
Hi Guys, Great explanations. Are you able to please further explain why no data transformation. I am assuming the transactional data starts in the CRM App and then the raw data is loaded to datalake Gen2. In almost all Power BI Report development instances, I have needed to format the column and/or clean the data etc. In my case using query editor in dataflows to clean data. Why is there no need to do any transformations in the sample data in the video. Thanks in advance.
@zxccxz164
@zxccxz164 2 жыл бұрын
Exactly!!!! Lol just open rowset.
@brendanbutler4815
@brendanbutler4815 Жыл бұрын
Does Azure Synapse Analytics effectively replace OLAP multidimensional cubes as they are basically faster cloud OLAP cloud services?
@andriyobukhovskyi9962
@andriyobukhovskyi9962 2 жыл бұрын
Correct me if I wrong, every time when user is working with PB report and changing some filters, SQL query will work on Azure Synapse resources? How about performance in case 100 users will work with report? I'm interested in performace.
@FrostSpike
@FrostSpike 2 жыл бұрын
How much are those jobs costing to run over that all those 1.6B rows of data? If no ETL/ELT is required, so no curation/standardisation being performed over that RAW data, is that being done each and every time and having to be replicated by every PBI job/model that runs over that data?
@vinayrana4664
@vinayrana4664 3 жыл бұрын
Does it cost if I add power bi in my workspace??
@DashcamRevenge
@DashcamRevenge Жыл бұрын
Will it support Query folding?
@denwo1982
@denwo1982 3 жыл бұрын
Why would you save your power bi reports on azure synapse and not publish them to power bi service? As we have users who would access the power bi dashboards we created via the Apps due to the licence they have
@andreasmohr7054
@andreasmohr7054 3 жыл бұрын
I am psyched.
@Crazy5exycurvy
@Crazy5exycurvy 4 жыл бұрын
I need to give Synapse more attention!
@akshaynm
@akshaynm 4 жыл бұрын
Hello Patrick, thanks for the nice video. Just wanted to request you for a topic. There is very limited stuff is available on Power BI Sentiment Analytics using Azure Cognitive Services. It would be best if you could create and share a video on that.
@kaveribhar8197
@kaveribhar8197 2 жыл бұрын
Hey Josh, does Azure Synapse supports Japanese. I really need this info. I know Azure cognitive services does. But I have to name Database or table or name in Japanese. Could I do those in Synapse or I had to handle this in Power BI alone. Please help.
@itsvtk
@itsvtk 3 жыл бұрын
Can you please give us the location of NYC Taxi data to play with Synapse Workspace along with the queries that you showed in this demo?
@GuyInACube
@GuyInACube 3 жыл бұрын
It's not publicly available to our knowledge. It's a large scale dataset that has cost associated with it.
@adorablecheetah2930
@adorablecheetah2930 3 жыл бұрын
Can anyone tell me why in Powerbi they entire data is downloaded? How is it safe from a security perspective and also how long should a user wait to download so many billions or rows on his own Machine
@yoshihirokawabataify
@yoshihirokawabataify 4 жыл бұрын
Thanks, nice movie, nice products, and more 🤩😍😘 I hope more movies
@visheshjjain
@visheshjjain 4 жыл бұрын
Newbie at SQL...what is the difference between Synapse and previously used SQL storages?
@zycbrasil2618
@zycbrasil2618 4 жыл бұрын
Synapse is an evolution of Azure SQL DW (Old on premise PDW).. It's based on MPP
@beaddy101
@beaddy101 4 жыл бұрын
Hi Patrick. From your scenario how is the aggregation table created. Thanks.
@GuyInACube
@GuyInACube 4 жыл бұрын
It's just an imported table. Can do it with Power Query, or have it pre-created on the data source side.
@ZenMel0dies
@ZenMel0dies 3 жыл бұрын
We can use dax requests on Synapse ?
@Fernando_Calero
@Fernando_Calero 4 жыл бұрын
A little estimate of the Azure cost of this deployment?
@JCap383
@JCap383 4 жыл бұрын
It all depends on which pricing method you use. If you go with the serverless on-demand option that I showed, you are charged per TB scan when you run queries. When you are not running queries, you are not being charged. There are also provisioned options where you purchase capacity. You can check you the different options here: azure.microsoft.com/en-us/pricing/calculator/?service=synapse-analytics
@johnfromireland7551
@johnfromireland7551 2 жыл бұрын
Looks like Synapse Analytics is an all-encompassing analytics tool. You don't need anything else for data engineering or analytics!!
@JackOfTrades12
@JackOfTrades12 4 жыл бұрын
Is synapse available today? Can I query data from a PowerBI dataset in synapse, using dax? Can I connect to an azure sql sever, write an view that gets incrementally refreshed, and store it in synapse? Thanks for this video, I've been looking forward to synapse for data cleansing for PowerBI since I saw it last summer!
@africa247
@africa247 4 жыл бұрын
Good video
@zxccxz164
@zxccxz164 2 жыл бұрын
Lol so the parquet file is clean.
@pravinupadhyay2046
@pravinupadhyay2046 3 жыл бұрын
Great video! Seems he has not faced Camera much,
@zoloturu6322
@zoloturu6322 4 жыл бұрын
Awesome, guys. I should try it. It consists of several services and practically all of them can be done at the Synapse Analytics portal (except Power BI dataset). Is the any service from Synapse Analytics which can be developed in Visual Studio?
@JCap383
@JCap383 4 жыл бұрын
We do support SSDT for your databases.
@FernandoThivanka
@FernandoThivanka 4 жыл бұрын
Can someone please tell me in what format is data stored in Azure data lake? ie: csv, json
@florianvandillen
@florianvandillen 4 жыл бұрын
You are free to use the format you like.
@vsnandy007
@vsnandy007 3 жыл бұрын
Parquet is probably the best format because it is compressed and quicker to query
@GregKramerTenaciousData
@GregKramerTenaciousData 4 жыл бұрын
awesome stuff @patrick & @josh! one nitpick-->geez, josh, get a better microphone!
@JCap383
@JCap383 4 жыл бұрын
Greg Kramer that is a good idea.
@pabtorre
@pabtorre 4 жыл бұрын
Demo was going great, until for some unknown reason, you have to go back to PBI desktop!!! Why would you want to go back to the desktop application??? Importing 1.6 billion records from the cloud to setup the PBI connections on the desktop only to push it back to the cloud... makes absolutely no sense
@JCap383
@JCap383 4 жыл бұрын
We never actually pulled 1.6 billion rows in the desktop. We queried them from the desktop. I happen to use direct query in this example. Had I used import, I would have aggregated and trim the rows in Power Query just like other data sources.
@pabtorre
@pabtorre 4 жыл бұрын
@@JCap383 having to go back to the desktop in order to build the dataset instead of having connect directly on the cloud, seems like a missed opportunity for a very significant feature change. Pulling and pushing datasets to/from the cloud is one of the biggest obstacles I find when using PBI.
FAST - Billions of rows with Azure Data Explorer (ADX)
10:32
Guy in a Cube
Рет қаралды 11 М.
Идеально повторил? Хотите вторую часть?
00:13
⚡️КАН АНДРЕЙ⚡️
Рет қаралды 17 МЛН
Challenge matching picture with Alfredo Larin family! 😁
00:21
BigSchool
Рет қаралды 28 МЛН
MASTERING Bar Charts in Power BI | No more Cut Labels
15:34
How to Power BI
Рет қаралды 111 М.
STOP publishing your Power BI report until you do these 5 things!
10:15
Learn the Fundamentals of Microsoft Fabric in 38 minutes
38:00
Learn Microsoft Fabric with Will
Рет қаралды 148 М.
Power BI + Azure Synapse Analytics: Beginning your journey
49:33
Efficient Power BI Dataset refresh with Azure Synapse Analytics Serverless SQL Pool
15:06
Top 5 things to get started with Azure Synapse Analytics
9:18
Guy in a Cube
Рет қаралды 11 М.
Azure Analysis Services Tutorial | Scale Power BI reports into hundreds of GBs
29:53
Adam Marczak - Azure for Everyone
Рет қаралды 112 М.
SQL Server in Microsoft Fabric? Use CETaS to move data!
10:15
Guy in a Cube
Рет қаралды 24 М.
Идеально повторил? Хотите вторую часть?
00:13
⚡️КАН АНДРЕЙ⚡️
Рет қаралды 17 МЛН