No video

Creating real time dashboards in Power BI with push datasets

  Рет қаралды 98,882

SQLBI

SQLBI

Күн бұрын

Though you can build real-time reports with DirectQuery, push datasets offer a more scalable, economical, and effective solution especially when combined with an Import model already in place. This video introduces the architecture of push datasets.
Article and download: www.sqlbi.com/...
How to learn DAX: www.sqlbi.com/...
The definitive guide to DAX: www.sqlbi.com/...

Пікірлер: 73
@humd3la167
@humd3la167 2 жыл бұрын
this is absolutely wonderful. thank you for the thorough introduction! Last time we needed "real time" for a report we resorted to direct query and "refreshing" - super slow and inconvenient... I hope to try this out soon!
@MrDaoneill
@MrDaoneill 2 жыл бұрын
What a foruitous video! I have recently been posed the challenge to find a way to provide real time dashboard within my organisation and this information and tool will be invaluable. Marco & Alberto thank you so so much for this video and the article.
@wexwexexort
@wexwexexort 2 жыл бұрын
This sounds really exciting. Great job, great explanation.
@EduardoAdanHuizaZarate
@EduardoAdanHuizaZarate Жыл бұрын
This is sooooo crazy, I am in a project that require push datasets. Going to read all the documentation right now :)
@emirtuncer
@emirtuncer 2 жыл бұрын
It sounds fantastic. Im very excited to watch the video. Thanks @Sqlbi team
@Milhouse77BS
@Milhouse77BS 2 жыл бұрын
Always good to see dashboards to get some love.
@yidaveding
@yidaveding 2 жыл бұрын
thanks for sharing. excellent video. This push data method also opens up a lot of new possibilities.
@BillyBob618
@BillyBob618 2 жыл бұрын
You guys did it again! Seriously this is great stuff
@Tschick
@Tschick 2 жыл бұрын
Great stuff! It might be very helpful for a specific dashboard of a client, where I am currently using direct query. Thanks for the video
@snehasamuel7426
@snehasamuel7426 19 күн бұрын
Mind-blowing 🤯
@federicocomito6815
@federicocomito6815 8 ай бұрын
Thanks for the amazing video! I apologize in advance for my ignorance but, is the api call still gonna work if my machine is off?
@meipolomiga471
@meipolomiga471 2 жыл бұрын
thanks for sharing this video..so wonderful..
@nhuananh
@nhuananh 2 жыл бұрын
Great solution!
@LuanNguyen-pl2wf
@LuanNguyen-pl2wf 2 жыл бұрын
this is just wow thank you !
@abhijithvm3166
@abhijithvm3166 18 күн бұрын
Very informative video, and I believe it's really helpful for aspiring data analysts. Currently, I am preparing a dashboard based on my day-by-day expenses. Sir, is there any possibility to make it a live dashboard based on my Excel data? Is it possible or not? Finally, thank you for the informative video.
@SQLBI
@SQLBI 16 күн бұрын
You should look at OneDrive publishing rather than push datasets for that. However, probably you have to schedule a refresh.
@joannak4329
@joannak4329 2 жыл бұрын
I love it!
@markqfairway7489
@markqfairway7489 2 жыл бұрын
Awesome Stuff! Thanks. Will this work with Oracle Database?
@SQLBI
@SQLBI 2 жыл бұрын
It is not related to the data source, it is up to you to build the technique to push updates, though.
@suvarchaladevikanchustambh8162
@suvarchaladevikanchustambh8162 2 жыл бұрын
Hi, For continuous running of dashboard do we need to run the powershell application continuously? Is it possible for running powershell command automatically?
@SQLBI
@SQLBI 2 жыл бұрын
You should use an automation process to run powershell, on Azure or on a PC - you can also look for Power Automate.
@pbihari0214
@pbihari0214 2 жыл бұрын
Thank you
@elecirlisboa9155
@elecirlisboa9155 2 жыл бұрын
Excel explanação, parabéns!
@GregKramerTenaciousData
@GregKramerTenaciousData 2 жыл бұрын
Is anyone else getting this error message: Confirm-PushDataset : The term 'Confirm-PushDataset' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again. At line:3 char:1 + Confirm-PushDataset -Model 'C:\Demo\Contoso.bim' + ~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (Confirm-PushDataset:String) [], CommandNotFoundException + FullyQualifiedErrorId : CommandNotFoundException
@ShekarCReddy
@ShekarCReddy 2 жыл бұрын
You are running a Command line utility (your Push app) in the shell on your desktop. How does that work in Power BI Service as a command line app? Do we need to install anything on the Service?
@SQLBI
@SQLBI 2 жыл бұрын
The command-line app is for testing purposes only. In order to push data into the dataset, you need to call the Power BI REST API through a service in the cloud that intercepts new data in your system. As a polling technique and no-code solution, you can use Power Automate to do the job.
@banklc7998
@banklc7998 2 жыл бұрын
show Confirm-PushDataset -Model 'C:\xxx.\desktop\Contoso.bim' no command name cmdlet: when i repeat your work in the powershell
@SQLBI
@SQLBI 2 жыл бұрын
Read the article, you should install the PowerShell cmdlet first.
@Baldur1005
@Baldur1005 2 жыл бұрын
I really like the idea, but... I got the feeling that you present Direct Query in pure form which - I guess is not best - but we have dual storage mode and aggregations which are increasing DirectQuery feasibility a lot. And soon the automatic aggregations will be introduced. Having it in mind is push dataset is really better solution?
@SQLBI
@SQLBI 2 жыл бұрын
Push dataset have the unique feature of updating the dashboard tiles as soon as new data arrive. Aggregations in DirectQuery should be refreshed periodically for the entire table (no partitioning) so currently there is no way to have an hybrid tables (part DirectQuery and part Import) that would solve the issue. However, even with hybrid tables you would still have a scalability issue with the DirectQuery part in case you have many users.
@Baldur1005
@Baldur1005 2 жыл бұрын
@@SQLBI Thanks a lot for explanation
@nachetdelcopet
@nachetdelcopet 2 жыл бұрын
Excelent video! Would you recommend use this tool if the original data source are json files in an Azure Blob Storage?
@SQLBI
@SQLBI 2 жыл бұрын
It's hard to implement real-time updates with that architecture: you should create a system (with Power Automate or a script/service) that polls the blob storage looking for new files very often. Not sure it would be much different than a frequent incremental update (but it would be every 1-2 hours, not every minute).
@nachetdelcopet
@nachetdelcopet 2 жыл бұрын
@@SQLBIThanks for the reply! Maybe it would make sense to add an additional layer to transform data into table. It would be a good idea to cover the implmentation of the library presented in this video for Azure Cloud DB connection
@Northsiderwoof
@Northsiderwoof 2 жыл бұрын
Hi thanks for a great solution. I've worked on it and made it working as far as incrementally refresh the push dataset after batch updating from DAX results from its twin in-memory model. Then I am hitting the wall with RLS - is RLS not supposed to be working with push dataset? I reviewed microsoft documentations etc, but couldn't find a definite answer for that.
@SQLBI
@SQLBI 2 жыл бұрын
Unfortunately, RLS is not supported in push datasets.
@AJ-alt
@AJ-alt 2 жыл бұрын
I haven't finished the video, but ist his possible when im getting the data from a connected database, say Smartsheet or other online data source?
@SQLBI
@SQLBI 2 жыл бұрын
Please, can you clarify the question?
@nagarjunaambati3141
@nagarjunaambati3141 Жыл бұрын
I have to build realtime dashboard. but for one of the visualization I need to perform Pivot transformation. is this achievable in though this process? (Pivot transformation not supporting when using direct query)
@SQLBI
@SQLBI Жыл бұрын
Push datasets don't have the limitations of DirectQuery. Power BI matrix visual (which is the way you obtain a pivot transformation) are compatible - but that's only at visual level, if you are talking about M, then push datasets don't have anything like that, you must transform the data upfront.
@jamesross5120
@jamesross5120 2 ай бұрын
is it just me or is the whole live dashboards really not worth the cost! i mean when would you benefit from live dashboards in business over a regular report.
@SQLBI
@SQLBI 2 ай бұрын
The "real-time" behavior could be good for a monitor that shows the information updated real-time. We call them "vanity projects" as they do not have a real business value, but they could be a motivational object. We've seen them used in monitor in the hall of a company (or in private areas when there are sensitive company data!).
@Matt_H_
@Matt_H_ 2 ай бұрын
Perhaps when it gets to the level of investor meetings? Other than that, as said, vanity.
@anilsharma-lu9cm
@anilsharma-lu9cm Жыл бұрын
Thanks looks interesting.. however just question/limitation with push datasets.. although looks fantastic to see live data but --for me it falls on duplicates.. its tough to dedup live data ..if the data coming in is not filtered properly and if there are data surprises. It can give erroneous dashboard. some existing tools which uses Push -data on ports like machine logs.. struggles with matching legacy and live data Over night pull data can be cleansed and filter through business rules ..etc.. but liked your diagrams and data flow presentations
@SQLBI
@SQLBI Жыл бұрын
The entire idea behind these real-time dashboards is to implement "vanity projects" without creating a dedicated infrastructure that is way more expensive to maintain. The daily cleanup partially solves the issue you mentioned. Clearly, you must have an error tolerance about the live data.
@papediouf1640
@papediouf1640 Жыл бұрын
​@@SQLBIi need your personal contact please. I have a project in mining industry
@yogeshdornala
@yogeshdornala 2 жыл бұрын
This is really helpful. Is there anyway that we can add slicers? At present, we cannot add slicers to dashboard. If we add it to the report, we have to refresh the page for the latest data. Is there any other way to do this?
@SQLBI
@SQLBI 2 жыл бұрын
No you cannot add slicers to a dashboard.
@narianu0
@narianu0 2 жыл бұрын
What about building a composite model with a similar architecture to what you described? You have your import tables and then a single DQ table that pulls in today's data? I know you have the DAX limitations, but it seems like the performance component would be mitigated in a large way.
@SQLBI
@SQLBI 2 жыл бұрын
You would not remove the scalability issue for DirectQuery and you would lose the automatic refresh of the dashboard tile when there is new data. Performance depends on the implementation.
@user-ze4xc6tu4f
@user-ze4xc6tu4f 2 жыл бұрын
amazing
@dfkgjdflkg
@dfkgjdflkg Жыл бұрын
One other solution is to FTP directory running a script that is running the same script that updates the data.
@willgart1
@willgart1 2 жыл бұрын
I think we can use power automate to push the data, no? there is a power automate "add rows to a dataset" connector.
@SQLBI
@SQLBI 2 жыл бұрын
Yes you can! Just pay attention to what you use in Power Automate to detect the presence of new rows, because a polling *might* be resource consuming for the server. But it definitely works!
@GagandeepSingh-db8eh
@GagandeepSingh-db8eh 2 жыл бұрын
I have a question regarding the tabular editor, Like We have some client laptops I made one model and deployed it in SSAS. Then The next day I am on another laptop can I load deployed model setting in the Tabular editor or do I need to bring saved local to new machine and open that file then only I can see all my work like CKF, Measures, relationships etc. thanks in advance
@SQLBI
@SQLBI 2 жыл бұрын
You can either copy the model.bim file manually or connect Tabular Editor to the server and retrieve the model from the server. However, we suggest that you use the Tabular Editor forum for these questions: github.com/TabularEditor/TabularEditor/issues
@GagandeepSingh-db8eh
@GagandeepSingh-db8eh 2 жыл бұрын
@@SQLBI Thanks a mill, I did it manually before will try how to retrieve it from the server. thanks again
@lifeisajourney2839
@lifeisajourney2839 2 жыл бұрын
thanks for the informative video. By any chance, is there an alternative tool that could generate .bim as tabular editor is not open source and neither it is a freeware. Your feedback and comments are appreciated
@SQLBI
@SQLBI 2 жыл бұрын
Tabular Editor 2 is free and open source: www.sqlbi.com/tools/tabular-editor/ and github.com/TabularEditor/TabularEditor
@sonsangsom
@sonsangsom 2 жыл бұрын
How is the Dual mode in Power BI different from this approach sir?
@SQLBI
@SQLBI 2 жыл бұрын
Push datasets update dashboards as soon as new data comes in. Dual mode can keep data in Power Query, but you always have to trigger a dashboard refresh in order to see new data.
@aphamila
@aphamila 2 жыл бұрын
Thanks for the great video, I get, Failed to get shared database name. Description: {"Message":"API is not accessible for application" When trying to refresh data from import DB to push DB
@SQLBI
@SQLBI 2 жыл бұрын
Please, write the technical questions in the comments of the related article and make sure you follow all the steps described here: www.sqlbi.com/articles/implementing-real-time-updates-in-power-bi-using-push-datasets-instead-of-directquery/
@AnandDwivedi
@AnandDwivedi 2 жыл бұрын
Is the hybrid table going toachive the same
@SQLBI
@SQLBI 2 жыл бұрын
It depends. You cannot have automatic updates as shown in the dashboard, you could implement automatic refresh in the report. However, every report sends query to the DirectQuery partition, potentially introducing DirectQuery latency to every report (where is your data source?) and creating a bigger workload on the datasource (read: consider scalability for your data source). The hybrid tables should be easier to implement if the data source fits well in the DirectQuery scenario and you are able to configure the partition wisely. The push datasets are a good alternative for data sources that would not perform well in DirectQuery and/or you have transformation that don't comply with the folding requirements for hybrid tables. As usual, every tool has pros and cons, having more options is better because you have more choices and depending on the scenario you can choose the best one.
@ugovirilli5834
@ugovirilli5834 2 жыл бұрын
where did you get your chair ?
@SQLBI
@SQLBI 2 жыл бұрын
You can find it here: www.sqlbi.com/blog/alberto/2021/02/14/on-my-recording-gear/
@user-ts9kk4nd4w
@user-ts9kk4nd4w 2 жыл бұрын
Will this tool work with power bi report server?
@SQLBI
@SQLBI 2 жыл бұрын
No, push datasets only works in Power BI service.
@jujubalismel
@jujubalismel 2 жыл бұрын
Thank you very much! So seems that don't work very well to "update" data.
@SQLBI
@SQLBI 2 жыл бұрын
What do you mean?
@jujubalismel
@jujubalismel 2 жыл бұрын
@@SQLBI I mean that works really well when we are increasing the database, but seems that when we need to update some figures don't work, is that right?
@SQLBI
@SQLBI 2 жыл бұрын
For the push dataset part, an update can be obtained by pushing a compensating transaction. The biggest issue is that you have to identify the change first. Once you know that a transaction changed from $10 to $8, you have to push a compensating transaction of $-2 Or, you can just ignore that, knowing that these adjustments will be reflected in the next daily refresh. For example, in an implementation we made, we just ignore refunds made in the day, because they will appear the following day in the regular daily refresh.
🚨 YOU'RE VISUALIZING YOUR DATA WRONG. And Here's Why...
17:11
Adam Finer - Learn BI Online
Рет қаралды 99 М.
Build THIS! Report DESIGN in Power BI | FULL TUTORIAL
44:14
How to Power BI
Рет қаралды 622 М.
Чёрная ДЫРА 🕳️ | WICSUR #shorts
00:49
Бискас
Рет қаралды 6 МЛН
Row context in DAX explained visually
21:45
SQLBI
Рет қаралды 12 М.
Real-time streaming in Power BI - Push Dataset
29:19
Learn Azure Data Services with Dinesh Priyankara
Рет қаралды 9 М.
Power BI Get Data: Import vs. DirectQuery vs. Live (2021)
8:31
Guy in a Cube
Рет қаралды 301 М.
5 DESIGN TRICKS that Make EVERY Power BI Report Look GREAT!
20:22
How to Power BI
Рет қаралды 572 М.
DAX by examples
1:11:50
SQLBI
Рет қаралды 218 М.
How to Build Power BI Reports from Start to Finish
41:07
MyOnlineTrainingHub
Рет қаралды 1,4 МЛН