DP-203: 43 - Azure Synapse Dedicated SQL Pool

DP-203: 43 - Azure Synapse Dedicated SQL Pool - Architecture and overview

Рет қаралды 591

Күн бұрын

Hey data engineers! We've finished transforming our data - it's now pure, pristine, and ready to be consumed as facts and dimensions. Great! The only issue is that it's located in a data lake using the delta format, and not every consumer can connect to it. Or maybe we just prefer a good, old-fashioned, traditional relational data warehouse instead of this "data lake thing." What should we do? How can we make it work?
Fortunately, Azure Synapse Analytics has a component that can be used as a "relational" data warehouse: the Dedicated SQL Pool.
Join me in the 43rd episode of my free DP-203 course, where I'll present an overview of the Dedicated SQL Pool, explain how to integrate it into our design, and discuss its architecture.
Enjoy!
▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬
My LinkedIn profile: / piotr-tybulewicz-81a8793
GitHub with my drawings: github.com/TybulOnAzure/DP-203
▬▬▬▬▬▬ MEMBERSHIP ▬▬▬▬▬▬
Join this channel to get access to perks:
/ @tybulonazure
▬▬▬▬▬▬ CHAPTERS ▬▬▬▬▬▬
00:00 Introduction
01:29 Synapse components
04:44 Where does it fit?
12:22 Architecture of Dedicated SQL Pool
26:50 Performance levels
30:18 Creating Dedicated SQL Pool
36:54 Hash distributed tables
55:34 Create a hash distributed table
58:36 Round-robin distribution
1:06:17 Replicated tables
1:11:04 Summary

Пікірлер: 20

@TheMapleSight 5 күн бұрын

It really like your style of teaching. You firstly provide use with some problem like "Not every consumer is able to connect to ADLSg2 and read Delta format" and then you explain how to solve it using tools and features that are required on the exam. It's phenomenal!

@TybulOnAzure 5 күн бұрын

Glad you liked it!

@TheMapleSight 4 күн бұрын

38:53 This look gave me chills XDDDDD

@TybulOnAzure 3 күн бұрын

@pst659 13 күн бұрын

thanks

@TybulOnAzure 13 күн бұрын

You're welcome!

@prabhuraghupathi9131 20 күн бұрын

Thanks Piotr for this great video on dedicated SQL pool and its distribution methods..!! Have subscribed to your channel as a Data Engineer to learn more about data engg to enhance my knowledge!!

@TybulOnAzure 20 күн бұрын

Great! Welcome aboard!

@jacklarrytairo5562 19 күн бұрын

Hello Piotr, thank you for the excellent excellent explanation, I am preparing for DP-203 and I took Microsoft test exams and I saw several questions about partitions and sharding , I am looking forward to that chapter,

@TybulOnAzure 19 күн бұрын

Thanks Jack. Episode about partitioning in Synapse Dedicated SQL Pool will be recorded quite soon.

@LongshengZhao 11 күн бұрын

@@TybulOnAzure Hello Piotr, I would like some help with this question on partitioning if possible: You have an Azure Synapse Analytics dedicated SQL pool. You plan to create a fact table named Table1 that will contain a clustered columnstore index. You need to optimize data compression and query performance for Table1. What is the minimum number of rows that Table1 should contain before you create partitions? A. 100,000 B. 600,000 C. 1 million D. 60 million. Most ppl (including me) go with D but I also saw many ppl choosing C, moreover on many website the editor's answer is even A. Appreciate it if you could provide some insight on this as my exam is approaching soon!

@TybulOnAzure 11 күн бұрын

@@LongshengZhao Due to Candidate Agreement (learn.microsoft.com/en-us/credentials/support/certification-exam-candidate-agreement) I'm not discussing any exam questions. As for partitioning - today I'm recording an episode about it and it will be available early next week for "Data Engineer" members of my channel. Remaining viewers will be able to watch it in two weeks.

@mission_possible 13 күн бұрын

Thank you man

@TybulOnAzure 13 күн бұрын

Any time

@TheMapleSight 4 күн бұрын

I'm just wondering... It seems to me that you said in one of the episodes that there are problems with Azure Synapse Analytics and Microsoft will not necessarily support it. Is it still worth learning or maybe concentrate on Fabric for example?

@TybulOnAzure 3 күн бұрын

Microsoft is supporting it and will support it as many customers built their solutions using Synapse Analytics. On the other hand, we should rather not expect many new features added to it. If I were you, I would focus on Fabric (unless you have an existing project where you use Synapse Analytics).

@TheMapleSight 3 күн бұрын

@@TybulOnAzure but I guess most of the concepts (if not all of them) that you talk about here are useful in Data Engineering workflow and by proxy in Fabric, so it's still worth every minute of my time to watch this series

@ryleyalexander8097 13 күн бұрын

Hi Piotr, Loving this course, im currently on episode 06 and its helping me understand so much easier than i thought i would. Im preparing to take my dp-203 exam. Ive already passed the az-900 and im currently a business intelligence analyst and aiming to move up to data engineer, ive been working with oracle sql developer with etls for regulatory reporting return creation. I was wondering if i follow all the episodes in your playlist will i have gained enough knowledge to take the exam and pass or should i learn more content that you havent made videos on, on a site like udemy? Which site did you use to go over the course and exam prep questions? Thank you😁

@TybulOnAzure 13 күн бұрын

Hi, based on the feedback I received from other students - yes, it is possible to pass the exam based on my playlist. However, I strongly recommend to practice the stuff I'm talking about and visit DP-203 page on MS Learn: learn.microsoft.com/en-us/credentials/certifications/azure-data-engineer/?practice-assessment-type=certification

@ryleyalexander8097 13 күн бұрын

@@TybulOnAzure Thank you so much for the response Piotr! Feels like a response from a celebrity :D haha im joking. will continue my studying using your content and let you know how the exam goes! :D will also join the membership because your explanations are the best on youtube! thank you sir