Introduction to Data Mesh - Zhamak Dehghani

  Рет қаралды 86,761

Thoughtworks

3 жыл бұрын

Today, data is ubiquitous. Data is the by-product of any and every action we take. Everything, every system, every process, every sensor generates data. Technology makes it easier for organizations to collect and store data, for businesses to leverage to make better decisions or create more tailored experiences for their customers.
However, organizations are struggling to enable and empower their employees to make the most informed and timely decisions possible. Centralized data platform architectures fail to deliver data with the speed and flexibility scaling organizations need.
Data mesh is a response to this problem. Data mesh applies the principles of modern software engineering and the learnings from building robust, internet-scale solutions to unlock the true potential of enterprise data.
Learn more at www.thoughtworks.com/what-we-do/data-and-ai/data-mesh

Пікірлер: 47
@TopperDillon
@TopperDillon Жыл бұрын
What a great presentation and introduction into the Data Mesh paradigm! Super helpful! Thanks a lot Zhamak!
@rabbanikajamohideen8075
@rabbanikajamohideen8075 2 жыл бұрын
Excellent introduction on Data Mesh.. I feel for the organisation which are already evolving towards Domain Oriented microservices this will be the perfect model to take care the Data Platform as well
@HealthcareMasterData
@HealthcareMasterData 2 жыл бұрын
To have the privilege of being the first to “like” the concepts on your video 🙂. From the Healthcare Industry, to begin the journey to “mesh” successfully via federated data governance, where the business defines the quality, the meaning, the value also requires teachers and guides who can speak the language of the clinician and the engineers. There are but a few of us, and it’s beyond exiting to help organizations succeed. But oh how I wish we could teach more quickly #DataScienceIsAmazingFun #SendTheStudents #RecruitTheCuriousYetDedicated
@BhanuPratap88
@BhanuPratap88 2 жыл бұрын
Thanks, Zhamak for introducing the intriguing concept of data mesh Thanks a lot!
@ghoshrajk
@ghoshrajk 2 жыл бұрын
Excellent introduction to the concepts of Data Mesh. Thank you.
@AB-xg1qb
@AB-xg1qb 3 жыл бұрын
Very well explained about Data Mesh . Thanks Zhama!
@SuperMakija
@SuperMakija 2 жыл бұрын
Definitely an insightful look into Data. I loved the notion of thinking and treating Data as a Product. Hope organisations will be mature enough to understand and embrace data mesh thinking. I am also looking to Data Mesh as a way to exchange data between organisations.
@AmitThakur-xq2cb
@AmitThakur-xq2cb 2 жыл бұрын
Insightful analyses. Thanks Zhamak!
@generousmeanuj
@generousmeanuj 2 жыл бұрын
Amazing concept and very beautifully explained concept.
@Liang-Lu
@Liang-Lu Жыл бұрын
This is such a great introduction! Thank you Zhamak!
@guybourne1284
@guybourne1284 Жыл бұрын
What a great explanation of why this sort of shift is needed. I'd love to compare the business cases for data lakes from 2010's to the business cases for data mesh in the 2020's.
@DanKeeley
@DanKeeley 2 жыл бұрын
for those short on time go straight to 13:00
@pliston1
@pliston1 Жыл бұрын
Really excellent. Thank you Zhamak!
@dzengiztafa510
@dzengiztafa510 2 жыл бұрын
Very good presentation... Needs a lot more views!
@grabberke2
@grabberke2 2 жыл бұрын
Very nice introduction. Thank you for this!
@vidviewer20
@vidviewer20 Жыл бұрын
Great presentation Zhamak. Very insightful and informative. I took a bunch of notes and would have been great to ask you questions. I think the industry and functional owners on the customer side generally talk in terms of solution space and capabilities, whereas what execs are thinking about in the problem space is all about how to make more effective use of data, trusting and understanding reports and creating a data-driven culture / decision making etc. I enjoyed learning about some of this from this video.
@i.y4275
@i.y4275 Жыл бұрын
Very impressive for the data evolution from warehouse, to lake and to mesh.
@wyhoe3140
@wyhoe3140 Жыл бұрын
Very clear and insightful presentation! This is going to provide more flexibilities to the organisation to cope with the latest business challenges
@farrukhhassan5656
@farrukhhassan5656 2 жыл бұрын
Definitely a next step vision for bringing consumers one step closer to sources.. however as you said it requires organisational change along with technology and architecture.
@gregorygreen6156
@gregorygreen6156 Жыл бұрын
awesome content and beautifully explained, thank you
@indumathipatalay3342
@indumathipatalay3342 9 ай бұрын
Data as a product is whats more fascinating piece for me❤
@morespinach9832
@morespinach9832 2 ай бұрын
And where do we find info about this dream?
@vibhaskashyap8247
@vibhaskashyap8247 3 ай бұрын
Awesome presentation !!
@ranjangupta5544
@ranjangupta5544 2 жыл бұрын
This is exactly what I am doing for 5g Data Mesh architecture .... where Network Data as a Service ... because simply centralized model is not going to work .... data is the oil .... but it is evaporating oil it needs to be used, monitored, serviced, secured reliably in a very short amount of time .... time is the essence .... But nice presentation .....
@angsh
@angsh Жыл бұрын
Just amazing !!
@leonidas7692
@leonidas7692 2 жыл бұрын
This is missing the business architecture topology that (sets the boundaries for domains, and therefore Data Products) and value streams which connect processes between capabilities (which domain context maps, and therefore Data Product interfaces, can be advised by). Without this, we'll recreate the mess of data silos where a "domain" is abused by following the existing org design (ala Conway). It's a great concept, and I have no doubt some enthusiastic early adopting enterprises would start doing this, but they will come back to some of the concepts that Data Mesh departs away from to undo the mess. I love the thinking here, and I believe would be a good fit with the abstractions in MIT CISR's Designed for Digital book. You have to think about the consequences of the topologies you create or grossly misalign the data landscape with the business and technological ones.
@BlueIced256
@BlueIced256 Жыл бұрын
I like the concept of data mesh, but I have to agree with you. From a theoretical perspective it is a really interesting idea. But it seems as if some assumptions are made that most likely will be very hard to get implemented in organisations. Maybe even impossible, when people have to learn things that they don't feel as part of their job or are not interested in to learn. It feels a bit like yet another solution to a problem that doesn't incorporate human behaviour. I think you will see quite a few hybrid solutions.
@AD-tx3gx
@AD-tx3gx 2 жыл бұрын
What are your thoughts on Master data management and data quality associated with that? Thats where some of the biggest challenges are in all the data platform projects.....
@DanKeeley
@DanKeeley 2 жыл бұрын
Do you find once you start building these different models or domains, that you actually end up finding / linking the domains more than ever before?
@bhanu4j
@bhanu4j 2 жыл бұрын
are these siloed domains.. You will only be addressing domain specific use cases using this approach.
@1riddles1
@1riddles1 2 жыл бұрын
I would say, trusting someone means believing that they have good/positive/empowering intentions for the context.
@rishiraj2548
@rishiraj2548 Жыл бұрын
Great
@relish1111
@relish1111 2 жыл бұрын
Extortionary detail information about data mesh. Thank u
@viniciusuliana
@viniciusuliana 3 ай бұрын
Great presentation. I have many questions to how implement Data Mesh in organization. How can I contact with you? Thank you!
@tpsadvocatekhanna611
@tpsadvocatekhanna611 8 ай бұрын
where you got this data for mesh ?
@arunlouie
@arunlouie Жыл бұрын
Very well articulated. Definitely not Data mesh 101, need some prior knowledge to see self in the implementation path with this video
@c147guitar7
@c147guitar7 2 жыл бұрын
I have one highly complex DB2 view that uses 60+ tables in a 7 level Select. It crosses 7 data domains that we are implementing, so it has to be split to multiple views, one per data domain, right? And a service will call other services in each data domain to merge data from the domains? Benchmarks say response time increases from 0.5 seconds to 10+ minutes. Any suggestions?
@BlueIced256
@BlueIced256 Жыл бұрын
Is that the typical view on top of a view problem, where the 'pipeline' is not optimised...?
@rqn2274
@rqn2274 Жыл бұрын
Finally I know how to pronounce your name :)
@user-eb1tq1ul1b
@user-eb1tq1ul1b 3 ай бұрын
AWS Datazone?
@ajithkannan522
@ajithkannan522 Жыл бұрын
listen at 1.5x speed and still u dont miss any thing
@itisgopsy
@itisgopsy 10 ай бұрын
How data Mesh is different to normal data engineering task is not super clear
@itisgopsy
@itisgopsy 10 ай бұрын
once the pipeline is automated is runs unless there is a data issue ... and those issues are not going away ...
@1paav
@1paav 2 жыл бұрын
Looks like fraud. A lot of nice words and nothing real.
@francescop1410
@francescop1410 2 жыл бұрын
Agreed, "we really need to move away from pipeline"... what?!? How are you going to move and process data then? Maybe I don't get it yet, but we'll see how this goes
@ivantang5795
@ivantang5795 Жыл бұрын
Lots of fluff, haven't seen any organization successfully implemented this so called "data mesh" yet.
@bartkl
@bartkl Жыл бұрын
@@francescop1410 She goes on to explain what she means in the next sentence: "pipeline as an architectural concept... it's quite challenging when you think about scaled decomposition.. and... let's move those pipelines to internal implementations of domains and define interfaces with clear contracts for sharing data across boundaries of domains" So, to paraphrase: it's fine to implement pipelines where this makes sense, but this should be a technological concern within the domain that creates the data, not part of the overarching architecture, since it inhibits the ease of scaling that data mesh enables.