No video

Fundamentals of Amazon DynamoDB Single Table Design with Rick Houlihan

  Рет қаралды 73,472

Serverless Land

Serverless Land

3 жыл бұрын

Пікірлер: 55
@StephenBlackstone
@StephenBlackstone 3 жыл бұрын
For all the "talks" and whatever people do trying to get the big idea out, theres something nice about being able to just watch a guy use the tool and do it. This was extremely helpful
@mystride
@mystride 2 жыл бұрын
100% agree!
@naraekim1833
@naraekim1833 2 жыл бұрын
01:31 client story dynamoDB cost 75% reduced after changing to single table design 05:19 Data modeling steps 06:08 Use case discovery phase 07:35 Entity relation diagram 08:43 NoSQL table example 11:08 Access patterns 12:00 Create data model in NoSQL workbench 19:16 Add relational pattern to data (partition key + sort key)
@neungviriyadamrongkij7046
@neungviriyadamrongkij7046 Жыл бұрын
thank you
@shashank_rajak
@shashank_rajak Жыл бұрын
So far the best video to understand single table design on dynamoDB, learnt a lot about this powerful db
@MrJ0Ker666
@MrJ0Ker666 2 жыл бұрын
Excellent video for DynamoDb single table modeling. The presentation style was very interesting, loved it, thanks!
@tejaschachcha
@tejaschachcha 3 жыл бұрын
You have got an enviable workplace Rick. One of the best sessions for DynamoDb single table modelling.
@explorer-xe7rt
@explorer-xe7rt 3 жыл бұрын
I assumed it was a background. The light on the face has cool colours, whereas the light in the background has warm colours.
@rishabhjain7781
@rishabhjain7781 Жыл бұрын
My team has been delaying this because of the complexity. Now I know how to convince them step by step. Crack the entity relation ship and then the access pattern and then start modeling. Now see if the model satisfies the access pattern. Boom !! Thanks for this video.
@AS-zw4lk
@AS-zw4lk 2 жыл бұрын
For a dolt like myself this video really helped me reason and make peace with my 'normalization' biased thinking. I still hate myself a little (for going to the dark side) but I am starting to see the advantages in single table design. Thank you for your efforts.
@neaumusic
@neaumusic 2 жыл бұрын
I really appreciate the workflow of writing the access patterns out and writing the model step by step. It’s really difficult to find design patterns for typical entity relationships, and this is extremely helpful. I’m still trying to figure out when a GSI/LSI is warranted, it seems like they make many to many relationships easy. It seems like Table/PK/SK are the only real hierarchies to index something, and if you needed to access it from the other direction you either need another index, duplicate/pointer nodes and multiple requests, or a scan. It seems dynamodb is really a cache for direct lookups and there are costs to maintain that cache for instant lookups. My current table model is to use an ID appended to sort keys so that I can use Queries, just haven’t seen this model anywhere and I’m trying to figure out if I’ll run into issues - EDIT it looks like Alex DeBrie’s single table design video shows this pattern around 53:00, exactly what I was thinking
@thobiaskarlsson1698
@thobiaskarlsson1698 2 жыл бұрын
Thank you Rick! Awesome stuff!
@cloud_architector
@cloud_architector 3 жыл бұрын
Thanks for your video, it's very helpful!! this topic is pretty complicated especially if you are from sql like me
@ruslanmuzhikov3962
@ruslanmuzhikov3962 3 жыл бұрын
Awesome lesson!
@MrJkreddy
@MrJkreddy 3 жыл бұрын
Awesome Rick
@NikhilSwamiExperimental
@NikhilSwamiExperimental 3 жыл бұрын
SUPER VIDEO!!!!!!!!!!!!!!!!!!!!!
@ryunosukehayashi2654
@ryunosukehayashi2654 2 жыл бұрын
I am japanese.I was impressed with this video.
@akashmaurya563
@akashmaurya563 3 жыл бұрын
would you please give the workbench data model files
@paracha3
@paracha3 3 жыл бұрын
It took me at least two watches of this video to wrap my head around it
@tarekbeb
@tarekbeb 4 ай бұрын
You are a GOD!!!!! Hands down, this is the best explanation out there!
@shailendraacharya
@shailendraacharya 2 жыл бұрын
I have seen almost all videos of ric and alex but there is a common use case related to get all data of a particular entity which is similar to select * from table. how should we approach that? if we keep it in a partition it can create a hot partition issue
@andreymashukov7576
@andreymashukov7576 7 ай бұрын
00:05 Focus on single table design benefits 01:52 Challenges with multi-table data model and large hierarchical documents. 05:12 Understanding use case and data modeling for single table 06:48 Entities include customers, products, orders, invoices, payments, warehouses, and shipments 10:05 Decorating objects with attributes and indexing for efficient querying 11:43 Designing the data model using NoSQL Workbench 15:13 Implementing access patterns for querying DynamoDB tables. 16:57 Iterating through result set and switching on object type for dynamic wrapping. 20:09 Query table by product id and warehouse id 21:43 Redesigning the data model to handle complex access patterns. 24:47 Consider adding individual items to separate partitions for efficient indexing. 26:28 Payments are added to orders and entered on the invoice item 29:53 Using partitions in DynamoDB Single Table Design 31:25 Consider consolidating related data into a single object when it makes sense. 34:36 Creating a new index and adding index attributes to the data model 36:20 Structuring data for different customer needs 39:36 Querying GSI1 for different access patterns 41:13 Iterating through shipments and updating status using secondary index 44:26 Implementing Time To Live (TTL) for automatic data removal after a specified time period 45:52 Different access patterns may require varied amounts of metadata, so it's important to design data models that minimize unnecessary data retrieval. 49:18 Using selective prefixes for querying customer data in Amazon DynamoDB 50:48 Understanding data reading and filtering in DynamoDB 54:10 Efficient data modeling is essential for scaling systems and reducing inefficiency. 55:46 Single table design model represents the design implemented thousands of times at Amazon and across the customer base. Crafted by Merlin AI.
@khanhdo3988
@khanhdo3988 3 жыл бұрын
At 10:26, does the sort key really allow querying with an "in" condition? AFAIK, we cannot currently do this.
@danixlt
@danixlt 3 жыл бұрын
Would be great if you would share data model for Nosql workbench.
@rossenpetrov3705
@rossenpetrov3705 3 жыл бұрын
The data model used is this: github.com/aws-samples/amazon-dynamodb-design-patterns/tree/master/examples/an-online-shop
@AnonYTChannelName
@AnonYTChannelName 5 ай бұрын
I'm curious about the ids for customer, product, warehouse, etc. How do we ensure there aren't id collisions across entities? Are these ids UUIDs and therefore all entities can take their ids from the same "stream" of generated ids, and we don't worry about separate sequences of ids for each entity?
@andreadiotallevi5780
@andreadiotallevi5780 Жыл бұрын
I feel like a kid who has just discovered a new amazing toy
@shobhitsharma3141
@shobhitsharma3141 7 ай бұрын
if i have to fetch details of a customer c#123 (like name, email etc) and all their orders, is it possible to fetch that in a single query? I can do that in two calls by first fetching customer data via PK=c#123 AND SK=c#123, and later via PK=o#123 AND SK=o#123
@HaibertBuilds
@HaibertBuilds Жыл бұрын
It was understandable in the first half, but got unclear real quick... thanks for the efforts still.
@jeremyllanta
@jeremyllanta 3 жыл бұрын
Hey, very interesting approach, what would be the best method for a get all Customer or get all Product for example?
@ahmadnabil2441
@ahmadnabil2441 3 жыл бұрын
I think you can add an attribute called "shop" for example on each customer and product record and create a gsi where pk = shop, sk starts with c# or p#
@guynicholas666
@guynicholas666 Жыл бұрын
I realize this is an old video now, but it would be very helpful if you provided a link to the final data model file you import. You move through the data pretty fast and being able to check it out at my leisure would be very helpful. It's still a great video though!
@vincentrouilhac4532
@vincentrouilhac4532 Жыл бұрын
Great video ! I have a question though, I'm new to DynamoDB so there is one missing piece I did not get, in 20:15 if I have a product Id I can query for that product all warehouses "begins with w" but how do I get both quantity available and warehouses' name in a single query (without having replication of attribute)? Do I need to query warehouses' name for each Id ?
@me-vn9wk
@me-vn9wk 24 күн бұрын
Create custom index or use scan with primary id and then sort the data. Second one is expensive and first one is custom pre build with the usecase in mind.
@matthewwoo6684
@matthewwoo6684 2 жыл бұрын
For your GSIs then, do you always use an ALL projection? It seems like to use patterns like this you can't limit the projection
@artificiallyunintelligent4537
@artificiallyunintelligent4537 Жыл бұрын
You could still create GSIs for quick actions that are done often like getting user account info
@XaphanMusic
@XaphanMusic 2 жыл бұрын
Is there any chance to get the DB modeling file?
@user-fn2fj5uw6t
@user-fn2fj5uw6t Жыл бұрын
I heard a saying that DDB transactions feature eliminates the need to design single table. Is that true?
@gearoidoconnor2584
@gearoidoconnor2584 3 жыл бұрын
What tool is he using that says AWS visualizer?
@a99881
@a99881 3 жыл бұрын
NoSQL Workbench - docs.aws.amazon.com/amazondynamodb/latest/developerguide/workbench.html
@tiagovinicius1941
@tiagovinicius1941 3 жыл бұрын
Is the sound ok over there? Here it is kinda breaking up
@PAD637
@PAD637 2 жыл бұрын
Is it possible to get an import file of this schema?
@zelinpu5022
@zelinpu5022 2 жыл бұрын
Hi Rick, maybe you could post the data model file so that we can play around on our own? :)
@ahmadnabil2441
@ahmadnabil2441 3 жыл бұрын
What if for example the customer for some reason edits his name how will i update the customer name in all other records for example in orders where it has the old customer name?
@ahmadnabil2441
@ahmadnabil2441 3 жыл бұрын
@4§4§r1 But how will this approach store the customer name in the order record? The customer might have multiple orders stored in a list or a map in the customer record and i can't make a gsi on a list or map to be able to project his name. I don't think there is a way around this except for fetching all customer orders and updating each order to change the customer name.
@ahmadnabil2441
@ahmadnabil2441 3 жыл бұрын
@Veerabramhendra Swamy Pothuluri I actually ended up considering this solution, i guess after all i can't optimize for both reads and writes. I will either store customer name in each order and optimze for reads but then i'll have to deal with updating each record if the customer edits his name, Or just store the customer id and for every order record i'll have to fetch the customer name which is also a bit too much. But anyways thanks for your response i really apperciate it.
@GrizzlyNetch
@GrizzlyNetch 3 жыл бұрын
@@ahmadnabil2441 also worth noting that in some cases you want to keep the previous customer name so copying it to record can actually be feature especially in case of invoices (eg. when customer is company and you need to keep the previous name intact on old records)
@ahmadnabil2441
@ahmadnabil2441 3 жыл бұрын
@@GrizzlyNetch Oh, I never thought of that my usecase is actually kinda similar. Thank you very much for the advice.
@huscachafe
@huscachafe 3 жыл бұрын
Excellent ! 👏🏻 is it possible that you can share the files of this example in workbench? To see in local environment. Pls 🙌🏻🙏🏻
@localGhost
@localGhost 3 жыл бұрын
This would be so wonderful! Please!
@dahooo2
@dahooo2 3 жыл бұрын
Yes please it would be great
@saurabhshri
@saurabhshri 3 жыл бұрын
They are available here: github.com/aws-samples/amazon-dynamodb-design-patterns/tree/master/examples/an-online-shop
@huscachafe
@huscachafe 3 жыл бұрын
@@saurabhshri thanks a lot !
@taylant1833
@taylant1833 6 ай бұрын
I dont understand why Rick creates a second GSI. Can someone please explain it to me? Thanks
@me-vn9wk
@me-vn9wk 24 күн бұрын
That helps to query data by not using primary keys and for custom purpose like ' give me the top warehouses running out of product data' etc. We can't build different tables for our custom usecases so effectively model data in few tables and build index to support custom query or use scan.
Migrating from relational databases to DynamoDB with Rick Houlihan
1:02:57
Single-Table Design with DynamoDB - Alex DeBrie, AWS Data Hero
1:09:32
AWS Portsmouth User Group
Рет қаралды 64 М.
Пранк пошел не по плану…🥲
00:59
Саша Квашеная
Рет қаралды 7 МЛН
Useful gadget for styling hair 🤩💖 #gadgets #hairstyle
00:20
FLIP FLOP Hacks
Рет қаралды 10 МЛН
What is a DynamoDB GSI (Global Secondary Index) ?
10:19
Be A Better Dev
Рет қаралды 116 М.
7 Common DynamoDB Patterns for Modeling and Building an App with Alex De Brie
1:11:16
AWS re:Invent 2022 - From RDBMS to NoSQL (PRT314)
52:47
AWS Events
Рет қаралды 15 М.
Amazon DynamoDB - A Crash Course (Demos Included)
1:15:13
Enlear Academy
Рет қаралды 84 М.