What is Database Sharding?

No video

What is Database Sharding?

Рет қаралды 47,855

Күн бұрын

🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com
▬▬▬▬▬ Experience & Location 💼 ▬▬▬▬▬
► I’m a Senior Software Engineer at Juniper Networks (12+ years of experience)
► Located in San Francisco Bay Area, CA (US citizen)
▬▬▬▬▬▬ Connect with me 👋 ▬▬▬▬▬▬
► LinkedIn: / anton-putra
► Twitter/X: / antonvputra
► GitHub: github.com/antonputra
► Email: me@antonputra.com
▬▬▬▬▬▬ Related videos 👨‍🏫 ▬▬▬▬▬▬
👉 [Playlist] Kubernetes Tutorials: • Kubernetes Tutorials
👉 [Playlist] Terraform Tutorials: • Terraform Tutorials fo...
👉 [Playlist] Network Tutorials: • Network Tutorials
👉 [Playlist] Apache Kafka Tutorials: • Apache Kafka Tutorials
👉 [Playlist] Performance Benchmarks: • Performance Benchmarks
👉 [Playlist] Database Tutorials: • Database Tutorials
▬▬▬▬▬▬▬ Timestamps ⏰ ▬▬▬▬▬▬▬
0:00 What is database sharding?
0:12 Why is database sharding important?
0:58 What are the benefits of database sharding?
1:57 How does database sharding work?
3:42 What are the methods of database sharding?
3:47 Range-based sharding
4:59 Hashed sharding
5:52 Directory sharding
6:38 Geo sharding
7:25 How to optimize database sharding for even data distribution?
7:45 Cardinality
8:05 Frequency
8:24 Monotonic change
▬▬▬▬▬▬▬ Source Code 📚 ▬▬▬▬▬▬▬
► GitHub: github.com/antonputra/tutorials
#kubernetes #devops #cloud

Пікірлер: 86

@AntonPutra Жыл бұрын

🔴 - To support my channel, I’d like to offer Mentorship/On-the-Job Support/Consulting - me@antonputra.com

@ashutoshtiwari4398 3 ай бұрын

No Nonsense, direct to point, covering all cases. Well-compiled video!

@AntonPutra 3 ай бұрын

thank you!

@jasper5016 6 ай бұрын

Hard to believe this top-notch content has very less views. Thanks a lot, Anton!!

@AntonPutra 6 ай бұрын

❤️

@jackfrost8969 5 ай бұрын

probably coz it basically has no volumn

@alonsoalcantar7847 4 ай бұрын

Ive tried to understand sharding for crypto purposes but every "crypto sharding" video display vague descriptions. I stumbled upon this by accident and it was great. Amazing work 🙌

@AntonPutra 4 ай бұрын

thanks!

@abdoulhamidcoulibaly2385 Ай бұрын

Very helpful. I just discovered sharding not long ago in my new position. I didn't understand a thing until your video.

@AntonPutra Ай бұрын

thank you!

@Xaoticex 7 ай бұрын

Nice, exhaustive and short video considering it covers a lot.

@aslan1504 Жыл бұрын

As for the hashing, you probably will take hash only of a subset of columns of a record, most probably - only primary key, because changing any field of any record will result the hash to change also, which leads to data losses.

@tonnytrumpet734 Ай бұрын

Thanks for this comment. Could you maybe clarify ? By data loss you mean the reverse proxy or whatever way of communicating with databases wont be able to know where to search for the information right ? however the information would still be here. Although it would be extremely computationally hard there would still be way to recover it right ? You could for example recalculate hashes for all the data and redistribute those that aren't belonging to the right database based on the sharding prefix ?

@aslan1504 Ай бұрын

@@tonnytrumpet734 oh yes, data will still be there, but it basically will become unusable. It's like creating yourself problems to solve.

@shanchessmetilda5550 4 ай бұрын

Great Explanation....Thanks for the efforts

@AntonPutra 4 ай бұрын

Thanks!

@agun21st Жыл бұрын

Thank you sir for detail explaination of database sharding. We hope a practical handson of Database sharding will publish soon.

@PhillyHank 9 ай бұрын

Very helpful. Very confused and to the point! I hope your colleagues who do technical videos would follow your framework. 👍🏾👍🏾👍🏾

@topoisonfungus Ай бұрын

Thank you much appreciated 👍

@AntonPutra Ай бұрын

thanks :)

@xardiannon5038 5 ай бұрын

Superb explanation, and never strayed off topic.

@GabrielPozo Жыл бұрын

Great video! Always an important topic when we think about scale our systems.

@AntonPutra Жыл бұрын

Thank you!

@bunnybal 2 ай бұрын

Really very well explained, thank you very much.

@AntonPutra 2 ай бұрын

thanks!

@yourname-scorpion 3 ай бұрын

Great video, thank you

@AntonPutra 3 ай бұрын

thanks!

@charlesopuoro5295 9 ай бұрын

Thank you so much for this clear, insightful explanation of Database Sharding.

@AntonPutra 9 ай бұрын

Thank you, Charles!

@ab_azmi5584 9 ай бұрын

Great video. Clear & easy to understand.

@AntonPutra 9 ай бұрын

Thanks Ab!

@nero4581 Жыл бұрын

Great Video, as always, Anton!

@AntonPutra Жыл бұрын

Thanks, if you think anything can be improved, please let me know!

@helciopandelo Жыл бұрын

Truly awesome and simple to learn!!! Thank you!!!

@AntonPutra Жыл бұрын

Thank you for the feedback!

@69k_gold 2 ай бұрын

Let's say I'm using a shard-nothing architecture, now let's say there's a relationship between customers table, payments table and orders table. Customers and orders tables are linked by the foreign key customers->id ~ orders->customer_id Orders table and payments table has the foreign key orders->id ~ payments->order_id Now how would you shard this database? You can't use a single shard key, because both customer_id and order_id are important that ensure all the related rows are in a single shard. So how would you solve this problem?

@erkanakgul6954 7 ай бұрын

Great explaination! Thanks

@AntonPutra 7 ай бұрын

thank you!

@vikasgoel7529 7 ай бұрын

Excellent described

@AntonPutra 7 ай бұрын

Thanks!

@meron6913 Жыл бұрын

Great video Anton.

@AntonPutra Жыл бұрын

Thank you!

@dmitriydiachenko5648 Жыл бұрын

Awesome video! thanks for explaining it

@AntonPutra Жыл бұрын

Thanks!

@bossgd100 Жыл бұрын

thank you for these explanations

@AntonPutra Жыл бұрын

thanks for visiting

@LinuxForLife Жыл бұрын

Very interesting! Thanks! 👍

@AntonPutra Жыл бұрын

welcome! my pleasure

@dhruvpatel6604 Жыл бұрын

Excellent tutorial

@AntonPutra Жыл бұрын

Thank you!

@egitim10 5 ай бұрын

Short,nice,clear

@nicgeorge6126 Жыл бұрын

Fortunately I’ve been able to get by with two read databases and a write by using table partitioning up until this point. Hopefully I don’t have to tackle sharding any time soon! Great video and thanks for sharing

@AntonPutra Жыл бұрын

Thanks! There a lot of distributed databases based on postgres that can shard for you.

@user-ui5yw6pf5p 5 ай бұрын

you said sharding have unique data sets if one sharding not respond then other sharding response you but if customer search record and that record will be in sharding 1 . After that sharding 1 will not respond then what we have to show for customer

@rayaalkhateeb874 9 ай бұрын

thanks a lot for explaining

@AntonPutra 9 ай бұрын

my pleasure!

@ricardorqr Жыл бұрын

How do you create the animation for your videos? They are so cool!!!!! 💪🏼

@AntonPutra Жыл бұрын

Thanks! I use adobe suite.

@EpoxyArt 6 ай бұрын

Hi, Anton! How can I search by the field that is not shard key? I need to go thru all the shards? And what if I need to scale it up or down (change shards number)?

@MagDag_ Ай бұрын

Классное видео, спасибо. Какая стоимость ваших услуг? Нужна консультация для Homelab.

@AntonPutra Ай бұрын

Privet, spasibo! At this point, I can offer one-on-one sessions. I ask that you send me any questions before the session so that I can prepare some examples, and we can go through them during the meeting. I charge $100 per 1 hour session. If you are interested, pls send me an email.

@thewaygoesup Жыл бұрын

your example confusing between database shard and table partitioning range-base sharding is about one table sharding not about database sharding.

@AntonPutra Жыл бұрын

Noted, will improve

@Jaguar1612 Ай бұрын

So how to solve hotspot problem?

@tejapolisettysai6950 10 ай бұрын

sir thanks for the video, what do you use for editing, its really good.

@AntonPutra 10 ай бұрын

thanks, adobe suite

@gcheese25 10 ай бұрын

great video! start subscribing now

@AntonPutra 10 ай бұрын

Thank you!

@noahgsolomon 11 ай бұрын

w video

@AntonPutra 11 ай бұрын

thanks

@ProSunnySharma 10 ай бұрын

Excellent! What tool do you use to do animations?

@AntonPutra 10 ай бұрын

Thanks Adobe suite

@user-yv6ti2wf7c 7 ай бұрын

thanks, but i have a question if i use range-based sharding and conside 4 shard what happen if i want to convert to 40 shard? what happen for previous data, and new data [ first i have 3 shard 1(a-h) 2(i-p) 3(q,z)) now need to make it 40.

@AntonPutra 7 ай бұрын

If you shard manually at the application level, you need to write logic to rebalance it yourself. It's easier to use built-in mechanisms for sharding.

@user-yv6ti2wf7c 7 ай бұрын

@@AntonPutra thanks

@andherium Жыл бұрын

I think 99.9% use case are served fine by a monolith database server. Heck even stack overflow is fully powered on a single server

@cariyaputta Жыл бұрын

Yes. Still, it's useful to have a knowledge of whatever jargons uppermamagements are throwing at you.

@AntonPutra Жыл бұрын

For personal projects, sure, but in the enterprise, you frequently have to deal with sharding.

@rahuldinesh2840 11 ай бұрын

Can I do sharding in WordPress database?

@AntonPutra 11 ай бұрын

WordPress uses a MySQL database under the hood. Take a look at Vitess.

@APoIIy 11 ай бұрын

So, sharding is a pain in the ass and requires a lot of configuration, analytics and also business logic to manage shards in an respectable way. This is also why NoSQL Databases come in handy as they can scale better horizontaly without this extensive configuration activities you have with traditional SQL databases. But to be said SQL Databases will probably cover 90% of all usecases anyway without you getting into sharding.

@AntonPutra 11 ай бұрын

Nowadays, once a year, I see a new distributed database based on PostgreSQL come out, lol.