Discord Stores BILLIONS of messages using this database

Рет қаралды 184,087

Жыл бұрын

How does HUGE companies like Discord handle MILLIONS of users AND store BILLIONS of messages per month!?! 👨‍💻 In this video, I explain how the software engineers and programmers at Discord made the transition that made their performance SKYROCKET. Even if you don't get programming, this will be easy to understand 😄.
My name is Lewis Menelaws, I am a 27 year old software engineer who has a software agency in Ontario, Canada. I have been programming professionally for 5-6 years and am looking to help inspire you to learn code :)
MY DISCORD:
/ discord
----
CONNECT WITH ME ON SOCIAL
📸 Instagram:
/ lewismenelaws
🎚 TikTok:
/ lewismenelaws
🐣 Twitter:
/ lewismenelaws
--
My gear 💻
liinks.co/lewismenelaws
---

Пікірлер: 315

@CodingWithLewis Жыл бұрын

Thank you so much everyone for watching this video! If you want to talk more about tech, you are welcome to join my Discord channel (5k+ members)! discord.gg/bY6N9z26et How do I get Discord to notice me?! 🤔

@opboi5498 Жыл бұрын

Hey Lewis, great video, Today WhatsApp crashed in INDIA for some time, can you dig deep into this issue and post a video explaining this, it would be interesting to know how engineers at these tech companies solve these issues in such a short time span

@nirmalkumard7102 Жыл бұрын

Hi Lewis! I'm a civil engineering graduate(2018 passed out) but i love coding. Now I'm studying python on my own aiming to study towards data science. Is it possible that I can get a job in the data science area. Or what should I do to simply enter the IT field? Can you give me any suggestions please.

@ertyking23 Жыл бұрын

Sooooo discord employees are seeing pretty much all the messages I sent... FUCK I'M GOING SHOULDN'T HAVE SAID THAT SHIT DAMN I'M GOING TO JAIL

@TheMrkipas Жыл бұрын

thanks alot for the great content matey.. would love to see video like this .. like how facebook or big social media store their data and how it works so on

@chidiebubeiroezindu Жыл бұрын

Hey Lewis thanks alot for all your videos they've really helped me in my tech journey. Could you please do a mobile app developer roadmap. I'd really appreciate it thank you

@CodingWithLewis Жыл бұрын

Thanks SO much everyone for the kind words ❤️… and the constructive criticism! My camera kept going out of focus… my mistake 😅. Also I will turn down the camera shake

@pxtrez Жыл бұрын

those 'effects' look cool actually

@sulaimandev Жыл бұрын

Forget about Discord, think about KZfaq 🤯

@CodingWithLewis Жыл бұрын

Literally wouldn’t be able to comprehend 😳😳

@BatmanFailedSociety Жыл бұрын

down

@sulaimandev Жыл бұрын

@@BatmanFailedSociety bruh

@EEBDivyanshgupta Жыл бұрын

@@CodingWithLewis a video on it??? Or probably a short

@BlueProgamer212 Жыл бұрын

@GGLVXD bro you're familiar... are u from ISTREEM's server?

@Pikachu-oo5ro Жыл бұрын

This is what KZfaq was made for. Congratulations, good sir. Bless your work.

@CodingWithLewis Жыл бұрын

Wow, thanks!

@Grimzor Жыл бұрын

@user-mx1ek4sl2m Жыл бұрын

I like turtles 🐢

@Ironyum99 Жыл бұрын

0:21

@SixOThree Жыл бұрын

Subscribed.

@weiSane Жыл бұрын

You should do more of these technical videos. Love the insights . Now I need to choose a weekend project just as an excuse for me to use Cassandra DB😂

@SomethingNick Жыл бұрын

Would love to see more of these videos, been studying system design and this is way more interesting than the videos I’ve been watching

@R5123 Жыл бұрын

Great video man! I'm a developer myself and this video made me realize I should be reading more articles like the one you based this video on. Great work!

@ShafterPlay Жыл бұрын

Really great video! Loved how technical but easy to understand it is. The camera shake was a bit too much for me tho.

@COOLUKGAMER Жыл бұрын

Loved to watch this video. I remember using Discord as a reference for an A-Level project I did and I ended up reading their development blog

@john95685 Жыл бұрын

Amazing video 😍. Being a beginner in software development, It shows me how much is there to explore in this field.

@nomadontherun_ Жыл бұрын

You lost me around 4:57 but well done. By far this is one of your greatest videos. I know it takes a lot to put these together. Again, great job. Keep up the hard work.

@onverion Жыл бұрын

Insane video! I found you through your Shorts but I love the longer videos!

@kristefanov Жыл бұрын

Great video. Would recommend diving into a Spotify-based topic. Their engineering blog is quite rich with similar examples and could be a good platform to explain basic concepts from.

@saramshshrestha7641 Жыл бұрын

Keep more videos like this coming. Loved it

@stuiedaman Жыл бұрын

Oh hey, it's one of those times KZfaq recommends a channel I've never heard of before but turns out to be really good educational content. *Subbed*

@dailydoseofhalal6767 Жыл бұрын

Love this type of video! Nice work :)

@mostlywhat Жыл бұрын

This is the style I like! Keep up the good work!

@eshaannilekar3851 Жыл бұрын

Amazingly explained!Great graphics and attention to detail.. Make more such vids like this

@INKILU Жыл бұрын

always wondered about this! Thanks for making a video about this 👍🏼

@CodingWithLewis Жыл бұрын

No problem!!

@hand-eye4517 Жыл бұрын

I love the way your content is made , flows well , gets the points of the topic clearly and concisely . Just had to give you the credit you deserve really quickly and also mention that i haven't been seeing notifications for your videos? strange ill just check the discord more often .

@eurotrucksimulator2741 Жыл бұрын

Nice video! Discord is insane. This video reminded me when like 6 months ago discord was down for 1-2 hours.

@-_-_-_-_ Жыл бұрын

I really like your longer content!

@CodingWithLewis Жыл бұрын

Im glad :) they take sooo much longer to make but love doing it

@-_-_-_-_ Жыл бұрын

@@CodingWithLewis But then again it's much more profitable :D

@jannatgaoshiqqalb3598 Жыл бұрын

@@CodingWithLewis Bro, what fundamental knowledges do you recommend before learn to code?(my aim is to get into big tech companies and become a good software engineer)

@ashharkausar413 Жыл бұрын

Brilliant stuff, ty for sharing!

@shivankchaturvedi210 Жыл бұрын

Wohh! Absolute bliss information I m data engineer and let me say you just described our months of work in just 10 mins very effectively . Nice video keep it up:)

@urvishjaiswal8222 Жыл бұрын

Beginner friendly and easy to grasp lewis❤

@jordanfarr3157 Жыл бұрын

You called out the YT algorithm for bringing me here, and uh, yep! Great video! I have been a programmer for 10 years but managed to only skirt around the sides of the database world. This helped clarify that a bit more!

@GMoyo Жыл бұрын

This was awesome, I’m subscribing rn

@iamcrabgod2809 Жыл бұрын

This video is gold. Though I wanted you to go more in-depth into this , but I liked it. I want more of this and please go more in-depth. I would love it.

@jaskidesigns8355 Жыл бұрын

Woah, great video mate!

@ash_ekka Жыл бұрын

Now this the content KZfaq should push for. Pure gold ♥️

@DiOnFire Жыл бұрын

Amazing video! 👍

@groundingtiming Жыл бұрын

Def nailed this vid Lewis. Follower from the start, you sure grew quick ! Talk about scalability huh!

@allezvenga7617 Жыл бұрын

Awesome video👍

@abhaysingh.632 Жыл бұрын

Very informative insights, and I subscribed 🎉

@PriyankRupareliya Жыл бұрын

Beautiful. Too much information in too little time. Pure gold

@grimjetram3866 Жыл бұрын

this was really really nice bro as an aspiring backend developer seeing stuff like this and knowing how it works on the backend its amazing lol i was glued to the the screen from the start thanks for an amazing video this makes me push more towards my goal of becoming a backend developer thats just amazing💯💯💯💯💯keep up the good work

@kacperkobuc9582 Жыл бұрын

Very interesting video, and the jokes are just a cherry on top!

@Tnvrahmed98 Жыл бұрын

Subbed, great Content.

@Marwin. Жыл бұрын

This was amazing. Thank you 🙏

@CodingWithLewis Жыл бұрын

You're so welcome!

@cesarlabastida1392 4 ай бұрын

Great video everything well explained

@lelouch9609 Жыл бұрын

This content what I want Thanks Lewis ☺️

@g4_684 Жыл бұрын

This video is amazing. Loads of information.

@SeanCassiere Жыл бұрын

Love these videos on "tech stories from companies".

@meu2121 Жыл бұрын

It's amazing!! how u people are helping the whole world . Thanks for ur existence ☺️

@JonBeeee Жыл бұрын

Very informative, thank you!

@UCezI1mEFSJR_GHO4Z7bs8YA Жыл бұрын

Great video dude.

@tusharverma4202 Жыл бұрын

Loved it Lewis, thank you so much!

@kephiob Жыл бұрын

👆send a direct message for support and guidance . . .

@lakshman587 Жыл бұрын

Thanks for the video!!!

@SportSync_official Жыл бұрын

great video!

@_Nonines Жыл бұрын

Great video!

@JustPyroYT Жыл бұрын

Great Video! 👍 Greetings from 🇩🇪 :D

@DevNetic Жыл бұрын

Interesting video. I had this question also for a really long time but never took the time to look into it. Now I have the answer

@kjl3080 Жыл бұрын

Really cool video!

@M-ABDULLAH-AZIZ Жыл бұрын

Amazing explanation mate❤️

@愛 Жыл бұрын

"Wow guys look how cool, Discord is logging all of my private messages and everything i say!!"

@connorradeloff5597 Жыл бұрын

Great video! I just found your channel, and I'll for sure be going down the rabbit hole. If you're willing to accept some feedback, I would consider significantly reducing (or IMO getting rid of) the amount of simulated camera shake in your profile shots. I personally found the amount used in this video just tolerable, but it was becoming hard to watch at some points. A little goes a long way!

@samuelstefancic8050 Жыл бұрын

Idk why i am thinking so much about this video, what a amazing work you've done here.

@kephiob Жыл бұрын

👆send a direct message for support and guidance . . .

@HeyFaheem Жыл бұрын

Good Way Of Explaining Sir....

@vitorm7236 Жыл бұрын

Excelent video!

@gabbyes Жыл бұрын

nice video bro

@MisterValiant Жыл бұрын

Videos keeps getting better

@xslvrxslwt Жыл бұрын

I already knew this since the very first day they switched to Cassandra but still watched the video, lol.

@artemisfowl127 Жыл бұрын

This was an amazing video, I'm a SWE myself, and everyday I've used this damn god send of an app, I've always wondered how it works so well with millions of users, servers, channels and billions of messages. Do ICloud next!

@kleptotrichy Жыл бұрын

Very informative video showing the ins and outs of a mega data management platforms!!!! I do still wonder what the hell did discord do for their september release on android that made it so much buggier haha Good video overall!

@trylaarsdam Жыл бұрын

Afaik they switched to react native from native apps which caused a lot of issues

@guillaume5623 Жыл бұрын

This is gold!

@FinickyHorseGameStudios Жыл бұрын

Very interesting!

@MrDesai Жыл бұрын

I am a mechanic engineer and I still enjoy your informative videos Thank you

@codewithguillaume Жыл бұрын

Soooo Interesting :)

@bob_pm Жыл бұрын

I really liked it! Thanks. Could you make another video but with the Telegram.

@bhaveshagarwal2795 Жыл бұрын

2:32 no pun intended 😂

@bpoole007 Жыл бұрын

I loved this one.

@Viralvlogvideos Жыл бұрын

Wow. This video is awesome. Please make a video on hotstar stack, It was really amazing to see hotstar handling more then 1crore streamers at once during cricket match.

@estebanduvanzzDelgado Жыл бұрын

You tell the story with so much joy while I as an engineer imagined a rollercoaster of tears, frustration and susfpense in the face of each one of the engineers working to deliver the migration. As you correcly said. DB Migrations are the worst of our nightmares

@kephiob Жыл бұрын

👆send a direct message for support and guidance . . .

@vigneshshanmugam Жыл бұрын

Hi Bro, I learnt how Big data to be managed ❤️🔥 keep doing and rocking Love from 🇮🇳

@CodingWithLewis Жыл бұрын

Love from 🇨🇦

@ZombieAssassin7777 Жыл бұрын

With MongoDB going through many major changes over the years, I wonder if Discord has tried to go back and test the performance of it today.

@javierflores09 Жыл бұрын

yeah, no. You don't go back to something that works and has been deeply integrated in your infrastructure like this. They either use Cassadra or fix Cassandra but that's what they got for the end of the times

@tonyhart2744 Жыл бұрын

mongodb is child toy company, if we talk about planet scaling we talk with vitess or cassandra

@ZombieAssassin7777 Жыл бұрын

@@javierflores09 That's blatantly false. You're never stuck with something till the end of times. In the tech field, there's always room for improvement.

@javierflores09 Жыл бұрын

@@ZombieAssassin7777 developers make improvement, companies don't necessarily go along with them. In production you are not changing your underlying infrastructure unless you have a damned good reason to, just look at banks still using Cobol; if it was a startup which isn't yet as established I'd understand but something like Discord can't really handle the weight of these changes anymore, so yes, you are in fact stuck with it

@ZombieAssassin7777 Жыл бұрын

@@javierflores09 Still wrong, I work at a bank. Yes we still use Cobol and Mainframe, but are working towards moving off of them and also in the process of changing our database configuration.

@phil7121 Жыл бұрын

So this is why tech companies ask interviewees to invert binary trees

@zhehuizhou Жыл бұрын

Have to point out that it's "sea-lla" instead of "sky-lla". Chatted with folks from ScyllaDB, and Discord is pretty much on Scylla now.

@mallikarjunhj8768 Жыл бұрын

Would like to see more of how they built it

@potsuaye69 Жыл бұрын

they're doing these amazing stuff yet im using their technology to send memes in general

@Sammysapphira Жыл бұрын

Puzzle & dragons lmao I loved that game

@YashSharma0605 Жыл бұрын

Crazy insights 😵😵

@kephiob Жыл бұрын

👆send a direct message for support and guidance

@billionaireno1 Жыл бұрын

interesting thanks for teaching

@CodingWithLewis Жыл бұрын

My pleasure

@ArBoooooo Жыл бұрын

Great video

@CodingWithLewis Жыл бұрын

🥰🥰🥰

@piyushsahu3841 Жыл бұрын

Want more videos like this

@bhaveshagarwal2795 Жыл бұрын

great explanation

@CodingWithLewis Жыл бұрын

Glad you think so!

@jannatgaoshiqqalb3598 Жыл бұрын

Hi Bro, keep going Love from Uzbekistan 🇺🇿 (whose grandfather is founder of Algorithm)

@TheHasanJr Жыл бұрын

This was soo-oo great! Keep making contents like these. Try WhatsApp or Facebook or KZfaq next please!!!!

@Valery0p5 Жыл бұрын

"and what if I fill a bucket with tombstones, wouldn't it lead to resource exhaustion?" 5 seconds later "Knew it "

@Valery0p5 Жыл бұрын

KZfaq, you are the only half decent social network left, these bots are so easy to see, please...

@caiocesarmelolopes2156 Жыл бұрын

GitHub! Me as a backend dev i am facineted how the github can do what they do 🤩

@rifkiardiansyah616 Жыл бұрын

In my mind now: lets build chat app with cassandra

@suharsh96 Жыл бұрын

@8:25 You sure about that? AFAIK reading is much more computationally expensive than writing to a DB, especially in NoSQL databases as the indexes has to be updated when data changes in one place.

@Kat_Boyy Жыл бұрын

You should do twitch next, how they moved from REST api to GraphQL in 2017

@stevepoper8073 Жыл бұрын

Can't believe an app that I use to frequently is so incredibly complex

@mushroomcraft Жыл бұрын

Back in the day, when Discord engineers focused on improving the platform instead of ruining the platform...

@Lark703 Жыл бұрын

I’m out of the loop. How are engineers ruining the platform these days?

@mushroomcraft Жыл бұрын

@@Lark703 Well, many ways like the new Android version which is horrible, and checking if your bot's description in the app directory contains "youtube" to unverify it.

@OatmealTheCrazy Жыл бұрын

@@mushroomcraft I'm unsure what's wrong with the new Android version other than maybe seeing too many image previews at once when trying to upload

@mushroomcraft Жыл бұрын

@@OatmealTheCrazy In a technical sense, it is just the IOS version slapped onto Android, which means that it won't be optimised for Android making it slow and it has lots of stupid problems like I sometimes have to wait up to a couple of minutes for it to connect. None of these things were issues pre-new-android-version

@chlorobyte_projects Жыл бұрын

bruh, my phone's specs are like close to top of the line, yet there's this one single gif that makes Discord run at 10 fps. Not even a problem with the gif itself, Firefox and the Photos app handle it very smoothly. Nice "native performance"

@jakeh5852 Жыл бұрын

Engineer here who worked on these systems. A lot has changed since 2016. We don't even use Cassandra anymore. We should probably write a new blog post. 😂

@givemeyourmangoes5040 Жыл бұрын

I love the way you broke it down but the artificial camera shake was a little disorienting since i was eating while watching Love ur content tho!

@Verulyrus Жыл бұрын

this is seriously interesting, as a developer and discord user. I've asked the question 'how the fuck do they do this all?' many many times, and you gave a well explained answer! Amazing work!