No video

Dropbox system design | Google drive system design | System design file share and upload

  Рет қаралды 257,400

Tech Dummies Narendra L

Tech Dummies Narendra L

Күн бұрын

Let's design a file hosting service like Dropbox or Google Drive. Cloud file storage enables users to store their data on remote servers. Usually, these servers are maintained by cloud storage providers and made available to users over a network
Diagram: imgur.com/a/pz...
#systemdesing #dropbox

Пікірлер: 302
@sumonmal009
@sumonmal009 3 жыл бұрын
idea scope 1:38 scale 2:10 HLD 2:41 problem to solve 4:55 6:57 solution 10:41 metadata file 15:26 HLD 17:38 messaging service detail 25:01 device sync feature metadata handling 28:40 metadata schema 31:48 edge store usage to serve metadata 36:16 search feature 40:01
@RakeshGajjar
@RakeshGajjar 5 жыл бұрын
Give this man the credit he deserves 👏🏼👏🏼👏🏼
@roopaschannel9731
@roopaschannel9731 4 жыл бұрын
Thanks for your channel Naren! Brings back my love for computer science. We need more such teachers that can break things down and explain it as simply as you have done here.
@chepaiytrath
@chepaiytrath 3 жыл бұрын
Clients described at 18:26, taking an example of Google Drive, refer to the various "Backup and Sync" desktop clients which you might have active on multiple devices. All these clients keep listening to a messaging queue. In case one device makes changes to a file, the change is propagated to S3 and all clients are notified of this by publishing the change to the messaging queue which they are listening to. The client which is the originator of the change doesn't care but other clients do and when they know of a change they update their local copies (download the whole file if not present). Update: It's not just one Q2. Each client will have its own queue on which the change is broadcasted. This is to have an asynchronous behaviour wherein the client can be offline for a period and then when it is online it starts listening to the queue for any changes This is my understanding. Correct me if I'm wrong
@helloworld7313
@helloworld7313 3 жыл бұрын
honestly as a swe working at dropbox, i don't feel like this is an answer i am looking for. It misses a lot of important stuff like how do you design your database schema for storing the metadata and how would your sync protocols looks like? what if there are write conflicts during sync how do you deal with that? and the search engine part i guess is the least likely bonus question i'll ask in an interview(probably makes more sense in design twitter) no offense to Narendra, i think you put in a lot of effort/research into this and even referenced dropbox's blog post on network edge infra. but i think this's a problem to almost all of these youtube system design videos, like, yes you will learn a little bit here and there, but it's not the same as a real interview and don't expect to memorize some sys design solution and pass the interview. better ways to learn system design: read DDIA, web scalability for startup engineers, take a distributed system class listen to real mock interviews if you somehow can(or some faang engineer does these mock interviews and post them somewhere i guess) design and implement projects at your job if you have the opportunity
@Doug-rv3nr
@Doug-rv3nr 7 күн бұрын
It depends on the system design interview you attend.
@eugenekim6937
@eugenekim6937 2 жыл бұрын
Great system design. I really wish he explained why file change sets need to be ordered and consistent, in which led him to use a relational database for the metadata. If you look at his design for google docs, it doesn't even use a relational database for massively concurrently updated files.
@rabindrapatra7151
@rabindrapatra7151 Жыл бұрын
Yes. He explained google docs using operational transformation.
@simpleurbanliving
@simpleurbanliving Жыл бұрын
Enjoyed this video more than others because of the cute doggo interruptions. :) Thank you!
@akinkanju9653
@akinkanju9653 5 жыл бұрын
Hello Naren! your channel is a goldmine. I've learned quite a lot. Please consider creating content that dives deep into data models/schemas/datasets. Thanks 🙏
@TechDummiesNarendraL
@TechDummiesNarendraL 5 жыл бұрын
Sure, Thanks
@waseemvadla5713
@waseemvadla5713 2 жыл бұрын
@@TechDummiesNarendraL Do you have code for this explained system?
@deepakzworld
@deepakzworld 4 жыл бұрын
The best part I like about your videos is you do a lot of research to put the information from various sources about a topic into one place. You are our Edgestore ;)
@vaibhavsingh9x
@vaibhavsingh9x 4 жыл бұрын
Another reason to use async queues: one cannot assume that only a single file will be uploaded. There could be a case in which multiple files could be uploaded and a queue ensures that chunks do not get mixed with each other. I guess one can also talk about failover (what happens when a chunk gets lost during transmission/gets corrupted) but that might not be required. Edit: NVM he covers this case as well LOL. Love the depth he goes into when covering different components.
@samahome
@samahome Жыл бұрын
Your explanations and approaches in explaining these System Design Problems is absolutely phenomenal.
@RandomShowerThoughts
@RandomShowerThoughts 5 жыл бұрын
15:39 LMFAO! great video man, you are my go to for system design prep
@viktorartemov2361
@viktorartemov2361 3 жыл бұрын
Needs an explanation of how exactly does one detect which chunk was changed. Because your applications, video editor, for example, doesn't know anything about chunks, it doesn't change a chunk, it changes your file. It's up to your Dropbox client to figure out which chunk the change corresponds to. And that is not immediately obvious especially for huge binary files.
@jyotsnamadhavi6203
@jyotsnamadhavi6203 8 ай бұрын
Hash computation can help
@Doug-rv3nr
@Doug-rv3nr 7 күн бұрын
@@jyotsnamadhavi6203 Yes, correlation ID is used in Kafka.
@stevemew6955
@stevemew6955 4 жыл бұрын
Great work Narendra. This is the best video I have found so far on KZfaq on the DropBox architecture.
@abhishekkapoor7955
@abhishekkapoor7955 2 жыл бұрын
separate queue for each client doesn't sound good additionally we are using queue as persistence storage which should be avoided because a large number of messages can pile up in queue without any proper ordering. instead, the client side can call the sync service to fetch the latest files index for the user
@yawar110
@yawar110 3 жыл бұрын
Salaams and respect from Pakistan for you sir! You are a hard working and a smart individual who is helping the IT community across the world using whatever best resources you have. Keep up the good work - Keep posting them system design videos. God Bless!
@druidclash9161
@druidclash9161 5 жыл бұрын
Shit, it's fucking perfect explanation. Thanks for all these stuff.
@TheDibyendusarkar
@TheDibyendusarkar 4 жыл бұрын
What if we send the diff only, what git does. Storing a tree like structure of changes.
@amoghasoda
@amoghasoda 3 жыл бұрын
Hey Naren. Great job! Few questions for you. 1. Why can't we expose a single service which takes chunks of data and make metadata entry into database and also stores chunks to S3 instead of client calling both services? 2. From your design, if sync service pushes notifications to a topic are we maintaining dedicated topics/partitions for different clients? Or are we pushing notifications via Websockets/HTTP Polling? Few comments: 1. If clients go offline they can still come back and establish connections via Websockets? 2. We can't have 'n' number of topics because creating Kafka topics/JMS queues need infrastructure support and is a costly operation. Also creating partitions in a live system is a costly affair. Pls let me know if I'm missing anything.
@jamesneesham70
@jamesneesham70 Жыл бұрын
Though this video is a good starter, its gets wrong at multiple places
@saiprajeeth
@saiprajeeth 4 жыл бұрын
WTF. only 633 likes out of 38,663 views for this gold? Come on viewers, you are beholden for this guy who is putting enormous effort to share knowledge beyond his boundaries.
@ragingpahadi
@ragingpahadi 4 жыл бұрын
Give this man Bharat Anmol Ratna : ]. Thanks for SD series it helps us broaden our thinking and not just defect fixing and small CR.
@pavankumaruppuluri4097
@pavankumaruppuluri4097 4 жыл бұрын
While explaining why we need queue instead of http call to sync service you mentioned we need it as client may not always be connected. My question is if client dont connect to internet for example, even that message also cant be transmitted to queue right ?
@rajkrishna8294
@rajkrishna8294 2 жыл бұрын
You don't have studio but you are delivering better content than those who have studio.
@rohittiwarirvt
@rohittiwarirvt 3 жыл бұрын
A Great Video on Understanding file storage service design like dropbox, Preparing for an interview and this content is helpfull
@joeyyu133
@joeyyu133 4 жыл бұрын
I am not quite clear about the response queue. Is it necessary? If each client maps to a response queue, and what if the client never comes back? Are we still posting messages to its queue? Meanwhile, why not just let each client periodically check the diff between the local metadata vs. the latest metadata? By doing this, we can get rid of the response queues, right?
@Maw0822
@Maw0822 4 жыл бұрын
What happens to the chunks when I add data to the file that would be contained in the first chunk causing it to go over it's limit? Wouldn't that cause a cascading effect where every chunk spills over into the next chunk? Our small change in one chunk would cause changes in every chunk no?
@sudhasravan92
@sudhasravan92 2 жыл бұрын
Exactly!! I have been struggling with the same question for the last few days but could not find an answer anywhere!
@mahee96
@mahee96 2 жыл бұрын
@@sudhasravan92 Haha at least at this point it was not just me scratching my head. Jokes apart, seriously I once had a discussion with my coworker why git scm was not being used, for that he reminded me how git works. which is by storing delta/diff between two files, so that when a file is modified, only the delta info is uploaded or downloaded. BUT, he explained me that this is exclusive to TEXT ENCODED files, and not for BINARY files because git can in no way know what is the delta because actual data is binary (such as .exe, .obj, .dat, .class etc). He confirmed that in case of binary files, git actually stores the new file completely. so this is equivalent to storing old file + new file which doubles size of storage required. HENCE git is not intended to store BINARY Files where delta info can't be determined. Considering this theory, you could see that the chunking current file to be uploaded can save you in terms of network errors so that you can re-upload erroneous chunk again, but it is completely not helpful in terms of using as delta information. Because when the file is modified, the whole file can't be chunked again as how the previous version was chunked and compared with previous version of chunks in 1:1 manner, nor it can be variably chunked such that we can deduce the exact chunk that has changed considering file is binary where data could be machine code(exe) of a processor. If someone can point me "THE OBVIOUSNESS" of the chunker design shown here and its purpose/usefulness, I would be much thankful!
@dhhsncnd6107
@dhhsncnd6107 4 жыл бұрын
Awesome video that comes down to details for real design not just for interviews 😄
@sananirajabov3
@sananirajabov3 5 жыл бұрын
Great system design and clear explanation, thank you !
@codinga-cx1nn
@codinga-cx1nn 7 ай бұрын
THE BEST OF THE BEST -> PLEASE, CONTINUE YOUR CHANNEL!
@codetolive27
@codetolive27 5 жыл бұрын
Very informative. You have covered each layer like front end, Middle tier and database layer effectively. Thanks
@Yan-rv8mi
@Yan-rv8mi 3 жыл бұрын
33:56 Here you threw the problem that we need to rebalance/re-shard as we get more and more data in one shard, but the subsequent mentioned approach "edgestore" does not seem to solve this, does it? It seems like the edge wrapper simply provides a better interface for developers to read/write data. How does the "edgestore" help in regards to the data sharding parts?
@ishanchopra7468
@ishanchopra7468 3 жыл бұрын
Yeah Naren, would like to know the answer to this - how is the cost of denormalization required due to sharding reduced by edgestore?
@Amin-wd4du
@Amin-wd4du 4 жыл бұрын
Very good content. I loved the dog barking.
@Icix1
@Icix1 3 жыл бұрын
just fyi, cassandra consistency model provides a higher chance of reads being consistent, but doesn't provide true linearizability. This is why it's better to use terms like linearizability and not consistency as DB providers can play games with their definition of "consistency". Cassandra and similar nosql variants are basically partitioned key value stores in disguise and cannot ever compete with a true relational database. Also, even within relational databases, configuring isolation levels is pretty important, and it's easy to get tripped up there.
@dimei4170
@dimei4170 5 жыл бұрын
Very nice video! Please do an Instagram system design for the next one! Thank you!
@raywu9685
@raywu9685 4 жыл бұрын
Without reference to original paper “Designing a Dropbox-like File Storage Service” by Alejandro Ramirez, Fariborz Khanzadeh, Hassaan Bukhari. this is unfair.
@leprofesseurshen
@leprofesseurshen 4 жыл бұрын
Man, I wish I discovered your channel sooner. I recently failed on a system design interview, Dropbox system design particularly. Thanks for your work. I will study every single of your video and prepare myself for my next interviews.
@lakshminarayanansairam2739
@lakshminarayanansairam2739 4 жыл бұрын
which company ?
@leprofesseurshen
@leprofesseurshen 4 жыл бұрын
@@lakshminarayanansairam2739 Ledger. The company that builds crypto wallets.
@archfitness2399
@archfitness2399 2 жыл бұрын
Excellent the way of explaining the concept. and really enjoyed the the dogs pictures while barking in the mid of presentation. 🙂👍
@ilyanaoumov5425
@ilyanaoumov5425 4 жыл бұрын
I'm not convinced by the argument for using queues in the design. If clients need to obtain the latest changes, they will need to establish a connection of some sort to some service. You could make a REST call asking for the latest data which could do a search against the metadata DB, or you could call a service that reads from the queue. I think the main argument for using a queue is latency. Your read/write path might take a long time, so could stand to gain by doing append only writes to a queue and having latest metadata responses pre-populated in multiple queues.
@manojbgm
@manojbgm 3 жыл бұрын
Nice explanation. Insightful
@dhruv4u9
@dhruv4u9 5 жыл бұрын
Nice explaination Naren. Why is messaging service sending information to clients. It should rather be pull model on client's front, where they periodically pulling from Server the chunks. It should be based on last synched chunk_id. So, even if any client is unavailable for an interim duration he can synch based on last synched chunk_id.
@armharish
@armharish 4 жыл бұрын
I agree
@psn999100
@psn999100 3 жыл бұрын
How does this work ? Lets say some client had last synched the chunk id = 10 of a particular file (this file has 10 chunks). Now lets say chunk_id = 5 has changed for this particular file. How will the other clients know that they need to get chunk_id = 5 from the object store ? Yoir idea only works if the file increases in size ,thereby increasing the chunk_id. Please correct me if I am wrong
@oscarmvl
@oscarmvl 3 жыл бұрын
@@psn999100 you store the last time there was a modification in the client #1, and let’s say another client #2 updates a chunk, then you update the latest modification time in the server. When client #1 asks if there has been any changes using the last time of client #1, the server compares it to its last modification time (done by client #2) and lets the client #1 know that there are some changes that client #1 is missing, these changes are the changes done by client #2.
@Anoopchaudhary36
@Anoopchaudhary36 3 жыл бұрын
We may need to pull only when client connects then for further updates queue can be used that can help us avoid constant polling to server
@sweetyb3287
@sweetyb3287 5 жыл бұрын
Awesome! Loved the explanation and learned a lot. Last part of the search design for this service could be expanded into another video.
@TechDummiesNarendraL
@TechDummiesNarendraL 5 жыл бұрын
Shooting a video for serchengine design
@chenx3838
@chenx3838 4 жыл бұрын
So clear and easy to understand, keep going!
@srikanth26mar
@srikanth26mar 5 жыл бұрын
Firstly, thanks for the video. it would have been interesting to know how the Edge Wrapper achieves transaction isolation level without explicit locking/transaction.
@Amandeep-bt4kl
@Amandeep-bt4kl 4 жыл бұрын
Thanks Narendra for this great resource of designing DropBox, could you please help me with following few doubts: 1. How we can implement this application to resolve any conflicts. For example, I made few changes in a document and same is synced into server and but other clients were offline for quite some time. Then, I started using another client which is offline; however, I made significant changes into it and then only realised that I am offline. Then what would happen, I don't want to loose any of my changes and the mentioned document is not a simple text file. 2. Should chunk size be dynamically determined depending upon an individual file size or it would be fixed for the entire application. If it is fixed, and then we have made necessary changes in a document's intermittent chunk number (say suppose 2nd chunk among 10 chunks in total). Now chunk size of 2nd chunk got tripled, should we break it and reorder all the chunks (my point is if there are 1000 chunks, then re-ordering would be an issues) or keep it as it is, what are your comments on this? Thanks for your inputs in advance.
@yakshitjain3048
@yakshitjain3048 4 жыл бұрын
There is a version control system called git which can help to accomplish the doubts you have.
@arnab_speaking
@arnab_speaking 2 жыл бұрын
sweetest part of the video at 15th Min
@AmdJunaid
@AmdJunaid 5 жыл бұрын
Truly amazing. Hats off to you. 🙏😍 Request you to upload more of such videos. It would be too awesome if we can have a system design tutorial for beginners and how to improve.
@TechDummiesNarendraL
@TechDummiesNarendraL 5 жыл бұрын
Sure, Some time soon
@vrushangdesai2813
@vrushangdesai2813 5 жыл бұрын
excellent video , thanks a ton . pls make a video on system design for decentralized applocations on ethereum and ipfs (like decentralized uber)
@ritwizsinha1261
@ritwizsinha1261 3 ай бұрын
One comment regarding the upload concurrency problem, s3 supports multi part uploads and you can upload the file part by part and it will be assembled on the s3 end at the end thus you can upload the file in parts without the s3 being aware about it
@karthiyogi93
@karthiyogi93 5 жыл бұрын
Wow. Amazing. U r doing a grt job.
@kristhiantiu4317
@kristhiantiu4317 2 жыл бұрын
for the length of video, i learned a ton
@yog2915
@yog2915 2 жыл бұрын
Amazing no nonsense serious designs which are really good hatsoff bro 👍 keep doing good work
@chilamakoorugangadevi9208
@chilamakoorugangadevi9208 4 жыл бұрын
Really you done a good & great job annaiah.....Awsome explanation,tq☺️
@tacowilco7515
@tacowilco7515 4 жыл бұрын
thank you for the video it gets the very general idea about how it works but without important details though once again thanks
@harshakada3374
@harshakada3374 4 жыл бұрын
Those are great videos that u r doing. Can you please start a course about system design basics n how to build from scratch to advanced level. Please do that course I would love to buy. Thank you 😀
@damluar
@damluar 5 жыл бұрын
Chunks idea is good, but your statement that if we change only some bytes we will only have to upload that chunk has a problem. If you add X characters to 4th chunk, all the following chunks change as well, they all will be shifted by X characters. So when your script checks the hash sum of a chunk, it will have to upload [4;n] chunks. Unless we can adjust how we split a file into chunks.
@chabhishyam
@chabhishyam 5 жыл бұрын
I think you just need to know how HDFS systems chunk the data into its default size. It won't separate the data from Line -L and Column - C 1L-1C to 10L 12C -chunk1 10L-13tC to 25L-30C - chunk2 . . so........
@arjun.s5112
@arjun.s5112 4 жыл бұрын
Thank you so much. The best system design video on this topic.
@ashleyspianoprogress1341
@ashleyspianoprogress1341 2 жыл бұрын
I have a question about the response queues. At the beginning you mention dropbox has 500 million users. If we assume a user has 3 devices on average, that leaves us maintaining 1.5 billion response queues. Am I misunderstanding something?
@yusufsipahi3916
@yusufsipahi3916 Жыл бұрын
I think this solution is not scalable.
@mohammedmohideen1756
@mohammedmohideen1756 2 жыл бұрын
Wonderful Explanation...!! Thanks for the work Naren.
@kunalbadole4211
@kunalbadole4211 Ай бұрын
best explaination
@vallimcts
@vallimcts 4 жыл бұрын
Thanks, you are doing a great job. Also, It would be really helpful if you could run the whole flow once at the end. So that we don't have to watch the full video when revisiting the video for the second time.
@amyzeng3816
@amyzeng3816 4 жыл бұрын
Do you mean there will be a queue maintained for each client? Actually two, one request and one response queue. Is this design efficient?
@ddtoledo
@ddtoledo 4 жыл бұрын
Amy Zeng I thought the same thing. Waiting on a comment from him about it
@nribackpacker
@nribackpacker 4 жыл бұрын
Sirji excellent video
@rahulchudasama
@rahulchudasama 5 жыл бұрын
First it was awesome explanation, one question clicked in my mind how git ver works with file history and lines changes detected?
@CharlesATH
@CharlesATH 3 жыл бұрын
Good Job Naren
@ameyapatil1139
@ameyapatil1139 4 жыл бұрын
Fabulous videos, excellent information and lots to learn ! Dogs were hilarious.
@sadihassan8407
@sadihassan8407 4 жыл бұрын
You are the best! Thank you so much for explaining this so nicely!!!
@bridgetp3733
@bridgetp3733 4 ай бұрын
Thank you so much. This was fascinating!
@T-Sparks208
@T-Sparks208 4 жыл бұрын
Amazing .. I am new in system design and I've learned a lot.. Thankyou so much
@devd5820
@devd5820 2 жыл бұрын
Very nice...keep it up..
@yishanlu3644
@yishanlu3644 4 жыл бұрын
The most handsome tech guy I have found in youtube! Thanks a lot !
@xuemingzhang8456
@xuemingzhang8456 3 жыл бұрын
The content is always great from this channel, but if you can use a microphone while talking that will bring the video to the next level.
@amitkabraiiit
@amitkabraiiit 5 жыл бұрын
Can you create videos on data models/schemas/datasets as has been asked some comment below as well.
@Mahesh-js6hp
@Mahesh-js6hp 5 жыл бұрын
Great job, Naren! Love your work. Keep it up!
@experience-engineering
@experience-engineering 3 жыл бұрын
Hello Narendra, Could you please make a video to design "google photos" like app? Or what architectural changes you would do in this existing design of drop box to limit it to "google photos"? By the way, your video has been real source of knowledge!
@palashmaran
@palashmaran Жыл бұрын
Narendra L Design looks good. But i doubt current design will work using browser as clients. Could you please update the documentation on how to optimize downloads upload/download using browsers as google chrome etc. Is it possible to add client side logic split and merge chunks in google chrome browser ?
@jagjotsingh3407
@jagjotsingh3407 3 жыл бұрын
Excellent Content
@messerschnitt7943
@messerschnitt7943 5 жыл бұрын
What I don't understand is how multithreading will help with the upload speed? If you are uploading the same file in single thread with your internet connection upload limit, I don't think you can upload faster with multiple threads in parallel - the threads will still compete for the total upload speed limit, wouldn't they? Except when you'd have multiple parallel internet connection (which is not that usual even for companies)...
@rujhanarora7892
@rujhanarora7892 3 жыл бұрын
Think in terms of context switching bwn threads and you will get your answer!
@veereshvik3521
@veereshvik3521 4 жыл бұрын
Doing great job Naren, keep up the spirit 👍🏻
@bephrem
@bephrem 4 жыл бұрын
15:38 hahahahaha, I do that sometimes too haha. I pause then forget to clip the video or I just can’t clip the video since noise is in a critical segment.
@karthikmucheli7930
@karthikmucheli7930 4 жыл бұрын
Hey back to back SWE. :D
@bephrem
@bephrem 4 жыл бұрын
@@karthikmucheli7930 hey man, wassup
@karthikmucheli7930
@karthikmucheli7930 4 жыл бұрын
@@bephrem yo, you are great. Like all your videos. :)
@jayachandradora5668
@jayachandradora5668 4 жыл бұрын
Nicely explained but its more time taken.. so instead of 45 min to cover it would have covered 10 min max.
@psn999100
@psn999100 4 жыл бұрын
I think his way of clearly explaining and laying out it in more verbose way is actually good. Most other youtube channels that have covered this topic have straightaway regurgitated the information from the "Grokking the System Design Interview" without much explanations. This video made by Naren is better in that you actually can think of what possible questions the interviewer can ask you to justify the design choices. You are definitely not expected to take 45 mins in a interview, but knowing a detailed information in your mind will help you tackle any follow up questions that the interviewer has in mind.
@avinashbole4827
@avinashbole4827 5 жыл бұрын
Amazing video, Very detailed and to the point!! If possible, please add Fault tolerance and Security related usecases to be incorporated in the design
@OmprakashYadav-nq8uj
@OmprakashYadav-nq8uj 5 жыл бұрын
Hey I really like the explanation and concept of solution you provide. Can you make a video of KZfaq system design. As there is no video on KZfaq yet.
@VenkeeN17
@VenkeeN17 4 жыл бұрын
Great system design video. Thank you !!!!
@JohnGummadi
@JohnGummadi 2 жыл бұрын
Just curious about the chunks, are you assuming the client to have the knowledge of and be able to handle various file formats?
@andrejab74
@andrejab74 2 жыл бұрын
Amazing video, great explanation!
@thekishanpatel
@thekishanpatel 2 жыл бұрын
Can we have a synchronizing service on the client side as well? The purpose would be the same--to ensure that the local internal storage is in sync with the global. Pros: Avoiding the excess network calls to check diff between client storage and global storage, now the sync service can just subscribe to the message bus and compare against the local. Is there a glaring--or not so obvious---drawback to this that I am missing?
@ZeeshanAmber
@ZeeshanAmber 4 жыл бұрын
Great work Narendra. I'm learning a lot from your videos. I have gone through almost all your system design videos. Just checking if you can create one on a Saas product like Salesforce. I didn't find any good video on Salesforce / Shopify like services.
@ilyanaoumov5425
@ilyanaoumov5425 4 жыл бұрын
I think the idea of syncing file chunks will reduce latency is a little too optimistic. If you have a 1 Gbps connection, chunks or no chunks, you will be moving 125 MB/s (at most).
@holatechm
@holatechm 2 жыл бұрын
Thanks for the useful information bro
@deepaknyool
@deepaknyool 5 жыл бұрын
Great job Nagendra, look forward to seeing more interesting content from you. A part of system design it would also be nice if you could do a couple of class design and DB design examples. Design a chess game (all the classes and design patterns) or Design the database schema for instagram would be good examples.
@TechDummiesNarendraL
@TechDummiesNarendraL 5 жыл бұрын
Sure I will as soon as I get more time to work on videos.
@hido4008
@hido4008 4 жыл бұрын
Queue is not necessary for response. Each client will get update when they send pull request when they connect to the network
@ashish161087
@ashish161087 3 жыл бұрын
How we will consider the scenario where we add a complete full chunk of data in the file. In that we will have to update the chunk_order info in metadata for all chunks of the file..
@bethmonka8741
@bethmonka8741 4 жыл бұрын
Trying to understand what is reading the Database? It seems the sync service is the only one interacting with the database and then pushing the same message it received back to the response queue?
@deepakmahtohan
@deepakmahtohan 5 жыл бұрын
believe me, ur channel will gonna have 50K+ subscribers within 3 months, keep up the good work
@TechDummiesNarendraL
@TechDummiesNarendraL 5 жыл бұрын
:) I wish, Thanks
@biboswanroy6699
@biboswanroy6699 4 жыл бұрын
dogs are also barking loudly and disturbing me here as well :) Btw you rocked!
@ekrem8847
@ekrem8847 4 жыл бұрын
So there will be a file in operating systems and copy of that files in terms of chunks in our chunked ? So we need extra space as mush as space the files that are to be sync?
@jeejon9107
@jeejon9107 3 жыл бұрын
Can you into details of how a file update is sync with server and other devices. If chunk 2 of file A is modified it can potentially "modify" all chunks. So how is update optimized?
@sivamunnaluri2263
@sivamunnaluri2263 3 жыл бұрын
We want to store the images and retrieve images with different resolution based on the devices. We need to achieve this scenario with less space . Can you suggest me ?
@shivaprasad.v.g7526
@shivaprasad.v.g7526 3 жыл бұрын
This is amazing video with lots of details. If you could add more details on which part runs where , it will be complete .
@thakursaurabh98
@thakursaurabh98 3 жыл бұрын
Why do we need a queue? This parts explanation confused me a little bit. If we are doing the queueing because of internet availability issue then there needs to be 2 queues. One at the client and the other at the server. Because if the client isn’t connected then the messages will be queued at the client so that they are dispatched once we have internet connectivity. And we need a queue at server because all the clients might not be connected to the internet and we can do the retries in a backoff manner (if we are doing a push based update) or the client can pull from the queue directly whenever it comes online.
@Tigerjz32
@Tigerjz32 4 жыл бұрын
I really like the idea of breaking the file into chunks but why not use document diff libraries to be sure to only capture the changes that have taken place?
@rabindrapatra7151
@rabindrapatra7151 Жыл бұрын
calculating diff is a complicated in a collbartive environment. He was explaining in one video about operational transformer. we can use libraries but how that works also we have to know.
Online games System design backend
33:23
Tech Dummies Narendra L
Рет қаралды 59 М.
CHOCKY MILK.. 🤣 #shorts
00:20
Savage Vlogs
Рет қаралды 27 МЛН
EVOLUTION OF ICE CREAM 😱 #shorts
00:11
Savage Vlogs
Рет қаралды 14 МЛН
The Joker saves Harley Quinn from drowning!#joker  #shorts
00:34
Untitled Joker
Рет қаралды 52 МЛН
Twitter system design | twitter Software architecture | twitter interview questions
36:56
In Memory databases internals for system design interviews
34:59
Tech Dummies Narendra L
Рет қаралды 39 М.
I forced EVERYONE to use Linux
22:59
NetworkChuck
Рет қаралды 344 М.
Paste bin system design | Software architecture for paste bin
31:20
Tech Dummies Narendra L
Рет қаралды 66 М.
Google system design interview: Design Spotify (with ex-Google EM)
42:13
IGotAnOffer: Engineering
Рет қаралды 1 МЛН
NETFLIX System design | software architecture for netflix
51:26
Tech Dummies Narendra L
Рет қаралды 427 М.
NGINX Tutorial - What is Nginx
14:32
TechWorld with Nana
Рет қаралды 87 М.
CHOCKY MILK.. 🤣 #shorts
00:20
Savage Vlogs
Рет қаралды 27 МЛН