Is there a Limit to Number of Connections a Backend can handle?

Рет қаралды 32,749

Күн бұрын

Someone asked me a question and I felt its interesting to make a video about, is there a limit to the maximum number of TCP connections a Client can make to the server? If there is what is it? and how does that make sense in all the configurations?
0:00 Intro
1:00 Is there a Max Connection Limit?
4:30 64K Connection Limit Explained
7:20 Max Connections on Reverse Proxies and Max Connections
14:30 How does Router get around Max Connections?
🏭 Backend Engineering Videos
• Backend Engineering (B...
💾 Database Engineering Videos
• Database Engineering
🛰 Network Engineering Videos
• Network Engineering
🏰 Load Balancing and Proxies Videos
• Proxies
🐘 Postgres Videos
• PostgresSQL
🚢Docker
• Docker
🧮 Programming Pattern Videos
• Programming Patterns
🛡 Web Security Videos
• Web Security
🦠 HTTP Videos
• HTTP
🐍 Python Videos
• Python by Example
🔆 Javascript Videos
• Javascript by Example
👾Discord Server / discord
Become a Member
/ @hnasr
Support me on PayPal
bit.ly/33ENps4
Become a Patreon
/ hnasr
Stay Awesome,
Hussein

Пікірлер: 77

@BeALeaderAndAnInspir 3 жыл бұрын

Man, subscribing to your channel is one of the best decisions I have made in 2020. You have touched on the topics that a few people discussed in detail on KZfaq. Thanks, keep doing these great work.

@hnasr 3 жыл бұрын

I appreciate you Tien! So glad the content helps, let me know how I can do better

@syedaffan3071 3 жыл бұрын

Man I am just hooked to your channel! Can,t just have enought of it. Thanks for producing such quality content!

@ryan-bo2xi 3 жыл бұрын

Sir you just leveling up my confidence for life with your videos .. huge thanks ! Will surely applaud !

@youd87 3 жыл бұрын

One of the best channel KZfaq ever recommended to me. Thank you for all your efforts🙏

@hnasr 3 жыл бұрын

Thank you dear!!

@utikevin5987 3 жыл бұрын

Can you tell me other KZfaq channels that was recommended to you?...It may be helpful. Thanks

@aaronputra5578 3 жыл бұрын

If my parents let me choose between university or watching your KZfaq videos, I'm too dumb if not choosing your videos! I super love your contents, really. I love the way you share and teach me what are the problems, what are the fundamentals, how to solve them until you gave the solution. Thank you, man!

@AllenLSY 3 жыл бұрын

We definitely like your videos. I have one suggestion: for the knowledges that can be better presented using diagrams or you need draw animations while expressing, get a ipad with apple pencil, draw on a sketch pad app, and record the screen. That way it's much more clear than waving your hands in the air.

@hnasr 3 жыл бұрын

Thanks for the tips! Appreciate you dear

@himanshutomar3512 3 жыл бұрын

Saw this video and listened to you, and immediately a like and a subscription 👍 You are awesome mate 🙂

@sawza0313 2 ай бұрын

I listen a lot of your explanations, and they help A LOT ! and I really really love how you use explain software engineering, and I really trust it! Some are new and some are like putting the missing puzzles at to place 🎉👏🏼👏🏼👏🏼👏🏼 Btw you look like a brother of mine, he’s civil engineer and very lever , like you 😊❤️

@DR-ee4wv 2 жыл бұрын

likes your all videos. A real backend engineer

@UgalabaguUgabagu 3 жыл бұрын

This video made life a bit simpler. Thanks!

@sudharajamanickam7534 3 жыл бұрын

Thanks for the dedicated video on this topic..

@hnasr 3 жыл бұрын

Sudha Rajamanickam Thank you for watching!

@MehdiShojaei 3 жыл бұрын

Nice clarification, Thanks. Actually, the real problem comes from the maximum file descriptor (FD) limit, that must be configured for servers.

@hnasr 3 жыл бұрын

Mahdi Shojaei that is something I missed thanks for sharing!

@hackwithharsha5228 3 жыл бұрын

@@hnasr Thanks Hussien, Is it possible for you to explain file descriptor (FD).. I have done little bit of research.. not able to understand the context here.

@saimanojch 3 жыл бұрын

Thanks for vids dude, a correction though. At 13:50, neither adding more backends nor listening on multiple ports on the backend solve the issue that you are addressing (i.e the number of TCP connections that the reverse proxy can open with your backend) If the ports on the reverse proxy exhausted, if you add one more backed at this point of time, reverse proxy can't open an extra connection just because you added a backed, reason being ports are already exhausted. Even If the backend is configured to listen on multiple ports, it doesn't help reverse proxy to open one more connections once the ports are exhausted.🙂

@amlanbiswas4526 7 ай бұрын

Hey sorry @saimanojch, I am two years too late to answer this and since no one answered here, I am. @hnasr can correct me, if necessary. Each TCP/IP packet has basically four fields for addressing. These are: source_ip source_port destination_ip destination_port Now, when the number of backend servers increase, along with source_port, destination_ip changes too. So the TCP connection can be registered uniquely.

@Oswee 3 жыл бұрын

Hi. Idea for the topic. Build systems. Why do we need one. History. Shell, Ant, Maven, Buck, Bazel. When do you don't need any. When you could look into Bazel or Buck or any other. Personally i currently digging into Bazel. Seems quite powerful and mono-repo centric. Probably will try to adopt for my workflow as i am working on 7+ related projects simultaneously.

@benjaminmella2407 2 жыл бұрын

Really good content, I appreciate your effort. Have a nice Day. Thanks

@ahmedtarek8792 3 жыл бұрын

Another great one!

@hnasr 3 жыл бұрын

Thank you Ahmed!

@familytamelo8140 Жыл бұрын

Thanks!

@namitpiriya 3 жыл бұрын

Great video. I am a beginner so didn't quite understand the last part. But rest was all good 😁. Keep making these videos great source of learning.

@anithaswaminathan996 2 жыл бұрын

You are awesome !!! Very interesting and you try to explain it in lame man terms - watching this gives the interest to learn more...

@ruhankhandakar 3 жыл бұрын

another great video.. thanx

@hnasr 3 жыл бұрын

Thanks Ruhan!

@benjaminmella2407 2 жыл бұрын

Thanks for the explanation, I would like to know if there is a limit in a mysql database connections, I really appreciate your answer

@ishandhar2851 3 жыл бұрын

Hi Hussein, on the backend machine aren't we limited with file descriptor limits on the OS?

@dragon_warrior_ 2 жыл бұрын

Question: you said that once the reverse proxy hit the max limit it can't open more connection and to solve this we need more backend. Question is that if the reverse proxy is exhausted, then on connecting to the reverse proxy we will be waiting till a socket is open and if we are still waiting here on the front end of the backend that is the reverse proxy and haven't even reached to the backend, then how our problem is resolved by adding more backend?

@debugmedia 3 жыл бұрын

Is there a limit to in-memory caching? Because I have a websocket connection where I store names of the active users in an array and emit that array to whoever opens the website, So that way they can see who all arw online. But is that a good idea? Or should I use something like redis?

@hnasr 3 жыл бұрын

Debug Media that is a good question, if you exceed the memory the OS will take over and page excess memory to disk which can slow things down.. i would implement an LRU cache where you dumb all pages from the cache

@Ooulu Жыл бұрын

This content is fire 🔥

@dexteraparicio6808 Жыл бұрын

With regards to limits at the backend, can you shed more info on what is the effect of /proc/sys/net/ipv4/ip_local_port_range? Thank you.

@pulkitkedia8679 3 жыл бұрын

If we add multiple backend servers with only one Load balancer then also the same issue of 64K connections running out would arise right ? It is only with multiple load balancers we can increase the number of connections(more than 64k) right ?

@hnasr 3 жыл бұрын

Pulkit Kedia correct the load balancer can only open 64k connections in total from its side.you can avoid this limit by listening on multiple ip addresses on the load balancer. Its equivalent to installing multiple network cards on the load balancer machine so you get multiple nics and as a result multiple ip network addresses and you can listen to all of them and use them accordingly

@titidantete 2 жыл бұрын

ah... I thought 64k connection is limited to server, so basically if I run websocket server, even 200k connections from client connected to server then it's no problem right ? the problem might be if the client send more than 64k request at same time to server, is that correct ?

@ripudamank 3 жыл бұрын

Hundred things run in his mind when he says "when you make a request"

@Flankymanga 3 жыл бұрын

13:59 Actually Hussein could another solution be to add more IPs to the single backend host? Like for example having a NIC with multiple IP addresses and backend is listening to both? While Proxy would have to have some load balancing functionality but the backend host will still be a single machine?

@hnasr 3 жыл бұрын

Flankymanga that is correct! The proxy can listen on multiple ip addresses .. i missed to talk about this , Someone else mentioned this in the comments

@StyleTrick 3 жыл бұрын

Great explanation. One question, how does a server establish a connection with a client? A server can handle millions of connections concurrently? After reading it seems like this optimisation is done at the kernel level?

@hnasr 3 жыл бұрын

thanks! the client is usually who initiate the connection not the server. I explain this here kzfaq.info/get/bejne/mL2Pdax90Z3SYI0.html

@StyleTrick 3 жыл бұрын

@@hnasr I watched this, yes I understand the 3 way handshake and the client sends packets to the server in order to achieve this. My question was more around how the server can handle 3 million concurrent requests like WhatsApp. From what it seems, the server assembles packets together to form ONE TCP Connection and the amount of memory each connection takes is the bottleneck. Is the TCP connection ephemeral or reused in this case?

@pk-bn1hc 3 жыл бұрын

8:38 shouldn't it be layer 7 proxy/load balancer , bcoz as u have explained in your layer 4 7 lb video that layer 7 proxy initiates a new tcp connection with the backend server ?????pls explain ??

@hnasr 3 жыл бұрын

p k layer 4 and layer 7 proxy usually has two tcp connection. There is one special case of layer 4 proxy where it acts like a gateway (router) and in that case it is a single TCP connection

@hackwithharsha5228 3 жыл бұрын

@@hnasr I watched both your Layer 4 and Layer 7 videos to understand this point, Got it.. Thank you !!

@vinny142 3 жыл бұрын

"One stiinki'n server" Not just "a" server, a *big* server. A *VERY* big server. Whatsapp is a bit of a utopian example because there are only a handfull of websites as big as whatsapp, the vast majority that us mere mortals have to deal with struggle to get 1Mln users, let alone 1M concurrent connections. About the proxies; a reverse proxy can ofcourse use as many internal IP's as it needs to, and each backend server can listen on as many internal IPs as it needs to, so there that 64k limit doesn't really exist there either. So a simple HAProxy setup would remove the TCP connections limit alltogether.

@hnasr 3 жыл бұрын

vinny142 that is a great point Vinny! So how would that work ? different network interfaces on different subnets? I haven’t played with this at all any resources? Cheers and thanks for those valuable info

@vinny142 3 жыл бұрын

@@hnasr No real resources off the top of my head but this support question does the same thing: discourse.haproxy.org/t/set-frontend-ip-as-source-ip-for-backend/320 What it boils down to is that you can instruct HAPRoxy to use a particular IP for outgoing traffic, so as long as that IP is defined on the NIC you can comunicate fro there to many backends and use all of the loadbalancer's IP's for all backends.Thre IP's = 60k connections,etc. Given that most hosted servers will have a private secondary network, and that IPv6 is getting more mainstream, this should solve the problem nicely.

@sahukarinaveenkumar3188 3 жыл бұрын

Nice hair style 😊😊

@athithyaraj1435 3 жыл бұрын

So at a moment a client can make 64k open connection with reverse proxy. What should be done to handle more than that with reverse proxy like nginx?. Clarify please

@hnasr 3 жыл бұрын

It is pretty bad idea to open 64k connections from the same client so I would want to know the use case (same IP address). I don't know anyone including big companies that hit that limit. But if you absolutely have to, nginx or any can listen to different IP addresses and put those behind a DNS as I explained in the video

@athithyaraj1435 3 жыл бұрын

@@hnasr Now I understand a bit more after your reply. From a single client , a reverse proxy can handle 64k connection which is more than required for any client server communication. Say , client A is engaged with reverse proxy with 64k connection (not a use case but curiosity) Now client B tries to connect with reverse proxy.. Question is : 1. What will happen if client A adds one more request to reverse proxy? Will it be in Q? 2.Say, multiple clients are having max 64k connection with reverse proxy.. can reverse proxy process a new client request without much performance issue?

@sharthakghosh970 3 жыл бұрын

How does client 2 open another set of 64k connections on the same server port ?

@hnasr 3 жыл бұрын

Sharthak Ghosh because client 2 has a different IP address and a whole set of a different local ports

@hnasr 3 жыл бұрын

This might help How Network Address Translation is used on Layer 4 Load Balancing and Port forwarding kzfaq.info/get/bejne/iK1patWn3JfJhaM.html

@sharthakghosh970 3 жыл бұрын

@@hnasr Makes sense

@section9999 3 жыл бұрын

Hey man you mentioned that whatsapp uses some 3 million ip connections. Do you mind sharing where you get this information? It would be interestingbto see how some other companies are handling on thier backend.

@hnasr 3 жыл бұрын

Oh yeah sure I discuss this in length in the WhatsApp video here kzfaq.info/get/bejne/rLdlopeotdy5iJs.html you can find the resources in the description box

@section9999 3 жыл бұрын

@@hnasr cool man thanks so much! Your efforts are muuch appreciated. :)

@hnasr 3 жыл бұрын

❤️❤️

@Flankymanga 3 жыл бұрын

6:08 it does make sense if you are doing DOS attack like slow lorris.

@hnasr 3 жыл бұрын

That is true, slow loris connections are long lived.

@hackwithharsha5228 3 жыл бұрын

@@hnasr As per wikipedia, Nginx and lighttpd server doesn't succumb to this attack. Not sure, how.. en.wikipedia.org/wiki/Slowloris_(computer_security)

@serhiihorun6298 3 жыл бұрын

I don’t understand who are those 5 guys who disliked your video. Seems some of your coworkers

@pajeetsingh 3 жыл бұрын

Limit is 2^bits-used-to-represent-int -1

@barebears289 2 жыл бұрын

So, is there?

@Hercules159 2 жыл бұрын

We are doing it[64k], We are doing file processing 64k is not enough for us, we are tyring to do same as WhatsApp

@rajeshkishore7171 3 жыл бұрын

sir layer 4 in your video kzfaq.info/get/bejne/p7-edJV1q6upfKs.html you talked about it wont terminate the connection , it will be the original tcp connection throughout but here I got confused as you mentioned layer 4 would terminate the connection and make a fresh connection to backend server?

@np2819 3 жыл бұрын

You are right. I wanted to bring that up as well. With Layer 4 this problem won't occur it is like a pipe. With layer 7 - this problem will occur.