With HTTP/1.0, there used to be a recommended limit of 2 connections per domain. More recent HTTP RFCs have relaxed this limitation but still warn to be conservative when opening multiple connections:
According to RFC 7230, section 6.4, "a client ought to limit the number of simultaneous open connections that it maintains to a given server".
More specifically, besides HTTP/2, these days, browsers impose a per-domain limit of 6-8 connections when using HTTP/1.1. From what I'm reading, these guidelines are intended to improve HTTP response times and avoid congestion.
Can someone help me understand what would happen with congestion and response times if many connections were opened by domain? It doesn't sound like an HTTP server problem since the amount of connection they can handle seems like an implementation detail. The explanation above seems to say it's about TCP performance? I can't find any more precise explanations for why HTTP clients limit the number of connections per domains.
The primary reasoning for this is resources on the server side.
Imagine that you have a server running Apache with the default of 256 worker threads. Imagine that this server is hosting an index page that has 20 images on it. Now imagine that 20 clients simultaneously connect and download the index page; each of these clients closes these connections after obtaining the page.
Since each of them will now establish connections to download the image, you likely see that the connections increase exponentially (or multiplicatively, I suppose). Consider what happens if every client is configured to establish up to ten simultaneous connections in parallel to optimize the display of the page with images. This takes us very quickly to 400 simultaneous connections. This is nearly double the number of worker processes that Apache has available (again, by default, with a pre-fork).
For the server, resources must be balanced to be able to serve the most likely load, but the clients help with this tremendously by throttling connections. If every client felt free to establish 100+ connections to a server in parallel, we would very quickly DoS lots of hosts. :)
Related
I've been googling around and I cannot seem to find a straight answer to this question, and some people offer contradictory answers.
Most browsers have a 6 connection limit for each domain. So for example, if your website is example.com and it initializes a persistent Server Sent Event connection on page load, then the end user can open that tab five more times, but the sixth tab won't load at all, because the 6 persistent TCP connection has been reached for that domain.
Now, I see some people saying that this is just a perennial problem with SSE, and the only alternative is hacky workarounds involving detecting this connection limit and then either closing the connections in hidden tabs or closing those connections in those tabs and switching to long polling.
However, some people claim that HTTP2 solves this with multiplexing, such that you can have as many open tabs of that website as you'd like, as all the tabs multiplex onto the same TCP connection. I cannot find a primary source for this claim, nor anyone with significant authority making it either.
So, is it true? Does HTTP2 multiplexing solve the issue of the common 6 connection limit for website domains? Or is one basically required to instead use websockets if they would like to support many open tabs to their site?
I have implemented HTTP/2 in Jetty.
As explained in this answer, with HTTP/2 the max number of concurrent requests that a browser can make to the server is largely increased - not infinite but increased from 6-8 to about 100.
So yes, multiplexing solves this issue in practice (unless you open more than 100 or so tabs).
Note that this value is configured by servers, so it's possible that a server sends to the client a configuration with max number of concurrent requests set to a small number, but in practice servers have settled on a number around 100.
Having said that, you want to also read this other answer for a discussion about SSE vs WebSocket.
The limit is applied in the browser (not server-side), and varies per browser.
At least it did as of http/1.1. I've been unable to find any configuration specific to http/2, so I think we have to assume the limits are still there. I would assume they've been kept to prevent abuse, or accidental DoS attacks.
https://developer.mozilla.org/en-US/docs/Mozilla/Preferences/Mozilla_networking_preferences
See network.http.max-persistent-connections-per-server. (You can also see it in about:config.)
Here are all the settings I see (in Firefox) filtering for "connections":
I wonder if SSE is governed by http.max-connections or the websocket.max-connections? (Ignoring any per-domain limit, for the moment.)
You could also just try it, of course (there were example scripts in my above-linked question). But unless you control the configuration of all clients, you have to assume some people will be running with a limit of 6.
We are trying to do Load test on our servers, for this we are currently using JMeter.
However we have decided to use golang's concurrency model to create simultaneous http requests to the server and perform the load test.
Is there any limitations on how many http requests or tcp connections a machine can open/send to any other machine, is there any way to find this number?
Edit----
We need this number since this will help us identify how many http request can be sent simultaneously to the server
Thanks
Is there any limitations on how many http requests or tcp connections a machine can open/send to any other machine, is there any way to find this number?
Yes. When connecting to a single target, you are limited by the number of outbound ports, which is 65535. In practice somewhat less, as not all ports are available for use as outbound ports.
We need this number since this will help us identify how many http request can be sent simultaneously to the server
From any one machine. It has nothing to do with the maximum number of connections from different machines.
Its said that one of the advantages of HTTP 2 over HTTP 1 is that HTTP2 has streams of data. It can be up to 256 different streams in ONE TCP/IP connection. However, in HTTP 1 there can be up to 6 parallel connections. It's an improvement that HTTP 2 enables reading data from 256 resources, but still I think that 6 connections (in HTTP 1) have a better throughput that one TCP/IP connection (in HTTP 2). Still, HTTP2 is considered faster than HTTP 1. So... what don't I understand correctly ?
6 physical connections would have more throughput than one physical connection, all else being equal.
However the same doesn't apply to 6 different TCP/IP connections between the same computers, as these are virtual connections (assuming you don't have two network cards). The limiting factor is usually the latency and bandwidth of your internet connection rather than TCP/IP protocol itself.
In fact, due to the way TCP connections are created and handled, its actually much more efficient to have one TCP/IP connection. This is because of the cost of the initial connection (a three way TCP handshake, the HTTPS handshake and the fact that TCP connections use a process called Slow Start to slowly build up its capacity to the maximum speed that the network can handle) but also in the ongoing upkeep of the connection (as the Slow Start process happens again periodically unless the connection is fully utilised all the time - which is much more likely to happen with one connection that is used for everything, than it is to happen when your requests are split across 6 connections).
Also HTTP/1.1 only allows one request in flight at a time, so the connection is not able to be used until the response is returned (ignoring pipelining which is not really supported at all in HTTP/1.1). This not only limits the usefulness of the 6 connections, but also means that it's more likely that the connections will be underused which, given the issues with underused connections in TCP mentioned above, means they are likely to be slower as they throttle back and have to go through the Slow Start process again to build up to maximum capacity. HTTP/2 however allows those 256 streams to allow requests to be in flight at the same time. Which is both better than just 6 connections, and allows true multiplexing.
If you want to know more, then Ilya Grigorik had written an excellent book on the subject called High Performance Browser Networking which is even available online for free.
I have several clients that constantly post data to a REST service. REST service is put behind a network load balancer. Each client sends 100 - 500 MB a day and I need to support 500+ clients.
I can POST either very large packets, this will reduce overhead for TCP/IP session set up and HTTP headers. This will, however, firmly tie one client to a particular server and limit my scalability options. Alternatively, I can send small HTTP packets, which I can load balance well, but I will get more overhead for TCP/IP session set up and HTTP headers.
What is the recommended packet size for HTTP POST? Or how can I calculate one for my environment?
There is no recommended size.
While HTTP POST size is not constrained by the RFCs, since HTTP is a commodity protocol implementing request / response type messaging, most of the infrastructure is configured around the idea that TCP connections are not particularly long lasting / does not carry significant amounts of data. i.e. there will be factors outside your control which may impact the service - although HTTP supports range requests for responses, there is no corollary for requests.
You can get around a lot of these (although not all) by using HTTPS. However you still need to think about how you detect/manage outages - are you happy to wait for a TCP timeout?
With 500+ clients presumably using the system quite heavily, the congestion avoidance limits shouldn't be a problem - whether TCP window scaling is likely to be an issue depends on how the system is used. HTTP handshakes should not be an issue unless you restrict the request size to something silly.
If the service is highly dependant on clients pushing lots of data on to your server, then I'd encourage you to look at parsing the data on the client (given the volume, presumably it's coming from files - implying a signed java applet or javascript with UniversalBrowserRead privilege) then sending it over a bi-directional communication channel (e.g. websocket).
Leaving that aside for now, the only way you can find out what the route between your clients and your server will support is to measure it - and monitor it. I would expect that a 2Mb upload size would work pretty much anywhere, while a 10Mb size would work most of the time within the US or Europe - and that you could probably increase this to 50Mb as long as there's no mobile clients.
But if you want to maintain the effectiveness of the service you'll need to monitor bandwidth, packet loss and lost connections.
So i thought with HTTP 1.1 your TCP connections are sustained for as long are you are communicating with that server? How does it actually work, how does the TCP connection know when you are done writing into the socket? Any formation would be awesome, i have done research but i cant find what im looking for short of reading the RFC.
The typical implementation is that the HTTP server will have a timeout (typically called KeepAliveTimeout or such) after which it will close an idle connection.
A server which reserves a thread or an entire process per connection (such as apache with the usual mpm_prefork or mpm_worker), keepalives are usually disabled entirely or kept quite short (a few seconds). For an event-based server such as nginx which uses much less memory per connection, the keepalive timeout can be left at a much higher value (typically a minute or so).
See section 8.1 of RFC 2616. Basically, HTTP 1.1 treats all connections as persistent but the langauage of the RFC doesn't mandate this behaviour, since it uses the word "SHOULD". If it was mandated, it would use "MUST".
However, the RFC does not specify in detail how an implementation does this. As can be seen from the HTTP Persistent Connection page on Wikipedia, Apache's default timeout (beyond which it returns persistent connections for other uses) may be as low as five seconds. (though this is almost certainly configurable, given all the other knobs and dials that Apache provides).
In other words, it's meant for numerous requests to the same address within a short time frame, so as to not waste time opening and closing a bucket-load of sessions where one will do. Increasing this timeout is not a "free ride", since resources are tied up while the connection is held open. In an environment where you expect lots of incoming clients, tying up these resources can be fatal to performance.