While I am configuring my nginx, I found two modules: ngx_http_limit_conn_module and ngx_http_limit_req_module
one is for limiting connection per defined key, and one for limiting request.
My question is what is the relationship (and difference) between
a HTTP connection and a request.
It seems that multiple HTTP requests can use one common HTTP connection, what is the principle under this?
Basically connections are established to make requests using it. So for instance endpoint for given key may accept 5 connections per hour from given IP address. But it doesn't mean only 5 requests can be made but much more - if the connection is not closed after a request (from HTTP 1.1 it's by default kept alive).
E.g. an endpoint accepts 5 connections and 10 requests from given IP address. If connection is established for every request only 5 requests overall can be made. If connection is kept alive single client may make all the requests. If there are 5 clients, every establishes a connection and keeps it alive there are 2 request approx. that can be made by each client - however one can make all the request if it's fast enough.
HTTP connections - client and server introduce themselves.
HTTP requests - client ask something from server.
Making a connection with server involves TCP handshaking and it is basically creating a socket connection with the server. To make a HTTP request you should be already established a connection with the server. If you established connection with a server you can make multiple request using the same connection(HTTP/1.0 by default one request per connection, HTTP/1.1 by default it is keep alive). As most of the web pages need multiple resources from the server(ex: 100 photos to load in the screen). It is a low burden to the server if we keep the connection and request those 100 images using the same connection(No need to go through the connection establishment process 100 times). That is why HTTP/1.0 came up with keep alive as default.
A request is a functional execution: "Do something for me, and return the result back to me" - which is made by the client over a channel that the server is listening on, the "connection". Think of it as making a phone call to a restaurant. When the restaurant picks up the phone, you have an established "connection" - and now can place multiple requests over the same connection. The restaurant can handle multiple, simultaneous customer calls, if it has multiple phone lines open to receive the calls. This is your "connection pool" - at any point in time, you can only have as many simultaneous open connections (max) as the size of your connection pool. The number of requests however will vary. Some client may make 3 requests, and hang up, while other client may make 10 requests before hanging up.
The size of your connection pool determines concurrency - how many simultaneous clients can you talk to at any point in time? The length of those conversations will be use case specific.
Related
Consider an application that access a remote HTTPS server, sending POST of JSON-formatted requests at an URL on the server, and receiving JSON-formatted answers. The server does not support HTTP/2 multiplexing.
There are many requests, with widely varying workload (from idle to hundreds TPS). JSON messages are in the order of 1 kbyte. Client and server are authenticated by certificates+private keys. The requests can be considered independent (in particular, the server treats requests alike for all HTTPS channels opened with the same client certificate).
HTTP/1.1 does not allow* multiple concurrent POST requests over the same connection. Therefore the throughput can't exceed N/(Tr+Ts) TPS, where N is the number of opened HTTPS/TLS channels in use, Tr is the network round trip delay, and Ts is processing time on the server side (in the order of 30 ms under low load, due to database access and other factors). Opening an HTTPS connection costs at least 4 Tr, and sizable CPU time on both sides. It looks like something is needed to manage a pool of HTTPS connections on the client side.
How is this issue usually handled?
What are common libraries or background daemon/services, automatically opening new HTTPS connections as needed, reusing them when possible?
It would be nice if that detected when the server becomes unresponsive, and handled fallback to a backup server at a different URL, with return to the main server when it is up again.
Note: Next step would be load balancing, but then my load-balancing layer must somewhat handle an affinity between the requests, since they are not fully independent (sending a dependent request to the wrong server is reliably detected by the server, though).
[*] Due to how RFC 2616 is interpreted, I'm told.
I understand that http2 uses one tcp connection to serve multiple requests, for example, if I request index.html which contains a.css and a.js, these three requests will be done in one tcp connection.
What happens if user clicks index2.html? does this request still use the same previous tcp connection? If so, will the browser keep the connection open until user closes the browser? And on the server side, does the server keep many connections open all the time?
When using HTTP/2, browsers typically open only one connection per domain.
In your example, index2.html will be sent on the same TCP connection that was used for index.html, a.css and a.js.
In HTTP/2 requests are multiplexed on the same TCP connection, so that the browser can send them concurrently, without waiting for a previous request to be responded to.
Both browsers and servers have an idle timeout for TCP connections.
If the connection is idle for long enough, it will be closed by either party - the one that has the shorter idle timeout, to save resources.
For example, you may open a connection to a wikipedia.org, perform a few requests, and then leave that tab and work on something else.
After a while (typically 30 seconds) the browser will close the TCP connection to wikipedia.org.
On the server side, the server will keep the connections from various clients open, until they are either closed by the client or until the server-side idle timeout fires, at which point it's the server that initiated the close of the TCP connection.
With HTTP/2, the number of connections that a server has to maintain is vastly less than it was with HTTP/1.1.
With HTTP/2, a server has to maintain just 1 TCP connection per client; with HTTP/1.1, the server had to maintain typically 2-8 TCP connections per client.
What happens if user clicks index2.html? does this request still use the same previous tcp connection?
Yes. On top of that, multiple browser tabs/windows also share a single HTTP/2 connection.
If so, will the browser keep the connection open until user closes the browser?
Below from RFC - connection management
For best performance, it is expected that clients will not close
connections until it is determined that no further communication with
a server is necessary (for example, when a user navigates away from a
particular web page) or until the server closes the connection.
Clients SHOULD NOT open more than one HTTP/2 connection to a given
host and port pair.
And on the server side, does the server keep many connections open all the time?
Servers are encouraged to maintain open connections for as long as
possible but are permitted to terminate idle connections if necessary.
When either endpoint chooses to close the transport-layer TCP
connection, the terminating endpoint SHOULD first send a GOAWAY
(Section 6.8) frame so that both endpoints can reliably determine
whether previously sent frames have been processed and gracefully
complete or terminate any necessary remaining tasks.
More info on connection error below.
RFC connection-error-handling
A connection error is any error that prevents further processing of
the frame layer or corrupts any connection state. An endpoint that
encounters a connection error SHOULD first send a GOAWAY frame with
the stream identifier of the last stream that it successfully received
from its peer. The GOAWAY frame includes an error code that indicates
why the connection is terminating. After sending the GOAWAY frame for
an error condition, the endpoint MUST close the TCP connection. It is
possible that the GOAWAY will not be reliably received by the
receiving endpoint. In the event of a connection error, GOAWAY only
provides a best-effort attempt to communicate with the peer about why
the connection is being terminated.
An endpoint can end a connection at any time. In particular, an
endpoint MAY choose to treat a stream error as a connection error.
Endpoints SHOULD send a GOAWAY frame when ending a connection,
providing that circumstances permit it.
I've read some articles comparing the differences between WebSocket and the other push methods like Long polling. All the conclusions tend to be WebSocket is better then HTTP with low latency in the server and client bidirectional communication process.
But if server push is not a must, for example, a client game program just make a few queries to the server for some information, does it still better to use WebSocket then HTTP? More specially, I have two doubts here:
1. In a single Request-Response procedure, which is more efficency ? (I establish a WebSocket connection each time querying in the above case.)
2. Will the server capacity (The total number of clients that the server can serve) be affected by the unnecessary long-lived connection if I keep an WebSocket connection during the life cycle of the client?
Added Question:
3. Suppose there is only one TCP connection between the server and the client, will the stability of the connection go down and down as time flows?
The basic thing behind both the WebSocket and HTTP is the socket. In HTTP, it opens a connection on request and closes on response. For WebSocket, concept is a 2 way communication (full duplex) rather than request-response cycle.
Answers to your question:
Either you can use HTTP server or can create request-response design
using WebSocket
That's obvious. Each connection is a socket object. Server capacity
will be affected if we are not managing connections.
In WebSocket, it's using ping-pong mechanism to make sure that the client or
the server is alive. For every ping requests from one end, other end is
subjected to reply a pong response. This mechanism helps to detect failures and hence to maintain stability.
I want to make simple HTTP proxy server.
Here, I have some problem of designing the program because of the number of connections.
When a client attempts to make connection to the 2 servers, there would be 2 connections; one from client to the server A and the other from client to the server B. It is natural; at least I think.
However, I'm confused when there is a proxy between client and server. I thought the client might make only 1 connection to the proxy, and send all of HTTP message (to server A and server B) via the connection. The first method is very natural (making 2 connections for 2 servers), but I want to double-check this before starting implementation!
Clients might make only one connection to your proxy server (using HTTP keepalive and/or pipelining to sequentially make more than one request through the same connection), or they might make multiple connections to your proxy server (especially if they want to make more than one HTTP request in parallel). You should be prepared for both eventualities because it's up to the client what it does.
The case of two HTTP requests coming over the same connection is semantically identical to the case of the same two HTTP requests coming over separate connections.
I just went through the specification of http 1.1 at http://www.w3.org/Protocols/rfc2616/rfc2616.html and came across a section about connections http://www.w3.org/Protocols/rfc2616/rfc2616-sec8.html#sec8 that says
" A significant difference between HTTP/1.1 and earlier versions of HTTP is that persistent connections are the default behavior of any HTTP connection. That is, unless otherwise indicated, the client SHOULD assume that the server will maintain a persistent connection, even after error responses from the server.
Persistent connections provide a mechanism by which a client and a server can signal the close of a TCP connection. This signaling takes place using the Connection header field (section 14.10). Once a close has been signaled, the client MUST NOT send any more requests on that connection. "
Then I also went through a section on http state management at https://www.rfc-editor.org/rfc/rfc2965 that says in its section 2 that
"Currently, HTTP servers respond to each client request without relating that request to previous or subsequent requests;"
A section about the need to have persistent connections in the RFC 2616 also said that prior to persistent connections every time a client wished to fetch a url it had to establish a new TCP connection for each and every new request.
Now my question is, if we have persistent connections in http/1.1 then as mentioned above a client does not need to make a new connection for every new request. It can send multiple requests over the same connection. So if the server knows that every subsequent request is coming over the same connection, would it not be obvious that the request is from the same client? And hence would this just not suffice to maintain the state and would this just nit be enough for the server to understand that the request was from the same client ? In this case then why is a separate state management mechanism required at all ?
Basically, yes, it would make sense, but HTTP persistent connections are used to eliminate administrative TCP/IP overhead of connection handling (e.g. connect/disconnect/reconnect, etc.). It is not meant to say anything about the state of the data moving across the connection, which is what you're talking about.
No. For instance, there might an intermediate (such as a proxy or a reverse proxy) in the request path that aggregates requests from multiple TCP connections.
See http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-21.html#intermediaries.