I have a server implementing a REST service using golang's net.http and gocraft.web package.
And I have a client sending JSON request via libcurl, the client is written in C++.
What I notice is that when there are a lot of client threads for sending concurrent JSON requests, there are times when the client gets "Couldn't connect to server" error, but then after a while it can resume sending again.
Since the observation points to rate-limiting, my question is where is rate-limitting happening? Is this at the client side (libcurl), Go's http package, or gocraft.web package?
Related
In the http2-spec, the scenario where the server half-closed the stream (server sent http2.END_STREAM), the client is still allowed to send data (since it's half-closed).
Consider the following gRPC scenario:
Client opens bidi-stream to server and starts sending data
Server closes the response stream and sends the status trailers (translates to sending http2.END_STREAM)
Client continues to send data
Are the semantics well-defined in gRPC?
Possible ways:
Follow the http2-spec: The client is allowed to continue to send data that is processed by the server.
Not follow the http2-spec: The client-connection is implicitly terminated if the server closes the stream.
NOTE: I just tested and it looks like gRPC for Java follows variant "not follow the http2-spec", i.e. if the server closes the downwards stream, also the upwards stream is closed.
In the semantics of gRPC, when the server sends a status, it means that the entire call is complete. The official servers implementations send RST_STREAM in addition to END_STREAM to shut down the stream in both directions at the HTTP/2 protocol level, in accordance with this part of Section 8.1 of the HTTP/2 RFC:
A server can send a complete response prior to the client sending an entire request if the response does not depend on any portion of the request that has not been sent and received. When this is true, a server MAY request that the client abort transmission of a request without error by sending a RST_STREAM with an error code of NO_ERROR after sending a complete response (i.e., a frame with the END_STREAM flag).
When servers do not do this, the gRPC protocol does not prohibit the client sending more data after receiving a status but the server will not process it so there is no reason to do so. Because of this, when the official gRPC clients receive a status they consider the call to be complete and stop sending data.
I understand an HTTP request will result in a response with a code and optional body.
If we call the originator of the request the 'client' and the recipient of the request the 'server'.
Then the sequence is
Client sends request
Server receives request
Server sends response
Client receive response
Is it possible for the Server to complete step 3 but step 4 does not happen (due to dropped connection, application error etc).
In other words: is it possible for the Server to 'believe' the client should have received the response, but the client for some reason has not?
Network is inherently unreliable. You can only know for sure a message arrived if the other party has acknowledged it, but you never know it did not.
Worse, with HTTP, the only acknowledge for the request is the answer and there is no acknowledge for the answer. That means:
The client knows the server has processed the request if it got the response. If it does not, it does not know whether the request was processed.
The server never knows whether the client got the answer.
The TCP stack does normally acknowledge the answer when closing the socket, but that information is not propagated to the application layer and it would not be useful there, because the stack can acknowledge receipt and then the application might not process the message anyway because it crashes (or power failed or something) and from perspective of the application it does not matter whether the reason was in the TCP stack or above it—either way the message was not processed.
The easiest way to handle this is to use idempotent operations. If the server gets the same request again, it has no side-effects and the response is the same. That way the client, if it times out waiting for the response, simply sends the request again and it will eventually (unless the connection was torn out never to be fixed again) get a response and the request will be completed.
If all else fails, you need to record the executed requests and eliminate the duplicates in the server. Because no network protocol can do that for you. It can eliminate many (as TCP does), but not all.
There is a specific section on that point on the HTTP RFC7230 6.6 Teardown (bold added):
(...)
If a server performs an immediate close of a TCP connection, there is
a significant risk that the client will not be able to read the last
HTTP response.
(...)
To avoid the TCP reset problem, servers typically close a connection
in stages. First, the server performs a half-close by closing only
the write side of the read/write connection. The server then
continues to read from the connection until it receives a
corresponding close by the client, or until the server is reasonably
certain that its own TCP stack has received the client's
acknowledgement of the packet(s) containing the server's last
response. Finally, the server fully closes the connection.
So yes, this response sent step is a quite complex stuff.
Check for example the Lingering close section on this Apache 2.4 document, or the complex FIN_WAIT/FIN_WAIT2 pages for Apache 2.0.
So, a good HTTP server should maintain the socket long enough to be reasonably certain that it's OK on the client side. But if you really need to acknowledge something in a web application, you should use a callback (image callback, ajax callback) asserting the response was fully loaded in the client browser (so another HTTP request). That means it's not atomic as you said, or at least not transactional like you could expect from a relational database. You need to add another request from the client, that maybe you'll never get (because the server had crash before receiving the acknowledgement), etc.
Say i make the web request(www.amazon.com) to amazon web server through browser. Browser makes the connection with Internet through Internet service providers.
Request reaches to amazon server which process it and send back the response. Two questions here :-
Does Amazon server makes new connection with internet to send the response back or incoming request(initiated by me) waits on socket till amazon process the response ?
Once my browser receives the response how does it map the response(sent from amazon) back to particular request . I believe there must be some unique identifier like
requestId must be present in response through which browser must be mapping to request. Is that correct ?
Does Amazon server makes new connection with internet to send the response back or incoming request(initiated by me) waits on socket
till amazon process the response ?
It uses the same connection. Most of the time it's not even possible to connect back to a web browser due to firewall restrictions or Network Address Translation (NAT).
Once my browser receives the request how does it map the response(sent from amazon) back to particular request . I believe
there must be some unique identifier like requestId must be present in
response through which browser must be mapping to request. Is that
correct ?
It receives the response on the same socket. So the socket is the identifier. If HTTP2 multiplexing is used, then each multiplexed stream has a stream identifier, which is used to map the response back to the request.
The client opens a TCP-connection to the server, sends an HTTP-request and the server sends the response using the same connection. So, the browser knows from the connection that the response belongs to a specific request. This applies to basic HTTP 1.
This has to be distinguished from the programming model of an AJAX web application which is asynchronous and not synchronous. The application does not actively wait for a response. It is instead triggered later when the response arrives. The connection handling described above is what happens "under the hood".
Back to the connection handling: There are optimizations of HTTP that make things more complicated. HTTP 1.1 has a feature called "keep alive" and HTTP 2 goes further into this direction. The idea is to send more data over a single TCP-connection because establishing a TCP-connection is expensive (-> three way handshake, slow start). So, multiple requests and responses are sent over a single TCP-connection. Your question arises again in case of this optimization. If e. g. there is a sequence of requests A, B and a sequence of corresponding responses B, A within a single HTTP-connection how does the browser know the request a response belongs to? HTTP 2 introduces the concept of streams (RFC 7540, section 5):
A single HTTP/2 connection can contain multiple concurrently open
streams, with either endpoint interleaving frames from multiple
streams.
The order in which frames are sent on a stream is significant.
Streams are identified by an integer.
So, the stream identifier and the order within a stream can be used by the browser to find out the request a response belongs to.
HTTP 2 introduces another interesting feature which is called "push". The client can proactively send resources to the client that the client has not even requested. So, resources like e. g. images can be already sent when the HTML is requested avoiding another communication roundtrip.
HTTP uses Transfer Control Protocol. This is how it happens-
Does Amazon server makes new connection with internet to send the response back or incoming request(initiated by me) waits on socket till amazon process the response ?
No. Most browsers use HTTP 1.1 so the connection between client and server is established only once until closed (Persistent connection).
Once my browser receives the request how does it map the response(sent from amazon) back to particular request . I believe there must be some unique identifier like requestId must be present in response through which browser must be mapping to request. Is that correct ?
There is a protocol(HTTP) on how the messages are exchanged. HTTP dictates that responses must arrive in the order they were requested. So it goes like-
Request;Response;Request;Response;Request;Response;...
And there is also a specific format of HTTP request (from your browser- HTTP client) and HTTP response message (from amazon HTTP server). There are response status codes that let the browser know if their request has been succeeded, otherwise tell the errors.
A few sample codes-
I am using HTTP long polling for pushing server events to a client.
On the client side, I send a long polling request to the server and block there waiting for a event from the server.
On the server side, we used the cometd framework (I am on the client side, do not really know much about the server side).
The problem is, after sometime, the connection is broken and the client can not detect this, so it blocks there forever. We are trying to implement some kind of heartbeat message, which will be sent every N minutes to keep the connection active. But this does not seem to work.
My question is: does HTTP long polling support heartbeat messages? As far as I understand, HTTP long polling only allows the server to send one event and will close the connection immediately thereafter. The client must reconnect and send a new request in order to receive the next event. Is it possible that the server sends heartbeat messages every N minutes while still keep the connection open until a real server event happens?
If you use the CometD framework, then it takes care of notifying the application (both on client and on server) about when the connection is broken, and it does send heartbeat messages.
What you call "HTTP long polling" is just a normal HTTP request, so in itself does not support heartbeat messages.
You can use HTTP long polling requests to implement heartbeat messages, and this is what CometD does for you under the covers.
In CometD, the response to a HTTP long poll request may deliver multiple messages, and the connection will not be closed afterwards. The client will send another HTTP long poll request without the need to reconnect, possibly reusing the previous connection.
CometD offers to your application a higher level API that is independent from the transport, so you can use WebSocket rather than HTTP, which is way more efficient, without changing a single line in your application.
You need to use the CometD libraries both on client (javascript and java) and on server, and everything will just work.
I need a way to detect a missing response to a long running HTTP POST request. This problem arises when the network infrastructure (firewalls, proxies, unplugged cables, etc.) drops the response packets. The server may detect this failure, but the client cannot send additional bytes after the POST to probe the state of the TCP connection. The failure may be limited to a single TCP connection. For example I may be able to subsequently open a new TCP connection to the server.
I'm looking for a solution that still uses HTTP POST and does not change the duration of the server side processing.
Some solutions that I can think of are:
Provide a side channel interface to retrieve request & response history. If the history lists the response as having been send (presumably resulting in a TCP error) but I have not yet received it within a reasonable time I can generate a local error.
Use an X header to request that the server deliver "spurious" 100 Continue provisional responses on a regular interval. If I fail to see an expected 100 Continue or a non-provisional response I can generate a local error.
Is there a state of the art solution for this problem?
It sounds to me like you are using Soap for something that would be much better done using a stateful connection, or a server side push technology.