As an HTTP 1.1 server, I reply to a GET request with 200 OK status code, then start sending the data to the client.
During this send, an error occurs, and I cannot finish.
I cannot send a new status code as a final status code has already been sent.
How should I behave to let the client know an error occurred and I cannot continue with this HTTP request?
I can think of only one solution: close the socket, but it's not perfect: it breaks the keep-alive feature, and no clear explanation of the error is given to the client.
The HTTP standard seems to suppose that the server already knows exactly what to reply before it starts replying.
But this is not always the case.
Examples:
I return a very large file (several GB) from disk, and I get an IO error at some point during the reading of the file.
Same example with a large DB dump.
I cannot construct my whole response in memory then send it.
The HTTP 1.1 standard helps for such usage with the chunked transfer encoding: I don't even need to know the final size before starting to send the reply.
So these usage are not excluded from HTTP 1.1.
I finally found a possible solution for this:
HTTP 1.1 Trailer headers.
In chunked encoded bodies, HTTP 1.1 allows the sender to add data after the last (empty) chunk, in the form of a block of headers.
The specification hints some use-cases like computing on the fly a md5 of the body, and send it after the body so the client can check its integrity.
I think it could be used for error reporting, even if I haven't found anything about this kind of usage.
The issues I see with this are:
this requires using chunked encoding (but it's not much of an issue)
trailers support is probably very low:
server-side (it could be bypassed by manually creating the chunked encoding, but since it's applied after the content-encoding (gzip), it would require a lot of reimplementation)
client-side (bugs fixed only in 2010 in curl for example)
and on proxies (that could then loose the trailers if not properly implemented)
I have pushed the similar question to be answered, so here you can find that there is no solution:
How to tell there's something wrong with the server during response that started as 200 OK. Fail gracefully
Related
Say I started writing to the response body, but there was some error, and I need to indicate that it's an HTTP 500 even if an HTTP 200 OK header was already written as a header...
How can I write something to the body of the response that's guaranteed to be malformed so that the response is interpreted as some sort of error by the client?
In general, this is impossible. Some clients only care about the response header, and may stop paying attention to what you send after the header.
But with certain clients, in certain cases, this may be possible.
I assume HTTP/1.1 here. HTTP/2 probably gives even more opportunities, because there’s more to screw up in the protocol, and the implementations are often stricter. Conversely, HTTP/1.0 is dumber and laxer, so harder to break.
Close the connection before the end of response, as indicated by your framing. If your response is framed with Content-Length: 100, close before you’ve sent the 100th byte of payload. If your response is framed with Transfer-Encoding: chunked, close before you’ve sent the final empty chunk. If the client expects to receive the entire payload, it may (and should) treat this as an error. But some won’t, including very popular client libraries.
If the payload is in a structured format, like JSON or XML, then do the same as 1, but before closing, send something that would disrupt that format. For example, no valid JSON text can end with {. Even if the client doesn’t recognize the incomplete payload as an error, it might then fail on trying to parse it.
Same as 1, but instead of closing the connection, just stop sending data. The client will “hang” until its receive operation times out, which it may treat as an error. This may be a bad idea if the client is operated by someone who is not prepared for such extravagant timeouts.
Only with Transfer-Encoding: chunked: Same as 3, but instead of hanging, send bogus very long chunks and/or keep sending chunks indefinitely, until the client gives up or crashes. Probably a very bad idea, bordering on malicious.
I am qurious if there is any standard method in HTTP 1.X protocol to tell there is a problem on the server during http response that started as 200 OK.
How to tell there's any error on the server if 200 OK header is already returned and we are currently sending the response body? In some standards-compilliant way.
UPD : There is a duplicate, but without a single answer (!) HTTP: error during reply after 200 OK status code.
To be specific: I can not use Content-Length for checking at response end, because the length can't be known at response start.
Additionaly, I can't cache the whole response on the server before sending (because it is too big and I will run out of memory, and it's too long to generate so the user can't wait, etc...).
There is no standard method to do what you want.
To be precise, the standard method is to buffer the response on the server, then send a 200 OK and the Content-Length, followed by the content. As stated, this does not work for you.
The only alternative I can think of, is to wrap the content in some format that makes it discoverable whether it was sent correctly. For example, you might end it with a hash or even a digital signature. But obviously, such mechanisms are not part of the HTTP standard.
All,
My requirement is fairly simple. I have to perform a simple HTTP POST to an IP:port combination. I used simple socket programming to do that and I have been successful in sending across my request to them and also get back response from them. The only problem being that the response is always a HTTP 400: Bad Request followed by my HTTP POST message. I am not sure if the problem is with the client or the server. My only guess being that there might be a problem with my data that I am sending. This is what my POST looks like
POST /<Server Tag> HTTP/5.1
Content-Length: xxx
--Content--
and the response from the server looks something like this
HTTP/1.1 400 Bad Request
Content-Length: xxx
--Same content that I sent them--
I was not sure If I could put in the IP of the server here so kept myself to using . I am pretty sure that the problem would not be there since I get back some response from the server and confident about the connection. Can someone help me ?
PS: Some pointers about my POST:
1) HTTP 5.1 was requested by the server and I am not sure if that is correct
2) I have played around with the number of line spaces after the content length. I have tried giving one and two lines. Not sure if that would make a difference. On wireshark though I see a difference with the number of line spaces as with a single line space the protocol is specified as TCP but with two it changes to HTTP. The response is always received on HTTP protocol. Some explanation on the difference would also help
Thanks
edit: the other thing that confuses me is that the response has a HTTP 1.1 and not a 5.1 that I had sent. I have also tried changing my post to 1.1 with no success
edit2: Based on suggestion form fvu and others, I used WebClient to Upload my request. Still got back a 400. The header that was generated by the WebClient looks like this
POST <server tag> HTTP/1.1
Host: <IP:PORT>
Content-Length: 484
Expect: 100-continue
Connection: Keep-Alive
The issue I see with this might be that the server was not expecting all the details in the header. The server has requested only the Content-Length from us. Would that be a problem?
Thanks
You can use a debugging proxy to view a client request and a server response to figure out what your client socket program needs to do.
But first you need to create a simple web page that a browser displays, allows you to do a POST from the browser to the web server, and get a simple response back from the server.
HTTP/5.1 is either wrong or misused by the programmer of the server application
You should get a valid example from the server api to check your protocol implementation first.
I am sending large amount of data in my response to the client in chunked transfer encoding format.
How should I be dealing with any errors that occur midway during writing of the response?
I would like to know if there is any HTTP Spec recommended practice regarding this for clients to know that indeed the response is not a successful one but that the server ran into some issue.
Once you have started sending the HTTP headers to the client, you can't send anything else. You have to finish sending the response you intended to send, ie the chunked data and associated headers. If an error occurs midway through that, there is no way to report that error to the client. All you can do is close the connection. Either the client does not receive all of the headers, or it does not receive the terminating 0-length chunk at the end of the response. Either way is enough for the client to know that the server encountered an error during sending.
Chunked responses can use trailer header with some integrity check or post-processing status. This may be better than closing the connection as described in accepted answer because you can have custom error message. However, you only benefit from that if you have control of both server and client.
I have an HTTP server that returns large bodies in response to POST requests (it is a SOAP server). These bodies are "streamed" via chunking. If I encounter an error midway through streaming the response how can I report that error to the client and still keep the connection open? The implementation uses a proprietary HTTP/SOAP stack so I am interested in answers at the HTTP protocol level.
Once the server has sent the status line (the very first line of the response) to the client, you can't change the status code of the response anymore. Many servers delay sending the response by buffering it internally until the buffer is full. While the buffer is filling up, you can still change your mind about the response.
If your client has access to the response headers, you could use the fact that chunked encoding allows the server to add a trailer with headers after the chunked-encoded body. So, your server, having encountered the error, could gracefully stop sending the body, and then send a trailer that sets some header to some value. Your client would then interpret the presence of this header as a sign that an error happened.
Also keep in mind that chunked responses can contain "footers" which are just like HTTP headers. After failing, you can send a footer such as:
X-RealStatus: 500 Some bad stuff happened
Or if you succeed:
X-RealStatus: 200 OK
you can change the status code as long as response.iscommitted() returns false.
(fot HttpServletResponse in java, im sure there exists an equivalent in other languages)