Consider a POST request, sending a big file. At some point - either after reading Content-Length or during reading the body, if chunked transfer - the server decides that the body length is too big.
According to http, the server should reply with 413 Payload Too Large. But in order to reply, the server has to read the whole POST request first, at least AFAIK it would break browser else and show cryptic error messages.
So where is the point then in 413 Payload Too Large if I anyway need to read the whole request in order to reply?
I am aware of the "option" to read the POST request to the end while discarding everything and then replying 413 Payload Too Large, but yeah that is kinda not an option keeping server busy for nothing...
So closing the connection without replying is basically the best I can do right?
Related
Say I started writing to the response body, but there was some error, and I need to indicate that it's an HTTP 500 even if an HTTP 200 OK header was already written as a header...
How can I write something to the body of the response that's guaranteed to be malformed so that the response is interpreted as some sort of error by the client?
In general, this is impossible. Some clients only care about the response header, and may stop paying attention to what you send after the header.
But with certain clients, in certain cases, this may be possible.
I assume HTTP/1.1 here. HTTP/2 probably gives even more opportunities, because there’s more to screw up in the protocol, and the implementations are often stricter. Conversely, HTTP/1.0 is dumber and laxer, so harder to break.
Close the connection before the end of response, as indicated by your framing. If your response is framed with Content-Length: 100, close before you’ve sent the 100th byte of payload. If your response is framed with Transfer-Encoding: chunked, close before you’ve sent the final empty chunk. If the client expects to receive the entire payload, it may (and should) treat this as an error. But some won’t, including very popular client libraries.
If the payload is in a structured format, like JSON or XML, then do the same as 1, but before closing, send something that would disrupt that format. For example, no valid JSON text can end with {. Even if the client doesn’t recognize the incomplete payload as an error, it might then fail on trying to parse it.
Same as 1, but instead of closing the connection, just stop sending data. The client will “hang” until its receive operation times out, which it may treat as an error. This may be a bad idea if the client is operated by someone who is not prepared for such extravagant timeouts.
Only with Transfer-Encoding: chunked: Same as 3, but instead of hanging, send bogus very long chunks and/or keep sending chunks indefinitely, until the client gives up or crashes. Probably a very bad idea, bordering on malicious.
I am qurious if there is any standard method in HTTP 1.X protocol to tell there is a problem on the server during http response that started as 200 OK.
How to tell there's any error on the server if 200 OK header is already returned and we are currently sending the response body? In some standards-compilliant way.
UPD : There is a duplicate, but without a single answer (!) HTTP: error during reply after 200 OK status code.
To be specific: I can not use Content-Length for checking at response end, because the length can't be known at response start.
Additionaly, I can't cache the whole response on the server before sending (because it is too big and I will run out of memory, and it's too long to generate so the user can't wait, etc...).
There is no standard method to do what you want.
To be precise, the standard method is to buffer the response on the server, then send a 200 OK and the Content-Length, followed by the content. As stated, this does not work for you.
The only alternative I can think of, is to wrap the content in some format that makes it discoverable whether it was sent correctly. For example, you might end it with a hash or even a digital signature. But obviously, such mechanisms are not part of the HTTP standard.
As an HTTP 1.1 server, I reply to a GET request with 200 OK status code, then start sending the data to the client.
During this send, an error occurs, and I cannot finish.
I cannot send a new status code as a final status code has already been sent.
How should I behave to let the client know an error occurred and I cannot continue with this HTTP request?
I can think of only one solution: close the socket, but it's not perfect: it breaks the keep-alive feature, and no clear explanation of the error is given to the client.
The HTTP standard seems to suppose that the server already knows exactly what to reply before it starts replying.
But this is not always the case.
Examples:
I return a very large file (several GB) from disk, and I get an IO error at some point during the reading of the file.
Same example with a large DB dump.
I cannot construct my whole response in memory then send it.
The HTTP 1.1 standard helps for such usage with the chunked transfer encoding: I don't even need to know the final size before starting to send the reply.
So these usage are not excluded from HTTP 1.1.
I finally found a possible solution for this:
HTTP 1.1 Trailer headers.
In chunked encoded bodies, HTTP 1.1 allows the sender to add data after the last (empty) chunk, in the form of a block of headers.
The specification hints some use-cases like computing on the fly a md5 of the body, and send it after the body so the client can check its integrity.
I think it could be used for error reporting, even if I haven't found anything about this kind of usage.
The issues I see with this are:
this requires using chunked encoding (but it's not much of an issue)
trailers support is probably very low:
server-side (it could be bypassed by manually creating the chunked encoding, but since it's applied after the content-encoding (gzip), it would require a lot of reimplementation)
client-side (bugs fixed only in 2010 in curl for example)
and on proxies (that could then loose the trailers if not properly implemented)
I have pushed the similar question to be answered, so here you can find that there is no solution:
How to tell there's something wrong with the server during response that started as 200 OK. Fail gracefully
I saw Expect: 100-continue in some PUT request(uploading a file), what does it mean?
What is supposed to happen, is you're supposed to send the request headers with an:
Expect: 100-continue
header. Then, after you have sent the headers, but before you send the payload, you should check to see if you get the 100 response, or a 417 response. If you get the 100 response, you can continue sending the payload. If you don't, you should stop.
The premise is that when you're getting ready to send that 10GB file, this gives the server an opportunity to say "Hold on, cowboy" and then you can handle the process more elegantly than the server simply slamming the socket shut on you.
The fact that you got a 100 back, and you weren't expecting it, says you probably got both the 100 and the 200 (or whatever) response. The 100 was sent to you after the headers were sent, then the final response when the request was finished.
That you weren't paying attention to it is really a detail.
But, ideally, in the future your processing can consider the proper mid-request response.
If you did NOT send the Expect header, the server should not have sent you the 100, since you weren't telling it that you were going to process it. If you DID send the Expect header, then the 100 should not have come as a surprise.
I have an HTTP server that returns large bodies in response to POST requests (it is a SOAP server). These bodies are "streamed" via chunking. If I encounter an error midway through streaming the response how can I report that error to the client and still keep the connection open? The implementation uses a proprietary HTTP/SOAP stack so I am interested in answers at the HTTP protocol level.
Once the server has sent the status line (the very first line of the response) to the client, you can't change the status code of the response anymore. Many servers delay sending the response by buffering it internally until the buffer is full. While the buffer is filling up, you can still change your mind about the response.
If your client has access to the response headers, you could use the fact that chunked encoding allows the server to add a trailer with headers after the chunked-encoded body. So, your server, having encountered the error, could gracefully stop sending the body, and then send a trailer that sets some header to some value. Your client would then interpret the presence of this header as a sign that an error happened.
Also keep in mind that chunked responses can contain "footers" which are just like HTTP headers. After failing, you can send a footer such as:
X-RealStatus: 500 Some bad stuff happened
Or if you succeed:
X-RealStatus: 200 OK
you can change the status code as long as response.iscommitted() returns false.
(fot HttpServletResponse in java, im sure there exists an equivalent in other languages)