Is it possible for a HTTP 1.1 client to set a header value that indicates the responses to requests should not be chunked? Or is the only way to prevent this, is to send a HTTP 1.0 request? I've tried googling around, but all I can find is ways to disable chunked transfers on HTTP 1.1 servers, so I am guessing it is not possible on a client, but I thought I'd ask anyways.
In HTTP (starting with HTTP/1.1), recipients MUST support chunked encoding. See http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p1-messaging-26.html#rfc.section.4.1.p.4.
To get Content-Length in bytes instead of chunked inside the response with HTTP 1.1,
you have to set Content-Length header and its size (long or int) based on the file you are expecting inside the response. long will be good so it can take care small as well as big file size.
response will be HttpServletResponse.
response.addHeader(Content-Length, Long.toString());
Thanks,
Related
A web application under test behaves in an odd way. A HEAD request returns the header Content-Length, but the consequent GET returns Transfer-Encoding: chunked. I expected the headers to be equal, and RFC says SHOULD, so my question is: how legit and how common is this behaviour?
UPDATE It turns out, that the root cause of the problem is HAProxy's behaviour. If that's a HEAD request, the response is propagated as is from the application underneath. But for GET it applies the compression and sets the chunked transfer. I'll close this question as an off-topic and perhaps will ask at ServerFault.
If the server use chunked encoding for GET, but returns Content-Length for HEAD this is IMHO an indication that the information returned for HEAD is unlikely to be correct.
The HEAD method response does not return entity-body but GET responds with an entity-body, if the HTTP server has the "Chunked transfer encoding" enabled does not send the "Content-Length" in the response because is not used, the server does not need to know the length of the content before it starts transmitting a response to the client. The server can begin transmitting dynamically-generated content to the client before knowing the total size of that content. Perhaps this is the most likely explanation.
I am trying to write a simple proxy. I just want to know whether it is possible to have chunked Http GET requests?
The answer is no and yes simultaneously. GET requests don't have any content, so they obviously cannot use chunked transfer encoding (there is nothing to transfer). However the response to a GET request can contain a body that is encoded using chunked transfer encoding. So whenever there is a body, chunked transfer encoding may be used. The wikipedia page has more information and also links to the corresponding RFC.
Hi i am writing a custom http server. At first i had only one request at one connection possible(Connection: close), everything was ok. But once i remade it connecton: keep-alive logic(more requests at one connection), my images stopped displaying. I think, it may be a problem with http response delimiters. Are there any? Or how can browser detect, that current http response is complete? thx
The size of the response is guided by the Content-Length header or by using Chunked Transfer Encoding.
Is it possible to set the content-length header and also use chunked transfer encoding? and does doing so solve the problem of not knowing the length of the response at the client side when using chunked?
the scenario I'm thinking about is when you have a large file to transfer and there's no problem in determining its size, but it's too large to be buffered completely.
(If you're not using chunked, then the whole response must get buffered first? Right??)
thanks.
No:
"Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non-identity transfer-coding, the Content-Length MUST be ignored." (RFC 2616, Section 4.4)
And no, you can use Content-Length and stream; the protocol doesn't constrain how your implementation works.
Well, you can always send a header stating the size of the file.
Something like response.addHeader("File-Size","size of the file");
And ignore the Content-Length header.
The client implementation has to be tweaked to read this value, but hey you can achieve both the things you want :)
You have to use either Content-Length or chunking, but not both.
If you know the length in advance, you can use Content-Length instead of chunking even if you generate the content on the fly and never have it all at once in your buffer.
However, you should not do that if the data is really large because a proxy might not be able to handle it. For large data, chunking is safer.
This headers can be cause of Postman Parse Error:
"Content-Length" and "Transfer-Encoding" can't be present in the response headers together.
Using parametrized ResponseEntity<?> except raw ResponseEntity in controller can fixed the issue.
The question asks:
Is it possible to set the content-length header and also use chunked transfer encoding?
The RFC HTTP/1.1 spec, quoted in Julian's answer, says:
Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding.
There is an important difference between what's possible, and what's allowed by a protocol. It is certainly possible, for example, for you to write your own HTTP/1.1 client which sends malformed messages with both headers. You would be violating the HTTP/1.1 spec in doing so, and so you'd imagine some alarm bells would go off and a bunch of Internet police would burst into your house and say, "Stop, arrest that client!" But that doesn't happen, of course. Your request will get sent to wherever it's going.
OK, so you can send a malformed message. So what? Surely on the receiving end, the server will detect the HTTP/1.1 protocol client-side violation, vanquish your malformed request, and serve you back a stern 400 response telling you that you are due in court the following Monday for violating the protocol. But no, actually, that probably won't happen. Of course, it's beyond the scope of HTTP/1.1 to prescribe what happens to misbehaving clients; i.e. while the HTTP/1.1 protocol is analogous to the "law", there is nothing in HTTP/1.1 analogous to the judicial system.
The best that the HTTP/1.1 protocol can do is dictate how a server must act/respond in the case of receiving such a malformed request. However, it's quite lenient in this case. In particular, the server does not have to reject such malformed requests. In fact, in such a scenario, the rule is:
If the message does include a non-identity transfer-coding, the Content-Length MUST be ignored.
Unfortunately, though, some HTTP servers will violate that part of the HTTP/1.1 protocol and will actually give precedence to the Content-Length header, if both headers are present. This can cause a serious problem, if the message visits two servers in sequence in the same system and they disagree about where one HTTP message ends and the next one starts. It leaves the system vulnerable to HTTP Desync attacks a.k.a. Request Smuggling.
Can I simply set the Transfer-Encoding header?
Will calling Response.Flush() at some point cause this to occur implicitly?
EDIT
No, I Cannot call Response.Headers.Add("Transfer-Encoding","anything"); That throws.
any other suggestions?
Related:
Enable Chunked Transfer Encoding in ASP.NET
TL;DR: Specifying the content-length is the best way to achieve a fast first byte; you'll allow chunking at TCP rather than HTTP level. If you don't know the content-length, setting context.Response.BufferOutput to false will send output as it's written the the output stream using chunked transfer-encoding.
Why do you want to set Transfer-Encoding: chunked? Chunked transfers are essentially a work-around to permit sending documents whose content-length is not known in advance. ASP.NET, however, by default buffers the entire output and hence does know the overall content length.
Of course, HTTP is layered over TCP, and behind the scene TCP is "chunking" anyhow by splitting even a monolithic HTTP response into packets - meaning that if you specify the content-length up front and disable output buffering, you'll get the best latency without requiring HTTP-level chunking. Thus, you don't need HTTP-level chunking to provide a fast first byte when you know the content-length.
Although I'm not an expert on HTTP, I have implemented a simple streaming media server with seeking support, dynamic compression, caching etc. and I do have a reasonable grasp of the relevance of a fast first byte - and chunking is generally an inferior option if you know the content-length - which is almost certainly why ASP.NET won't let you set it manually - it's just not necessary.
However, if you don't know the HTTP content length before transmission and buffering is too expensive, you turn off output buffering and presumably the server will use a chunked transfer encoding by necessity.
When does the server use chunked transfer encoding? I just tested, and indeed if context.Response.BufferOutput is set to false, and when the content length is not set, the response is chunked; such a response is 1-2% larger in my entirely non-scientific quick test of a 1.7MB content-encoding: gzip xml document. Since gzip relies on context to reduce redundancy, I'd expected the compression ratio to suffer more, but it seems that chunking doesn't necessarily greatly reduce compression ratios.
If you look at the framework code in reflector, it seems that the transfer encoding is indeed set automatically as needed - i.e. if buffering is off AND no content length is known AND the response is to an HTTP/1.1 request, chunked transfer encoding is used. However, if the server is IIS7 and this is a worker request (?integrated mode?), the code branches to a native method - probably with the same behavior, but I can't verify that.
It looks like you need to setup IIS for this. IIS 6 has a property AspEnableChunkedEncoding in the metabase and you can see the IIS 7 mappings for this on MSDN at http://msdn.microsoft.com/en-us/library/aa965021(VS.90).aspx.
This will enable you to set TRANSFER-ENCODING: chunked in your header. I hope this helps.
Although you set Buffer to false and leave empty the content length, you need to make sure that you have disabled "Dynamic Content Compressing" feature for IIS7 to make chunked response working. Also, client browser should have at least HTTP 1.1 .. Chunked mode won't be working for HTTP 1.0
Response.Buffer = False
This will set HTTP Header "Tranfer-Encoding:Chuncked" and send the response each callled response.write