If not, how accurate is it?
I want to know the size of the image before I download it.
Can the HTTP Content-length header be malformed? Yes.
Should you trust it to be a fair representation of the size of the message body? Yes.
It should be, and usually is, accurate. However it is entirely possible for a web server to report a incorrect content length although this obviously doesn't happen often (I recall old versions of apache retuning nonsensical content lengths on files > 2GB).
It is also not mandatory to provide a Content-Length header
It had better be - otherwise why have it at all?
If it can't be reliably determined in advanced, it shouldn't be served by the server at all. (When dealing with dynamically generated text, for example, something like chunked transfer encoding may be used - which doesn't require the final length to be known when the HTTP header is written at the beginning of the stream.)
Content-Length can be sent by the server code (or) by the apache layer itself.
When the code is not sending apache will send it.
There are known client-crashes when the client connects and closes the socket when the
content-length is sent smaller.
Since the images are usually not generated by code in run-time, you can rely on it.
Browsers can be unforgiving if the content-length is incorrect.
I was having a problem here, where the server was sometimes returning a content-length that was too low. The browsers just wouldn't handle it.
So yes, you can assume that the server is setting the content-length correctly, based on the knowledge that browser clients work on the same assumption.
A sharing for what I've discovered recently.
My case was using NodeJS to post http request to another server, which I'm able to set the content-length value by my self into the header (the value is lower than the actual size).
When the other server received my http request, it only process the request param/body size up to the point that the content-length told the server.
E.g. my request actual length is 100, but content-length mentioned 80, the server who received the request will only process 1-80 and the last 20 wasn't being processed and caused some error like "Invalid parameter".
Nowadays sever side will automatically insert the content-length for you, unless there are needs and you know what you are doing, else you don't want to change it as it may cause you trouble.
Related
Can max-stale be set in the response header for it to be used by the request on the client side? The documentation here - Cache-Control: Syntax, specifically the "Cache response directives" section makes it appear as if max-stale is not part of the response header. Is this only used by client to make decision how much longer it will use the stale resource and server/application has no say in it? If so, what can be set on the response to simulate the functionality of max-stale?
No max-stale cannot be used in the response. It’s meant to be used by the client to override the cache defaults. “Gimme this resource even if it’s technically a bit past it’s expiry date”. It would be used if there is a caching server between the client and the origin server.
To be honest, in my experience, request cache-control headers are rarely used, except to force refresh all the way back to origin server (max-age=0) for example when doing a “hard reload” with dev tools open. I’ve never seen a real world instance of max-stale as far as I can recall.
There is not equivalent on a response header. If a server is happy for a resource to be used for longer then it should just increase the max-age amount.
There is the stale-while-revalidate response option which allows a stale resource to be used for a limited period, to allow a quick reload of the page, while the browser checks and downloads a new version in the background for the next time. However support of it is limited at this time as shown at the bottom of that page you linked.
Simulating the behaviour of max-stale on the response does not really make much sense. Server owns the resources and has an understanding of how those resources change over time. Server has to decide how critical a resource is and if it is ok for it be served stale. Also decide on some reasonable time limits for the resource to be re-validated such that the clients get fresh data most of the time. It is a balancing act on the server side. Too strict the setting, you are overloading your server with requests and client with network traffic. Too loose, your clients see an old representation.
A client can use max-stale to avoid any re-validation and get what is in the cache. you don't want to generate network requests unless it is really necessary. for example must-revalidate overrides max-stale so if the response has that header, even with max-stale you will hit the origin server no matter. Similarly with no-cache and no-store. So in that sense, server has a say in it. It can identify resources that CANNOT be used stale, even with max-stale and marks with the appropriate header.
I understand that one can set Accept-Ranges: none on the server to advise the client not to attempt a range request.
I am wondering if there is a way to tell a browser not to attempt a range request without having to make any changes on the server.
For instance, is there a setting in Chrome or Firefox that I can toggle to deter my browser from making range requests?
You answered the question in the first sentence.
The relevant RFC is 7233, Hypertext Transfer Protocol (HTTP/1.1): Range Requests:
2.3. Accept-Ranges
A client MAY generate
range requests without having received this header field for the
resource involved.
A server that does not support any kind of range request for the
target resource MAY send
Accept-Ranges: none
to advise the client not to attempt a range request.
If you mean you want to know how to disable range requests in a browser altogether, consult the specific browser's documentation. A quick web search yielded no options for me to do this for common browsers.
The HTTP/1.0 specs state that the server can indicate the length of a response containing an entity body by
sending a Content-Length header, or
closing the connection when the entire response has been sent.
So the Content-Length header, for normal responses like an image, Html page or Ajax data is optional.
As that header has been provided there are cases in which using it give some benefits.
So the question is: what are those cases and what are the benefits?
(For example: if the response is a file that is being downloaded the browser displays a progress bar if Content-Length is specified.)
When the content length is predictable, communicating it to the client helps, because then the required memory and disk space can be allocated at once, allowing for a more efficient transfer, and also an early abortion if the required resources are not available to the client.
This optimizes the transfer, which benefits both the server and the client.
When the content length cannot be predicted (e.g. because it is dynamically generated) these optimizations are obviously not possible.
Is it possible to set the content-length header and also use chunked transfer encoding? and does doing so solve the problem of not knowing the length of the response at the client side when using chunked?
the scenario I'm thinking about is when you have a large file to transfer and there's no problem in determining its size, but it's too large to be buffered completely.
(If you're not using chunked, then the whole response must get buffered first? Right??)
thanks.
No:
"Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding. If the message does include a non-identity transfer-coding, the Content-Length MUST be ignored." (RFC 2616, Section 4.4)
And no, you can use Content-Length and stream; the protocol doesn't constrain how your implementation works.
Well, you can always send a header stating the size of the file.
Something like response.addHeader("File-Size","size of the file");
And ignore the Content-Length header.
The client implementation has to be tweaked to read this value, but hey you can achieve both the things you want :)
You have to use either Content-Length or chunking, but not both.
If you know the length in advance, you can use Content-Length instead of chunking even if you generate the content on the fly and never have it all at once in your buffer.
However, you should not do that if the data is really large because a proxy might not be able to handle it. For large data, chunking is safer.
This headers can be cause of Postman Parse Error:
"Content-Length" and "Transfer-Encoding" can't be present in the response headers together.
Using parametrized ResponseEntity<?> except raw ResponseEntity in controller can fixed the issue.
The question asks:
Is it possible to set the content-length header and also use chunked transfer encoding?
The RFC HTTP/1.1 spec, quoted in Julian's answer, says:
Messages MUST NOT include both a Content-Length header field and a non-identity transfer-coding.
There is an important difference between what's possible, and what's allowed by a protocol. It is certainly possible, for example, for you to write your own HTTP/1.1 client which sends malformed messages with both headers. You would be violating the HTTP/1.1 spec in doing so, and so you'd imagine some alarm bells would go off and a bunch of Internet police would burst into your house and say, "Stop, arrest that client!" But that doesn't happen, of course. Your request will get sent to wherever it's going.
OK, so you can send a malformed message. So what? Surely on the receiving end, the server will detect the HTTP/1.1 protocol client-side violation, vanquish your malformed request, and serve you back a stern 400 response telling you that you are due in court the following Monday for violating the protocol. But no, actually, that probably won't happen. Of course, it's beyond the scope of HTTP/1.1 to prescribe what happens to misbehaving clients; i.e. while the HTTP/1.1 protocol is analogous to the "law", there is nothing in HTTP/1.1 analogous to the judicial system.
The best that the HTTP/1.1 protocol can do is dictate how a server must act/respond in the case of receiving such a malformed request. However, it's quite lenient in this case. In particular, the server does not have to reject such malformed requests. In fact, in such a scenario, the rule is:
If the message does include a non-identity transfer-coding, the Content-Length MUST be ignored.
Unfortunately, though, some HTTP servers will violate that part of the HTTP/1.1 protocol and will actually give precedence to the Content-Length header, if both headers are present. This can cause a serious problem, if the message visits two servers in sequence in the same system and they disagree about where one HTTP message ends and the next one starts. It leaves the system vulnerable to HTTP Desync attacks a.k.a. Request Smuggling.
Can I simply set the Transfer-Encoding header?
Will calling Response.Flush() at some point cause this to occur implicitly?
EDIT
No, I Cannot call Response.Headers.Add("Transfer-Encoding","anything"); That throws.
any other suggestions?
Related:
Enable Chunked Transfer Encoding in ASP.NET
TL;DR: Specifying the content-length is the best way to achieve a fast first byte; you'll allow chunking at TCP rather than HTTP level. If you don't know the content-length, setting context.Response.BufferOutput to false will send output as it's written the the output stream using chunked transfer-encoding.
Why do you want to set Transfer-Encoding: chunked? Chunked transfers are essentially a work-around to permit sending documents whose content-length is not known in advance. ASP.NET, however, by default buffers the entire output and hence does know the overall content length.
Of course, HTTP is layered over TCP, and behind the scene TCP is "chunking" anyhow by splitting even a monolithic HTTP response into packets - meaning that if you specify the content-length up front and disable output buffering, you'll get the best latency without requiring HTTP-level chunking. Thus, you don't need HTTP-level chunking to provide a fast first byte when you know the content-length.
Although I'm not an expert on HTTP, I have implemented a simple streaming media server with seeking support, dynamic compression, caching etc. and I do have a reasonable grasp of the relevance of a fast first byte - and chunking is generally an inferior option if you know the content-length - which is almost certainly why ASP.NET won't let you set it manually - it's just not necessary.
However, if you don't know the HTTP content length before transmission and buffering is too expensive, you turn off output buffering and presumably the server will use a chunked transfer encoding by necessity.
When does the server use chunked transfer encoding? I just tested, and indeed if context.Response.BufferOutput is set to false, and when the content length is not set, the response is chunked; such a response is 1-2% larger in my entirely non-scientific quick test of a 1.7MB content-encoding: gzip xml document. Since gzip relies on context to reduce redundancy, I'd expected the compression ratio to suffer more, but it seems that chunking doesn't necessarily greatly reduce compression ratios.
If you look at the framework code in reflector, it seems that the transfer encoding is indeed set automatically as needed - i.e. if buffering is off AND no content length is known AND the response is to an HTTP/1.1 request, chunked transfer encoding is used. However, if the server is IIS7 and this is a worker request (?integrated mode?), the code branches to a native method - probably with the same behavior, but I can't verify that.
It looks like you need to setup IIS for this. IIS 6 has a property AspEnableChunkedEncoding in the metabase and you can see the IIS 7 mappings for this on MSDN at http://msdn.microsoft.com/en-us/library/aa965021(VS.90).aspx.
This will enable you to set TRANSFER-ENCODING: chunked in your header. I hope this helps.
Although you set Buffer to false and leave empty the content length, you need to make sure that you have disabled "Dynamic Content Compressing" feature for IIS7 to make chunked response working. Also, client browser should have at least HTTP 1.1 .. Chunked mode won't be working for HTTP 1.0
Response.Buffer = False
This will set HTTP Header "Tranfer-Encoding:Chuncked" and send the response each callled response.write