This question already has answers here:
Why is GZIP Compression of a Request Body during a POST method uncommon?
(3 answers)
Compressing HTTP Post Data sent from browser
(3 answers)
Closed last month.
When I send a request to HTTP server I can specify header
Accept-Encoding: br, gzip, deflate, and then server can compress the response body.
Is there a way to compress sent request the same way using curl or any other http client?
Related
I'm writing a simple HTTP server that will serve content from the file system.
I'm a little confused as to how the client and server negotiate content type.
After doing some research, I found that Content-Type specifies the content type of the HTTP message being sent, while the Accept header specifies what the program expects to receive as a response.
When I visit my server from my browser, and read the initial GET request (when visited with a null URI), I get the following:
GET / HTTP/1.1
Host: 127.0.0.1:1234
User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:50.0) Gecko/20100101 Firefox/50.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Connection: keep-alive
Upgrade-Insecure-Requests: 1
As you can see, the accept header doesn't specify it will accept pdfs, judging by the fact that I can't see the MIME type application/pdf in the accept header value.
Yet, when I send a pdf's bytes along with a content type set to application/pdf, the browser magically displays it.
So, what am I missing? I originally thought the browser might be doing some basic inference on the URI to see if it ends it .pdf, and then accept the corresponding MIME type.
But, when I visit it with a link to a pdf, the Accept header stays the same.
Any help would be really appreciated.
I'm writing a simple HTTP server
Then you should learn to find your way around the various RFCs that describe HTTP.
The relevant one here is RFC 7231, 5.3.2. Accept:
If the header field is
present in a request and none of the available representations for
the response have a media type that is listed as acceptable, the
origin server can either honor the header field by sending a 406 (Not
Acceptable) response or disregard the header field by treating the
response as if it is not subject to content negotiation.
A browser in principle wants to display HTML-formatted documents, for whatever variant of (X)HTML the server is willing to serve, so by default it sends the accept header you observed.
If the request is for another kind of resource however, the server is free to respond with that type of content.
What encoding does browser use when sending HTTP requests?
I mean when browser sends the very first request how can it be sure that the encoding it uses will be understood by the server?
Example:
GET /hello.htm HTTP/1.1
User-Agent: Mozilla/4.0 (compatible; MSIE5.01; Windows NT)
Host: www.tutorialspoint.com
Accept-Language: en-us
Accept-Encoding: gzip, deflate
Connection: Keep-Alive
A browser can tell the server explicitly which encoding is used thanks to Content-type header. Content-type might contains charset, but it's possible to infer the encoding by type. For example, application/json:
Content-type: application/json; charset=utf-8 designates the content
to be in JSON format, encoded in the UTF-8 character encoding.
Designating the encoding is somewhat redundant for JSON, since the
default (only?) encoding for JSON is UTF-8. So in this case the
receiving server apparently is happy knowing that it's dealing with
JSON and assumes that the encoding is UTF-8 by default, that's why it
works with or without the header.
What about the situation that Content-type is not defined in request?
A sender that generates a message containing a payload body SHOULD
generate a Content-Type header field in that message unless the
intended media type of the enclosed representation is unknown to the
sender. If a Content-Type header field is not present, the recipient
MAY either assume a media type of "application/octet-stream"
([RFC2046], Section 4.5.1) or examine the data to determine its type.
When the field Connection: Keep-Alive is set inside a HTTP request, the connection is not closed after the response is sent.
How does the client and the server knows that the data is over?
The only way I believe this works is by setting the field Content-Length at the HTTP request and the response.
Is there any other way?
Let's say a client makes a request like the following (pulled from iOS):
GET /test.mp4 HTTP/1.1
Host: example.com:80
Range: bytes=0-1
X-Playback-Session-Id: 3DFA3BE3-CB22-4EC5-808F-B59A735DCECE
Accept-Encoding: identity
Accept: */*
Accept-Language: en-us
Connection: keep-alive
User-Agent: AppleCoreMedia/1.0.0.11B554a (iPad; U; CPU OS 7_0_4 like Mac OS X; en_us)
There other such requests out there, I believe Chrome might test the waters by asking for blank Range.
How can the server respond to any such request so that it does not need to honor Range , but rather treat it as a standard HTTP delivery, and the client will play the file?
Sending a regular header response and the data as though the client were not asking for Range does not seem to work.
EDIT: Conversely, if the client does not request a Range, is it okay to respond with HTTP 206 with full filesize in Content-Length and also Content-Range header (which client will ignore)?
If the server does not support the Range header, it would send a normal 200 reply to send the entire file. If the server supports the Range header, it would send a 206 or 416 reply, depending on whether the requested range can be satisfied or not. This is covered in RFC 2616 Section 14.35.
It is not OK to respond with 206 if the client did not request a Range.
Try responding with HTTP 1.0 - it doesn't support range requests at all.
Maybe the client will treat such a reply more gracefully.
I'm writing a HTTP client which doesn't have any encoding algorithms built into it yet. Therefore, I was wondering if there is a value for Accept-Encoding header to indicate this? like: "none" for example, or "text/plain" or similar ?
You can just omit the Accept-Encoding header if you don't support compression:
http://www.httpwatch.com/httpgallery/compression/
However, this will not prevent some servers sending you chunked encoded responses:
http://www.httpwatch.com/httpgallery/chunked/
on persistent "keep-alive" HTTP connections. You can disable persistent connections by adding a "Connection: close" request header and the server will simply close the connection after all the content has been returned in the response message.