I am using a 3G UMTS connection. I am trying to implement HTTP tunneling to a server of mine
which listens on port 80 (this is done in order to bypass client's firewall). The problem is that the ISP's proxy server supports HTTP/1.0 which doesn't support persistent HTTP connection.
As a result, after one http request/response iteration between my client/server the ISP's proxy tears down the underlying TCP connection.
my client receives the following HTTP response:
HTTP/1.0 200 OK
Content-Type: application/octet-stream
Content-Length: yyy
X-Cache: MISS from ipmr5
Proxy-Connection: close
Content data
while my server actually sends:
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Type: application/octet-stream
Content-Length: yyy
Content data
Is there any workaround?
You could always use HTTPS. You will lose any benefits offered by the proxies (such as caching), but all of your HTTP headers will arrive at the server exactly as you sent them.
HTTP 1.0 proxies (which it seems your ISP uses) shouldn't be used in connection with Connection: Keep-Alive for persistent connections. The reasons for this are outlined in RFC-2068 (section 19.7.1). The short version, basically, is that your server is sending an invalid header for the kind of proxy you are using.
Related
I was inspecting some of the HTTP exchanges between my browser and Google and it triggered this question.
In short, my browser (Firefox 36.0.4) is making HTTP/1.1 requests and Google is responding with HTTP/2.0; there is no attempt to respond in the requested protocol. I am aware that much of the HTTP/2.0 spec has already been implemented in a haphazard way through SPDY, but this seems like a poor neogitation with the client.
I thought that the purpose of declaring protocols in the header was that a server would be able to determine how it should respond to the client, which is in one of three ways:
1. the client has requested the server's preferred protocol, so the server continues with the request as normal
2. The client has requested another protocol version that the server supports, the server responds in the request protocol but includes an upgrade header indicating its preferred protocol. The client MAY request an upgrade at which point the server will send a 101 Switching Protocols response and switch to the preferred protocol.
3. The client has requested an unsupported or outdated protocol, the server sends a 426 Upgrade Required response with supported protocols (in descending order of preference) in the upgrade header; the client must repeat the request with a supported protocol.
4. The client reuested a major protocol version that is wholly unsupported; e.g. HTTP/2.x while the server only supports HTTP/1.x. The server responds with 505 HTTP Version Not Supported
The exchange with Google is not doing this; is this poor practice or am I missing something?
An example, selected at random:
https://plus.google.com/u/0/_/notifications/frame?querystring=blahblahblah
GET /u/0/_/notifications/frame?querystring=blahblahblah HTTP/1.1
Host: plus.google.com
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:36.0) Gecko/20100101 Firefox/36.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-GB,en;q=0.5
Accept-Encoding: gzip, deflate
Referer: https://www.google.co.uk/?gfe_rd=cr&ei=Lc8bVcXFOKbj8we_uIKYDg&gws_rd=ssl
Cookie: NID=67=iZxcMVTvg-6PsQIUpZ5tSPL-7-uJdls3vdci3afLmoLCpD5JOq0NfzhTnnpcCW9ymbXsn3GRGxfSgYlXGEk9XmnbUne0LCPrUc_ahhpc5wV6n-GZ8F7s-JS-JWgZWEwri-GaWXK1vgyRw7jMbqEiAUSRCzs1Fr1K6ZUIH0EpJdlwZD-K26MJNazpyHL_vZ5k4m8NrtFDkAoYPw; OTZ=2759671_52_56_123900_52_436380; SID=DQAAAP0AAAAqKgGz5aFNESd464Z_jUsmTi7JQfEKsuWkGZVJe8QvdbOPTZpL5ZNjKSsSSg9QvJglP-aMNLrgn2b7MsDF_4Z7Ebe1X347Cd3-j3ktLedgmq9nRO92hxEseqf974VNumrst-XqMj9Oq_xf-KDz-CDEJ1XiqWZYVHurV-IrXib5ei7x9dqlLF2NSPYLaCxlrwKdjCQX-FDDB03FWEuE7dIMYs3BQ-_NU5fG9os6I6r6ABy9mkiy84rraZFVthd38VJF5z2WYmgQ55QJPr9EDpSA5VKH1tbW6XyLjZLt5EEEj1xoqRF4EguRkIOiG8IiqRs49GnwqQSCpTw3ROW-jNDI; HSID=A7u8vyQI-v7jJSEbS; SSID=AOojY4hDLYgnSjUrK; APISID=z23KH1a0VsBukvMu/ARaOeOni08HfbGg6R; SAPISID=5iTgyxKDRPP7fNtF/AdiFbKNYN04h7n6cu; PREF=ID=cc54787f58f50d42:U=8e10581450dbe3b5:FF=0:LD=en:TM=1416091562:LM=1418086819:GM=1:S=0KVfl2hqkG8Psvwv; OGP=-5061451:-5061492:; OGPC=4061155-1:
Connection: keep-alive
HTTP/2.0 200 OK
Alternate-Protocol: 443:quic,p=0.5
Cache-Control: private, max-age=0
content-security-policy-report-only: script-src 'unsafe-inline' 'unsafe-eval' 'self' https://*.googleapis.com https://*.gstatic.com https://apis.google.com https://www.google-analytics.com https://www.googletagmanager.com https://*.talkgadget.google.com https://pagead2.googleadservices.com https://pagead2.googlesyndication.com https://tpc.googlesyndication.com https://s.ytimg.com https://www.youtube.com https://clients1.google.com https://www.google.com;report-uri /_/cspreport/es_oz_20150330.18_p0
Content-Type: text/html; charset=utf-8
Date: Wed, 01 Apr 2015 10:57:55 GMT
Expires: Wed, 01 Apr 2015 10:57:55 GMT
Server: GSE
x-content-type-options: nosniff
x-ua-compatible: IE=edge, chrome=1
X-XSS-Protection: 1; mode=block
X-Firefox-Spdy: h2-15
This is a https request. The client announced the support for HTTP/2.0 with the ALPN (formerly NPN) extension in the SSL handshake. Therefore the server knows that the client can do HTTP/2.0. If this extension is not given the server is not allowed to reply with a higher major HTTP version compared to the client request.
The HTTP version in the response is an advertisement of the capabilities of the server, not the actual protocol version of the response.
The protocol version of the response is the one that has been sent with the request.
In the past (but perhaps even nowadays) it was common for an old client to send a HTTP/1.0 request, and have the server respond in this way:
GET / HTTP/1.0
User-Agent: Netscape/1.0
HTTP/1.1 200 OK
Content-Length: 0
<connection closed>
The server advertised that it was able to speak HTTP/1.1, but behaved as HTTP/1.0 in the response (by closing the connection).
The same is happening in your case: you make a HTTP/1.1 request, the server advertises that it can speak HTTP/2.0 and responds with the HTTP/1.1 response format.
A smart client receiving that response could start speaking HTTP/2.0 to that server.
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
HTTP/1.1 proxies MUST parse the Connection header field before a message is forwarded and, for each connection-token in this field, remove any header field(s) from the message with the same name as the connection-token.
Could somebody please give an example of a common scenario the above paragraph is referring to?
Does that have anything to do with Connection: close header?
A good example, in HTTP/1.1, is Upgrade, to indicate that a client wishes to move from HTTP/1.1 to another protocol:
GET http://www.example.com/hello.txt HTTP/1.1
Connection: upgrade
Upgrade: HTTP/2.0, SHTTP/1.3, IRC/6.9, RTA/x11
If this were a proxy, the Upgrade header should not be passed to any upstream servers, as it only makes sense for this connection.
The Keep-Alive header could also appear here in HTTP/1.0 but is now obsolete with HTTP/1.1.
Request:
POST / HTTP/1.0
Content-Type: text/xml; charset=UTF-8
User-Agent: Axis2
Host: localhost:8000
Content-Length: 539
Response from tomcat:
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Content-Type: text/xml;charset=UTF-8
Date: Sat, 19 Oct 2013 00:28:57 GMT
Connection: close
From tomcat website it says:
If the client (typically a browser) supports only HTTP/1.0, the
Connector will gracefully fall back to supporting this protocol as
well. No special configuration is required to enable this support.
How Tomcat gracefully fall back to HTTP 1.0? From my example it still reply HTTP 1.1. Can anyone explain to me?
The protocol version indicates the protocol capability of the sender. It does not specify the version of the response itself. So as long as the response can be understood by the HTTP 1.0 client, Tomcat is doing exactly what it should.
It's all in RFC2616...
Edit: And it's even in the Tomcat documentation itself, right after the part you quoted:
This Connector supports all of the required features of the HTTP/1.1 protocol, as described in RFC 2616, including persistent connections, pipelining, expectations and chunked encoding. If the client (typically a browser) supports only HTTP/1.0, the Connector will gracefully fall back to supporting this protocol as well. No special configuration is required to enable this support. The Connector also supports HTTP/1.0 keep-alive.
RFC 2616 requires that HTTP servers always begin their responses with the highest HTTP version that they claim to support. Therefore, this Connector will always return HTTP/1.1 at the beginning of its responses.
format of a https get request
I was trying to implement a HTTPS get request.
This is my sample HTTP get request.
GET example.com HTTP/1.1
Date: Mon, 22 Feb 1857 12:27:07 GMT
Host: xyz.com:5901
User-Agent:
Content-Length: 0
I have used SSL library to encrypt the message, but is there anyway to differentiate a HTTP message and https message with the request?
Right now I have put a condition to call the SSL library, but I havent found any example where http request varies from https request.
Can anyone guide me here.
HTTPS is HTTP over SSL/TLS. Essentially, the SSL/TLS connection is initiated by the client. Once the SSL/TLS connection is established, it's just like normal HTTP. As the RFC says:
Conceptually, HTTP/TLS is very simple. Simply use HTTP over TLS
precisely as you would use HTTP over TCP.
Solved: pasting the bytes here made me realise that I was missing empty lines between chunks...
Does an HTTP/1.1 request need to specify a Connection: keep-alive header, or is it always keep-alive by default?
This guide made me think it would; that, when my http server gets a 1.1 request, it is keep-alive unless explicitly receiving a Connection: close header.
I ask since my the different client behaviour of ab and httperf is driving me mad enough to wonder my sanity on this one...
Here's what httperf --hog --port 42042 --print-reply body sends:
GET / HTTP/1.1
User-Agent: httperf/0.9.0
Host: localhost
And here's my server's response:
HTTP/1.1 200 OK
Connection: keep-alive
Transfer-Encoding: chunked
Content-Length: 18
12
Hello World 1
0
httpref promptly prints out the response, but then just sits there, neither side closing the connection and httpref not exiting.
Where's my bug?
From RFC 2616, section 8.1.2:
A significant difference between HTTP/1.1 and earlier versions of HTTP is that persistent connections are the default behavior of any HTTP connection. That is, unless otherwise indicated, the client SHOULD assume that the server will maintain a persistent connection, even after error responses from the server.