we know that we use tcp in http. In tcp connection Origin and destination are specify.according to this sentence why we specify host in Http header?
for example:
GET /index.html HTTP/1.1\r\n
Host: www-net.cs.umass.edu\r\n
User-Agent: Firefox/3.6.10\r\n
Accept: text/html,application/xhtml+xml\r\n
Accept-Language: en-us,en;q=0.5\r\n
Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n
Keep-Alive: 115\r\n
Connection: keep-alive\r\n
\r\n
The reason it is specified is to specify the name of the host. Since the request itself is made to the IP address in the IP header, the web server has no real way to know which domain was requested.
Under HTTP 1.0, this was not needed. After HTTP 1.0 was in existence for a number of years, the prevalence of virtual hosts become very common. Virtual hosting, in case you are unaware, allows you to host multiple sites on a single server, even the same IP, responding to each domain differently.
Since virtual hosting had become very common, the HTTP 1.1 standard made the Host header a requirement, simplifying the logic necessary to figure out which site the request should be routed to.
Related
I came across a HTTP HELP method (https://portswigger.net/research/cracking-the-lens-targeting-https-hidden-attack-surface chapter "Invalid Host") and asked myself:
Are there any more systems that offer something like that?
I was wondering how did the pentester come up with this method.
Google couldn't help me here.
In the specific case, it was about an Apache Traffic Server, whose help could be queried as follows:
HELP / HTTP / 1.1
Host: XX.X.XXX.XX: 8082
HTTP / 1.1 200 Connection Established
Date: Tue, 07 Feb 2017 16:33:59 GMT
Transfer encoding: chunked
Connection: keep-alive
OK
Traffic Server Overseer Port
commands:
get <variable-list>
set <variable-name> = "<value>"
help
exit
example:
OK
get proxy.node.cache.contents.bytes_free
proxy.node.cache.contents.bytes_free = "56616048"
OK
Variable lists are conf / yts / stats records, separated by commas
And then applied specifically as follows:
GET / HTTP / 1.1
Host: XX.X.XXX.XX: 8082
Content-Length: 34
GET proxy.config.alarm_email
HTTP / 1.1 200 Connection Established
Date: Tue, 07 Feb 2017 16:57:02 GMT
Transfer encoding: chunked
Connection: keep alive
...
proxy.config.alarm_email = "nobody#yahoo-inc.com"
I figured out the answer:
This is a protocol specially customized for an Apache Traffic Server by Yahoo.
Apache Traffic Server allows you to create your own protocols using the "New Protocols Plugin": https://docs.trafficserver.apache.org/en/latest/developer-guide/plugins/new-protocol-plugins.en.html.
The protocol created here appears to be line-based.
The scenario was as follows:
An initial load balancer evaluated the host header in the incoming HTTP request in such a way that it forwarded the incoming request to the location entered there. This means that the attacker could determine to which internal location the request should be routed, in this case to an Apache traffic server sitting at IP:Port XX.X.XXX.XX: 8082. The underlying attack was a host header injection (https://portswigger.net/web-security/host-header).
The line-based self-made protocol now evaluated the individual lines of the HTTP request. This is how the information shown was achieved (like explained here https://www.youtube.com/watch?v=zP4b3pw94s0&feature=youtu.be&t=12m40s)
.
This means that the attacker was able to address the internal Apache traffic server via an HTTP request and the individual lines of the request were each understood as individual commands.
A HELP command has now been implemented by Yahoo here.
If a webserver serves multiple virtual hosts (which can be selected by the client in the HTTP request header Host) and supports Keep-Alive, is the client allowed to send subsequent requests over the same connection with a different Host header?
From performance/efficiency point of view, both browser and server should Keep-Alive base on IP, not Hostname. Anyway, the reused connection is on TCP level, not HTTP level.
Although no information is found in RFC2068's Persistent Connections section. It seems Apache Keep-Alive between different virtual hosts (Is http keep-alive effective with different domain on the same webserver?). Also, #Michael Neale reports that connection is reused for different virtual hosts, on Chrome.
GET /calcuapp/calculator.jsp HTTP/1.1
Host: 192.168.1.66:8080
I'm using PuTTy and the host destination is already set up on the settings. Why do I need again to type the host destination as you can see above?
The short answer is Virtual Hosts.
For many years now, it has been quite common to host multiple sites/domains from a single server. HTTP 1.1 supports this by requiring the host header. If you use HTTP 1.0 you may leave this out.
The Host HTTP header is mandatory since HTTP/1.1 and it's used for virtual hosting.
It must include the domain name of the server, and the TCP port number on which the server is listening. The port number may be omitted if the port is the standard port for the service requested (80 for HTTP and 443 for HTTPS).
A HTTP/1.1 request that lacks the Host header should be responded with a 400 (Bad Request) status code.
The RFC 7230, the current reference message syntax and routing in HTTP/1.1, tells the whole story about this header:
5.4. Host
The Host header field in a request provides the host and port
information from the target URI, enabling the origin server to
distinguish among resources while servicing requests for multiple
host names on a single IP address.
Host = uri-host [ ":" port ]
A client MUST send a Host header field in all HTTP/1.1 request
messages. If the target URI includes an authority component, then a
client MUST send a field-value for Host that is identical to that
authority component, excluding any userinfo subcomponent and its #
delimiter. If the authority component is missing or
undefined for the target URI, then a client MUST send a Host header
field with an empty field-value.
Since the Host field-value is critical information for handling a
request, a user agent SHOULD generate Host as the first header field
following the request-line.
For example, a GET request to the origin server for
http://www.example.org/pub/WWW/ would begin with:
GET /pub/WWW/ HTTP/1.1
Host: www.example.org
A client MUST send a Host header field in an HTTP/1.1 request even if
the request-target is in the absolute-form, since this allows the
Host information to be forwarded through ancient HTTP/1.0 proxies
that might not have implemented Host.
When a proxy receives a request with an absolute-form of
request-target, the proxy MUST ignore the received Host header field
(if any) and instead replace it with the host information of the
request-target. A proxy that forwards such a request MUST generate a
new Host field-value based on the received request-target rather than
forward the received Host field-value.
Since the Host header field acts as an application-level routing
mechanism, it is a frequent target for malware seeking to poison a
shared cache or redirect a request to an unintended server. An
interception proxy is particularly vulnerable if it relies on the
Host field-value for redirecting requests to internal servers, or for
use as a cache key in a shared cache, without first verifying that
the intercepted connection is targeting a valid IP address for that
host.
A server MUST respond with a 400 (Bad Request) status code to any
HTTP/1.1 request message that lacks a Host header field and to any
request message that contains more than one Host header field or a
Host header field with an invalid field-value.
Your local resolver (DNS etc) converts the host name on the command line to an IP address before connecting; there is no way for the remote server to know which host name you gave on the command line if there are multiple host names which resolve to the same IP address (this is what's called "virtual hosting"; with HTTP 1.0, you needed a separate IP address for each distinct HTTP host, which is extremely wasteful, but saves you from needing to transmit the Host: header).
Say I make an HTTP request to: foosite.com but the port I actually send the request to is 6103 and I DON'T put that port in the Host header for example:
GET /barpage HTTP/1.1
Host: foosite.com
Method: GET
Should http server then recognize that I'm trying to talk to it on port 6103? Or since it was omitted in the request header am I gambling on if the server actually recognizes this?
I ask that question to say this: I've found that browsers, at least firefox + chrome, put the port in the Host header. But the Java app I'm using does not. And when the port is not passed in the Host the server responds back thinking I'm on port 80. So who do I need to badger? The server operator, or the Java programmer?
See section 14.23 of the HTTP spec which specifies that the port # should be included if it's not the default port (80 for HTTP, 443 for HTTPS).
UPDATED for modern day browsers:
Browsers (and curl) will add the port only when it is not the standard port, as required by the HTTP spec and noted in #superfell's answer.
Browsers this day (2013), will actually strip the port from the Host Header when the port is the standard (http port 80, https port 443). Some clients, which use their own method, like the Baidu Spider, include the port number even when the port is 80.
Whether this is proper or not, I don't know. The spec doesn't say whether it's OK or not to include the port number when the port used IS the default.
To answer your comment, servers will do whatever they need to do to comply with the spec, and the spec suggests only the cases WHEN it's needed. Because of this, I feel It's not really a question of how the server deals with it - it's more how the client issues the request: includes the port number in the Host Header, or not.
RFC2616 states that
A "host" without any trailing port information implies the default
port for the service requested (e.g., "80" for an HTTP URL). For
example, a request on the origin server for
http://www.w3.org/pub/WWW/ would properly include:
GET /pub/WWW/ HTTP/1.1
Host: www.w3.org
This means that https://example.com would not need a trailing port as well since the default port is known for https. I have checked the HTTP requests from Firefox, Chrome and Edge and found that none of them added the port number for the host header when the domain protocole was https. For sure the port number is added when the port number was also added to the URL. The following screenshots below come from Google chrome
Sample headers of an actual request to a hopefully non existent server 'http://myhost.com:3003/content/page.htm'
Accept: */*
Accept-Encoding: gzip, deflate
Accept-Language: en-US;q=0.9,en;q=0.8,nb;q=0.7,de;q=0.6
Connection: keep-alive
Host: myhost.com:3244
Referer: http://myhost.com:3244/content/page.htm
The RFC https://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html requires some training to read.
Section 14:24 not so easy to translate all elements to the simple reality:
Host = "Host" ":" host [ ":" port ] ;
Host Header Syntax:
Host: :
if its not default than put port after host:
Host: example.com:1337
I am using a 3G UMTS connection. I am trying to implement HTTP tunneling to a server of mine
which listens on port 80 (this is done in order to bypass client's firewall). The problem is that the ISP's proxy server supports HTTP/1.0 which doesn't support persistent HTTP connection.
As a result, after one http request/response iteration between my client/server the ISP's proxy tears down the underlying TCP connection.
my client receives the following HTTP response:
HTTP/1.0 200 OK
Content-Type: application/octet-stream
Content-Length: yyy
X-Cache: MISS from ipmr5
Proxy-Connection: close
Content data
while my server actually sends:
HTTP/1.1 200 OK
Connection: Keep-Alive
Content-Type: application/octet-stream
Content-Length: yyy
Content data
Is there any workaround?
You could always use HTTPS. You will lose any benefits offered by the proxies (such as caching), but all of your HTTP headers will arrive at the server exactly as you sent them.
HTTP 1.0 proxies (which it seems your ISP uses) shouldn't be used in connection with Connection: Keep-Alive for persistent connections. The reasons for this are outlined in RFC-2068 (section 19.7.1). The short version, basically, is that your server is sending an invalid header for the kind of proxy you are using.