What, at the bare minimum, is required for an HTTP request? - http

I'm trying to issue a GET command to my local server using netcat by doing the following:
echo -e "GET / HTTP/1.1\nHost: localhost" | nc localhost 80
Unfortunately, I get a HTTP/1.1 400 Bad Request response for this. What, at the very minimum, is required for a HTTP request?

if the request is: "GET / HTTP/1.0\r\n\r\n" then the response contains header as well as body, and the connection closes after the response.
if the request is:"GET / HTTP/1.1\r\nHost: host:port\r\nConnection: close\r\n\r\n"
then the response contains header as well as body, and the connection closes after the response.
if the request is:"GET / HTTP/1.1\r\nHost: host:port\r\n\r\n" then the response contains header as well as body, and the connection will not close even after the response.
if your request is: "GET /\r\n\r\n" then the response contains no header and only body, and the connection closes after the response.
if your request is: "HEAD / HTTP/1.0\r\n\r\n" then the response contains only header and no body, and the connection closes after the response.
if the request is: "HEAD / HTTP/1.1\r\nHost: host:port\r\nConnection: close\r\n\r\n" then the response contains only header and no body, and the connection closes after the response.
if the request is: "HEAD / HTTP/1.1\r\nHost: host:port\r\n\r\n" then the response contains only header and no body, and the connection will not close after the response.

It must use CRLF line endings, and it must end in \r\n\r\n, i.e. a blank line. This is what I use:
printf 'GET / HTTP/1.1\r\nHost: www.example.com\r\nConnection: close\r\n\r\n' |
nc www.example.com 80
Additionally, I prefer printf over echo, and I add an extra header to have the server close the connection, but those aren’t needed.

See Wiki: HTTP Client Request (Example).
Note the following:
A client request (consisting in this case of the request line and only one header) is followed by a blank line, so that the request ends with a double newline, each in the form of a carriage return followed by a line feed. The "Host" header distinguishes between various DNS names sharing a single IP address, allowing name-based virtual hosting. While optional in HTTP/1.0, it is mandatory in HTTP/1.1.
The absolute minimum (if removing the Host is allowed ;-) is then GET / HTTP/1.0\r\n\r\n.
Happy coding

I was able to get a response from my Apache server with only the requested document, no response header, with just
GET /\r\n
If you want response headers, including the status code, you need one of the other answers here though.

The fact of the 400 Bad Request error itself does not imply that your request violates HTTP. The server very well could be giving this response for another reason.
As far as I know the absolute minimum valid HTTP request is:
GET / HTTP/1.0\r\n\r\n

Please, please, please, do not implement your own HTTP client without first reading the relevant specs. Please read and make sure that you've fully understood at least RFC 2616. (And if you're ambitious, RFC 7230 through 7235).
While HTTP looks like an easy protocol, there are actually a number of subtle points about it. Anyone who has written an HTTP server will tell you about the workarounds he had to implement in order to deal with incorrect but widely deployed clients. Unless you're into reading specifications, please use a well-established client library; Curl is a good choice, but I'm sure there are others.
If you're going to implement your own:
do not use HTTP/0.9;
HTTP/1.0 requires the query line and the empty line;
in HTTP/1.1, the Host: header is compulsory in addition to the above.
Omitting the Host: header in HTTP/1.1 is the most common cause of 400 errors.

You should add an empty line: \r\n\r\n
http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Client_request

The really REALLY BARE minimum, is not using netcat, but using bash itself:
user#localhost:~$ exec 3<>/dev/tcp/127.0.0.1/80
user#localhost:~$ echo -e "GET / HTTP/1.1\n" >&3
user#localhost:~$ cat <&3
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/2.7.6
Date: Mon, 13 Oct 2014 17:55:55 GMT
Content-type: text/html; charset=UTF-8
Content-Length: 514
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"><html>
<title>Directory listing for /</title>
<body>
<h2>Directory listing for /</h2>
<hr>
<ul>
</ul>
<hr>
</body>
</html>
user#localhost:~$

Related

How uTorrent can read the body of a http response message without a specified size or "chunked" option?

I've used the SmartSniff tool to catch the http messages between the uTorrent application and a server. I found one server that sends "HTTP/1.0 200 OK" response messages with a body and no headers that can tell the length of the body or the "chunked" option, but uTorrent seems not having trouble with that, it works fine. And I wander how did it do it ?
I think that, maybe, uTorrent knows about this "server error" and when it is expecting a body, after it reads the response line and headers (in this case none), it reads until the server close the connection. Is this posible ?
Catched communication:
GET /announce.php?(a list of parameters here) HTTP/1.1
Host: some.server.here:1234
User-Agent: uTorrent/3320(30416)
Accept-Encoding: gzip
Connection: Close
HTTP/1.0 200 OK
(empty line)
d8:completei176e10:incompletei0e8:intervali3600e5:peers0:e
I researched the link provided and I find the answer. It is as follow:
In a response message without a declared message body length, the message body length is determined by the number of octets received prior to the server closing the connection.

How to get the response content of an HTTP 404 response

Is there an easier way of getting the content of an HTTP 404 response than directly accessing the host via tcp?
This is a sample of a 404 response with content:
HTTP/1.1 404 Object Not Found
Server: CouchDB/1.3.0 (Erlang OTP/R15B03)
Date: Wed, 24 Jul 2013 08:32:50 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 41
Cache-Control: must-revalidate
{"error":"not_found","reason":"missing"}
The Rebol HTTP scheme really isn't designed with this in mind, it's geared toward reading content the way you would in a browser, not services over HTTP.
In saying that, you can hack the protocol to subvert how Rebol 2 handles different response codes:
in-http-scheme: func [does [block!]][
do bind :does bind? last body-of get in system/schemes/http/handler 'open
]
in-http-scheme [
remove-each [code response] response-actions [find [400 403 404] code]
append response-actions [400 success 403 success 404 success]
]
The caveat here is that the HTTP protocol has to have been initiated (any http port opened/read). response-actions can still be accessed when http has not been initiated:
select body-of get in system/schemes/http/handler 'open quote response-actions:
You can get the last response line thus:
in-http-scheme [response-line]
Alternatively you are going to need a scheme designed for services over HTTP. I have a REST protocol (two versions, one that uses cURL, and one that uses a customised HTTP scheme that works, but isn't as good). Though are for Rebol 2. I have plans for a Rebol 3 version.
Christopher Ross-Gill has created a REST protocol for Rebol which allows simple access to all headers and even handles OAuth. Have a look at the details here.
http://www.ross-gill.com/page/REST_Protocol
Unfortunately it is only for Rebol 2 at the moment and it depends on the use of curl for the http requests.

HTTP 400 - Hard to understand error code with minimal description

All,
My requirement is fairly simple. I have to perform a simple HTTP POST to an IP:port combination. I used simple socket programming to do that and I have been successful in sending across my request to them and also get back response from them. The only problem being that the response is always a HTTP 400: Bad Request followed by my HTTP POST message. I am not sure if the problem is with the client or the server. My only guess being that there might be a problem with my data that I am sending. This is what my POST looks like
POST /<Server Tag> HTTP/5.1
Content-Length: xxx
--Content--
and the response from the server looks something like this
HTTP/1.1 400 Bad Request
Content-Length: xxx
--Same content that I sent them--
I was not sure If I could put in the IP of the server here so kept myself to using . I am pretty sure that the problem would not be there since I get back some response from the server and confident about the connection. Can someone help me ?
PS: Some pointers about my POST:
1) HTTP 5.1 was requested by the server and I am not sure if that is correct
2) I have played around with the number of line spaces after the content length. I have tried giving one and two lines. Not sure if that would make a difference. On wireshark though I see a difference with the number of line spaces as with a single line space the protocol is specified as TCP but with two it changes to HTTP. The response is always received on HTTP protocol. Some explanation on the difference would also help
Thanks
edit: the other thing that confuses me is that the response has a HTTP 1.1 and not a 5.1 that I had sent. I have also tried changing my post to 1.1 with no success
edit2: Based on suggestion form fvu and others, I used WebClient to Upload my request. Still got back a 400. The header that was generated by the WebClient looks like this
POST <server tag> HTTP/1.1
Host: <IP:PORT>
Content-Length: 484
Expect: 100-continue
Connection: Keep-Alive
The issue I see with this might be that the server was not expecting all the details in the header. The server has requested only the Content-Length from us. Would that be a problem?
Thanks
You can use a debugging proxy to view a client request and a server response to figure out what your client socket program needs to do.
But first you need to create a simple web page that a browser displays, allows you to do a POST from the browser to the web server, and get a simple response back from the server.
HTTP/5.1 is either wrong or misused by the programmer of the server application
You should get a valid example from the server api to check your protocol implementation first.

HTTP HEAD chunked-encoded

I have implemented a HTTP 1.1 server. It is an embedded server so I only support the mandatory features of the RFC. All responses are sent chunked-encoded.
As HEAD is mandatory it is also supported.
HEAD is a GET without body. So the server is sending a response like following in response to a HEAD request:
HTTP/1.1 200 OK
Server: testServer
Connection: keep-alive
Transfer-Encoding: chunked
What I am wondering is do have to add a "0\r\n" as it required to signal the end of chunks:
HTTP/1.1 200 OK
Server: testServer
Connection: keep-alive
Transfer-Encoding: chunked
0
I have tried to collect the relevant parts in the RFC:
"The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response."
"All responses to the HEAD request method MUST NOT include a message-body, even though the presence of entity- header fields might lead one to believe they do."
"1.Any response message which "MUST NOT" include a message-body (such as the 1xx, 204, and 304 responses and any response to a HEAD request) is always terminated by the first empty line after the header fields, regardless of the entity-header fields present in the message."
So far I understand it that my first solution (without 0) is the correct one. But it seems to be strange to send a message with Transfer-encoding: chunked which does not terminate with the the chunk style 0\r\n.
I guess the problem stated is solved as its quite old,
Recently I developed a test http server for streaming video and I wanted to put forth my learnings so that it might help others.
First, "0\r\n" is not the marker for end of chunks, the chunked trailer is '0\r\n\r\n'.
And for the header the string should look like this
"HTTP/1.1 200 OK\r\n" \
"Server: testServer\r\n" \
"Connection: keep-alive\r\n" \
"Transfer-Encoding: chunked\r\n" \
"\r\n"
Note the last CRLF, its indicating end of header.
Hope this help.

Difference between Content-Range and Range headers?

What is the difference between HTTP headers Content-Range and Range? When should each be used?
I am trying to stream an audio file from a particular byte offset. Should I use Content-Range or Range header?
Actually, the accepted answer is not complete. Content-Range is not only used in responses. It is also legal in requests that provide an entity body.
For example, an HTTP PUT provides an entity body, it might provide only a portion of an entity. Thus the PUT request can include a Content-Range header indicating to the server where the partial entity body should be merged into the entity.
For example, let's first create and then append to a file using HTTP:
Request 1:
PUT /file HTTP/1.1
Host: server
Content-Length: 1
a
Request 2:
PUT /file HTTP/1.1
Host: server
Content-Range: bytes 1-2/*
Content-Length: 1
a
How, let's see the file's contents...
Request 3:
GET /file HTTP/1.1
Host: server
HTTP/1.1 200 OK
Content-Length: 2
aa
This allows random file access, both READING and WRITING over HTTP. I just wanted to clarify, as I was researching the use of Content-Range in a WebDAV client I am developing, so perhaps this expanded information will prove useful to somebody else.
Range is used in the request, to ask for a particular range (or ranges) of bytes. Content-Range is used in the response, to indicate which bytes the server is giving you (which may be different than the range you requested), as well as how long the entire content is (if known).

Resources