I am testing the Last.fm api using row socket interface.
Now i noticed that some of api's http response have not contain a field Content-Length .
But I want to know is there a way to ask the server presenting it?
Because i can't take good care of this in my program elegantly.
Quoth the RFC:
7.2.2 Length
When an Entity-Body is included with a message, the length of that body may be determined in one of two ways. If a Content-Length header field is present, its value in bytes represents the length of the Entity-Body. Otherwise, the body length is determined by the closing of the connection by the server.
The right RFC to look at is RFC 7230 (Section 3.3.2).
And no, in HTTP/1.1 a client has to be able to process chunked encoding (which would be the only legitimate reason not to provide a Content-Length header field).
I am using the HTTPListener class to implement a basic web server. My response uses a Content-Type of "multipart/x-mixed-replace" to return a stream of JPEG images. I am able to construct my response correctly, however my web client does not properly interpret the response due to the way in which the response is broken across IP packet boundaries.
Using a separate server implemented in python, I am able to generate a good, working case. The response to the client's HTTP GET request looks like this:
packet 1:
HTTP/1.1 200 OK
packet 2:
Server: (myServer)
Date: (the date)
Connection: close
(other header info)
Content-Type: multipart/x-mixed-replace; boundary=myboundary
packet 3:
--myboundary
Content-Length: 1042
Content-Type: image/jpeg
(jpeg data)
In the failed case, using the HTTPListener, everything gets sent in a single packet
packet 1:
HTTP/1.1 200 OK
Server: (myServer)
Date: (the date)
Connection: close
(other header info)
Content-Type: multipart/x-mixed-replace; boundary=myboundary
--myboundary
Content-Length: 1042
Content-Type: image/jpeg
(jpeg data)
So my question is, how do I manipulate the HTTPListenerResponse's OutputStream to force a packet boundary? I want to be able to specify some data, manually tell the OutputStream to push out a packet, then specify some more data and push out another packet. Does the HTTPListener offer this level of control, or do I need to instead use a TCPListener? I've not been able to find a solution; please help!
If your client doesn't work because of IP packet boundaries it is severely broken. Fix the client, don't add crutches for it in a place where there isn't a problem. HTTP is defined over TCP, and TCP is a byte-stream protocol. Period. Any correctly written TCP program, let alone an HTTP client, doesn't care where the packet boundaries are. If your client misbehaves in this way it will misbehave in other ways as well. Fix it.
Is there an easier way of getting the content of an HTTP 404 response than directly accessing the host via tcp?
This is a sample of a 404 response with content:
HTTP/1.1 404 Object Not Found
Server: CouchDB/1.3.0 (Erlang OTP/R15B03)
Date: Wed, 24 Jul 2013 08:32:50 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 41
Cache-Control: must-revalidate
{"error":"not_found","reason":"missing"}
The Rebol HTTP scheme really isn't designed with this in mind, it's geared toward reading content the way you would in a browser, not services over HTTP.
In saying that, you can hack the protocol to subvert how Rebol 2 handles different response codes:
in-http-scheme: func [does [block!]][
do bind :does bind? last body-of get in system/schemes/http/handler 'open
]
in-http-scheme [
remove-each [code response] response-actions [find [400 403 404] code]
append response-actions [400 success 403 success 404 success]
]
The caveat here is that the HTTP protocol has to have been initiated (any http port opened/read). response-actions can still be accessed when http has not been initiated:
select body-of get in system/schemes/http/handler 'open quote response-actions:
You can get the last response line thus:
in-http-scheme [response-line]
Alternatively you are going to need a scheme designed for services over HTTP. I have a REST protocol (two versions, one that uses cURL, and one that uses a customised HTTP scheme that works, but isn't as good). Though are for Rebol 2. I have plans for a Rebol 3 version.
Christopher Ross-Gill has created a REST protocol for Rebol which allows simple access to all headers and even handles OAuth. Have a look at the details here.
http://www.ross-gill.com/page/REST_Protocol
Unfortunately it is only for Rebol 2 at the moment and it depends on the use of curl for the http requests.
I am writing a HTTP Webserver. My server has to handle Http multipart requests. In my previous implementation, I was extracting the data with the help of content length header present in every part of request. The client which I was using give content-length header with every part part(file) in the multipart request.
But another client is not giving content-length of each file. In my implementation I use content-length header to extract that much bytes and save them into a file.
Please tell me how can I extract data now.
The Headers which I am getting now are:
POST xxxxxxxxxxxxxxxxxxxxxxx¤tTab=PHOTOxxxxxxxxxxxxxxxx HTTP/1.1
Content-Length: 6829
Content-Type: multipart/form-data; boundary=SnlCg9JqTpQIl6t_mPzByTjZ8bD24kUj; charset=UTF-8
Host: host
Connection: Keep-Alive
User-Agent: Apache-HttpClient/xxxxxxxx
Accept-Encoding: gzip
--SnlCg9JqTpQIl6t_mPzByTjZ8bD24kUj
Content-Disposition: form-data; name="file"; filename="imagesCA5L2CL6_jpg(2)_jpg.jpg"
Content-Type: photo/jpg
**Some Data byte array**
--SnlCg9JqTpQIl6t_mPzByTjZ8bD24kUj--
In this request, there is now content-length header in part data.
EDIT:
Earlier this client used to send content-length header in every part. But for some reason it is not sending it any more. Can anybody suggest any reason for that.
thanks
Like this : Reading file input from a multipart/form-data POST
Reading file input from a multipart/form-data POST
Take a look at RFC 2616 if you want to implement a HTTP/1.1 server. See section 4.4 on how to determine message length. See RFC 2388 on how to implement multipart/form-data.
The real answer is: don't reinvent the wheel, or you'll have to reimplmement a few hundred pages of RFC's. There are tons of libraries and servers out there.
If you do want to write your own web server, for example as an exercise, you would have found those RFC's already, right?
I'm trying to issue a GET command to my local server using netcat by doing the following:
echo -e "GET / HTTP/1.1\nHost: localhost" | nc localhost 80
Unfortunately, I get a HTTP/1.1 400 Bad Request response for this. What, at the very minimum, is required for a HTTP request?
if the request is: "GET / HTTP/1.0\r\n\r\n" then the response contains header as well as body, and the connection closes after the response.
if the request is:"GET / HTTP/1.1\r\nHost: host:port\r\nConnection: close\r\n\r\n"
then the response contains header as well as body, and the connection closes after the response.
if the request is:"GET / HTTP/1.1\r\nHost: host:port\r\n\r\n" then the response contains header as well as body, and the connection will not close even after the response.
if your request is: "GET /\r\n\r\n" then the response contains no header and only body, and the connection closes after the response.
if your request is: "HEAD / HTTP/1.0\r\n\r\n" then the response contains only header and no body, and the connection closes after the response.
if the request is: "HEAD / HTTP/1.1\r\nHost: host:port\r\nConnection: close\r\n\r\n" then the response contains only header and no body, and the connection closes after the response.
if the request is: "HEAD / HTTP/1.1\r\nHost: host:port\r\n\r\n" then the response contains only header and no body, and the connection will not close after the response.
It must use CRLF line endings, and it must end in \r\n\r\n, i.e. a blank line. This is what I use:
printf 'GET / HTTP/1.1\r\nHost: www.example.com\r\nConnection: close\r\n\r\n' |
nc www.example.com 80
Additionally, I prefer printf over echo, and I add an extra header to have the server close the connection, but those aren’t needed.
See Wiki: HTTP Client Request (Example).
Note the following:
A client request (consisting in this case of the request line and only one header) is followed by a blank line, so that the request ends with a double newline, each in the form of a carriage return followed by a line feed. The "Host" header distinguishes between various DNS names sharing a single IP address, allowing name-based virtual hosting. While optional in HTTP/1.0, it is mandatory in HTTP/1.1.
The absolute minimum (if removing the Host is allowed ;-) is then GET / HTTP/1.0\r\n\r\n.
Happy coding
I was able to get a response from my Apache server with only the requested document, no response header, with just
GET /\r\n
If you want response headers, including the status code, you need one of the other answers here though.
The fact of the 400 Bad Request error itself does not imply that your request violates HTTP. The server very well could be giving this response for another reason.
As far as I know the absolute minimum valid HTTP request is:
GET / HTTP/1.0\r\n\r\n
Please, please, please, do not implement your own HTTP client without first reading the relevant specs. Please read and make sure that you've fully understood at least RFC 2616. (And if you're ambitious, RFC 7230 through 7235).
While HTTP looks like an easy protocol, there are actually a number of subtle points about it. Anyone who has written an HTTP server will tell you about the workarounds he had to implement in order to deal with incorrect but widely deployed clients. Unless you're into reading specifications, please use a well-established client library; Curl is a good choice, but I'm sure there are others.
If you're going to implement your own:
do not use HTTP/0.9;
HTTP/1.0 requires the query line and the empty line;
in HTTP/1.1, the Host: header is compulsory in addition to the above.
Omitting the Host: header in HTTP/1.1 is the most common cause of 400 errors.
You should add an empty line: \r\n\r\n
http://en.wikipedia.org/wiki/Hypertext_Transfer_Protocol#Client_request
The really REALLY BARE minimum, is not using netcat, but using bash itself:
user#localhost:~$ exec 3<>/dev/tcp/127.0.0.1/80
user#localhost:~$ echo -e "GET / HTTP/1.1\n" >&3
user#localhost:~$ cat <&3
HTTP/1.0 200 OK
Server: SimpleHTTP/0.6 Python/2.7.6
Date: Mon, 13 Oct 2014 17:55:55 GMT
Content-type: text/html; charset=UTF-8
Content-Length: 514
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN"><html>
<title>Directory listing for /</title>
<body>
<h2>Directory listing for /</h2>
<hr>
<ul>
</ul>
<hr>
</body>
</html>
user#localhost:~$