I'm sending data to a server with an Arduino which requires constructing an HTML POST line-by-line. I don't necessarily know the Content-Length a-priori, so I am using "chunked" encoding.
When I tried this example post from Wikipedia with the "Transfer-Encoding" option as specified in rfc2616
client.println("POST /myurl HTTP/1.1");
client.println("Host: 12.345.679.999"); // replaced with the test server's IP
client.println("User-Agent: Arduino/1.0");
client.println("Transfer-Encoding: chunked");
client.println();
client.println("4");
client.println("test");
client.println("0");
client.println();
or, with escape characters explicit:
client.print("4\r\ntest\r\n0\r\n\r\n");
I receive the error from my server:
HTTP/1.1 411 Length Required
A request of the requested method POST requires a valid Content-length.
Server: Apache/2.2.22 (Ubuntu)
However, "chunked" encoding shouldn't require a Content-Length header field, see 4.4 - Message Length in rfc2616
Am I missing a field? Why doesn't this call work?
For the record, the non-Chunked-Encoding works:
if(client.connect(server, 80)){
String PostData = "test";
Serial.println("POST /myurl HTTP/1.1");
client.println("Host: 12.345.679.999"); // replaced with the test server's IP
Serial.println("User-Agent: Arduino/1.0");
Serial.print("Content-Length: ");
Serial.println(PostData.length());
Serial.println();
Serial.println(PostData);
}
UPDATE
From the apache2 error.log: "chunked Transfer-Encoding forbidden"
After finding
chunked Transfer-Encoding forbidden
in my Apache2 log I concluded that the error was not in the POST that I was making.
I found that modwsgi (the middle-layer between apache and django) does not enable chunked transfer-encoding by default. In the past, chunked wasn't supported at all
Refering to the change-log in the new version of modwsgi, I found that writing
WSGIChunkedRequest On
in my apache httpd.conf file allowed chunked requests (no more 411 error)
Related
I found that a get message header looks like:
:method: GET
:scheme: https
:authority: server.net
:path: /config
accept: */*
accept-encoding: gzip,deflate
What a connect message header should look like?
This example is from the RFC of http2:
GET /resource HTTP/1.1 HEADERS
Host: example.org ==> + END_STREAM
Accept: image/jpeg + END_HEADERS
:method = GET
:scheme = https
:path = /resource
host = example.org
accept = image/jpeg
I want to know the equivalent of the connect header in http2.
In Http1 is:
CONNECT example.org:443 HTTP/1.1
Host: example.org:443
The format of the CONNECT method in HTTP/2 is specified in section 8.3.
With the formatting you used above looks like:
:method: CONNECT
:authority: proxy.net:8080
As specified, :scheme and :path must be omitted.
The HTTP/2 CONNECT method can also be used for bootstrapping other protocols (see for example WebSocket over HTTP/2), so that, additionally, the :protocol pseudo-header may also be present.
Remember however that this is only a textual representation of HTTP/2; the bytes that actually travel over the network are different since you must encode them using HPACK.
Unless you are actually writing an HTTP/2 implementation, it is better that you use existing libraries (available in virtually any programming language) to send HTTP/2 requests (of any kind): the libraries will take care of converting your CONNECT request into the proper bytes to send over the network.
I'm trying to test writing correct HTTP headers to understand
the syntax. Here I'm trying to PUT some text into httpbin.org/put and I expect the response body content to be the same.
PUT /HTTP/1.1
Host: httpbin.org
Accept-Language: en-us
Connection: Keep-Alive
Content-type: text/plain
Content-Length: 12
Hello jerome
However I'm getting the following bad request 400 response:
HTTP/1.1 400 Bad Request
Server: nginx
Date: Tue, 01 Mar 2016 12:34:02 GMT
Content-Type: text/html
Content-Length: 166
Connection: close
Response:
<html>
<head><title>400 Bad Request</title></head>
<body bgcolor="white">
<center><h1>400 Bad Request</h1></center>
<hr><center>nginx</center>
</body>
</html>
What syntactical errors have I done?
NOTE: newlines are \r\n not \n in the request.
Apparently the correct syntax goes like this for PUT:
PUT /put HTTP/1.1\r\n
Content-Length: 11\r\n
Content-Type: text/plain\r\n
Host: httpbin.org\r\n\r\n
hello lala\n
I believe I didn't say much on how I connected to httpbin.org; it was via sockets in C. So the connection was already established before sending the header + message.
You miss the destination url following the PUT verb, the first line must be:
PUT http://httpbin.org/ HTTP/1.1
This will probably also fail, you need one of their handler urls so they know what to reply with:
PUT http://httpbin.org/put HTTP/1.1
The general form of the first line, or Request Line, in an HTTP request is as follows:
<method> <path component of URL, or absolute URL> HTTP/<Version>\r\n
Where for your example, the method is PUT. Including an absolute URL (so, starting with http:// or https:// is only necessary when connecting to a proxy, because the proxy will then attempt to retrieve that URL, rather than attempt to serve a local resource (as found by the path component).
As presented, the only change you should have needed to make was ensuring there was a space between the / and HTTP/1.1. Otherwise, the path would be "/HTTP/1.1"... which would be a 404, if it weren't already a badly formed request. /HTTP/1.1 being interpreted as a path means the HTTP server that's parsing your request line doesn't find the protocol specifier (the HTTP/1.1 bit) before the terminating \r\n... and that's one example of how 400 response codes are born.
Hope that helped. Consult the HTTP 1.1 RFC (2616), section 5.1 for more information and the official definitions.
Actually I am trying to send PUT request to a remote server the request is successfully send to the remote server but unfortunately the request is considered a bad request it can not see the body of the request I hope my discription is clear and here is the code for sending the request in Arduino programming language:
Serial.println("Sending to Server: ");
client.print("PUT /***/***/sensor/uod/1/A/1 HTTP/1.1\n");
Serial.print("PUT /***/***/sensor/uod/1/A/1 HTTP/1.1");
client.print("Host: ********.koding.io\n");
client.print("Cache-Control: no-cache\n");
client.print("Content-Type: application/x-www-form-urlencoded\r\n\r\n");
client.print("status=1");
one thing I want to mention that from the server side I used a tool that is called postman and from it it works successfully without any error and actually I have taken it's http request and convert it into the code to be send and you can find the http request below:
PUT /***/***/sensor/uod/1/A/1 HTTP/1.1
Host: *******.koding.io
Cache-Control: no-cache
Postman-Token: 8740ccb4-c0f2-a916-fd52-3089e4f3cbc9
Content-Type: application/x-www-form-urlencoded
status=0
I have solve the problem by adding content-Length: 10 header
Serial.println("Sending to Server: ");
client.print("PUT /***/***/sensor/uod/1/A/1 HTTP/1.1\n");
Serial.print("PUT /***/***/sensor/uod/1/A/1 HTTP/1.1");
client.print("Host: *********.koding.io\n");
client.print("Cache-Control: no-cache\n");
client.print("Content-Type: application/x-www-form-urlencoded\r\n");
client.print("Content-Length: 10\r\n\r\n");
client.print("status=1\r\n");
When trying to execute an http request that causes a 400 Bad Request error, an nginx server returns the "400 Bad Request", a Content-Encoding: gzip, and the response in a non-encoded format.
So because of the Content-Encoding: gzip my code wraps the response with a GZIPInputStream which throws an error.
My question is: is gzip a valid encoding when the status code is not "200 OK"? if not, I can simply check for the status code and not try to gzip if anything other than 200 was returned.
Are there rules/guidelines about this? Is this documented anywhere?
TIA
Yes, it is a valid encoding. Content-Encoding (and Transfer-Encoding) are orthogonal to status codes.
Given this snippet:
(defroutes main-routes
(POST "/input/:controller" request
(let [buff (ByteArrayOutputStream.)]
(copy (request :body) buff)
;; --- snip
The value of buff will be a non-empty byte array iff there's the Content-Type header in the request. The value can be nonsencial, the header just has to be there.
However, I need to dump the body (hm... that came out wrong) if the request came without a content type, so that the client can track down the offending upload. (The uploading software is not under my control and its maintainers won't provide anything extra in the headers.)
Thank you for any ideas on how to solve or work around this!
EDIT:
Here are the headers I get from the client:
{
"content-length" "159",
"accept" "*/*",
"host" (snip),
"user-agent" (snip)
}
Plus, I discovered that Ring, using an instance of Java's ServletRequest, fills in the content type with the standard default, x-www-form-urlencoded. I'm now guessing that HTTPParser, which supplies the body through HTTPParser#Input, can't parse it correctly.
I face the same issue. It's definitely one of the middleware not being able to parse the body correctly and transforming :body. The main issue is that the Content-Type suggest the body should be parsable.
Using ngrep, I found out how curl confuses the middleware. The following, while intuitive (or rather sexy) on the command line sends a wrong Content-Type which confuses the middleware:
curl -nd "Unknown error" http://localhost:3000/event/error
T 127.0.0.1:44440 -> 127.0.0.1:3000 [AP]
POST /event/error HTTP/1.1.
Authorization: Basic SzM5Mjg6ODc2NXJkZmdoam5idmNkOQ==.
User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3.
Host: localhost:3000.
Accept: */*.
Content-Length: 13.
Content-Type: application/x-www-form-urlencoded.
.
Unknown error
The following however forces the Content-Type to being opaque and the middleware will not interfere with the :body.
curl -nd "Unknown error" -H "Content-Type: application/data" http://localhost:3000/event/error
T 127.0.0.1:44441 -> 127.0.0.1:3000 [AP]
POST /event/error HTTP/1.1.
Authorization: Basic SzM5Mjg6ODc2NXJkZmdoam5idmNkOQ==.
User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3.
Host: localhost:3000.
Accept: */*.
Content-Type: application/data.
Content-Length: 13.
.
Unknown error
I'm considering replacing the middleware with a more liberal one because even though the request is wrong, I'd still like to be able to decide what to do with the body myself. It's a really weird choice to zero the request body when the request doesn't make sense. I actually think a more correct behavior would be to pass it to an error handler which by default would return a 400 Bad Request or 406 Not Acceptable.
Any thoughts on that? In my case I might propose a patch to Compojure.
According to:
http://mmcgrana.github.com/ring/ring.middleware.content-type-api.html
the default content type is application/octet-stream. Unless you actively support that content type, can't you just check if the content type matches that one, and then dump whatever you need based on that?