Go Webserver Not Handling POST Requests - http

I'm creating a simple web server using Go's http package. I'm registering just one handler, for requests to the path "/requests/".
It can handle GET requests just fine but when I send a POST request, the handler is never invoked and the client gets a 301 Moved Permanently response.
I've tried searching for this but it appears this isn't a problem people are commonly facing.
My handler is:
func requestHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello")
}
main function:
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/requests/", requestHandler)
http.ListenAndServe(":8000", mux)
}
Curl gives the following output:
>> curl -i -X POST http://localhost:8000/requests
HTTP/1.1 301 Moved Permanently
Location: /requests/
Date: Thu, 12 Jan 2017 08:51:10 GMT
Content-Length: 0
Content-Type: text/plain; charset=utf-8
Go's own http client returns a similar response object:
&{301 Moved Permanently 301 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8] Location:[/requests/] Date:[Thu, 12 Jan 2017 08:51:58 GMT] Content-Length:[0]] 0x339190 0 [] false false map[] 0xc4200cc0f0 <nil>}
Again, GET requests behave just as I'd expect them to and invoke the handler function. Do I need to do something differently to handle POST requests? Thanks for any help on this!

You are querying for /requests.
The redirect is pointing you to /requests/
You used curl like this :
curl localhost:8000/requests
You need to either use /requests instead of /requests/ in mux.HandleFunc.
or use
curl localhost:8000/requests/
Also note that if you request would work on a browser without any change as it handles redirects automatically.
And the route with trailing slash would return 404, if the route in mux.HandleFunc does not have a trailing slash.
PS : Your requestHandler handles all methods, not just POST requests. You need to check for r.Method to handle the methods differently.

Related

How to not pass authorization header in httr for redirects

Issue mentioned and followed up at:
https://github.com/r-lib/httr/issues/626
https://community.rstudio.com/t/possible-to-drop-authentication-header-on-redirects-in-httr/44358
This issue happens when:
httr sends a GET call to a URL which requires an "Authorization" header
the API constructs a redirect to a pre-signed URL which is returned as the response to the original GET request, for which no authentication is required
the httr client sends the same "Authorization" header on the redirect URL as it did on the URL of the original GET call
a 401 is returned because the bearer token is not valid for the redirect URL
It seems that "Authorization" headers should not be sent when a redirect is followed.
But this is a common issue even in e.g. browsers (so this might be broader than httr):
Authorization header when following redirects
The following example works (http redirect to https), but requires writing a function separate from httr.
Can httr handle this type of use case directly or is it best practice to write custom wrappers for this ?
Example (adapted from https://github.com/r-lib/httr/issues/626) :
library(httr)
# Ensure curl version is higher than 7.58: https://curl.se/docs/CVE-2018-1000007.html
curl::curl_version()$version
# Call URL without following to redirect
resp <- GET(
"http://github.com",
config(followlocation = FALSE),
authenticate("user", "pwd", type = "basic")
)
# Check response of URL, which contains the redirect within the location field
resp
redirected_url <- headers(resp)[["location"]]
# Call redirect URL without authentication
resp_redirect <- GET(redirected_url)
Console outputs:
> curl::curl_version()$version
[1] "7.60.0"
> resp
Response [http://github.com/]
Date: 2022-08-24 09:11
Status: 301
Content-Type: <unknown>
<EMPTY BODY>
> redirected_url
[1] "https://github.com/"
> resp_redirect
Response [https://github.com/]
Date: 2022-08-24 09:12
Status: 200
Content-Type: text/html; charset=utf-8
Size: 301 kB

How can I customize HTTP 400 responses for parse errors?

I've written a REST API service that requires that all responses be JSON. However, when the Go HTTP request parser encounters an error, it returns 400 as a plain text response without ever calling my handlers. Example:
> curl -i -H 'Authorization: Basic hi
there' 'http://localhost:8080/test' -v
* Trying ::1...
* TCP_NODELAY set
* Connected to localhost (::1) port 8080 (#0)
> GET /test HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.54.0
> Accept: */*
> Authorization: Basic hi
> there
>
< HTTP/1.1 400 Bad Request
HTTP/1.1 400 Bad Request
< Content-Type: text/plain; charset=utf-8
Content-Type: text/plain; charset=utf-8
< Connection: close
Connection: close
<
* Closing connection 0
Note the invalid Authorization header. Of course 400 is the proper response, but it's text/plain, of course. Is there some way to configure the Go http parser to use custom error response media types and bodies?
You can't. You can find this in net/http source, it only happens if the request was malformed:
https://github.com/golang/go/blob/master/src/net/http/server.go#L1744
I think your problem might be a new line in the header you're adding in curl?
401, 403, 404, 500 errors you'll be able to respond with json, but bad requests or bad headers (too long, malformed) are handled within server.go.
There is at present no way to intercept such errors though it is under consideration, so your only solution in go would be to patch the stdlib source (I don't recommend this). However, since this error only presents if the client has made a mistake and the request is malformed, it's probably not a huge problem. The reason for the text response is so that a browser or similar client (like curl without -v) doesn't just see an empty response. You could put a proxy like nginx in front of your app but then you'd never see the request either as it is a bad request, your proxy would handle it.
Possibly you'd be able to do it with a proxy like nginx in front though if you set a specific static error page for it to serve for 400 errors and serve a 400.json file that you specify? That's the only solution I can think of. A directive something like this might work for nginx:
error_page 400 /400.json;
If you'd like to be able to customise these errors, perhaps add a comment to the issue linked to let them know you had this specific problem.
If you are using the standard net/http library you can use the following code. Take a look at this answer Showing custom 404 error page with standard http package #Mostafa to which I got this example from
func homeHandler(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/" {
errorHandler(w, r, http.StatusNotFound)
return
}
fmt.Fprint(w, "welcome home")
}
func errorHandler(w http.ResponseWriter, r *http.Request, status int) {
w.WriteHeader(status)
if status == http.StatusNotFound {
// JSON Out here
}
}

How to send a POST request using HTTPie?

I have a basic silex application, and I try to test it using HTTPie. Yet when posting using:
http POST http://localhost:1337 data="hello world"
The data, that I get from the Request object via:
$data = $request->request->get('data');
will always be empty. What is the problem here?
It was an httpie usage problem as the form flag was necessary, as silex requires the parameters to be form-encoded, yet the default of HTTPie is to pass a JSON object.
$ http --form POST http://localhost:1337 data="hello world"
HTTP/1.1 200 OK
Cache-Control: no-cache
Connection: close
Content-Type: application/json
Date: Wed, 14 Oct 2015 15:04:09 GMT
Host: localhost:1337
X-Powered-By: PHP/5.5.9-1ubuntu4.13
{
"message": "hello world"
}
Just to clarify what kOpernikus said, when you are making a POST request using httpie, use the following syntax:
http --form post :3000/register username="gilbert" password="stackoverflow!"
Alternatively, since forms are for post requests you can leave out post and also abbreviate --form to -f like so:
http -f :3000/register username=gilbert password=stackoverflow!
EDIT (thanks to Aerials)
To pass csrf token as header in the post request do:
http --form POST http://localhost:8000/login/ username=user password=pass X-CSRFToken:assQ$%auxASDLSIAJSd

Redirect to same URL with Lua on Nginx (openresty setup)

I'm looking to modify a request header and redirect it in Lua, I've tried
ngx.redirect("/")
and
ngx.exec("/")
but I'm getting following error:
attempt to call ngx.redirect after sending out the headers
Is there a straightforward way to add a header value and redirect it somewhere else in Lua? In the documentation I didn't find any appropriate directive, Is there a way something like this can be done while still using content_by_lua_file ?
I'm using openresty.
From the redirect method documentation:
Note that this method call terminates the processing of the current request and that it must be called before ngx.send_headers or explicit response body outputs by either ngx.print or ngx.say.
So check that or use another request phase handler such as rewrite_by_lua.
As for setting a header, use ngx.header
E.g:
location /testRedirect {
content_by_lua '
ngx.header["My-header"]= "foo"
return ngx.redirect("http://www.google.com")
';
}
curl http://127.0.0.1/testRedirect
output:
HTTP/1.1 302 Moved Temporarily
Server: openresty
Date: Tue, 30 Jun 2015 17:34:38 GMT
Content-Type: text/html
Content-Length: 154
Connection: keep-alive
My-header: foo
Location: http://www.google.com
<html>
<head><title>302 Found</title></head>
<body bgcolor="white">
<center><h1>302 Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
Note: Most sites will NOT accept custom header coming from a redirect, so consider using a cookie on that case.

Compojure: getting the body from a POST request from which the Content-Type header was missing

Given this snippet:
(defroutes main-routes
(POST "/input/:controller" request
(let [buff (ByteArrayOutputStream.)]
(copy (request :body) buff)
;; --- snip
The value of buff will be a non-empty byte array iff there's the Content-Type header in the request. The value can be nonsencial, the header just has to be there.
However, I need to dump the body (hm... that came out wrong) if the request came without a content type, so that the client can track down the offending upload. (The uploading software is not under my control and its maintainers won't provide anything extra in the headers.)
Thank you for any ideas on how to solve or work around this!
EDIT:
Here are the headers I get from the client:
{
"content-length" "159",
"accept" "*/*",
"host" (snip),
"user-agent" (snip)
}
Plus, I discovered that Ring, using an instance of Java's ServletRequest, fills in the content type with the standard default, x-www-form-urlencoded. I'm now guessing that HTTPParser, which supplies the body through HTTPParser#Input, can't parse it correctly.
I face the same issue. It's definitely one of the middleware not being able to parse the body correctly and transforming :body. The main issue is that the Content-Type suggest the body should be parsable.
Using ngrep, I found out how curl confuses the middleware. The following, while intuitive (or rather sexy) on the command line sends a wrong Content-Type which confuses the middleware:
curl -nd "Unknown error" http://localhost:3000/event/error
T 127.0.0.1:44440 -> 127.0.0.1:3000 [AP]
POST /event/error HTTP/1.1.
Authorization: Basic SzM5Mjg6ODc2NXJkZmdoam5idmNkOQ==.
User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3.
Host: localhost:3000.
Accept: */*.
Content-Length: 13.
Content-Type: application/x-www-form-urlencoded.
.
Unknown error
The following however forces the Content-Type to being opaque and the middleware will not interfere with the :body.
curl -nd "Unknown error" -H "Content-Type: application/data" http://localhost:3000/event/error
T 127.0.0.1:44441 -> 127.0.0.1:3000 [AP]
POST /event/error HTTP/1.1.
Authorization: Basic SzM5Mjg6ODc2NXJkZmdoam5idmNkOQ==.
User-Agent: curl/7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3.
Host: localhost:3000.
Accept: */*.
Content-Type: application/data.
Content-Length: 13.
.
Unknown error
I'm considering replacing the middleware with a more liberal one because even though the request is wrong, I'd still like to be able to decide what to do with the body myself. It's a really weird choice to zero the request body when the request doesn't make sense. I actually think a more correct behavior would be to pass it to an error handler which by default would return a 400 Bad Request or 406 Not Acceptable.
Any thoughts on that? In my case I might propose a patch to Compojure.
According to:
http://mmcgrana.github.com/ring/ring.middleware.content-type-api.html
the default content type is application/octet-stream. Unless you actively support that content type, can't you just check if the content type matches that one, and then dump whatever you need based on that?

Resources