Go http cannot handle HTTP requests with no PATH - http

I am writing a small HTTP server that receives HTTP POSTs from some embedded devices. Unfortunately these devices send malformed POST request that contain no PATH component:
POST HTTP/1.1
Host: 192.168.13.130:8080
Content-Length: 572
Connection: Keep-Alive
<?xml version="1.0"?>
....REST OF XML BODY
Due to this the Go http never passes the request to any of my handlers and always responds with 400 Bad Request.
Since these are embedded devices and changing the way they send the request is not an option I though maybe I could intercept the HTTP requests and if no PATH is present add one (e.g. /) to it before it passes to the SeverMux.
I tried this by creating my own CameraMux but Go always responds with 400 Bad Request even before calling the ServeHTTP() method from my custom ServeMux (see code below).
Is there a way to modify the Request object at some point before Go http responds Bad Request or there is a way to make Go accept the request even if it has no PATH?
package main
import (
"net/http"
"log"
"os"
)
type CameraMux struct {
mux *http.ServeMux
}
func (handler *CameraMux) ServeHTTP(w http.ResponseWriter, r *http.Request) {
// Try to fix URL.Path here but the server never reaches this method.
log.Printf("URL %v\n", r.URL.Path)
handler.mux.ServeHTTP(w, r)
}
func process(path string) error {
log.Printf("Processing %v\n", path)
// Do processing based on path and body
return nil
}
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
path := r.URL.Path[1:]
log.Printf("Processing path %v\n", path)
err := process(path)
if err != nil {
w.WriteHeader(http.StatusBadRequest)
} else {
w.WriteHeader(http.StatusOK)
}
})
err := http.ListenAndServe(":8080", &CameraMux{http.DefaultServeMux})
if err != nil {
log.Println(err)
os.Exit(1)
}
os.Exit(0)
}

The error you are seeing occurs within the request parsing logic, which happens before ServeHTTP is called.
The HTTP request is read from the socket by the ReadRequest function from the net/http package. It will tokenize the first line of the request with an empty URL portion, but then goes on to parse the URL:
if req.URL, err = url.ParseRequestURI(rawurl); err != nil {
return nil, err
}
Unfortunately this function will return an error for an empty URL string, which will in turn aborts the request reading process.
So it doesn't look like there is an easy way to achieve what you're after without modifying the standard library code.

I'm unsure if Go's HTTP parser will allow requests with no URI path element. If it doesn't then you're out of luck. If it does however; you could overwrite the request's path like this:
type FixPath struct {}
func (f *FixPath) ServeHTTP(w http.ResponseWriter, r *http.Request) {
r.RequestURI = "/dummy/path" // fix URI path
http.DefaultServeMux.ServeHTTP(w, r) // forward the fixed request to http.DefaultServeMux
}
func main() {
// register handlers with http.DefaultServeMux through http.Handle or http.HandleFunc, and then...
http.ListenAndServe(":8080", &FixPath{})
}

Related

Go returning http status codes when using rs.cors

When using rs.cors to enable Cors on a go web service, returning http status codes other than 200 (for instance, returning http.StatusTooManyRequests) results in a CORS error:
Access to XMLHttpRequest at 'http://localsrv:3021/test?status=toomany' from origin 'http://localhost:8080' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
Using rs.cors is not a requirement, but I was unable to successfully get CORS working in some cases without it.
Here is some sample code:
package main
import (
"log"
"net/http"
"github.com/gorilla/mux"
"github.com/rs/cors"
)
func main() {
rtr := mux.NewRouter()
rtr.HandleFunc("/test", test).Methods("GET")
handler := cors.Default().Handler(rtr)
if err := http.ListenAndServe(":3021",
(limitMiddleware(handler))); err != nil {
log.Fatalf("unable to start server: %s", err.Error())
}
}
func limitMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
params := r.URL.Query()
if params["status"][0] == "toomany" {
http.Error(w, http.StatusText(http.StatusTooManyRequests), http.StatusTooManyRequests)
return
}
next.ServeHTTP(w, r)
})
}
func test(w http.ResponseWriter, req *http.Request) {
log.Println("test()")
}
JavaScript request that should generate a StatusTooManyRequests ends up with a CORS error instead of the HTTP 429.
axios.get("http://okami:3021/test?status=toomany")
.then(res => {
console.log("stuff http://okami:3021/test result", res);
}); //axios
Is there a way to use rs.cors so that CORS kicks in when there is a CORS issue, and http error codes can return when there is not a CORS issue?
Basically, unless the system returns HTTP 200, CORS kicks in.
This should be simple, but I'm scratching my head on this one.
Let me know if it would be better to abandon rs.cors and work on manually enabling cors instead and I can start another question for that!
The function returned by limitMiddleware executes before the cors middleware (the next argument). Reorder your handlers:
rtr := mux.NewRouter()
rtr.HandleFunc("/test", test).Methods("GET")
handler := cors.Default().Handler(limitMiddleware(rtr))
http.ListenAndServe(":3021", handler)
You need to specify Cors, Please try this code inside your main(){ ... }
port := ":9000"
headers := handlers.AllowedHeaders([]string{"X-Requested-With", "Content-Type", "Authorization"})
methods := handlers.AllowedMethods([]string{"GET", "POST", "PUT", "DELETE"})
origins := handlers.AllowedOrigins([]string{"*"})
err = http.ListenAndServe(port, handlers.CORS(headers, methods, origins))
if err != nil {
log.Errorf("Error starting server: %s\n", err)
os.Exit(1)
}
Is there a way to use rs.cors so that CORS kicks in when there is a CORS issue, and http error codes can return when there is not a CORS issue?
Unless you handle cors, you will get that as error, so the fix of this issue is to fix cors.
Using rs.cors is not a requirement, but I was unable to successfully get CORS working in some cases without it.
you can handle cors by setting http.ResponseWriter Header value. you can do it so
func getGoogle(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "*")
}
or replace "*" with your front end url.

How to send response to the client before executing the calculations that are not required in response

Can we send response(or write on user side)with statement like:
json.NewEncoder(w).Encode("some data")
in api before doing some calculation part which are not required in response but needed to store in database. I am thinking like we can give response in less time to the user and other part of function will be continue to work until the return statement.
Correct me if I am thinking in wrong direction.
One way would be to do your additional work which is not required for the response in another goroutine:
func someHandler(w http.ResponseWriter, r *http.Request) {
go func() {
// Do anything here, this won't delay the response
// But don't touch the writer or request, as they may not be available here
}()
if err := json.NewEncoder(w).Encode("some data"); err != nil {
log.Printf("Error sending response: %v", err)
}
}
Note that in the launched gorotuine you can't use the http.ResponseWriter nor the http.Request, as they are only valid to use until you return from your handler. If you need something from them, you must make a copy of the needed parts before you launch the goroutine.
If you want to complete the additional task before you return from the handler, you can still use a goroutine, and use a sync.WaitGroup to wait for it to complete and only then return from the handler. You may or may not flush the response:
func someHandler(w http.ResponseWriter, r *http.Request) {
wg := &sync.WaitGroup{}
wg.Add(1)
go func() {
defer wg.Done()
// You may use the writer and request here
}()
if err := json.NewEncoder(w).Encode("some data"); err != nil {
log.Printf("Error sending response: %v", err)
}
// Optionally you may flush the data written so far (icnluding HTTP headers)
if flusher, ok := w.(http.Flusher); ok {
flusher.Flush()
}
wg.Wait()
}
Note that here the goroutine is allowed to use the http.ResponseWriter and http.Request, because the handler does not return until the additional task is completed.

Not handling GET in net/http golang

I am trying to turn off handling GET requests in golang.
I just want to handle POST.
Is it possible to do?
Reason for doing so is that i can see more and more memory being allocated by golang whenever i go to localhost:8080 and refresh page multiple times.
Here is my test code:
package main
import (
"fmt"
"net/http"
"encoding/json"
)
type test_struct struct {
Test string
}
var t test_struct
func handlePOST(rw http.ResponseWriter, req *http.Request) {
switch req.Method {
case "POST":
decoder := json.NewDecoder(req.Body)
decoder.Decode(&t)
defer req.Body.Close()
fmt.Println(t.Test)
}
}
func main() {
http.HandleFunc("/", handlePOST)
http.ListenAndServe(":8080", nil)
}
You cannot not handle GET requests, Go's HTTP server (or rather its http.ServeMux) only allows you to specify a path pattern before dispatching the request to your handler. HTTP method related routing can only happen at the handler level.
Note that some external mux libraries allow you to register handlers to specific HTTP methods only, but the decision and routing based on that also happens in "hidden" handlers of those libraries.
What you're doing is the best: simply do nothing in the handler if the HTTP method is not the one you intend to handle, or even better: send back a http.StatusMethodNotAllowed error response:
func myHandler(w http.ResponseWriter, r *http.Request) {
if r.Method != http.MethodPost {
http.Error(w, "Only POST is allowed", http.StatusMethodNotAllowed)
return
}
var t test_struct // Use local var not global, else it's a data race
decoder := json.NewDecoder(r.Body)
if err := decoder.Decode(&t); err != nil {
fmt.Println("Error decoding:", err)
}
fmt.Println(t.Test)
}

Terminating or aborting an HTTP request

What's the way to abort my API serving with some error message?
Link to call my service:
http://creative.test.spoti.io/api/getVastPlayer?add=
{"Json":Json}&host=api0.spoti.io&domain=domain&userAgent=userAgent&mobile=true
To call my service the client need to send a Json and some params.
I want to test if the params that I get are correct, if not I want send a error message.
The response should be a Json Code {"Result":"Result","Error":"error message"}
I tried log.fatal and os.Exit(1) they stop the service, not just the call request. panic aborts the call but it prevents me to send a http.ResponseWriter which is my error message.
I read something about panic, defer, recover but I don't really know how can I use them to solve this problem.
return works:
mobile :=query.Get("mobile")
if mobile=="mobile" {
str:=`{"Resultt":"","Error":"No valide Var"}`
fmt.Fprint(w, str)
fmt.Println("No successfull Operation!!")
return}
But I can use it just in the main function, because in the other functions it exits just the func not the caller function (request).
Terminating the serving of an HTTP request is nothing more than to return from the ServeHTTP() method, e.g.:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// examine incoming params
if !ok {
str := `{"Result":"","Error":"No valide Var"}`
fmt.Fprint(w, str)
return
}
// Do normal API serving
})
panic(http.ListenAndServe(":8080", nil))
Notes:
If the input params of your API service are invalid, you should consider returning an HTTP error code instead of the implied default 200 OK. For this you can use the http.Error() function, for example:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// examine incoming params
if !ok {
http.Error(w, `Invalid input params!`, http.StatusBadRequest)
return
}
// Do normal API serving
})
For a more sophisticated example where you send back JSON data along with the error code:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// examine incoming params
if !ok {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(http.StatusBadRequest)
str := `{"Result":"","Error":"No valide Var"}`
fmt.Fprint(w, str)
return
}
// Do normal API serving
})
Example showing how to propagate "returning"
If the error is detected outside of ServeHTTP(), e.g. in a function that is called from ServeHTTP(), you have to return this error state so that ServeHTTP() can return.
Let's assume you have the following custom type for your required parameters and a function which is responsible to decode them from a request:
type params struct {
// fields for your params
}
func decodeParams(r *http.Request) (*params, error) {
p := new(params)
// decode params, if they are invalid, return an error:
if !ok {
return nil, errors.New("Invalid params")
}
// If everything goes well:
return p, nil
}
Using these:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
p, err := decodeParams(r)
if err != nil {
http.Error(w, `Invalid input params!`, http.StatusBadRequest)
return
}
// Do normal API serving
})
Also see this related question: Golang, how to return in func FROM another func?

How Can I Make the Go HTTP Client NOT Follow Redirects Automatically?

I'm currently writing some software in Go that interacts with a REST API. The REST API endpoint I'm trying to query returns an HTTP 302 redirect along with an HTTP Location header, pointing to a resource URI.
I'm trying to use my Go script to grab the HTTP Location header for later processing.
Here's what I'm currently doing to achieve this functionality:
package main
import (
"errors"
"fmt"
"io/ioutil"
"net/http"
)
var BASE_URL = "https://api.example.com/v1"
var STORMPATH_API_KEY_ID = "xxx"
var STORMPATH_API_KEY_SECRET = "xxx"
func noRedirect(req *http.Request, via []*http.Request) error {
return errors.New("Don't redirect!")
}
func main() {
client := &http.Client{
CheckRedirect: noRedirect
}
req, err := http.NewRequest("GET", BASE_URL+"/tenants/current", nil)
req.SetBasicAuth(EXAMPLE_API_KEY_ID, EXAMPLE_API_KEY_SECRET)
resp, err := client.Do(req)
// If we get here, it means one of two things: either this http request
// actually failed, or we got an http redirect response, and should process it.
if err != nil {
if resp.StatusCode == 302 {
fmt.Println("got redirect")
} else {
panic("HTTP request failed.")
}
}
defer resp.Body.Close()
}
This feels like a bit of a hack to me. By overriding the http.Client's CheckRedirect function, I'm essentially forced to treat HTTP redirects like errors (which they aren't).
I've seen several other places suggesting to use an HTTP transport instead of an HTTP client -- but I'm not sure how to make this work since I need the HTTP Client as I need to use HTTP Basic Auth to communicate with this REST API.
Can any of you tell me a way to make HTTP requests with Basic Authentication -- while not following redirects -- that doesn't involve throwing errors and error handling?
There's a much simpler solution right now:
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
This way, the http package automatically knows: "Ah, I shouldn't follow any redirects", but does not throw any error. From the comment in the source code:
As a special case, if CheckRedirect returns ErrUseLastResponse,
then the most recent response is returned with its body
unclosed, along with a nil error.
Another option, using the client itself, without the RoundTrip:
// create a custom error to know if a redirect happened
var RedirectAttemptedError = errors.New("redirect")
client := &http.Client{}
// return the error, so client won't attempt redirects
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
return RedirectAttemptedError
}
// Work with the client...
resp, err := client.Head(urlToAccess)
// test if we got the custom error
if urlError, ok := err.(*url.Error); ok && urlError.Err == RedirectAttemptedError{
err = nil
}
UPDATE: this solution is for go < 1.7
It is possible, but the solution inverts the problem a little. Here's a sample written up as a golang test.
package redirects
import (
"github.com/codegangsta/martini-contrib/auth"
"github.com/go-martini/martini"
"net/http"
"net/http/httptest"
"testing"
)
func TestBasicAuthRedirect(t *testing.T) {
// Start a test server
server := setupBasicAuthServer()
defer server.Close()
// Set up the HTTP request
req, err := http.NewRequest("GET", server.URL+"/redirect", nil)
req.SetBasicAuth("username", "password")
if err != nil {
t.Fatal(err)
}
transport := http.Transport{}
resp, err := transport.RoundTrip(req)
if err != nil {
t.Fatal(err)
}
// Check if you received the status codes you expect. There may
// status codes other than 200 which are acceptable.
if resp.StatusCode != 200 && resp.StatusCode != 302 {
t.Fatal("Failed with status", resp.Status)
}
t.Log(resp.Header.Get("Location"))
}
// Create an HTTP server that protects a URL using Basic Auth
func setupBasicAuthServer() *httptest.Server {
m := martini.Classic()
m.Use(auth.Basic("username", "password"))
m.Get("/ping", func() string { return "pong" })
m.Get("/redirect", func(w http.ResponseWriter, r *http.Request) {
http.Redirect(w, r, "/ping", 302)
})
server := httptest.NewServer(m)
return server
}
You should be able to put the above code into it's own package called "redirects" and run it after fetching the required dependencies using
mkdir redirects
cd redirects
# Add the above code to a file with an _test.go suffix
go get github.com/codegangsta/martini-contrib/auth
go get github.com/go-martini/martini
go test -v
Hope this helps!
To make request with Basic Auth that does not follow redirect use RoundTrip function that accepts *Request
This code
package main
import (
"fmt"
"io/ioutil"
"net/http"
"os"
)
func main() {
var DefaultTransport http.RoundTripper = &http.Transport{}
req, _ := http.NewRequest("GET", "http://httpbin.org/headers", nil)
req.SetBasicAuth("user", "password")
resp, _ := DefaultTransport.RoundTrip(req)
defer resp.Body.Close()
contents, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Printf("%s", err)
os.Exit(1)
}
fmt.Printf("%s\n", string(contents))
}
outputs
{
"headers": {
"Accept-Encoding": "gzip",
"Authorization": "Basic dXNlcjpwYXNzd29yZA==",
"Connection": "close",
"Host": "httpbin.org",
"User-Agent": "Go 1.1 package http",
"X-Request-Id": "45b512f1-22e9-4e49-8acb-2f017e0a4e35"
}
}
As an addition of top rated answer,
You can control the particle size
func myCheckRedirect(req *http.Request, via []*http.Request, times int) error {
err := fmt.Errorf("redirect policy: stopped after %d times", times)
if len(via) >= times {
return err
}
return nil
}
...
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return myCheckRedirect(req, via, 1)
},
}
ref: https://golangbyexample.com/http-no-redirect-client-golang/

Resources