I am trying to implement a Reverse Proxy in Go that proxies traffic to different hosts based on some tenant embedded in the URL. The implementation looks like this:
type Offloader struct {
tenantHostMap map[string]string // Map a tenant to its host:port
tenantProxyMap map[string](*httputil.ReverseProxy) // Map a tenant to its reverse proxy
}
func (o *Offloader) OnCreate() {
// Tenants Map
o.tenantHostMap = make(map[string]string)
o.tenantProxyMap = make(map[string]*httputil.ReverseProxy)
o.PopulateTenantHostMap()
// Rx
http.HandleFunc("/", o.ServeHTTP)
go http.ListenAndServe(":5555", nil)
}
// ServeHTTP is the callback that is called each time a Http Request is received.
func (o *Offloader) ServeHTTP(w http.ResponseWriter, req *http.Request) {
incomingUrl := req.URL.RequestURI()
tenant := o.GetTenantFromUrl(incomingUrl)
if proxy, ok := o.tenantProxyMap[tenant]; ok {
proxy.ServeHTTP(w, req)
}
if remoteHostAddr, ok := o.tenantHostMap[tenant]; ok {
remoteUrl, err := url.Parse(fmt.Sprintf("http://%s", remoteHostAddr))
if err != nil {
return
}
proxy := httputil.NewSingleHostReverseProxy(remoteUrl)
o.tenantProxyMap[tenant] = proxy
proxy.ServeHTTP(w, req) // non blocking
} else {
panic("Unknown Tenant")
}
}
When receiving a new HTTP request, I get the tenant from the URL. If this is the first time I am seeing this tenant I create a new ReverseProxy, otherwise I try to use the one I created before and stored in the tenantProxyMap.
When I test this, I get the following error:
2022/04/05 12:31:01 http: proxy error: readfrom tcp ****: http: invalid Read on closed Body
2022/04/05 12:31:01 http: superfluous response.WriteHeader call from net/http/httputil.(*ReverseProxy).defaultErrorHandler (reverseproxy.go:190)
If I create a new Reverse Proxy for each request rather than reusing the same proxy, the error doesn't happen.
I thought the proxy is per host and not per request (as the name suggests), so I am wondering why this error happens?
I know I need to protect the maps from concurrent reads/writes however that is irrelevant at the moment.
Thanks,
The problem is that in the scenario where a previous proxy already existed, you first pass the request on to that - and then still recreate the proxy, and again pass the request. In other words: you are making two proxied requests for each incoming request, when the tentantProxyMap is already populated for that tenant.
The ReverseProxy implementation closes the req.Body, so the second time you pass the request on to the proxy, it attempts reading from an already closed body. You're seeing the http: invalid Read on closed Body error as a result.
What you should try is to return after proxying the request, e.g. by adding a return:
if proxy, ok := o.tenantProxyMap[tenant]; ok {
proxy.ServeHTTP(w, req)
return
}
I'm working on my project which includes Angular 9 application using firebase functions. I deployed some function, which should update some values on the Firebase Realtime Database. Unfortunately, I encountered a problem with CORS policy if I use my API.
After adding middleware to my express server, where I added these code, the problem still occurs.
app.post('/changeQuantity', permit(), (request: any, response: any) => {
response.status(200).send({ data: { Message: 'Changing order quantity - success.' } });
})
export default function permit() {
return (req: any, _res: any, next: any) => {
_res.setHeader('Access-Control-Allow-Origin', '*');
_res.setHeader('Access-Control-Allow-Methods', 'POST, GET, OPTIONS');
next();
};
}
I noticed, that I get this error when frequently requesting API.
Here is the error which server returns:
Access to fetch at [here is a link to my API endpoint] from origin [my origin] has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
Did anybody have the same issues with the firebase functions API? Could it be related to frequent requests of my API endpoints from the development environment? Thanks in advance.
Try using response.header to set the CORS properties, like this
app.use((request, response, next) => {
response.header("Access-Control-Allow-Origin", "*");
response.header("Access-Control-Allow-Methods", "GET, POST, OPTIONS");
next();
});
I am having trouble getting access to a custom respoone header being provided by our backend server. Server is written in Go. I am attempting to access it using axios as my HTTP client. I can see the header in the chrome dev-tools console, but I cannot get access to the header through axios.
chrome dev tools response headers. I am looking to get the x-***-token at the bottom
headers I have access to through axios (cors enabled)
I can, however, access the header through axios when I run chrome with cors disabled:
open -n -a /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --args --user-data-dir="/tmp/chrome_dev_test" --disable-web-security
headers I have access to through axios (cors disabled)
This tells me that its not an issue with axios, but with the header configuration I have.
Below is how we have configured our headers, I am trying to get access the x-custom-token header:
func allowCORS(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if origin := r.Header.Get("Origin"); origin != "" && contains(cors, origin) {
w.Header().Set("Access-Control-Allow-Origin", origin)
}
if contains(cors, "*") {
w.Header().Set("Access-Control-Allow-Origin", "*")
}
if r.Method == "OPTIONS" || r.Method == "GET" && r.Header.Get("Access-Control-Request-Method") != "" &&
w.Header().Get("Access-Control-Allow-Origin") != "" {
headers := []string{"Content-Type", "Accept-Encoding", "X-CSRF-Token", "Authorization", "accept", "origin", "Cache-Control", "X-Requested-With", "x-custom-token"}
w.Header().Set("Access-Control-Allow-Headers", strings.Join(headers, ","))
methods := []string{"GET", "HEAD", "POST", "PUT", "DELETE"}
w.Header().Set("Access-Control-Allow-Methods", strings.Join(methods, ","))
expose := []string{"x-custom-token"}
w.Header().Set("Access-Control-Expose-Headers", strings.Join(expose, ","))
return
}
h.ServeHTTP(w, r)
})}
Can anyone tell me what header I have set incorrectly?
Your issue is that you are only exposing for OPTIONS requests.
In your if statement, one of your criteria are r.Header.Get("Access-Control-Request-Method") != "".
This header is typically only ever used in pre-flight requests and so your expose headers are not getting written on GET or other methods.
In addition, your statement returns after writing the headers. This is correct for pre-flight but would cause no content to be sent for other methods.
Solution - adjust your logic to handle exposing headers separately from the pre-flight headers.
I use axios to communicate with my own API (not written in NodeJS).
When I post a non simple request axios always goes directly to the catch block displaying a network error in the console, even with 2 successful Http Requests.
Error: Network Error
Stack trace:
createError#http://localhost:3000/static/js/bundle.js:1634:15
handleError#http://localhost:3000/static/js/bundle.js:1170:14
There is also a CORS warning about a missing header
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at http://127.0.0.1:8080. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).
However it is included in the Options Request!
When I add 'Access-Control-Allow-Origin': '*' in the Axios request headers, the warning is gone, but the browser doesn't fire a Post request after the successful Options request.
For the sake of being complete here are the post request headers.
The code:
postForm = () => {
axios.post("http://127.0.0.1:8080/",
myComplexObj, {
headers: {
//'Access-Control-Allow-Origin': '*',
'Content-Type': 'application/json'
},
timeout: 15000
}
).then(res => {
console.log(res);
alert('success');
})
.catch(function(error) {
//code always end up here
console.log(error);
/*Error: Network Error
Stack trace:
createError#http://localhost:3000/static/js/bundle.js:1634:15
handleError#http://localhost:3000/static/js/bundle.js:1170:14
*/
console.log(error.response); //undefined
console.log(error.response.data); //undefined
}
})
Any help is gladly appreciated.
What I have tried:
Remove the timeout //no change
Remove the Catch block //still no success
Return status code 204 on Options and/or Post requests //no difference
You are confusing because status 200, however, the browser will not allow you to access the response of a CORS request if the Access-Control-Allow-Origin header is missing.
Here are some great articles that explain how CORS works:
https://www.html5rocks.com/en/tutorials/cors/
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
Anyway, I think that you are using Django. So, you need add to settings.py:
CORS_ORIGIN_WHITELIST = (
'localhost:8080',
'localhost'
)
Or wherever you have the axios code.
I'm trying to set a header in my Go web server. I'm using gorilla/mux and net/http packages.
I'd like to set Access-Control-Allow-Origin: * to allow cross domain AJAX.
Here's my Go code:
func saveHandler(w http.ResponseWriter, r *http.Request) {
// do some stuff with the request data
}
func main() {
r := mux.NewRouter()
r.HandleFunc("/save", saveHandler)
http.Handle("/", r)
http.ListenAndServe(":"+port, nil)
}
The net/http package has documentation describing sending http request headers as if it were a client - I'm not exactly sure how to set response headers?
Never mind, I figured it out - I used the Set() method on Header() (doh!)
My handler looks like this now:
func saveHandler(w http.ResponseWriter, r *http.Request) {
// allow cross domain AJAX requests
w.Header().Set("Access-Control-Allow-Origin", "*")
}
Maybe this will help someone as caffeine deprived as myself sometime :)
All of the above answers are wrong because they fail to handle the OPTIONS preflight request, the solution is to override the mux router's interface. See AngularJS $http get request failed with custom header (alllowed in CORS)
func main() {
r := mux.NewRouter()
r.HandleFunc("/save", saveHandler)
http.Handle("/", &MyServer{r})
http.ListenAndServe(":8080", nil);
}
type MyServer struct {
r *mux.Router
}
func (s *MyServer) ServeHTTP(rw http.ResponseWriter, req *http.Request) {
if origin := req.Header.Get("Origin"); origin != "" {
rw.Header().Set("Access-Control-Allow-Origin", origin)
rw.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
rw.Header().Set("Access-Control-Allow-Headers",
"Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
}
// Stop here if its Preflighted OPTIONS request
if req.Method == "OPTIONS" {
return
}
// Lets Gorilla work
s.r.ServeHTTP(rw, req)
}
Do not use '*' for Origin, until You really need a completely public behavior.
As Wikipedia says:
"The value of "*" is special in that it does not allow requests to supply credentials,
meaning HTTP authentication, client-side SSL certificates, nor does it allow cookies
to be sent."
That means, you'll get a lot of errors, especially in Chrome when you'll try to implement for example a simple authentication.
Here is a corrected wrapper:
// Code has not been tested.
func addDefaultHeaders(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
if origin := r.Header.Get("Origin"); origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
}
w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
w.Header().Set("Access-Control-Allow-Headers", "Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token")
w.Header().Set("Access-Control-Allow-Credentials", "true")
fn(w, r)
}
}
And don't forget to reply all these headers to the preflight OPTIONS request.
If you don't want to override your router (if you don't have your app configured in a way that supports this, or want to configure CORS on a route by route basis), add an OPTIONS handler to handle the pre flight request.
Ie, with Gorilla Mux your routes would look like:
accounts := router.Path("/accounts").Subrouter()
accounts.Methods("POST").Handler(AccountsCreate)
accounts.Methods("OPTIONS").Handler(AccountsCreatePreFlight)
Note above that in addition to our POST handler, we're defining a specific OPTIONS method handler.
And then to actual handle the OPTIONS preflight method, you could define AccountsCreatePreFlight like so:
// Check the origin is valid.
origin := r.Header.Get("Origin")
validOrigin, err := validateOrigin(origin)
if err != nil {
return err
}
// If it is, allow CORS.
if validOrigin {
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Set("Access-Control-Allow-Methods", "POST")
w.Header().Set("Access-Control-Allow-Headers",
"Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
}
What really made this all click for me (in addition to actually understanding how CORS works) is that the HTTP Method of a preflight request is different from the HTTP Method of the actual request. To initiate CORS, the browser sends a preflight request with HTTP Method OPTIONS, which you have to handle explicitly in your router, and then, if it receives the appropriate response "Access-Control-Allow-Origin": origin (or "*" for all) from your application, it initiates the actual request.
I also believe that you can only do "*" for standard types of requests (ie: GET), but for others you'll have to explicitly set the origin like I do above.
Set a proper golang middleware, so you can reuse on any endpoint.
Helper Type and Function
type Adapter func(http.Handler) http.Handler
// Adapt h with all specified adapters.
func Adapt(h http.Handler, adapters ...Adapter) http.Handler {
for _, adapter := range adapters {
h = adapter(h)
}
return h
}
Actual middleware
func EnableCORS() Adapter {
return func(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if origin := r.Header.Get("Origin"); origin != "" {
w.Header().Set("Access-Control-Allow-Origin", origin)
w.Header().Set("Access-Control-Allow-Methods", "POST, GET, OPTIONS, PUT, DELETE")
w.Header().Set("Access-Control-Allow-Headers",
"Accept, Content-Type, Content-Length, Accept-Encoding, X-CSRF-Token, Authorization")
}
// Stop here if its Preflighted OPTIONS request
if r.Method == "OPTIONS" {
return
}
h.ServeHTTP(w, r)
})
}
}
Endpoint
REMEBER! Middlewares get applyed on reverse order( ExpectGET() gets fires first)
mux.Handle("/watcher/{action}/{device}",Adapt(api.SerialHandler(mux),
api.EnableCORS(),
api.ExpectGET(),
))
I create wrapper for this case:
func addDefaultHeaders(fn http.HandlerFunc) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Access-Control-Allow-Origin", "*")
fn(w, r)
}
}
I had the same issue as described above the solutions given above are correct, the set up I have is as follows
1) Angularjs for the Client
2) Beego framework for GO server
Please following these points
1) CORS settings must be enabled only on GO server
2) Do NOT add any type of headers in angularJS except for this
.config(['$httpProvider', function($httpProvider) {
$httpProvider.defaults.useXDomain = true;
delete $httpProvider.defaults.headers.common['X-Requested-With'];
}])
In you GO server add the CORS settings before the request starts to get processed so that the preflight request receives a 200 OK after which the the OPTIONS method will get converted to GET,POST,PUT or what ever is your request type.
I know this is a different twist on the answer, but isn't this more of a concern for a web server? For example, nginx, could help.
The ngx_http_headers_module module allows adding the “Expires” and “Cache-Control” header fields, and arbitrary fields, to a response header
...
location ~ ^<REGXP MATCHING CORS ROUTES> {
add_header Access-Control-Allow-Methods POST
...
}
...
Adding nginx in front of your go service in production seems wise. It provides a lot more feature for authorizing, logging,and modifying requests. Also, it gives the ability to control who has access to your service and not only that but one can specify different behavior for specific locations in your app, as demonstrated above.
I could go on about why to use a web server with your go api, but I think that's a topic for another discussion.