Return the incoming header as is | Nginx - nginx

We have a downstream application which is setting some custom headers to the requests from browser before hitting nginx. nginx serves only static contents.
ie browser >> application A >> nginx
The requirement is that the nginx should be able to return all the headers which it receives as is to the downstream server which would give it back to the browser. by default its returning only the generic headers ( cookies etc, expiry etc ) and not retuning the custom ones sent by the downstream server.
For instance, there is a header with name appnumber which nginx receives with value app01. I tried to explicitly set it with the following rule to set it manually if it exist, but did not help as it throws error that variables are not allowed.
if ($appnumber) {
add_header appnumber $appnumber;
}
Can someone please guide me here?

The request headers are stored under $http_ variable. You could try something like
if ($appnumber) {
add_header appnumber $http_appnumber;
}
Refer http://nginx.org/en/docs/http/ngx_http_core_module.html and nginx - read custom header from upstream server

Related

No ‘Access-Control-Allow-Origin’ header is present using angular + WordPress

when i post the data using POST method in woocommerce api. i am getting cors issue
Access to fetch at 'http://localhost/wordpress/wc-api/v3/customers?oauth_consumer_key=ck_64d88e1fa3516e9f5a06b6053f02976a534d3f8f&oauth_nonce=zsu3ysEnFHhvrZt4Nc7H66Dgu28H20K7&oauth_signature_method=HMAC-SHA256&oauth_timestamp=1562587817&oauth_version=1.0&oauth_signature=KtFxvyQNklUlfCi6rNWyJ0DEJ6AS2ZbwbO44u%2FEqxG4%3D' from origin 'http://localhost:8100' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: It does not have HTTP ok status.
You have to set a Access-Control-Allow-Origin header on each request to the server, if your server is on a different domain than the app on which you are making those requests (the server sets it as a response header). Adding that header tells the system that the external domain "localhost:8100" is allowed to make those requests.
You cannot circumvent this requirement in vanilla browsers, because it is a built in security feature to reduce CORS attacks
PS. different ports on the same domain are considered to be different domains. Thus example.com will get a 401 error, if you are making a request to example.com:8100. Same goes for localhost, or any other domain.
Example code from an Apache2 web server .conf file, that I personally use to set these headers.
SetEnvIf Origin "^http(s)?://(.+\.)?(staging.\xxx\.com|xxx\.com|xxx\.local|xxx\.local:4200|a2\.local)$" origin_is=$0
Header always set Access-Control-Allow-Origin %{origin_is}e env=origin_is
Just replace the xxx.com domains with localhost:8100 or whatever else you need in that array. (if you are using Apache web server)
As a result, the Chrome network tab should have an Access-Control-Allow-Origin header on attached to the request

Override nginx request headers with uwsgi_pass

Is there any way I can get nginx to not forward a specific request header to uwsgi?
I want to enable nginx basic auth, but if the Authorization header gets forwarded to my app it breaks things (for reasons, I won't go into). If it was just a simple proxy_pass I would be able to do proxy_set_header Authorization ""; but I don't think this works with uwsgi_pass and there's no equivalent uwsgi_set_header as far as I can see.
Thanks.
Try hide header and ignore header directives:
uwsgi_hide_header
Syntax: uwsgi_hide_header field; Default: — Context: http, server,
location
By default, nginx does not pass the header fields “Status” and
“X-Accel-...” from the response of a uwsgi server to a client. The
uwsgi_hide_header directive sets additional fields that will not be
passed. If, on the contrary, the passing of fields needs to be
permitted, the uwsgi_pass_header directive can be used.
uwsgi_ignore_headers
Syntax: uwsgi_ignore_headers field ...; Default: —
Context: http, server, location Disables processing of certain
response header fields from the uwsgi server. The following fields can
be ignored: “X-Accel-Redirect”, “X-Accel-Expires”,
“X-Accel-Limit-Rate” (1.1.6), “X-Accel-Buffering” (1.1.6),
“X-Accel-Charset” (1.1.6), “Expires”, “Cache-Control”, “Set-Cookie”
(0.8.44), and “Vary” (1.7.7).
If not disabled, processing of these header fields has the following
effect:
“X-Accel-Expires”, “Expires”, “Cache-Control”, “Set-Cookie”, and
“Vary” set the parameters of response caching; “X-Accel-Redirect”
performs an internal redirect to the specified URI;
“X-Accel-Limit-Rate” sets the rate limit for transmission of a
response to a client; “X-Accel-Buffering” enables or disables
buffering of a response; “X-Accel-Charset” sets the desired charset of
a response.
It's probably too late for you but for anyone who would have the same problem, this answer provides a valid solution.
In this case the Authorization header could be passed by using the following directive:
uwsgi_param HTTP_Authorization "";

Why is an OPTIONS request sent and can I disable it?

I am building a web API. I found whenever I use Chrome to POST, GET to my API, there is always an OPTIONS request sent before the real request, which is quite annoying. Currently, I get the server to ignore any OPTIONS requests. Now my question is what's good to send an OPTIONS request to double the server's load? Is there any way to completely stop the browser from sending OPTIONS requests?
edit 2018-09-13: added some precisions about this pre-flight request and how to avoid it at the end of this reponse.
OPTIONS requests are what we call pre-flight requests in Cross-origin resource sharing (CORS).
They are necessary when you're making requests across different origins in specific situations.
This pre-flight request is made by some browsers as a safety measure to ensure that the request being done is trusted by the server.
Meaning the server understands that the method, origin and headers being sent on the request are safe to act upon.
Your server should not ignore but handle these requests whenever you're attempting to do cross origin requests.
A good resource can be found here http://enable-cors.org/
A way to handle these to get comfortable is to ensure that for any path with OPTIONS method the server sends a response with this header
Access-Control-Allow-Origin: *
This will tell the browser that the server is willing to answer requests from any origin.
For more information on how to add CORS support to your server see the following flowchart
http://www.html5rocks.com/static/images/cors_server_flowchart.png
edit 2018-09-13
CORS OPTIONS request is triggered only in somes cases, as explained in MDN docs:
Some requests don’t trigger a CORS preflight. Those are called “simple requests” in this article, though the Fetch spec (which defines CORS) doesn’t use that term. A request that doesn’t trigger a CORS preflight—a so-called “simple request”—is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (for example, Connection, User-Agent, or any of the other headers with names defined in the Fetch spec as a “forbidden header name”), the only headers which are allowed to be manually set are those which the Fetch spec defines as being a “CORS-safelisted request-header”, which are:
Accept
Accept-Language
Content-Language
Content-Type (but note the additional requirements below)
DPR
Downlink
Save-Data
Viewport-Width
Width
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
No event listeners are registered on any XMLHttpRequestUpload object used in the request; these are accessed using the XMLHttpRequest.upload property.
No ReadableStream object is used in the request.
Have gone through this issue, below is my conclusion to this issue and my solution.
According to the CORS strategy (highly recommend you read about it) You can't just force the browser to stop sending OPTIONS request if it thinks it needs to.
There are two ways you can work around it:
Make sure your request is a "simple request"
Set Access-Control-Max-Age for the OPTIONS request
Simple request
A simple cross-site request is one that meets all the following conditions:
The only allowed methods are:
GET
HEAD
POST
Apart from the headers set automatically by the user agent (e.g. Connection, User-Agent, etc.), the only headers which are allowed to be manually set are:
Accept
Accept-Language
Content-Language
Content-Type
The only allowed values for the Content-Type header are:
application/x-www-form-urlencoded
multipart/form-data
text/plain
A simple request will not cause a pre-flight OPTIONS request.
Set a cache for the OPTIONS check
You can set a Access-Control-Max-Age for the OPTIONS request, so that it will not check the permission again until it is expired.
Access-Control-Max-Age gives the value in seconds for how long the response to the preflight request can be cached for without sending another preflight request.
Limitation Noted
For Chrome, the maximum seconds for Access-Control-Max-Age is 600 which is 10 minutes, according to chrome source code
Access-Control-Max-Age only works for one resource every time, for example, GET requests with same URL path but different queries will be treated as different resources. So the request to the second resource will still trigger a preflight request.
Please refer this answer on the actual need for pre-flighted OPTIONS request: CORS - What is the motivation behind introducing preflight requests?
To disable the OPTIONS request, below conditions must be satisfied for ajax request:
Request does not set custom HTTP headers like 'application/xml' or 'application/json' etc
The request method has to be one of GET, HEAD or POST. If POST, content type should be one of application/x-www-form-urlencoded, multipart/form-data, or text/plain
Reference:
https://developer.mozilla.org/en-US/docs/Web/HTTP/Access_control_CORS
When you have the debug console open and the Disable Cache option turned on, preflight requests will always be sent (i.e. before each and every request). if you don't disable the cache, a pre-flight request will be sent only once (per server)
Yes it's possible to avoid options request. Options request is a preflight request when you send (post) any data to another domain. It's a browser security issue. But we can use another technology: iframe transport layer. I strongly recommend you forget about any CORS configuration and use readymade solution and it will work anywhere.
Take a look here:
https://github.com/jpillora/xdomain
And working example:
http://jpillora.com/xdomain/
For a developer who understands the reason it exists but needs to access an API that doesn't handle OPTIONS calls without auth, I need a temporary answer so I can develop locally until the API owner adds proper SPA CORS support or I get a proxy API up and running.
I found you can disable CORS in Safari and Chrome on a Mac.
Disable same origin policy in Chrome
Chrome: Quit Chrome, open an terminal and paste this command: open /Applications/Google\ Chrome.app --args --disable-web-security --user-data-dir
Safari: Disabling same-origin policy in Safari
If you want to disable the same-origin policy on Safari (I have 9.1.1), then you only need to enable the developer menu, and select "Disable Cross-Origin Restrictions" from the develop menu.
As mentioned in previous posts already, OPTIONS requests are there for a reason. If you have an issue with large response times from your server (e.g. overseas connection) you can also have your browser cache the preflight requests.
Have your server reply with the Access-Control-Max-Age header and for requests that go to the same endpoint the preflight request will have been cached and not occur anymore.
I have solved this problem like.
if($_SERVER['REQUEST_METHOD'] == 'OPTIONS' && ENV == 'devel') {
header('Access-Control-Allow-Origin: *');
header('Access-Control-Allow-Headers: X-Requested-With');
header("HTTP/1.1 200 OK");
die();
}
It is only for development. With this I am waiting 9ms and 500ms and not 8s and 500ms. I can do that because production JS app will be on the same machine as production so there will be no OPTIONS but development is my local.
You can't but you could avoid CORS using JSONP.
you can also use a API Manager (like Open Sources Gravitee.io) to prevent CORS issues between frontend app and backend services by manipulating headers in preflight.
Header used in response to a preflight request to indicate which HTTP headers can be used when making the actual request :
content-type
access-control-allow-header
authorization
x-requested-with
and specify the "allow-origin" = localhost:4200 for example
After spending a whole day and a half trying to work through a similar problem I found it had to do with IIS.
My Web API project was set up as follows:
// WebApiConfig.cs
public static void Register(HttpConfiguration config)
{
var cors = new EnableCorsAttribute("*", "*", "*");
config.EnableCors(cors);
//...
}
I did not have CORS specific config options in the web.config > system.webServer node like I have seen in so many posts
No CORS specific code in the global.asax or in the controller as a decorator
The problem was the app pool settings.
The managed pipeline mode was set to classic (changed it to integrated) and the Identity was set to Network Service (changed it to ApplicationPoolIdentity)
Changing those settings (and refreshing the app pool) fixed it for me.
OPTIONS request is a feature of web browsers, so it's not easy to disable it. But I found a way to redirect it away with proxy. It's useful in case that the service endpoint just cannot handle CORS/OPTIONS yet, maybe still under development, or mal-configured.
Steps:
Setup a reverse proxy for such requests with tools of choice (nginx, YARP, ...)
Create an endpoint just to handle the OPTIONS request. It might be easier to create a normal empty endpoint, and make sure it handles CORS well.
Configure two sets of rules for the proxy. One is to route all OPTIONS requests to the dummy endpoint above. Another to route all other requests to actual endpoint in question.
Update the web site to use proxy instead.
Basically this approach is to cheat browser that OPTIONS request works. Considering CORS is not to enhance security, but to relax the same-origin policy, I hope this trick could work for a while. :)
One solution I have used in the past - lets say your site is on mydomain.com, and you need to make an ajax request to foreigndomain.com
Configure an IIS rewrite from your domain to the foreign domain - e.g.
<rewrite>
<rules>
<rule name="ForeignRewrite" stopProcessing="true">
<match url="^api/v1/(.*)$" />
<action type="Rewrite" url="https://foreigndomain.com/{R:1}" />
</rule>
</rules>
</rewrite>
on your mydomain.com site - you can then make a same origin request, and there's no need for any options request :)
It can be solved in case of use of a proxy that intercept the request and write the appropriate headers.
In the particular case of Varnish these would be the rules:
if (req.http.host == "CUSTOM_URL" ) {
set resp.http.Access-Control-Allow-Origin = "*";
if (req.method == "OPTIONS") {
set resp.http.Access-Control-Max-Age = "1728000";
set resp.http.Access-Control-Allow-Methods = "GET, POST, PUT, DELETE, PATCH, OPTIONS";
set resp.http.Access-Control-Allow-Headers = "Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since";
set resp.http.Content-Length = "0";
set resp.http.Content-Type = "text/plain charset=UTF-8";
set resp.status = 204;
}
}
What worked for me was to import "github.com/gorilla/handlers" and then use it this way:
router := mux.NewRouter()
router.HandleFunc("/config", getConfig).Methods("GET")
router.HandleFunc("/config/emcServer", createEmcServers).Methods("POST")
headersOk := handlers.AllowedHeaders([]string{"X-Requested-With", "Content-Type"})
originsOk := handlers.AllowedOrigins([]string{"*"})
methodsOk := handlers.AllowedMethods([]string{"GET", "HEAD", "POST", "PUT", "OPTIONS"})
log.Fatal(http.ListenAndServe(":" + webServicePort, handlers.CORS(originsOk, headersOk, methodsOk)(router)))
As soon as I executed an Ajax POST request and attaching JSON data to it, Chrome would always add the Content-Type header which was not in my previous AllowedHeaders config.

Nginx proxy_cache_key $request_body is ignored for large request body

I use nginx as a reverse proxy and I would like it to cache POST requests. My back-end is correctly configured to return appropriate cache-control headers for POST requests. In nginx I have configured:
proxy_cache_methods POST;
proxy_cache_key "$request_method$request_uri$request_body";
This works great for small HTTP POST requests. However I started noticing that for large requests (e.g. file uploads) it seems like the $request_body is ignored in the proxy_cache_key. When a form containing a file upload is submitted twice with completely different data, nginx will return the cached result.
What could cause this? How can I configure nginx to use the $request_body (or a hash of $request_body) in the proxy_cache_key even for large POST requests?
So it turns out that when $content_length > client_body_buffer_size,
then the request body is written to a file and the variable $request_body == "".
See also http://mailman.nginx.org/pipermail/nginx/2013-September/040442.html
Rather than using the $request_body within the proxy_cache_key, you may more simply use $content_length.
Of course, it comes with its own limitation, but if you know which query you will receive, it can be also a very interesting workaround.
proxy_cache_key "$scheme$request_method$host$request_uri$content_length";
You may alternatively use $request_body as well to keep the desired behavior for smaller request payload:
proxy_cache_key "$scheme$request_method$host$request_uri$request_body$content_length";

fiddler: how to disable overwrite Header Host

When using Fiddler, it pops up an alert dialog.
Fiddler has detected a protocol violation in session #14.
The Request's Host header did not match the URL's host component.
URL Host: proxy.music.pp.com
Header Host: 119.147.22.41
And it shows that Fiddler changed HTTP Header's host to "proxy.music.pp.com", is there any way to disable Fiddler changing it?
From my book:
Swap the Host Header
When Fiddler gets a request whose URL doesn’t match its Host header, the original Host value is stored in the session flag X-Original-Host and then the Host value is replaced with the host parsed from the URL. The following script, placed inside your FiddlerScript's BeforeRequest function, reverses behavior by routing the request to the host specified by the original Host header.
if (oSession.BitFlags & SessionFlags.ProtocolViolationInRequest)
{
var sOverride = oSession["X-Original-Host"];
if (!String.IsNullOrEmpty(sOverride))
{
oSession["X-overrideHost"] = sOverride;
oSession["ui-backcolor"] = "yellow";
// Be sure to bypass the gateway, otherwise overrideHost doesn't work
oSession.bypassGateway = true;
}
}
You can do this with rules.
Go into Customize rules, and find the function: OnBeforeRequest(oSession: Session)
Then add the following as the last line of that function:
if (oSession.HostnameIs("proxy.music.pp.com")) { oSession.host = "119.147.22.41"; }
I didn't understand what the "url host" was supposed to be - I mean, normally, there is only the Host header in HTTP.
Looking closer, it appears that this violation occurs for an initial proxy "setup" request, which looks like this:
CONNECT targaryen:45633 HTTP/1.1
Host: targaryen
This is where the error makes sense to me.
I couldn't get what I needed for the existing answers, but Eric Law's answer gave me what I needed to fix my issue. I was having a problem where I was calling a web server by IP address, and I had to add a Host header to get the right endpoint. Fiddler was changing the Host header to the IP address I was calling and removing my Host header value, which caused the call to fail. I added a line to Eric's script to solve this. I put the rule into OnBeforeRequest in the Fiddler Rules Script.
if (oSession.BitFlags & SessionFlags.ProtocolViolationInRequest)
{
var sOverride = oSession["X-Original-Host"];
if (!String.IsNullOrEmpty(sOverride))
{
oSession["X-overrideHost"] = sOverride;
oSession["ui-backcolor"] = "yellow";
oSession.oRequest["Host"] = sOverride;
// Be sure to bypass the gateway, otherwise overrideHost doesn't work
oSession.bypassGateway = true;
}
}
Took this Error:
And changed it to this:

Resources