I have setup a server on AWS everything working fine, i am able to fetch data from server but unable to POST, POST data is continuously giving 403 forbidden response,
i have check nginx logs, call never even reach there, but GET request are shows up on access.log.
error.log are empty and nothing happening there.
server {
listen 80;
listen [::]:80 default_server;
server_name example.com www.example.com;
location / {
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $http_host;
proxy_pass "http://127.0.0.1:3004";
}
}
and yes there is a Load Balancer which is running on 443 port and forward to port 80 on the instance.
I get it after looking all the way arounds, there was a rule defined on Load Balancer, post call methoud be defined in lower case like post not POST.
I was getting 403 Forbidden errors from the AWS Load Balancer WAF when the body of my POST request contained "http://127.0.0.1". When I change it to "http://localhost" instead, it doesn't fail. That's pretty lame.
Related
I am configuring Nginx load balance with Nginx upstream module, configuration as follow:
upstream load {
server loadapi.example.com;
server loadapi.anotherdomain.com down;
}
server {
listen 80;
server_name api.example.com;
location / {
proxy_pass http://load;
# proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
server {
listen 80;
server_name loadapi.example.com;
root /disk/projects/load/loadapi;
index index.html index.htm index.shtml index.php;
...
...
error_page 404 /404.html;
}
Notice that the api.example.com and loadapi.example.com are on the same server. loadapi.anotherdomain.com is resolved to another server which provides the same service.
Everything works fine with loadapi.anotherdomain.com, which are on another server.
But when I use the loadapi.example.com as the backend, it seems that Nginx cannot handle it correctly. I can get my service up and running on loadapi.example.com. But it is not working with the upstream.(look like Nginx cannot resolve the subdomain name correctly).
any advice? thx in advance.
nginx uses the Host header to determine which server block to use to process a request.
When the request passes through the proxy_pass http://load; statement, the Host header is set to the value load by default.
To make nginx choose the server block containing the server_name loadapi.example.com; statement, it either needs to be the default_server server, or include the name load in its server_name, or set the Host header using:
proxy_set_header Host loadapi.example.com;
Of course, using upstream for load balancing means that both servers receive the same value for the Host header, and must both respond correctly to it.
See this document for more.
As a prefix, I have been using the following stack for some time with great success:
NGINX - web proxy
SSL - configured in nginx
Pyramid web application, served by gunicorn
The above combo works great, here is a working configuration.
server {
# listen on port 80
listen 80;
server_name portalapi.example.com;
# Forward all traffic to SSL
return 301 https://www.portalapi.example.com$request_uri;
}
server {
# listen on port 80
listen 80;
server_name www.portalapi.example.com;
# Forward all traffic to SSL
return 301 https://www.portalapi.example.com$request_uri;
}
#ssl server
server {
listen 443 ssl;
ssl on;
ssl_certificate /usr/local/etc/letsencrypt/live/portalapi.example.com/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/portalapi.example.com/privkey.pem;
server_name www.portalapi.example.com;
client_max_body_size 10M;
client_body_buffer_size 128k;
location ~ /.well-known/acme-challenge/ {
root /usr/local/www/nginx/portalapi;
allow all;
}
location / {
proxy_set_header Host $host;
proxy_pass http://10.1.1.16:8005;
#proxy_intercept_errors on;
allow all;
}
error_page 404 500 502 503 504 /index.html;
location = / {
root /home/luke/ecom2/dist;
}
}
Now, this is how I serve my public facing apps, it works very well. For all my internal applications, I used to simply direct users to an internal domain example: http://subdomain.company.domain , again this worked well for a long time.
Now in the wake of KRACK attack although we have some very thorough firewall rules to prevent a lot of attacks, I want to force all internal traffic through SSL, and I don't want to use a self signed certificate, I want to use lets encrypt so I can auto-renew certificates which makes administration much easier (and cheaper).
In order to use lets encrypt, I need to have a public facing DNS and server to perform the ACME challenge (for auto renewing). Now again this was a very easy thing to setup in nginx, and the below config works perfectly for serving static content:
What it does is if a user from the internet accesses intranet.example.com it simply shows a forbidden message. However, if a local user tries, they get forwarded to intranet.example.com:8002 and the port 8002 is only available locally, so there is no way external users can access a webpage on this site
geo $local_user {
192.168.155.0/24 0;
172.16.10.0/28 1;
172.16.155.0/24 1;
}
server {
listen 80;
server_name intranet.example.com;
client_max_body_size 4M;
client_body_buffer_size 128k;
# Space for lets encrypt to perform challenges
location ~ /\.well-known/ {
root /usr/local/www/nginx/intranet;
}
if ($local_user) {
# If user is local, redirect them to SSL proxy only available locally
return 301 https://intranet.example.com:8002$request_uri;
}
# Default block all non local users see
location / {
root /home/luke/forbidden_html;
index index.html;
}
# This server block is only available to local users inside geo $local_user
# this block listens on an internal port only, so it is never availble to
# external networks
server {
listen 8002 default ssl; # listen on a port only accessible locally
server_name intranet.example.com;
ssl_certificate /usr/local/etc/letsencrypt/live/intranet.example.com/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/intranet.example.com/privkey.pem;
client_max_body_size 4M;
client_body_buffer_size 128k;
location / {
allow 192.168.155.0/24;
allow 172.16.10.0/28; # also add in allow/deny rules in this block (extra security)
allow 172.16.155.0/24;
root /home/luke/ecom2/dist;
index index.html;
deny all;
}
}
Now, here comes the pyramid/nginx marrying problem, if I use the same above configuration, but have the below settings for my server on 8002:
server {
listen 8002 default ssl; # listen on a port only accessible locally
server_name intranet.example.com;
ssl_certificate /usr/local/etc/letsencrypt/live/intranet.example.com/fullchain.pem;
ssl_certificate_key /usr/local/etc/letsencrypt/live/intranet.example.com/privkey.pem;
client_max_body_size 4M;
client_body_buffer_size 128k;
location / {
allow 192.168.155.0/24;
allow 172.16.10.0/28; # also add in allow/deny rules in this block (extra security)
allow 172.16.155.0/24;
# Forward all requests to python application server
proxy_set_header Host $host;
proxy_pass http://10.1.1.16:6543;
proxy_intercept_errors on;
deny all;
}
}
I run into all sorts of problems, first off inside pyramid I was using the following code in my views/templates
request.route_url # get route url for desired function
Now using request.route_url with the above settings should cause https://intranet.example.com:8002/login to route tohttps://intranet.example.com:8002/welcome but in reality, this setup would forward a user to: http://intranet.example.com/welcome Again this is not correct.
And if I use route_url with the NGINX proxy setting:
proxy_set_header Host $http_host;
I get the error: NGINX to return a 400 error:
400: The plain HTTP request was sent to HTTPS port
And a request to: https://intranet.example.com:8002/ gets reverted to: http://intranet.example.com/login (omitting port and https)
Then I used the same nginx settings (header $htto), but thought I would change to using:
request.route_path
My theory was this should force everything to stay on the same url prefix, and just forward a user from https://intranet.example.com:8002/login to https://intranet.example.com:8002/welcome but in reality, this setup performed the same way as using route_url.
proxy_set_header Host $http_host;
I then get an error when navigating to https://intranet.example.com:8002
400: The plain HTTP request was sent to HTTPS port
And a request to: https://intranet.example.com:8002/ gets reverted to: http://intranet.example.com/login (omitting port and https)
Can anyone assist with the correct setup in order for me to serve my application on https://intranet.example.com:8002
EDIT:
Have also tried:
location / {
allow 192.168.155.0/24;
allow 172.16.10.0/28; # also add in allow/deny rules in this block (extra security)
allow 172.16.155.0/24;
# Forward all requests to python application server
proxy_set_header Host $host:$server_port;
proxy_pass http://10.1.1.16:8002;
proxy_intercept_errors on;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
# root /home/luke/ecom2/dist;
# index index.html;
deny all;
}
Which gives the same result.
I’ve checked a similar configuration and your last example seems correct,
at least for a simplistic gunicorn/pyramid app combination.
Seems something is missing in your puzzle )
Here’s my code (I’m new to Pyramid so something might be done better)
helloworld.py
from pyramid.config import Configurator
from pyramid.renderers import render_to_response
def main(request):
return render_to_response('templates:test.pt', {}, request=request)
with Configurator() as config:
config.add_route('main', '/')
config.add_view(main, route_name='main')
config.include('pyramid_chameleon')
app = config.make_wsgi_app()
templates/test.pt
<html>
<body>
Route url: ${request.route_url('main')}
</body>
</html>
My nginx config
server {
listen 80;
server_name pyramid.lan;
location / {
return 301 https://$server_name:8002$request_uri;
}
}
server {
listen 8002;
server_name pyramid.lan;
ssl on;
ssl_certificate /usr/local/etc/nginx/cert/server.crt;
ssl_certificate_key /usr/local/etc/nginx/cert/server.key;
location / {
proxy_set_header Host $host:$server_port;
proxy_pass http://127.0.0.1:5678;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
This is how I run gunicorn:
gunicorn -w 1 -b 127.0.0.1:5678 helloworld:app
And yes, it works:
$ curl --insecure https://pyramid.lan:8002/
<html>
<body>
Route url: https://pyramid.lan:8002/
</body>
</html>
$ curl -D - http://pyramid.lan
HTTP/1.1 301 Moved Permanently
Server: nginx/1.12.2
Date: Thu, 02 Nov 2017 20:41:50 GMT
Content-Type: text/html
Content-Length: 185
Connection: keep-alive
Location: https://pyramid.lan:8002/
Lets figure out what might go wrong in your case
http 400 usually pops up when you go over httP instead of httpS to a server awaiting httpS requests. If there’s no typo in the post and it indeed occurs when you navigate to https://intranet.example.com:8002 it would be nice to see a curl request showing this and a tcpdump showing what’s happening. Actually you can easily reproduce it by simply typing http://intranet.example.com:8002
another idea is that you’re doing a redirect from your app and the link gets broken when the redirect occurs. I better description on how the user may navigate from https://intranet.example.com:8002/login to .../welcome would be helpful
one more idea is that your app is not that simple and you use some middlewares / customization that makes the default logic work differently and your X-Forwarded-Proto header gets ignored - in this case the behavior would be just as you described
The issue here is, obviously, the missing port within the Location directives that your backend produces.
Now, why is the port missing? Most certainly, because of the following code:
proxy_set_header Host $host;
Note that $host itself does not contain $server_port, unlike $http_host, so, your backend would have no way of knowing which port you meant if you just use $host all by itself.
Note that proxy_redirect default of default expects Location to correspond with the value from proxy_pass in order to do its magic (according to documentation), so, your explicit header setting likely interferes with such logic.
As such, from the nginx point of view, I see multiple possible independent solutions:
remove proxy_set_header Host, and let proxy_redirect do its magic;
set proxy_set_header Host appropriately, to include the port number, e.g., using $host:$server_port or $http_host as you see fit (if that doesn't work, then perhaps the deficiency is actually within your upstream app itself, but fear not -- read below);
provide a custom proxy_redirect setting, e.g., proxy_redirect https://pyramid.lan/ / (equivalent to proxy_redirect https://pyramid.lan/ https://pyramid.lan:8002/), which will ensure that all the Location responses will have the proper port; the only way this wouldn't work is if your upstream does non-HTTP redirects with the missing port.
My app runs on Google Compute Engine. Nginx used as a proxy server. Nginx was configured to use SSL. Below is the content of /etc/nginx/sites-available/default:
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name mywebapp.com;
return 301 https://$server_name$request_uri;
}
server {
listen 443 ssl http2 default_server;
listen [::]:443 ssl http2 default_server;
include snippets/ssl-mywebapp.com.conf;
include snippets/ssl-params.conf;
root /home/me/MyWebApp/wwwroot;
location /.well-known/ {
}
location / {
proxy_pass http://localhost:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
In Startup.cs I have:
app.UseGoogleAuthentication(new GoogleOptions()
{
ClientId = Configuration["Authentication:Google:ClientId"],
ClientSecret = Configuration["Authentication:Google:ClientSecret"],
});
Now in Google Cloud Platform I need to specify Authorized redirect URIs. If I enter the following, my web app works as expected:
http://mywebapp.com/signin-google
But, it won't work if https is used; browser displays the following error:
The redirect URI in the request, http://mywebapp.com/signin-google, does
not match the ones authorized for the OAuth client.
In this case, is it safe to use http as authorized redirect uri? What configuration do I need if I want it to be https?
This happens because your application which is running behind a reverse proxy server doesn't have any idea that originally request came over HTTPS.
SSL/TLS Termination Proxy
The configuration of the reverse proxy described in the question is called SSL/TLS Termination reverse proxy. That means that secure traffic is established between a client and a proxy server. The proxy server decrypts a request and then forwards it to an application over HTTP protocol.
The issue with this configuration is that an application behind it is not aware that client sent request over HTTPS. So when it comes to redirect to itself it uses HttpContext.Request.Scheme, HttpContext.Request.Host and HttpContext.Request.Port to build a valid URL for redirect.
X-Forwarded-* HTTP Headers
This is where X-Forwarded-* headers come into play. To let the application know that request is originally coming through a proxy server over HTTPS we have to configure the proxy server to set X-Forwarded-For and X-Forwarded-Proto HTTP headers.
location / {
proxy_pass http://localhost:5000;
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
OK, now if we get back to ASP.NET Core application and take a look at incoming HTTP request we will see both X-Forwarded-* headers set, however a redirect URL still uses HTTP scheme.
Forwarded Headers Middleware
Basically this middleware overrides HttpContext.Request.Scheme and HttpContext.Connection.RemoteIpAddress to values which were provided by X-Forwarded-Proto and X-Forwarded-For headers appropriately. To make it happen let's add it to pipeline by adding the following line somewhere in the beginning of the Startup.Configure() method.
var forwardedHeadersOptions = new ForwardedHeadersOptions
{
ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto,
RequireHeaderSymmetry = false
};
forwardedHeadersOptions.KnownNetworks.Clear();
forwardedHeadersOptions.KnownProxies.Clear();
app.UseForwardedHeaders(forwardedHeadersOptions);
This should eventually make your application construct valid URLs with HTTPS scheme.
My Story
The code above looks different to what Microsoft suggests. If we take a look in documentation their code looks a bit shorter:
app.UseForwardedHeaders(new ForwardedHeadersOptions
{
ForwardedHeaders = ForwardedHeaders.XForwardedFor | ForwardedHeaders.XForwardedProto
});
However this didn't work for me. Also according to the comments under this issue I'm not alone.
I have a nginx set up as reverse proxy for ASP.NET Core application running in Docker container. It became more complicated after I put everything behind Amazon Load Balancer (ELB).
I followed advice from the documentation first, but it didn't work for me. I have got the following warning in my app:
Parameter count mismatch between X-Forwarded-For and X-Forwarded-Proto
Then I looked at my X-Forwarded-* headers and realized that they had different length. X-Forwarded-For header was containing 2 records (comma separated IP addresses), while X-Forwarded-Proto only one record https. This is how I came up to setting the property RequireHeaderSymmetry to false.
Well, I got rid of 'Parameter count...' warning message, but immediately after that I faced another odd debug message:
Unknown proxy: 172.17.0.6:44624
After looking into the source code of ForwardedHeadersMiddleware I have finally figured out that I have to either clean up both KnownNetworks and KnownProxies collections of the ForwardedHeadersOptions or add my docker network 172.17.0.1/16 to the list of known networks. Right after that I have finally got it working.
PS: For those who sets up a SSL/TLS termination on load balancer (e.g. Amazon Load Balancer or ELB) DON'T set header X-Forwarded-Proto in nginx configuration. This will override correct https value which came from load balance to the http scheme and redirect url will be wrong. I have not found yet how to just append scheme used in nginx to the header instead of overriding it.
For apache users, need to add only one header:
RequestHeader set X-Forwarded-Proto "https"
Firstly need to be sure that mod_headers is enabled.
I have a web app running on port 8081. I want to use nginx to redirect incoming http traffic to this port. So I have this configuration
server {
listen 80;
server_name myserver.thing.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $http_host;
proxy_pass http://127.0.0.1:8081;
}
}
When I try to open the address, I get a ERR_CONNECTION_REFUSED error.
However, if I change the port to, say, 81 and connect there, everything goes smoothly. I don't understand why. There is nothing running on the machine, only nginx and Artifactory. Firewall is turned off for the time being.
Is there a problem with interacting on port 80? What can cause this?
EDIT
I have been using this nginx a few days ago to redirect from 80 to 443 and then from 443 to 8081. (http -> https -> 8081). For debugging purposes now I want to ignore https.
However, for some reason, the port 80 seems to be still redirected to 443, despite no such config being there for nginx.
...
Turns out there must have been a cached redirect in Chrome (originally when the web was visited on port 80 I returned a 301 redirect to port 443. This must've been saved, because when I open an incognito window, it works as expected.
I am struggling to open a connection from https://siteA.com to https://siteB.com using Meteor's DDP.connect() where both servers reside behind nginx reverse proxies forwarding from http to https.
Things work fine in development. In production, when I run DDP.connect('siteB.com') in the console of siteA, I receive:
Mixed Content: The page at 'https://siteA.com/' was loaded over HTTPS, but requested an insecure XMLHttpRequest endpoint 'http://siteB.com/sockjs/info?cb=zw6j36l90y'. This request has been blocked; the content must be served over HTTPS.
In my current nginx configuration for siteB, I have the following (in relevant part, lmk if more is required):
server {
listen 80 default_server;
location / {
rewrite ^ https://$server_name$request_uri? permanent;
}
}
server {
listen 443 ssl spdy;
add_header Access-Control-Allow-Origin 'https://siteA.com';
proxy_pass http://localhost:3000;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Nginx-Proxy true;
proxy_redirect off;
}
I get that it's complaining that the request terminates in an http endpoint (http://localhost:3000) but I'm not sure what to do about it.
If I change siteB's nginx config to proxy_pass https://localhost:3000 it results in a 502 Bad Gateway.
I have tried things both with and without Meteor's force-ssl package on siteB.
Neither site includes Meteor's browser-policy package - based on my reading the default setting without the package should allow me to connect anywhere as is.
I also tried DDP.connect("ws://siteB.com"); but this resulted in:
XMLHttpRequest cannot load ws://siteB.com/sockjs/info?cb=9lahswe7_9. Cross origin requests are only supported for protocol schemes: http, data, chrome, chrome-extension, https, chrome-extension-resource.
Should I be listening on a different port? Is there an nginx config setting to whitelist this request? Any help is appreciated.
After some initial googling here a small hint: Mixed Content: The page at 'XYZ' was loaded over HTTPS, but requested an insecure over https
...even though it talks about stylesheets not XMLHttpRequest it is definately related