I have the following nginx configuration for one of my virtual servers:
upstream app_example_https {
server 127.0.0.1:1340;
}
proxy_cache_path /Users/jaanus/dev/nginxcache levels=1:2 keys_zone=S3CACHE:10m;
proxy_cache_key "$scheme$request_method$host$request_uri";
server {
listen 0.0.0.0:1338;
server_name localhost;
ssl on;
ssl_certificate /Users/jaanus/dev/devHttpsCert.pem;
ssl_certificate_key /Users/jaanus/dev/devHttpsKey.pem;
location = / {
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host 'something.s3-website-us-east-1.amazonaws.com';
proxy_set_header Authorization '';
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header Set-Cookie;
proxy_ignore_headers Set-Cookie;
proxy_cache S3CACHE;
proxy_cache_valid any 60m;
add_header X-Cached $upstream_cache_status;
proxy_pass http://something.s3-website-us-east-1.amazonaws.com/;
}
location /static/ {
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host 'something.s3-website-us-east-1.amazonaws.com';
proxy_set_header Authorization '';
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header Set-Cookie;
proxy_ignore_headers Set-Cookie;
proxy_cache S3CACHE;
proxy_cache_valid any 60m;
add_header X-Cached $upstream_cache_status;
proxy_pass http://something.s3-website-us-east-1.amazonaws.com/static/;
}
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://app_example_https/;
proxy_redirect off;
}
}
What this does in English:
There’s an nginx frontend which serves requests either from a static Amazon S3 site, or an application server.
All requests to / (site root) and /static are reverse-proxied from Amazon S3. All other requests are reverse-proxied from the application server.
Now, the problem: there are two almost identical Location blocks for the S3. This was the only way how I could make this configuration work, where two specific folders (root and /static) are served from S3, and everything else goes to the application server.
Two almost-identical blocks look dumb and are not scalable. When I add such folders, I don’t want to keep duplicating the blocks.
How do I merge the two locations into one Location block, while keeping everything working the same way?
You could put repeating part into external file and include it.
amazon.inc
proxy_http_version 1.1;
proxy_set_header Connection "";
proxy_set_header Host 'something.s3-website-us-east-1.amazonaws.com';
proxy_set_header Authorization '';
proxy_hide_header x-amz-id-2;
proxy_hide_header x-amz-request-id;
proxy_hide_header Set-Cookie;
proxy_ignore_headers Set-Cookie;
proxy_cache S3CACHE;
proxy_cache_valid any 60m;
add_header X-Cached $upstream_cache_status;
proxy_pass http://something.s3-website-us-east-1.amazonaws.com;
your config
location = / {
include amazon.inc;
}
location /static/ {
include amazon.inc;
}
location / {
# proxy to you app
}
If you prefer to keep all in one file, you could use this trick:
error_page 470 = #amazon;
location = / {
return 470;
}
location /static/ {
return 470;
}
location #amazon {
# proxy to amazon
}
You could use regexp to merge several locations together, but I would not recommend to do that because it's hard to read and understand and is less efficient than simple prefix locations. But, just as an example:
# NOT RECOMMENDED
location ~ ^/($|static/) {
# proxy to amazon
}
Related
I want to disable all caching for a specific folder and all it's descendants in my application. The folder has some files in the root and also other subdirectories.
printscreen of my folder in Windows explorer
This is what I tried to far in my nginx configuration file.
I added two map directives and used them in my server directive.
If I check the response headers for any file with curl like so
curl -I http://example.com/api/folder/index.html
I don't see the added Cache-Control header.
This is my nginx file:
events {
}
http {
include /etc/nginx/mime.types;
map $uri $cache_control {
/api/folder/(l|m|r|s)/ "no-cache, no-store, must-revalidate";
}
map $uri $expire {
/api/folder/(l|m|r|s)/ off;
}
upstream api {
server api:80;
}
upstream web {
server web:80;
}
server {
listen 80;
server_name localhost;
root /usr/share/nginx/html;
index index.html;
expires $expire;
add_header Cache-Control $cache_control;
location /api {
rewrite /api/(.*) /$1 break;
proxy_pass http://api;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
location /swagger {
proxy_pass http://api;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
location / {
proxy_pass http://web;
proxy_set_header X-Real-IP \$remote_addr;
proxy_set_header X-Forwarded-For \$proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
}
}
}
I have below configuration. I only want to authorize / because the only one that has a UI. Other URLs are already apis like /report/, /group/, /delete/ and so on.
upstream gofastdfs{
server localhost:8081;
keepalive 32;
}
server {
listen 8080;
server_name localhost;
gzip on;
gzip_types '*';
location / {
auth_basic "Caution";
auth_basic_user_file /etc/nginx/conf.d/.htpasswd;
keepalive_timeout 620;
proxy_redirect off;
proxy_buffering off;
proxy_pass http://gofastdfs;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
But my configuration authorizes every URL, I only want to limit to /
location / matches any URI that is not handled by some other location block - i.e. it is the default location.
location = / only matches the single URI /.
See this document for details.
You can split your configuration into two location blocks using one with authentication and the other without. Some statements will need to be duplicated between both location blocks, but most can be moved into the outer context.
For example:
keepalive_timeout 620;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host:$server_port;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
location = / {
auth_basic "Caution";
auth_basic_user_file /etc/nginx/conf.d/.htpasswd;
proxy_pass http://gofastdfs;
}
location / {
proxy_pass http://gofastdfs;
}
I have following nginx config. if I remove the cache config for css everything works and all css files load perfectly via reverse proxy. but when I put in cache config for .css that results in 404 for all my css resources:
location ~* \.css {
add_header Cache-Control public;
add_header Pragma public;
add_header Vary Accept-Encoding;
expires 1M;
}
location / {
proxy_pass http://localhost:8080;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
}
Nginx chooses a single location to process a request. That location needs to be complete. See how Nginx processes a request.
Your location ~* \.css block is missing the proxy_pass statement.
The proxy_set_header statements can be moved to the outer block and inherited by both location blocks.
For example:
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Port $server_port;
location / {
proxy_pass http://localhost:8080;
}
location ~* \.css {
add_header Cache-Control public;
add_header Pragma public;
add_header Vary Accept-Encoding;
expires 1M;
proxy_pass http://localhost:8080;
}
I have a NGINX server as front-end cache server and I'd like to disable cache on specific urls.
Here is the configuration on NGINX:
proxy_cache_path /tmp/nginx levels=1:2 keys_zone=my_zone:10m inactive=120m max_size=1000m;
proxy_cache_key "$scheme$request_method$host$request_uri$is_args$args";
server {
listen 10.0.0.45:80 default_server;
server_name proxy2.jjd;
include /etc/nginx/default.d/*.conf;
location / {
client_max_body_size 20m;
proxy_cache my_zone;
proxy_cache_bypass $http_cache_control;
proxy_no_cache $http_pragma $http_authorization $cookie_nocache $arg_nocache;
add_header X-Proxy-Cache-NGINX $upstream_cache_status;
add_header X-Real-IP $remote_addr;
add_header Cache-Control "public";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
}
}
Add the following location to avoid an url:
location ^~ /your-url/ {
add_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Port 443;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:8080;
proxy_read_timeout 90;
proxy_connect_timeout 90;
proxy_redirect off;
}
It just assigns this location to the proxy and doesn't enable caching for it.
As I get it, you just need a nested location with a single string proxy_cache off; inside to disable caching for nested URLs. Like this:
location / {
proxy_cache my_zone;
proxy_cache_bypass $http_cache_control;
// other stuff related to proxying or other processing
location /do/not/cache/this/url/ {
proxy_cache off;
}
}
you can just specify location do proxy_pass only for disable cache
location /will/not/cache {
proxy_pass http://127.0.0.1:8080;
..set_header ..
}
I've got an Elasticsearch cluster plus Logstash and Kibana, and I only want to expose a read-only window into the indexes, with the exception of the index kibana-int so that dashboards can be saved.
I've found a suitable ES proxy config, and I've modified it to use limit_except to disallow write/modify to other indexes, but much of the config is needlessly duplicated. Is there a cleaner way to define this?
upstream elasticsearch {
server es-01.iad.company.com:9200;
server es-02.iad.company.com:9200;
}
server {
listen 9200;
server_name elasticsearch.proxy;
client_max_body_size 50m;
location / {
limit_except GET POST HEAD OPTIONS {
deny all;
}
proxy_pass http://elasticsearch;
proxy_redirect off;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass_header Access-Control-Allow-Origin;
proxy_pass_header Access-Control-Allow-Methods;
proxy_hide_header Access-Control-Allow-Headers;
add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type';
add_header Access-Control-Allow-Credentials true;
}
location /kibana-int/ {
proxy_pass http://elasticsearch;
proxy_redirect off;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_pass_header Access-Control-Allow-Origin;
proxy_pass_header Access-Control-Allow-Methods;
proxy_hide_header Access-Control-Allow-Headers;
add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type';
add_header Access-Control-Allow-Credentials true;
}
}
There are several ways:
Solution 1
You could put repeating config into file and include it.
Your config:
upstream elasticsearch {
server es-01.iad.company.com:9200;
server es-02.iad.company.com:9200;
}
server {
listen 9200;
server_name elasticsearch.proxy;
client_max_body_size 50m;
location / {
limit_except GET POST HEAD OPTIONS {
deny all;
}
include proxy.inc;
}
location /kibana-int/ {
include proxy.inc;
}
}
proxy.inc:
proxy_pass http://elasticsearch;
proxy_redirect off;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_hide_header Access-Control-Allow-Headers;
add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type';
add_header Access-Control-Allow-Credentials true;
Solution 2
Other way is use nginx's directive inheritance.
upstream elasticsearch {
server es-01.iad.company.com:9200;
server es-02.iad.company.com:9200;
}
server {
listen 9200;
server_name elasticsearch.proxy;
client_max_body_size 50m;
proxy_redirect off;
proxy_set_header Connection "";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_hide_header Access-Control-Allow-Headers;
add_header Access-Control-Allow-Headers 'X-Requested-With, Content-Type';
add_header Access-Control-Allow-Credentials true;
location / {
limit_except GET POST HEAD OPTIONS {
deny all;
}
proxy_pass http://elasticsearch;
}
location /kibana-int/ {
proxy_pass http://elasticsearch;
}
}
BTW, your proxy_pass_header directives are needless. Nginx proxies almost all headers by default.