Nginx setting expires header with proxy - nginx

I have the following base Nginx configuration (pre-installed Ghost platform on DigitalOcean droplet):
server {
listen 80;
server_name xxx.com;
client_max_body_size 10M;
location / {
proxy_pass http://localhost:2368/;
proxy_set_header Host $host;
proxy_buffering off;
}
}
Now I tried to set the following expiry header for my assets but without success:
location ~ ^/assets/ {
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
}
Based on the information I've found, Nginx only uses one location path at a time so have to copy the proxy_* parameters inside the assets location block. If I just copy them I get an error (regex with proxy_pass) that can be solved by rewriting the URL before passing it to the proxy. I already did some experiments with that but I don't get it to work either.
Does anyone have an example of how to set expiry headers when a proxy_pass is present? I simply want all files under xxx.com/assets/ to have the proper expiry date.

location /assets/ {
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
proxy_pass http://localhost:2368/assets/;
# or proxy_pass http://localhost:2368;
proxy_set_header Host $host;
proxy_buffering off;
}
Nginx doc for proxy_pass say that:
If the proxy_pass directive is specified with a URI, then when a request is passed to the server, the part of a normalized request URI matching the location is replaced by a URI specified in the directive.
I your case /assets/ get replaced by / (which is an URI). To avoid that either use proxy_pass with URI equal to location prefix (proxy_pass http://localhost:2368/assets/;) or don't use URI at all (proxy_pass http://localhost:2368;). But in latter case nginx will proxy unnormalized URI.

Managed to solve it by using the script below (note the /assets/ after the proxy).
server {
listen 80;
server_name xaviertalpe.be;
client_max_body_size 10M;
location /assets/ {
expires 30d;
add_header Pragma public;
add_header Cache-Control "public";
proxy_pass http://localhost:2368/assets/;
proxy_set_header Host $host;
proxy_buffering off;
}
location / {
proxy_pass http://localhost:2368/;
proxy_set_header Host $host;
proxy_buffering off;
}
}

Related

Correct proxy configuration for nginx server to access rest api

I have a nginx-server configured as follows:
server {
listen 3000;
listen [::]:3000;
server_name .+;
add_header Strict-Transport-Security "max-age=15552000; includeSubDomains" always;
...
index index.html;
root /usr/share/nginx/html;
location /cam/ {
proxy_pass http://cam:8000/;
}
location /api {
proxy_pass https://some_ip:some_port;
proxy_pass_request_headers on;
proxy_set_header X-API-KEY xxxxx;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $server_name;
proxy_ssl_server_name on;
}
location /share/ {
alias /usr/src/share;
}
location / {
try_files $uri $uri/ /index.html;
}
add_header 'Access-Control-Allow-Origin' '*' always;
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS' always;
add_header 'Access-Control-Allow-Headers' 'DNT,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Range' always;
client_max_body_size 1M;
keepalive_timeout 10;
}
For displaying my frontend, this configuration works fine. The proxy for the cam is also working correctly.
I am getting problems in the configuration of the proxy for the api. The api has a sanity-check under "https://some_ip:some_port", which just returns "200: ok". Otherwise I should be able to make request using a fetch request such as
const res = await fetch(`api/subaddress/${system_id.toString()}/base`, {
method: 'POST',
headers: {
'Content-type': 'application/json',
'X-API-KEY': xxxxx,
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Methods': 'HEAD, GET, POST, PUT, PATCH, DELETE',
'Access-Control-Allow-Headers': 'Origin, Content-Type, X-Auth-Token'
},
body: JSON.stringify(data_body),
});
Which, if successfull, returns "202: ok".
If I make such a request using curl, it works just fine (both of the requests), so the api should not be the problem.
At the current configuration, I get a "200: ok", which tells me that I have connected to the api, but the proxy does not take in account my subaddresses.
Besides the given configuration, I tried the following:
location /api {
proxy_pass https://some_ip:some_port$request_uri;
...
}
which always gives me "502: bad gateway"
location /api/ {
proxy_pass https://some_ip:some_port/;
...
}
which always gives me "403: forbidden"
location /api {
rewrite ^/api/(.*) /$1 break;
proxy_pass https://some_ip:some_port/$1;
...
}
which causes nginx to crash, as it tells me that it cannot resolve it. I would like to avoid using a resolver, unless definitly necessary (I have tried it using a resolver before too and it did not work either).
What is the correct way to accomplish this?
I solved this. Apparently the 403: forbidden was caused my some of the configurations of the proxy. After removing those, I did not get that error anymore.
In the end I used
location /api {
rewrite ^/api/(.*) /$1 break;
proxy_pass https://some_ip:some_port/$1;
...
}
which I made work by putting
resolver 1.1.1.1 valid=30s;
at the top of the configuration file.

Nginx | headers with add_header ignored when proxy_pass is used for S3 hosted file

I've a Nginx configuration, where I get certain files from AWS S3 bucket, like call from *.my.api.com/file.js will get the file from X folder in S3.
I've an exceptional domain (like xx.my.api.com) for which I will add the
Cache-Control "no-store, no-cache";
Pragma "no-cache";
headers and for the rest of *.my.api.com the headers will be default (it's cache-control: public now).
On my local environment, the file is hosted on my machine, so the headers are set correctly. However, on production, the headers come as default as cache-control: public.
I've read answers like this saying there should be no trouble with it, but it's not working for me.
Is there anything I'm doing wrong? Is it related to the file being hosted on AWS?
My Nginx configuration is as below:
server {
listen 80;
root /var/xyz/public;
index index.html index.htm;
server_name my.api.com *.my.api.com;
add_header Access-Control-Allow-Origin "*";
if ($http_host ~* "^(.*).my.api.com$"){
set $myName $1;
}
location ~ /myfile.js {
resolver 8.8.8.8;
proxy_buffering off;
proxy_set_header Content-Length "";
proxy_set_header Cookie "";
proxy_method GET;
proxy_pass_request_body off;
proxy_max_temp_file_size 0;
if ($myName = "mySpecialName") {
proxy_pass http://path/to/aws/s3/bucket/file.js;
add_header Cache-Control "no-store, no-cache";
add_header Pragma "no-cache";
add_header X-XSS-Protection "1";
add_header X-Frame-Options "SAMEORIGIN";
add_header X-Content-Type-Options nosniff;
}
if ($query_string !~* "myQueryString=([^&]*)") {
proxy_pass http://path/to/aws/s3/bucket/file.js;
}
if ($query_string ~* "myQueryString=([^&]*)") {
proxy_pass http://path/to/some/other/aws/s3/bucket/file.js;
}
}
}
I've tried:
always
proxy_pass_request_headers on
proxy_set_header
copying the server code and adjusting for xx.my.api.com only
proxy_hide_header (can't be used because of if block)
more_set_headers
but none of them worked.
Any help would be appreciated, thanks in advance.
We've solved it by adding the headers from our DNS panel, which was used for caching the file stored in S3.

How to cache NextJS 10.0 images using NGINX

We would like to launch a NextJS 10 app using NGINX so we use a configuration similar to:
location /_next/static/ {
alias /home/ec2-user/my-app/.next/static/;
expires 1y;
access_log on;
}
It works great, it caches for a year our statics but as we use NextJS images I'm failing to add an expires tag on on-the-fly resized images.
If I do:
location /_next/image/ {
alias /home/ec2-user/my-app/.next/image;
expires 1y;
access_log on;
}
It just returns a 404 on images.
Here is my server part NGINX config :
server {
listen 80;
server_name *.my-website.com;
# root /usr/share/nginx/html;
# root /home/ec2-user/my-app;
charset utf-8;
client_max_body_size 20M;
client_body_buffer_size 20M;
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
underscores_in_headers on;
add_header X-Frame-Options SAMEORIGIN always;
add_header X-Content-Type-Options nosniff always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "same-origin" always;
location = /robots.txt {
proxy_pass https://api.my-website.com/robots.txt;
}
location /_next/static/ {
alias /home/ec2-user/my-app/.next/static/;
expires 1y;
access_log on;
}
location / {
# reverse proxy for merchant next server
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass_request_headers on;
proxy_cache_bypass $http_upgrade;
proxy_buffering off;
}
}
Here is an example how you can rely of upstream Content-Type header to set up the Expires and Cache-Control headers:
map $upstream_http_content_type $expire {
~^image/ 1y; # 'image/*' content type
default off;
}
server {
...
location / {
# reverse proxy for merchant next server
proxy_pass http://localhost:3000;
...
expires $expire;
}
}
The same way you can tune cache control headers for any other content type of proxied response. The $upstream_http_<name> nginx variable is described here.
Update
To add cache control headers only by specific URIs you can use two chained map blocks:
map $uri $expire_by_uri {
~^/_next/image/ 1y;
default off;
}
map $upstream_http_content_type $expire {
~^image/ $expire_by_uri;
default off;
}
And if you don't expect anything but the images from /_next/image/... URIs, you can just use the
map $uri $expire {
~^/_next/image/ 1y;
default off;
}

How to check substring using regular expression in Nginx config?

I try to check some parameters in request. Here is my url:
http://localhost:8080/wms?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&LAYERS=004C0000064F&
STYLES=&WIDTH=256&HEIGHT=256&FORMAT=image%2fjpeg&CRS=EPSG%3a100000&DPI=96&
MAP_RESOLUTION=96&FORMAT_OPTIONS=dpi%3a96&
BBOX=1530569.52624839870259166%2c524135.21126760687911883%2c1531064.27656850102357566%2c524629.96158770937472582
I trying to get REQUEST parameter. Here is my nginx 1.12.1 config:
server {
listen 8080;
server_name 127.0.0.1 localhost;
set $site_backend localhost:56297;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
location /favicon.ico {
error_page 403 404 = #tomcat_static_mapping;
}
location ~* /wms {
internal;
add_header URI $request_uri;
add_header X-debug-message1 "$request_uri" always;
if ($request_uri ~* REQUEST=([^&]*)) {
add_header X-debug-message2 "hi" always;
set $requesttype $1;
}
}
}
And in browser i got header:
X-debug-message1: /wms?SERVICE=WMS&VERSION=1.3.0&REQUEST=GetMap&LAYERS=004C0000064F&STYLES=&WIDTH=256&HEIGHT=256&FORMAT=image%2fjpeg&CRS=EPSG%3a100000&DPI=96&MAP_RESOLUTION=96&FORMAT_OPTIONS=dpi%3a96&BBOX=1530569.52624839870259166%2c524135.21126760687911883%2c1531064.27656850102357566%2c524629.96158770937472582
But not get X-debug-message2 header. I check regular expression here https://rubular.com/ and it's found match and return GetMap as like i want.
What can be wrong here?
Something is not complete / matching in your post. I got X-debug-message2: hi only which does match to how nginx has to behave:
These directives are inherited from the previous level if and only if there are no add_header directives defined on the current level
For more intuitive outcome, use Headers-More module.
more_set_headers "URI: $request_uri";
more_set_headers 'X-debug-message1: "$request_uri"';
location ~* /wms {
if ($request_uri ~* REQUEST=([^&]*)) {
more_set_headers 'X-debug-message2: hi';
set $requesttype $1;
}
}

Nginx - reverse proxy a Ghost blog with /subfolder redirect

I have a working nginx instance with the rules below. But I'm having difficulties pointing all the requests to domain.com/ghost
I tried modifying the location / {} block to location /ghost/ {} but with no success. I just get a 404 from the ghost app. Any suggestions?
server {
listen 80;
server_name domain.com;
root /home//user/ghost/;
index index.php;
# if ($http_host != "domain.com") {
# rewrite ^ http://domain.com$request_uri permanent;
# }
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header Host $host;
proxy_pass http://127.0.0.1:2368;
}
location ~* \.(?:ico|css|js|gif|jpe?g|png|ttf|woff)$ {
access_log off;
expires 30d;
add_header Pragma public;
add_header Cache-Control "public, mustrevalidate, proxy-revalidate";
proxy_pass http://127.0.0.1:2368;
}
location = /robots.txt { access_log off; log_not_found off; }
location = /favicon.ico { access_log off; log_not_found off; }
location ~ /\.ht {
deny all;
}
}
I'm using a regexp location directive for a similar proxy setup. This is the minified configuration file:
worker_processes 1;
pid /path/to/file.pid;
worker_priority 15;
events {
worker_connections 512;
accept_mutex on;
}
http {
server {
error_log /path/to/log/error.log error;
listen 127.0.0.1:9000;
server_name example.com;
location ~* (/ghost) {
expires epoch;
proxy_no_cache 1;
proxy_pass http://localhost:1234;
}
location / {
proxy_pass http://localhost:1234;
}
}
}
Have solved similar problem with other apps which have no support for subfolders. Both apps are built on one platform, so they both tries to work in /fx dir. I had to place one of them in to subfolder /gpms .
The idea is to redirect requests with referer from subfolder to destinations which links outside of subfolder - i just add subfolder to beginning of such uris. It is not ideal, but it works.
Here is my nginx config:
server {
listen 80;
server_name mydomain.com;
location / {
rewrite ^/$ /fx/;
proxy_pass http://127.0.0.1:56943/;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_read_timeout 300;
}
error_log /var/log/nginx/debug.log debug;
set $if_and_hack "";
if ( $http_referer ~ '^http://mydomain.com/gpms/.*$' ) {
set $if_and_hack "refgpms";
}
if ( $uri !~ '^/gpms/.*$' ) {
set $if_and_hack "${if_and_hack}_urinogpms";
}
if ( $if_and_hack = "refgpms_urinogpms" ) {
rewrite ^/(.*)$ http://$host/gpms/$1;
}
location /gpms/ {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cookie_path /fx /;
proxy_pass http://127.0.0.1:12788/fx/;
proxy_redirect default;
}
}
External links will be broken, but it is not critical for me and i guess it may be corrected.
$if_and_hack is for overcome nginx limitation on nested conditions.
By the way i have got a cookies issue, because they was set with path, and hit browser bug with not sending cookies for a new path after redirect, so i just remove path from cookies.
Note full link form in rewrite - this form of rewrite immediately redirects browser to new page, you should not change it to just "/gpms/$1".
As alternative, i guess, it may be possible to use nginx module to inspect html content and modify links. I have not tried this. Or consider to use subdomains instead of subfolders.
Good news! As of version 0.4.0 Ghost now supports subdirectory installation. And there are already people who've figured this out and created tutorials.

Resources