Nginx is not caching the path I setup to cache - nginx

proxy_cache_path /tmp/nginx_team_alert_cache keys_zone=team_alerts:10m levels=1:2 max_size=1g use_temp_path=off;
server{
...
location /api/timeentry/timeentry/team_alerts/ {
proxy_cache team_alerts;
proxy_ignore_headers Cache-Control Set-Cookie;
proxy_hide_header "Set-Cookie";
proxy_cache_valid 200 5s;
proxy_cache_key $scheme$host$request_method$request_uri;
proxy_buffering on;
add_header X-Cached $upstream_cache_status;
include uwsgi_params;
uwsgi_pass unix:/tmp/app.sock;
}
}
I have been searching on stackoverflow etc and added all the recommended options but still not caching.

I just realised I am uwsgi_pass so the proxy_* directives won't work, simply replace proxy_* to uwsgi_* worked

Related

Different proxy_cache_valid depending on request_uri for nginx

I'm using nginx as a cache for googleapis.com. Currently all responses are cached for 5m:
proxy_cache_path /var/cache/nginx/xxx_cache keys_zone=xxx_cache:10m;
server {
location ~ /blog/ {
proxy_pass https://www.googleapis.com/blogger/v3/blogs/;
proxy_cache xxx_cache;
proxy_cache_lock on;
proxy_cache_valid 5m;
}
I'd like to change this interval depending on the request_uri. Defining a $proxy_cache_valid variable via the map directive, and using it for proxy_cache_valid fails with invalid time value "$proxy_cache_valid" in ....
map $request_uri $proxy_cache_valid {
default 5m;
~^/blog/[0-9]+/posts/[0-9]+ 1h;
}
proxy_cache_path /var/cache/nginx/xxx_cache keys_zone=xxx_cache:10m;
server {
location ~ /blog/ {
proxy_pass https://www.googleapis.com/blogger/v3/blogs/;
proxy_cache xxx_cache;
proxy_cache_lock on;
proxy_cache_valid $proxy_cache_valid;
}
How can I realise this in nginx (nginx version: nginx/1.16.1)?

How to cache NextJS 10.0 images using NGINX

We would like to launch a NextJS 10 app using NGINX so we use a configuration similar to:
location /_next/static/ {
alias /home/ec2-user/my-app/.next/static/;
expires 1y;
access_log on;
}
It works great, it caches for a year our statics but as we use NextJS images I'm failing to add an expires tag on on-the-fly resized images.
If I do:
location /_next/image/ {
alias /home/ec2-user/my-app/.next/image;
expires 1y;
access_log on;
}
It just returns a 404 on images.
Here is my server part NGINX config :
server {
listen 80;
server_name *.my-website.com;
# root /usr/share/nginx/html;
# root /home/ec2-user/my-app;
charset utf-8;
client_max_body_size 20M;
client_body_buffer_size 20M;
proxy_connect_timeout 600;
proxy_send_timeout 600;
proxy_read_timeout 600;
send_timeout 600;
underscores_in_headers on;
add_header X-Frame-Options SAMEORIGIN always;
add_header X-Content-Type-Options nosniff always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Referrer-Policy "same-origin" always;
location = /robots.txt {
proxy_pass https://api.my-website.com/robots.txt;
}
location /_next/static/ {
alias /home/ec2-user/my-app/.next/static/;
expires 1y;
access_log on;
}
location / {
# reverse proxy for merchant next server
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass_request_headers on;
proxy_cache_bypass $http_upgrade;
proxy_buffering off;
}
}
Here is an example how you can rely of upstream Content-Type header to set up the Expires and Cache-Control headers:
map $upstream_http_content_type $expire {
~^image/ 1y; # 'image/*' content type
default off;
}
server {
...
location / {
# reverse proxy for merchant next server
proxy_pass http://localhost:3000;
...
expires $expire;
}
}
The same way you can tune cache control headers for any other content type of proxied response. The $upstream_http_<name> nginx variable is described here.
Update
To add cache control headers only by specific URIs you can use two chained map blocks:
map $uri $expire_by_uri {
~^/_next/image/ 1y;
default off;
}
map $upstream_http_content_type $expire {
~^image/ $expire_by_uri;
default off;
}
And if you don't expect anything but the images from /_next/image/... URIs, you can just use the
map $uri $expire {
~^/_next/image/ 1y;
default off;
}

Nginx Static content caching proxy_cache_bypass proxy_no_cache

I have a problem with using nginx as a load balancer. I could configure it to work as a load balancer but I don't how to make it cache static contents from the proxied servers in the backend such as html,css,js, etc... This means I want nginx to
weather to cache or not based on the content of the response from the backend servers if it changed to bypass cache and send requests to the backend and if not to serve from cache. I tried and seached a lot in the internet to make it using many directives such as proxy_cache_bypass and proxy_no_cache but I couldn't. Is there any means to do this if anyone has experience in such topic. these are the configurations:
upstream backend {
server www.webserver1.com:443 max_fails=3 fail_timeout=15s;
server www.webserver2.com:443 max_fails=3 fail_timeout=15s;
}
server {
listen 443 ssl;
rewrite_log on;
error_log /var/log/nginx/lb.error.log;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Proxy-Cache $upstream_cache_status;
ssl_certificate /etc/nginx/client.crt;
ssl_certificate_key /etc/nginx/client.key;
ssl on;
location / {
proxy_cache backcache;
#proxy_cache_methods GET HEAD POST;
#proxy_cache_bypass $cookie_nocache $arg_nocache;
#proxy_no_cache $cookie_nocache $arg_nocache;
proxy_cache_min_uses 1;
#proxy_cache_revalidate on;
#proxy_cache_valid 200 4m;
proxy_cache_lock on;
proxy_cache_background_update on;
add_header X-Proxy-Cache $upstream_cache_status;
proxy_pass https://backend;
}
}
server {
listen 80 ;
if ($http_x_forwarded_proto != 'https') {
rewrite ^(.*) https://$host$1 redirect;
}
}
these are the contents of a config. file under /etc/nginx/conf.d/ which is included in the main config. file which is /etc/nginx/nginx.conf and also those 2 lines are in the main config. file :
proxy_cache_path /var/lib/nginx/cache keys_zone=backcache:20m max_size=100m;
proxy_cache_key "$scheme$request_method$host$request_uri$is_args$args$cookie_user";
Your backend servers could be the root cause of that problem, if those servers were improperly configured. For example sending Cache-Control headers on requests to static files.
According to that docs by default, NGINX respects the Cache-Control headers from origin servers. It does not cache responses with Cache-Control set to Private, No-Cache, or No-Store or with Set-Cookie in the response header.
You can permanently change this behavior by adding those directives:
proxy_ignore_headers Cache-Control;
proxy_cache_valid any 30m;
So the config will look like:
location / {
proxy_cache backcache;
proxy_cache_revalidate on;
proxy_cache_min_uses 3;
proxy_cache_valid 200 302 10m;;
proxy_cache_lock on;
proxy_cache_background_update on;
proxy_ignore_headers Cache-Control;
proxy_cache_valid any 30m;
add_header X-Proxy-Cache $upstream_cache_status;
proxy_pass https://backend;
}
Hope it will help you to figure out.

nginx caching few upstreams on same server

I'm trying to test to build a nginx server to cache few servers.
My nginx conf is like that :
...
http {
upstream srv1 {
ip_hash;
server srv1.domain1.fr:443;
}
upstream srv2 {
ip_hash;
server srv2.domain2.fr:443;
}
...
proxy_cache_path /nginx/cache/cache_temp use_temp_path=off keys_zone=cache_temp:10m max_size=10g inactive=10m;
proxy_cache cache_temp;
...
#srv1
server {
listen 443 ssl http2;
server_name srv1.domain1.fr;
all ssl settings...
location ~* \.(gif|jpg|jpeg|png|wmv|avi|mpg|mpeg|mp4|htm|html|js|css|mp3|swf|ico|flv|woff|woff2|ttf|svg)$ {
proxy_cache_valid 12h;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
add_header X-Cache $upstream_cache_status;
proxy_pass https://srv1;
}
location / {
proxy_cache_valid 12h;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
add_header X-Cache $upstream_cache_status;
proxy_pass https://srv1;
}
}
#srv2
server {
listen 443 ssl http2;
server_name srv2.domain2.fr;
all ssl settings...
location ~* \.(gif|jpg|jpeg|png|wmv|avi|mpg|mpeg|mp4|htm|html|js|css|mp3|swf|ico|flv|woff|woff2|ttf|svg)$ {
proxy_cache_valid 12h;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
add_header X-Cache $upstream_cache_status;
proxy_pass https://srv2;
}
location / {
proxy_cache_valid 12h;
proxy_ignore_headers X-Accel-Expires Expires Cache-Control;
add_header X-Cache $upstream_cache_status;
proxy_pass https://srv2;
}
}
so in my dns, I put the same IP for srv1.domain1.fr and srv2.domain2.fr
that works well but when I switch between both, issue occured : cache is the same
so I try to find a way to get separated cache
any idea ?
thanks
add more conf :
proxy_redirect off;
proxy_http_version 1.1;
proxy_read_timeout 10s;
proxy_send_timeout 10s;
proxy_connect_timeout 10s;
proxy_cache_path /nginx/cache/cache_temp use_temp_path=off keys_zone=cache_temp:10m max_size=10g inactive=10m;
proxy_cache cache_temp;
proxy_cache_methods GET HEAD;
proxy_cache_key $uri;
proxy_cache_valid 404 3s;
proxy_cache_lock on;
proxy_cache_lock_age 5s;
proxy_cache_lock_timeout 1h;
proxy_ignore_headers Cache-Control;
proxy_ignore_headers Set-Cookie;
proxy_cache_use_stale updating;
Change proxy_cache_key
From
proxy_cache_key $uri;
To
proxy_cache_key $scheme$proxy_host$request_uri;
Definitely it will solve this behavior.

Nginx "Proxy_Cache" Configuration Help Needed

I'm running in to a really weird issue. I only want to enable Proxy Cache for "new-site.com". However, when doing so, Nginx is proxy caching all of my websites.
I've went through all my vhost / config files and made sure that all "http" and "server" blocks were opened and closed correctly. It's my understanding that Proxy_Cache is only enabled for a site when you include (for example) "proxy_cache new-site;" in your websites "server" block.
In my "http" block, I load all of my websites .conf files, but none of them include any proxy_cache directives.
What am I doing wrong?
Here is a snippet of my config file :
http {
...
...
# nginx cache
proxy_cache_path /www/new-site.com/httpdocs/cache levels=1:2
keys_zone=new-site:10m
max_size=50m
inactive=1440m;
proxy_temp_path /www/new-site.com/httpdocs/cache/tmp 1 2;
# virtual hosting
include /etc/nginx/vhosts/*.conf;
}
Then here is my "new-site.com" vhost conf file:
server {
listen xxx.xxx.xxx.xxx:80;
server_name new-site.com;
root /www/new-site.com/httpdocs;
index index.php;
...
...
proxy_cache new-site;
location / {
try_files $uri #backend;
}
location ~* \.php {
include /usr/local/etc/nginx/proxypass.conf;
proxy_ignore_headers Expires Cache-Control;
proxy_cache_use_stale error timeout invalid_header updating http_502;
proxy_cache_bypass $cookie_session $http_secret_header;
proxy_no_cache $cookie_session;
add_header X-Cache $upstream_cache_status;
proxy_cache_valid 200 302 5m;
proxy_cache_valid 404 1m;
proxy_pass http://127.0.0.1:80;
}
location #backend {
include /usr/local/etc/nginx/proxypass.conf;
proxy_ignore_headers Expires Cache-Control;
proxy_cache_use_stale error timeout invalid_header updating http_502;
proxy_cache_bypass $cookie_session $http_secret_header;
proxy_no_cache $cookie_session;
add_header X-Cache $upstream_cache_status;
proxy_cache_valid 200 302 5m;
proxy_cache_valid 404 1m;
proxy_pass http://127.0.0.1:80;
}
location ~* \.(jpg|jpeg|gif|png|bmp|ico|pdf|flv|swf|css|js)$ {
....
}
}
Once I moved the line "proxy_cache new-site;" in to a "location" block, that resolved the issue for me.
Not sure why I have this issue when it sits outside a block though.

Resources