Nginx keeps sending old files (from cache?) - nginx

Question

I have on a production server an Angular app (using Universal for server-side rendering) running on Node Express localhost:4000, and I configured Nginx reverse proxy for the app. The production server is using HTTPS for its domain name.
Here is nginx config in /etc/config/sites-enabled:
location ~ (/landing/home|/landing/company|/landing/contact|/landing/support) {
proxy_pass http://127.0.0.1:4000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_cache_bypass $http_upgrade;
add_header X-Cache-Status $upstream_cache_status;
}
location / {
proxy_pass http://127.0.0.1:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
Here is nginx.conf
user ubuntu;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 2048;
multi_accept on;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
server_names_hash_bucket_size 64;
server_name_in_redirect off;
# include /etc/nginx/mime.types;
# default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
gzip_disable "msie6";
gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
In Chrome Dev Tools - Network, here is sample request & response for an image (SVG file). The image that nginx sent is older version, and has since updated (file name unchanged) on the Angular side. Please note that this file is just a sample, the issue I'm facing is not just this one file, but all static files (including css and js files).
request header
response header
To verify, I did curl on a client and on the server, to the same image file. Here are result of curl:
curl result from a client browser, result was from nginx
curl result on the server, comparing between curl to localhost:4000 and curl to the actual public url
We can see that in response from localhost:4000 it is the latest version of the image, where in response from the public url it is older version of the same image.
I checked in /etc/nginx, there is no chache folder in there. I thought about clearing nginx's cache, but I couldn't find it there.
I have tried adding many things in config, including:
add_header Last-Modified $date_gmt;
add_header Cache-Control 'no-store, no-cache';
if_modified_since off;
expires off;
etag off;
and
proxy_cache off;
And somehow even the X-Cache-Status doesn't show up in response header neither, but comparing the curl result from localhost and the public url, it is clear to me that it must be something to do with nginx.
Anyone have suggestion on what to do to make nginx sends response from the actual output of localhost:4000, instead of from cache? Thank you.
UPDATE #1:
Sorry I only included partial nginx conf. I have found the root cause: I actually have two Node Express running on the same domain, one is on port 4000 (Angular Universal) and the other is on port 5000 (non-Universal). I have updated the excerpt of nginx conf above to include the other location directive for the one on port 5000. Please see my answer below for further explanation of what I did wrong to cause the problem in this question.

Answer 1

I found out the root cause of the problem.
I actually have two Node Express running on the same server, same domain. One in on port 4000 (uses Angular Universal), and the other on port 5000 (non-Universal). I have edited my question to include the second location directive in nginx conf.
The way I had my nginx conf made it looked like the whole page came as response from localhost:4000, but some parts within the page (images, style sheets, etc) actually were response from localhost:5000, due to the url of the request did not match the pattern in nginx conf for localhost:4000. So localhost:5000 got to respond the the request, and the files that localhost:5000 had was older version (not all, but the one I tested with curl happened to be older version).
I only realized this situation when I disabled the second location directive in the nginx conf, effectively disabled localhost:5000 from responding to any request, and then I saw many 404 errors because of that.
To solve this problem, meaning to have both localhost:4000 and localhost:5000 active, and still getting the correct respond, I had to make some adjustments to the routings in my Angular code.