Analyzing an online shop (Shopware) with GooglePageSpeed results in many "expiration not specified"-Lines on every image.
I am wondering about because the webserver (nginx) adds Last-Modified-Timestamps and ETAG headers to the response of all images, resulting in an expected 304-Response on the second request.
Is ETAG/LastModified not supported by Google Page Speed?
I will provide the appropriate parts of the nginx-configuration:
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
expires 1M;
access_log off;
add_header Cache-Control "public";
}
## All static files will be served directly.
location ~* ^.+\.(?:css|cur|js|jpe?g|gif|ico|png|html|xml)$ {
## Defining rewrite rules
rewrite files/documents/.* /engine last;
rewrite backend/media/(.*) /media/$1 last;
expires 1w;
add_header Cache-Control "public, must-revalidate, proxy-revalidate";
access_log off;
# The directive enables or disables messages in error_log about files not found on disk.
log_not_found off;
tcp_nodelay off;
## Set the OS file cache.
open_file_cache max=3000 inactive=120s;
open_file_cache_valid 45s;
open_file_cache_min_uses 2;
open_file_cache_errors off;
## Fallback to shopware
## comment in if needed
#try_files $uri #shopware;
}
Is there anythong wrong or missing?
Finally we've found out, that there was another expire-statement in the vhost-config. Reduce both to one single statement solved our issue
Related
I'd like to set up nuxt with NGINX as load balancer. Additionally, I'd like to add some caching to the images.
Now I get 404 for my images using this:
location ~ ^/img.+\.(?:ico|gif|jpe?g|webp|png|woff2?|eot|otf|ttf|svg|js|css)$ {
expires 365d;
add_header Pragma public;
add_header Cache-Control "public";
try_files $uri $uri/ #proxy;
}
location #proxy {
expires 365d;
add_header Content-Security-Policy "default-src 'self' 'unsafe-inline';";
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
add_header X-Frame-Options "SAMEORIGIN";
add_header X-Cache-Status $upstream_cache_status;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_ignore_headers Cache-Control;
proxy_http_version 1.1;
proxy_read_timeout 1m;
proxy_connect_timeout 1m;
proxy_pass http://1649681_app/$request_uri;
#proxy_cache nuxt-cache;
#proxy_cache_bypass $arg_nocache; # probably better to change this
#proxy_cache_valid 200 302 60m; # set this to your needs
#proxy_cache_valid 404 1m; # set this to your needs
#proxy_cache_lock on;
#proxy_cache_use_stale error timeout http_500 http_502 http_503 http_504;
#proxy_cache_key $uri$is_args$args;
}
It seems like $request_uri is wrong in the line proxy_pass http://1649681_app/$request_uri - but how can I pass the requested path to the #proxy-location?
No, it isn't completely wrong to specify an URI using variables for the proxy_pass directive inside the named (as well as the regex) locations. However such a thing will have a drawback - if you can't specify your upstream address using IP rather than hostname, you'll need either a resolver defined in your configuration (worse) or an additional upstream block to define your 1649681_app backend (better). More details can be found here (exactly the same is applicable to named locations too).
Being that said, how do you think, what a request URI will be passed to the upstream specified in the proxy_pass directive if you won't specify any explicitly? It will be exactly the URI being processed (and if for some reason you'd need to pass a modified URI instead, you'd need to to modify it via the rewrite rules).
For the given configuration, assuming you don't need to modify the request URI or query arguments, you should simply use the
proxy_pass http://1649681_app;
And, do you understand what the $uri/ parameter does exactly mean? It makes the try_files directive to check if the given URI is a directory to search an index file inside it. I really doubt you need it using that kind of regex pattern. Remove it, it is only an extra (some kind of expensive) system kernel stat call. Use
try_files $uri #proxy;
instead.
We are facing issue with nginx memory leak it seems.
Setup:
Nginx running as deployment in GKE
Nginx version 1.20.2
Nginx is used to stream HLS. We write chunk file to a google filestore(NFS service). It is mounted on /var/www/html/.
Nginx never ever recovers memory it just grows on increasing. Nginx confiuration
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
}
Default.conf
server {
listen 80;
server_name localhost;
proxy_buffering off;
proxy_buffer_size 4k;
proxy_buffers 64 4k;
proxy_busy_buffers_size 16k;
proxy_cache_valid 200 302 1m;
proxy_cache_valid 404 60m;
proxy_cache_use_stale error timeout invalid_header updating;
proxy_redirect off;
#charset koi8-r;
#access_log /var/log/nginx/host.access.log main;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Methods' '*';
add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type';
add_header 'Access-Control-Max-Age' 1728000;
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
location /stub_status {
stub_status on;
access_log on;
allow all;
}
}
}
Screenshot of our internal monitoring system
First of all, what does your chart show? Memory usage of nginx worker processes? Or memory utilization of whole system?
In case of nginx memory growth it may relate to known issue (basically with OpenSSL either), see https://trac.nginx.org/nginx/ticket/2316
So either try to apply patch suggested by Maxim in that issue, or try the workaround he suggested in the last comment or upgrade to newer version of OpenSSL (PKCS11 engine) or even nginx (especially if it is linked statically).
There are enough OpenSSL-related leak issues, see also for example https://github.com/kubernetes/ingress-nginx/issues/7647 or linked within. So to veryfy it is not affected by OpenSSL, try to test it without SSL/TLS/https and check whether you'd see growth of memory usage.
Although I don't see any memory leak trying vanilla nginx 1.20.2 (without any patch, built with OpenSSL 1.1.1k) testing similar configuration (I don't see proxy_pass directive in your config so I was simply proxying to http/https upstream too). No leak reproducible at all.
In case of high system memory usage, it may be common OS cache or even some buffering of NFS, see https://askubuntu.com/a/1393696/1384131 for similar question.
On mobile 📱
I'm trying to improve the page load of my site
I added
# Media: images, icons, video, audio, HTC
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|mp3|ogg|ogv|webm|htc|woff2|woff)$ {
expires 1M;
access_log off;
add_header Cache-Control "public";
}
# CSS and Javascript
location ~* \.(?:css|js)$ {
expires 1y;
access_log off;
add_header Cache-Control "public";
}
For some reasons, I feel like the changes that I just added to my Nginx is not taking any effect.
https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fwww.bunlongheng.com%2F
How should I debug this further ?
You're missing the max-age directive, from http://nginx.org/en/docs/http/ngx_http_headers_module.html
I don't think you really want CSS and JS files to expire so far out, but I could be wrong.
Also, no logging on images and all these file types? What if you're getting hotlinked or serving CSS/JS files that can't be found.
I would rethink your cache control a bit more.
# Media: images, icons, video, audio, HTC
location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|mp3|ogg|ogv|webm|htc|woff2|woff)$ {
expires 1M;
access_log off;
# max-age must be in seconds
add_header Cache-Control "max-age=2629746, public";
}
# CSS and Javascript
location ~* \.(?:css|js)$ {
expires 1y;
access_log off;
add_header Cache-Control "max-age=31556952, public";
}
How do I set up Nginx to never cache sitemaps?
this was my idea:
location ~* \.(xml)$ {
expires -1;
log_not_found off;
}
thoughts?
Yes, thats correct if you have a add_header directive at previous levels. If not then
location ~* .(xml)$ {
add_header Expires -1;
log_not_found off;
}
A negative time sets the cache-control header to no-cache.
A better way to play with headers is to use nginx v1.4.3 that has the module more_set_headers and more_clear_headers in order to replace or set the headers from origin.
You can download the module from here.
More information HERE
I have a Wordpress site running in a Vagrant development environment. When I load up http://localhost:8080, the site comes up fine, but as soon as I try to access the admin by going to http://localhost:8080/wp-admin, I'm redirected to http://localhost/wp-admin/.
Two things here:
Something (Wordpress?) is forcing the trailing slash (which would be fine, except...).
In the trailing slash redirection, the port is getting lost (which is very much not fine).
I've tried adding the port_in_redirect directive (using both values, to be honest) that I've seen in other answers to similar questions, but it changed nothing. This seems to be a Wordpress, ahem, feature, but I can't find anything to explain its purpose or that helps me keep it from breaking things. I'm hoping someone can help.
My Nginx server block:
server {
listen 80;
server_name localhost;
root /vagrant/www;
index index.php;
access_log /var/log/nginx/project.vm.access.log;
error_log /var/log/nginx/project.vm.error.log;
location / {
try_files $uri $uri/ /index.php?$args;
}
# Add trailing slash to */wp-admin requests so the admin interface
# works correctly. I've tried with and without this. No difference.
rewrite /wp-admin$ $scheme://$host$uri/ permanent;
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
fastcgi_pass 127.0.0.1:9000;
fastcgi_index index.php;
include fastcgi_params;
}
# Cache static files
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
expires max;
log_not_found off;
}
location = /favicon.ico {
access_log off;
log_not_found off;
}
location = /robots.txt {
access_log off;
log_not_found off;
allow all;
}
location ~ /\. {
access_log off;
log_not_found off;
deny all;
}
location ~ ~$ {
access_log off;
log_not_found off;
deny all;
}
}
In my wp-config.php file, I have the WP_HOME and WP_SITEURL constants set.
define('WP_HOME','http://localhost:8080');
define('WP_SITEURL','http://localhost:8080');
Any idea what I'm missing?
I recently face the same problem after moving a WordPress installation from a server to a local Vagrant instance and solved it adding:
define('RELOCATE',true);
to wp-config.php file and going to http://localhost:8080/wp-login.php
I found the solution at http://codex.wordpress.org/Changing_The_Site_URL#Relocate_method, I think there you can find a few other ways to deal with the issue of moving a WP installation from one server to another.