I'm using nginx + passenger. I'm trying to understand the nginx response 200 and 304. What does this both means? Sometimes, it responses back in 304 and others only 200. Reading the YUI blog, it seems browser needs the header "Last-Modified" to verify with the server. I'm wondering why the browser need to verify the last modified date. Here is my nginx configuration:
location / {
root /var/www/placexpert/public; # <--- be sure to point to 'public'!
passenger_enabled on;
rack_env development;
passenger_use_global_queue on;
if ($request_filename ~* ^.+\.(jpg|jpeg|gif|png|ico|css|js|swf)$) {
expires max;
break;
}
}
How would I add the header "Last-Modified" to the static files? Which value should I set?
Try this.
if (...){
expires max;
add_header Last-Modified $sent_http_Expires;
break;
}
Related
I have a PHP CMS as an upstream/origin server that serves content. It is placed behind a Nginx web server configured as a reverse proxy cache with proxy_pass and proxy_cache.
While implementing 304 Not Modified response on the upstream/origin server, I noticed that I sometimes get 304 Not Modified response from the proxy without my upstream/origin server being called.
My proxy configuration looks like this (notice that I don't cache 304 responses):
http {
#...
proxy_cache_path /usr/share/nginx/cache/ levels=1:2 keys_zone=microcache:10m max_size=1024m inactive=1h;
}
server {
server_name _;
#...
location / {
proxy_cache microcache;
proxy_cache_revalidate on;
add_header X-Cache-Status $upstream_cache_status;
#...
proxy_cache_valid 200 301 302 1m;
proxy_pass http://upstream;
}
}
When hitting a URL for the first time, I get a 200 with a X-Cache-Status: MISS and the ETag and Last-Modified headers returned by the origin.
The second time I hit the URL, I get a X-Cache-Status: HIT and my origin doesn't get called, so all is great!
However, if I hit the URL again, this time with the If-Modified-Since and If-None-Match headers with the values received with the first call, I get a 304 Not Modified, with X-Cache-Status: HIT and still no call on my origin server!
I seems then that Nginx has the content in its cache, and figures with the incoming headers that it should return a 304. Meaning my origin never gets to return the 304 itself if the content is in the Nginx cache.
Am I right? I cannot find any documentation on this behavior. I, however found a module in the Nginx source code that seems the be handling something similar: https://github.com/nginx/nginx/blob/master/src/http/modules/ngx_http_not_modified_filter_module.c#L78
Could you please provide me with insights on this behavior, how it works, and what enables it in my configuration?
Thank you very much!
proxy_cache_path /usr/share/nginx/cache/ levels=1:2 keys_zone=microcache:10m max_size=1024m inactive=1h;
proxy_cache microcache;
Meaning that nginx server stores cache data to /usr/share/nginx/cache/
and
proxy_cache_valid 200 301 302 1m;
This means any HTTP API calls that have HTTP statuses 200, 301, or 302 use nginx local cache (which is stored in /usr/shre/nginx/cache) for one minute.
I am using NGINX as a reverse proxy and have the following setup that redirects all requests to my SPA container
location / {
proxy_pass http://172.x.x.x;
proxy_intercept_errors on;
error_page 404 = /index.html;
}
This works but all images have their expiry set to the default (max-age=3600).
When I add the following block that is supposed to set the expiry for all image types then none of the images work anymore (404 NOT FOUND).
location ~* \.(jpg|png)$ {
add_header Cache-Control public;
expires 1M;
}
I tried nesting this block inside the first one but then I also get the 404 responses.
Why doesn't this work and what should I change so that all images have an extended max-age?
Nginx chooses one location to process a request. It uses the directives within the location block or inherited from the surrounding block.
If the requests for .jpg and .png URIs are to be passed upstream, the location also needs to contain a proxy_pass statement.
Alternatively, use a map to set the expires value. See this document for details.
For example:
map $request_uri $expires {
default off;
~*\.(jpg|png)(\?|$) 1M;
}
server {
...
expires $expires;
location / {
...
}
}
I think that expires already sets the cache-control header, but you can also use another map to set the value of an add_header directive.
I'm trying to follow this guide: https://www.digitalocean.com/community/tutorials/how-to-implement-browser-caching-with-nginx-s-header-module-on-ubuntu-16-04
but every time I execute curl -I http://myjsfile.com/thejsfile.js it doesn't return the cache property
i.e this one:
Expires: Thu, 01 Jan 1970 00:00:01 GMT
Cache-Control: no-cache
this is what I have in my sites-available file. though there are 2 in there the default and our custom one for Certbot SSL certs. I did apply this to those 2 files.
# Expires map
map $sent_http_content_type $expires {
default off;
text/html epoch;
text/css max;
application/javascript max;
~image/ epoch;
}
So I'm not sure it's caching it and when I checked it using gtmetrix it still gets an F for browser caching.
I also tried this one: NGINX cache static files
and I have this in my nginx.conf inside http
server {
location ~* \.(?:ico|css|js)$ {
expires 30d;
add_header Vary Accept-Encoding;
access_log off;
}
}
but it still didn't work when I checked using the curl command.
so can someone enlighten me on what I'm doing wrong here or is this not the best approach to cache JS and CSS files?
I'm trying to tell nginx to cache some of my assets (js, css) forever, or at least for a very long time.
The idea is that once an asset bundle is compiled and published with an /assets/ URI prefix (e.g. /assets/foo-{fingerprint}.js) it stays there and doesn't ever need to change.
The internets told me I should write the following rule:
location ~ ^/assets/.*-([^.]+)\.(js|css)$ {
gzip_static on; # there's also a .gz of the asset
expires max;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
break;
}
I would expect this would result in responses with HTTP code 304 "Not Modified", but what I get is a consistent HTTP 200 (OK) every time.
I have tried some other approaches, for instance:
a) explicitly setting modification time to a constant point in time in the past;
add_header Last-Modified "Thu, 01 Jan 1970 00:00:00 GMT";
b) switching to If-None-Match checks;
add_header ETag $1;
if_modified_since off;
However, the only thing that really worked as needed was this:
add_header Last-Modified "Thu, 01 Jan 2030 00:00:00 GMT";
if_modified_since before;
I'm lost. This is contrary to everything I thought was right. Please help.
You should change your internets, since they give you wrong advices.
Just remove all add_header lines from your location (as well as surplus brake):
location ~ ^/assets/.*-([^.]+)\.(js|css)$ {
gzip_static on; # there's also a .gz of the asset
expires max;
}
and read the docs from the true Internet: http://nginx.org/r/expires and https://www.rfc-editor.org/rfc/rfc2616
It seems part of my configuration. During my researching I realized that browser uses heuristic analysis to validate requests with ConditionalGet headers (E-Tag, Last-Modified). It makes a lot of sense for back-end responses, so you can handle that to save server resources.
But in terms of static files (js, css, images), you can tell browser to serve them straight away without any Conditional Get validation. It is helpful if you update file name if any change takes place.
This part of configuration makes it happen:
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
I am looking for a nginx config setup that does setup the Access-Control-Allow-Origin to the value received in the Origin.
It seems that the * method doesn't work with Chrome and the multiple URLs doesn't work with Firefox as it is not allowed by CORS specification.
So far, the only solution is to setup the Access-Control-Allow-Origin to the value received in the origin (yes some validation could be implemented).
The question is how to do this in nginx, preferably without installing additional extensions.
set $allow_origin "https://example.com"
# instead I want to get the value from Origin request header
add_header 'Access-Control-Allow-Origin' $allow_origin;
Using if can sometimes break other config such as try_files. You can end up with unexpected 404s.
Use map instead
map $http_origin $cors_header {
default "";
"~^https?://[^/]+\.example\.com(:[0-9]+)?$" "$http_origin";
}
server {
...
location / {
add_header Access-Control-Allow-Origin $cors_header;
try_files $uri $uri/ /index.php;
}
...
}
If is evil
I'm starting to use this myself, and this is the line in my current Nginx configuration:
add_header 'Access-Control-Allow-Origin' "$http_origin";
This sets a header to allow the origin of the request as the only allowed origin. So where ever you are coming from is the only place allowed. So it shouldn't be much different than allowing "*" but it looks more specific from the browser's perspective.
Additionally you can use conditional logic in your Nginx config to specify a whitelist of hostnames to allow. Here's an example from https://gist.github.com/Ry4an/6195025
if ($http_origin ~* (whitelist\.address\.one|whitelist\.address\.two)$) {
add_header Access-Control-Allow-Origin "$http_origin";
}
I plan to try this technique in my own server to whitelist the allowed domains.
Here is a part of a file from conf.f directory where people always describes their virtual hosts of Nginx.
$http_origin compares with list of allowed_origins and then in second map block the system decides what will write to "header Access-Control-Allow-Origin" according to allowed list.
Here is a part of code.
#cat /etc/nginx/conf.d/somehost.conf
map $http_origin $origin_allowed {
default 0;
https://xxxx.yyyy.com 1;
https://zzz.yyyy.com 1;
}
map $origin_allowed $origin {
default "";
1 $http_origin;
}
server {
server_name somehost.com;
#...[skipped text]
add_header Access-Control-Allow-Origin $origin always;
#....[skipped text]
}
I test it om my servers. All works fine.
Have a nice day & be healthy,
Eugene.