I've a Nginx configuration, where I get certain files from AWS S3 bucket, like call from *.my.api.com/file.js will get the file from X folder in S3.
I've an exceptional domain (like xx.my.api.com) for which I will add the
Cache-Control "no-store, no-cache";
Pragma "no-cache";
headers and for the rest of *.my.api.com the headers will be default (it's cache-control: public now).
On my local environment, the file is hosted on my machine, so the headers are set correctly. However, on production, the headers come as default as cache-control: public.
I've read answers like this saying there should be no trouble with it, but it's not working for me.
Is there anything I'm doing wrong? Is it related to the file being hosted on AWS?
My Nginx configuration is as below:
server {
listen 80;
root /var/xyz/public;
index index.html index.htm;
server_name my.api.com *.my.api.com;
add_header Access-Control-Allow-Origin "*";
if ($http_host ~* "^(.*).my.api.com$"){
set $myName $1;
}
location ~ /myfile.js {
resolver 8.8.8.8;
proxy_buffering off;
proxy_set_header Content-Length "";
proxy_set_header Cookie "";
proxy_method GET;
proxy_pass_request_body off;
proxy_max_temp_file_size 0;
if ($myName = "mySpecialName") {
proxy_pass http://path/to/aws/s3/bucket/file.js;
add_header Cache-Control "no-store, no-cache";
add_header Pragma "no-cache";
add_header X-XSS-Protection "1";
add_header X-Frame-Options "SAMEORIGIN";
add_header X-Content-Type-Options nosniff;
}
if ($query_string !~* "myQueryString=([^&]*)") {
proxy_pass http://path/to/aws/s3/bucket/file.js;
}
if ($query_string ~* "myQueryString=([^&]*)") {
proxy_pass http://path/to/some/other/aws/s3/bucket/file.js;
}
}
}
I've tried:
always
proxy_pass_request_headers on
proxy_set_header
copying the server code and adjusting for xx.my.api.com only
proxy_hide_header (can't be used because of if block)
more_set_headers
but none of them worked.
Any help would be appreciated, thanks in advance.
We've solved it by adding the headers from our DNS panel, which was used for caching the file stored in S3.
I used the Negated Regular Expressions in location but it does add the header but removes everything else that existed before. Even if I add it doesn’t consider the rest only hsts. I am not sure what is the best way to do this. add a header for anything else but "don't add this HSTS header if we're on API::P".
location ~ (?!^/p/) {
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
}
or
location ~ ^(/p/) {
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
}
Here is the logic that I have in mind.
if location == "/p/":
pass
else:
add_header ...HSTS...
Because of performance considerations, avoid using regex whenever possible. Either use two locations duplicating everything else that cannot be moved one level up:
location / {
add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
... common configuration
}
location /p/ {
... common configuration
}
or use the map block (however this actually will make at least one PRCE library call):
map $uri $hsts {
~^/p/ "";
default "max-age=31536000; includeSubDomains";
}
server {
...
add_header Strict-Transport-Security $hsts always;
...
}
If evaluated variable used in the add_header directive will be empty, nginx won't add a header with an empty value - instead it won't add such a header at all.
For the two-locations configuration, every request started with /p/ will be handled with the location /p/ { ... }, and every other request will be handled with the location / { ... }. There is absolutely no need to use any regex locations for this particular case.
Please note that add_header directives are inherited from the previous configuration level if and only if there are no add_header directives defined on the current level.
I have a website on an EC2 instance with NGINX and Ubuntu 18.0.4. I have setup the server block for the site the site is loading correctly. I have also enabled caching for images, javascript and css files.
I want to exclude certain paths from getting cached and I tried some config examples available on the internet. None of them are working and I am getting a 404 error for the locations that I am trying to exclude.
My server block is as follows:
# Expires map
map $sent_http_content_type $expires {
default off;
text/html epoch;
text/css max;
application/javascript max;
~image/ max;
~font/ max;
}
server {
listen 80 default_server;
listen [::]:80 default_server;
expires $expires;
location /path1/ {
add_header Cache-Control "private";
}
location /path2/subpath1/ {
add_header Cache-Control "private";
}
location /path2/subpath2/ {
add_header Cache-Control "private";
}
location /path2/subpath3/ {
add_header Cache-Control "private";
}
}
The paths (path1, path2) which I have tried to exclude from caching are returning a 404 not found error. Can someone please help?
I want to setup a general cache-control for all NGINX-Sites.
Here is an example of it:
location ~* \.(?:ico|gif|jpe?g|png|svg?z)$ {
expires 1y;
add_header Pragma public;
add_header Cache-Control public;
}
If I try to add this "location" setting to /etc/nginx/nginx.conf I got an error, that I can't set it here.
So is there a way to set this cache handling for all sites in nginx by default, without the need to edit each single host in /sites_available/ ?
No, you can't use location directive in contexts other than server or another location. However you can use the following workaround (this one can be added to the http context):
map $cache $public {
1 public;
}
map $cache $expires {
1 1y;
default off; # or some other default value
}
map $uri $cache {
~*\.(?:ico|gif|jpe?g|png|svg?z)$ 1;
}
expires $expires;
add_header Pragma $public;
add_header Cache-Control $public;
nginx won't add the header at all (or modify an existing header) if the value calculated via the map expression will be an empty string. But be aware of this documentation excerpt:
There could be several add_header directives. These directives are inherited from the previous configuration level if and only if there are no add_header directives defined on the current level.
I am having an intriguing problem where whenever I use add_header in my virtual host configuration on an ubuntu server running nginx with PHP and php-fpm it simply doesn't work and I have no idea what I am doing wrong. Here is my config file:
server {
listen 80; ## listen for ipv4; this line is default and implied
#listen [::]:80 default ipv6only=on; ## listen for ipv6
root /var/www/example.com/webroot/;
index index.html index.htm index.php;
# Make site accessible from http://www.example.com/
server_name www.example.com;
# max request size
client_max_body_size 20m;
# enable gzip compression
gzip on;
gzip_static on;
gzip_min_length 1000;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Headers' 'Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, PUT, DELETE';
add_header PS 1
location / {
# First attempt to serve request as file, then
# as directory, then fall back to index.html
try_files $uri $uri/ /index.php?$query_string;
# Uncomment to enable naxsi on this location
# include /etc/nginx/naxsi.rules
}
location ~* \.(css|js|asf|asx|wax|wmv|wmx|avi|bmp|class|divx|doc|docx|eot|exe|gif|gz|gzip|ico|jpg|jpeg|jpe|mdb|mid|midi|mov|qt|mp3|m4a|mp4|m4v|mpeg|mpg|mpe|mpp|odb|odc|odf|odg|odp|ods|odt|ogg|ogv|$
# 1 year -> 31536000
expires 500s;
access_log off;
log_not_found off;
add_header Pragma public;
add_header Cache-Control "max-age=31536000, public";
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
# NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
# With php5-cgi alone:
#fastcgi_pass 127.0.0.1:9000;
# With php5-fpm:
fastcgi_pass unix:/var/run/example.sock;
fastcgi_index index.php?$query_string;
include fastcgi_params;
# instead I want to get the value from Origin request header
}
# Deny access to hidden files
location ~ /\. {
deny all;
access_log off;
log_not_found off;
}
error_page 403 /403/;
}
server {
listen 80;
server_name example.com;
rewrite ^ http://www.example.com$request_uri? permanent;
}
I've tried adding the headers to the other location sections but the result is the same.
Any help appreciated!!
There were two issues for me.
One is that nginx only processes the last add_header it spots down a tree. So if you have an add_header in the server context, then another in the location nested context, it will only process the add_header directive inside the location context. Only the deepest context.
From the NGINX docs on add_header:
There could be several add_header directives. These directives are inherited from the previous level if and only if there are no add_header directives defined on the current level.
Second issue was that the location / {} block I had in place was actually sending nginx to the other location ~* (\.php)$ block (because it would repath all requests through index.php, and that actually makes nginx process this php block). So, my add_header directives inside the first location directive were useless, and it started working after I put all the directives I needed inside the php location directive.
Finally, here's my working configuration to allow CORS in the context of an MVC framework called Laravel (you could change this easily to fit any PHP framework that has index.php as a single entry point for all requests).
server {
root /path/to/app/public;
index index.php;
server_name test.dev;
# redirection to index.php
location / {
try_files $uri $uri/ /index.php?$query_string;
}
# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
location ~ \.php$ {
fastcgi_split_path_info ^(.+\.php)(/.+)$;
# NOTE: You should have "cgi.fix_pathinfo = 0;" in php.ini
# With php5-fpm:
fastcgi_pass unix:/var/run/php5-fpm.sock;
fastcgi_index index.php;
include fastcgi_params;
# cors configuration
# whitelist of allowed domains, via a regular expression
# if ($http_origin ~* (http://localhost(:[0-9]+)?)) {
if ($http_origin ~* .*) { # yeah, for local development. tailor your regex as needed
set $cors "true";
}
# apparently, the following three if statements create a flag for "compound conditions"
if ($request_method = OPTIONS) {
set $cors "${cors}options";
}
if ($request_method = GET) {
set $cors "${cors}get";
}
if ($request_method = POST) {
set $cors "${cors}post";
}
# now process the flag
if ($cors = 'trueget') {
add_header 'Access-Control-Allow-Origin' "$http_origin";
add_header 'Access-Control-Allow-Credentials' 'true';
}
if ($cors = 'truepost') {
add_header 'Access-Control-Allow-Origin' "$http_origin";
add_header 'Access-Control-Allow-Credentials' 'true';
}
if ($cors = 'trueoptions') {
add_header 'Access-Control-Allow-Origin' "$http_origin";
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Max-Age' 1728000; # cache preflight value for 20 days
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
add_header 'Access-Control-Allow-Headers' 'Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken,Keep-Alive,X-Requested-With,If-Modified-Since';
add_header 'Content-Length' 0;
add_header 'Content-Type' 'text/plain charset=UTF-8';
return 204;
}
}
error_log /var/log/nginx/test.dev.error.log;
access_log /var/log/nginx/test.dev.access.log;
}
The gist for the above is at: https://gist.github.com/adityamenon/6753574
I had the issue of not getting the response header due to the response code not within the allowed range, unless you specify the "always" keyword after the header value.
From the official docs:
Adds the specified field to a response header provided that the response code equals 200, 201, 204, 206, 301, 302, 303, 304, 307, or 308. The value can contain variables.
When I test the above add_header settings with:
# nginx -t && service nginx reload
I get
nginx: [emerg] directive "add_header" is not terminated by ";" in
/etc/nginx/enabled-sites/example.com.conf:21
nginx: configuration file /etc/nginx/nginx.conf test failed
So the complain is reagarding this line:
add_header PS 1
missing the semi-colon (;)
To test the headers I like to use
# curl -I http://example.com
According to the ngx_http_headers_module manual
syntax: add_header name value;
default: —
context: http, server, location, if in location
I further tried
add_header X-test-A 1;
add_header "X-test-B" "2";
add_header 'X-test-C' '3';
in the context of http, server and location, but it only showed up in the server context.
Firstly, let me say that after looking around the web, I found this answer popping up everywhere:
location ~* \.(eot|ttf|woff|woff2)$ {
add_header Access-Control-Allow-Origin *;
}
However, I have decided to answer this question with a separate answer as I only managed to get this particular solution working after putting in about ten more hours looking for a solution.
It seems that Nginx doesn't define any [correct] font MIME types by default. By following this tuorial I found I could add the following:
application/x-font-ttf ttc ttf;
application/x-font-otf otf;
application/font-woff woff;
application/font-woff2 woff2;
application/vnd.ms-fontobject eot;
To my etc/nginx/mime.types file. As stated, the above solution then worked. Obviously, this answer is aimed at sharing fonts but it's worth noting that the MIME types may not be defined in Nginx.
Evidently the add_header inheritance quirk/gotcha applies to the upstream layer as well.
I had a script pre-authorizing requests meant for another service, and was therefore returning all of the headers from the other service.
Once I started adding an 'Access-Control-Allow-Origin' entry along with these relayed headers, the browser would actually get the entry and allow the request.
I don't think it works properly by reloading ==> nginx -s reload
When I used add_header and then reloaded, nothing changed in response.
But when I made a deliberate error and saw a 404 error on the client side,
and then fixed my deliberate error and reloaded again, Add_header worked.
What does your nginx error log say?
Do you know which add_header lines are breaking the configuration? If not, comment them all out then enable them 1 by 1, reloading nginx to see which one(s) is/are the problem. I would begin by commenting out the block:
add_header 'Access-Control-Allow-Origin' '*';
add_header 'Access-Control-Allow-Credentials' 'true';
add_header 'Access-Control-Allow-Headers' 'Authorization,Content-Type,Accept,Origin,User-Agent,DNT,Cache-Control,X-Mx-ReqToken';
add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS, PUT, DELETE';
add_header PS 1
The problem could be that you're setting headers not supported by the core httpHeaders module. Installing the NginxHttpHeadersMoreModule may be helpful.
Also, try replacing the two add_header lines int the location ~* \... with the following:
add_header Pragma '';
add_header Cache-Control 'public, max-age=31536000'
Is there a reason you have the gzip configuration here and not in your global nginx.conf?
It turns out that trying to update nginx to the newest version was causing this. I had tried previously to reinstall which seemed to reinstall it correctly but actually Ubuntu wasn't properly removing nginx. So all I had to do is reinstall Ubuntu server and install everything anew using just the standard ubuntu repositories.