Nginx Variable Cache Time - nginx

I'm currently working with fastcgi_cache and wanted to pass a variable to fastcgi_cache_valid so I could have variable amount of cache time depending on the file. But it seems that it will not accept a variable.
I tried the following:
set $cache_time 15s;
fastcgi_cache_valid 200 ${cache_time};
fastcgi_cache_valid 200 $cache_time;
set $cache_time "15s";
fastcgi_cache_valid 200 ${cache_time};
fastcgi_cache_valid 200 $cache_time;
set $cache_time 15;
fastcgi_cache_valid 200 ${cache_time}s;
fastcgi_cache_valid 200 $cache_time;
But I receieved the following errors:
nginx: [emerg] invalid time value "$cache_time" in /etc/nginx/conf.d/www.com.conf:118
nginx: [emerg] directive "fastcgi_cache_valid" is not terminated by ";" in /etc/nginx/conf.d/www.com.conf:118

fastcgi_cache_valid does not accept variable but there is a work around if you have only two options cached and not-cached.
In this example we want cached.php to return the cached version, and the non-cached version if the url contains params. meaning cached.php will always reaturn the cached version and cached.php?id=1 will always return the non-cached version.
location = /cached.php {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
set $disable_cache 1;
if ( $request_uri = "/cached.php" ){
set $disable_cache 0;
}
fastcgi_cache phpcache;
fastcgi_cache_valid 200 1h;
fastcgi_cache_methods GET HEAD;
fastcgi_cache_bypass $disable_cache;
fastcgi_no_cache $disable_cache;
add_header X-Is-Cached "Cache disabled $disable_cache";
}
This the content of cached.php used for test
<?php echo "The time is " . date("h:i:sa")."\n";?>
This is the results if tests using curl
curl --head "http://www.example.com/cached.php"
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Thu, 31 Dec 2020 17:07:55 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Fastcgi-Cache: HIT
X-Is-Cached: Cache disabled 0
curl --head "http://www.example.com/cached.php?id=1"
HTTP/1.1 200 OK
Server: nginx/1.14.0 (Ubuntu)
Date: Thu, 31 Dec 2020 17:09:15 GMT
Content-Type: text/html; charset=UTF-8
Connection: keep-alive
X-Fastcgi-Cache: BYPASS
X-Is-Cached: Cache disabled 1
X-Is-Cached is used only for debuging and it is not necessary.

Related

Double forward proxy situation client(A) -> nginx(B) -> squid(C) -> <internet> -> server (D)

My end server URL is app.sample.com/v1/get/health
Now my client will reach to server to get the health.
But Client Machine (A) and the machine(B) on which nginx is running doesn't have the internet (air gapped) but machine (C) on which squid is running is connected to the internet and server (D).
A(client) is able to talk to B(nginx)
B(nginx) is able to talk to C (squid)
C(squid) is able to talk to D(server)
but when I am configuring the nginx on B and squid on C, request is not forwarding from C to server (D).
Nginx config
http {
error_log /etc/nginx/error_log.log warn;
server {
server_name localhost;
location / {
proxy_pass http://<squid-host>:<squid-port>/;
}
listen 32708;
}
}
Squid
http_port 3128 accel allow-direct
Curl request from machine B (this is not working)
curl <nginx-ip>:32708/www.google.com
Nginx logs
20.1.1.2 - - [15/Feb/2022:10:03:10 +0000] "GET /www.google.com HTTP/1.1" 400 3544 "-" "curl/7.58.0"
Squid
The following error was encountered while trying to retrieve the URL: <a href="www.google.com">www.google.com
<p>Some aspect of the requested URL is incorrect.</p>
<p>Some possible problems are:</p>
<ul>
<li><p>Missing or incorrect access protocol (should be <q>http://</q> or similar)</p></li>
<li><p>Missing hostname</p></li>
<li><p>Illegal double-escape in the URL-Path</p></li>
<li><p>Illegal character in hostname; underscores are not allowed.</p></li>
</ul>
ERR_INVALID_URL
1644919643.574 0 20.1.1.2 TAG_NONE/400 3913 GET www.google.com - HIER_NONE/- text/html
When I tried curl using this way from machine B, I am able to get the response. (this is working)
http_proxy="http://<squid-host>:<squid-port>" curl -I http://www.google.com
Response:
HTTP/1.1 200 OK
Content-Type: text/html; charset=ISO-8859-1
P3P: CP="This is not a P3P policy! See g.co/p3phelp for more info."
Date: Tue, 15 Feb 2022 09:57:42 GMT
Server: gws
X-XSS-Protection: 0
X-Frame-Options: SAMEORIGIN
Expires: Tue, 15 Feb 2022 09:57:42 GMT
Cache-Control: private
Set-Cookie: 1P_JAR=2022-02-15-09; expires=Thu, 17-Mar-2022 09:57:42 GMT; path=/; domain=.google.com; Secure
Set-Cookie: NID=511=j5ZU92fz-JUCQxj__gIdZavA8mOLyiUe_2gp-ZPePQXcV7RgruMLwf36C3Imdt5oTm__xQdGkcNgHBliaEhB7eeXR6gtjaQidHWcebo3w0YXhgUlmZ2YFC3dx5vb4YbpZIr8gmCUcWdPlz8tmSA2c7FYYnZNbegkb9QWe5ybSUU; expires=Wed, 17-Aug-2022 09:57:42 GMT; path=/; domain=.google.com; HttpOnly
X-Cache: MISS from anilhost
X-Cache-Lookup: MISS from sample:3128
Via: 1.1 sample (squid/3.5.27)
Connection: keep-alive
Squid
1644919591.994 50 20.1.1.2 TCP_MISS/200 804 HEAD http://www.google.com/ - HIER_DIRECT/216.58.195.68 text/html

Nginx http_accept cannot be mapped to variable for content-negotiation

The goal is to get map the Content-Type value to a file type using -
# nginx.conf
map $http_accept $suffix {
"~*turtle" ".ttl";
default "DEFAULT";
}
which is handled by -
# sites-available/site
location /ontologies {
root /folder;
add_header Vary Accept;
add_header X-debug-accept "$http_accept";
add_header X-debug-suffix "$suffix";
try_files $uri$suffix $uri =404;
}
However, the response always gives -
# curl -I -H "Content-type: text/turtle" URL
http_accept = */*
suffix = DEFAULT, which is actually (blank)
Your problem is that you are accessing the wrong header. You should be using $http_content_type and not $http_accept
# curl -v -I -H "Content-type: text/turtle" URL
* Trying ::1...
* Connected to localhost (::1) port 80 (#0)
> GET / HTTP/1.1
> Host: localhost
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Type: text/turtle
As you can see you passed Content-Type and not Accept. So based on which header you need use the correct variable. Either $http_content_type or $http_accept

change nginx response code from 413

Is there any way to change the response code nginx sends? When the server receives a file that exceeds its client_max_body_size as defined in the config, can I have it return a 403 code instead of a 413 code?
Below works fine for me
events {
worker_connections 1024;
}
http {
server {
listen 80;
location #change_upload_error {
return 403 "File uploaded too large";
}
location /post {
client_max_body_size 10K;
error_page 413 = #change_upload_error;
echo "you reached here";
}
}
}
Results for posting a 50KB file
$ curl -vX POST -F file=#test.txt vm/post
Note: Unnecessary use of -X or --request, POST is already inferred.
* Trying 192.168.33.100...
* TCP_NODELAY set
* Connected to vm (192.168.33.100) port 80 (#0)
> POST /post HTTP/1.1
> Host: vm
> User-Agent: curl/7.54.0
> Accept: */*
> Content-Length: 51337
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------67df5f3ef06561a5
>
< HTTP/1.1 403 Forbidden
< Server: openresty/1.11.2.2
< Date: Mon, 11 Sep 2017 17:58:55 GMT
< Content-Type: text/plain
< Content-Length: 23
< Connection: close
<
* Closing connection 0
File uploaded too large%
and nginx logs
web_1 | 2017/09/11 17:58:55 [error] 5#5: *1 client intended to send too large body: 51337 bytes, client: 192.168.33.1, server: , request: "POST /post HTTP/1.1", host: "vm"
web_1 | 192.168.33.1 - - [11/Sep/2017:17:58:55 +0000] "POST /post HTTP/1.1" 403 23 "-" "curl/7.54.0"

nginx = / location pattern not working

I am trying to configure nginx to serve a static html page on the root domain, and proxy everything else to uwsgi. As a quick test I tried to divert to two different static pages:
server {
server_name *.example.dev;
index index.html index.htm;
listen 80;
charset utf-8;
location = / {
root /www/src/;
}
location / {
root /www/test/;
}
}
This seems to be what http://nginx.org/en/docs/http/ngx_http_core_module.html#location says you can do. But I'm always getting sent to the test site, even on the / request by visiting http://www.example.dev in my browser.
Curl output:
$ curl http://www.example.dev -v
* Rebuilt URL to: http://www.example.dev/
* Hostname was NOT found in DNS cache
* Trying 192.168.50.51...
* Connected to www.example.dev (192.168.50.51) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.37.1
> Host: www.example.dev
> Accept: */*
>
< HTTP/1.1 200 OK
* Server nginx/1.8.0 is not blacklisted
< Server: nginx/1.8.0
< Date: Tue, 19 May 2015 01:11:10 GMT
< Content-Type: text/html; charset=utf-8
< Content-Length: 415
< Last-Modified: Wed, 15 Apr 2015 02:53:27 GMT
< Connection: keep-alive
< ETag: "552dd2a7-19f"
< Accept-Ranges: bytes
<
<!DOCTYPE html>
<html>
...
And the output from the nginx access log:
192.168.50.1 - - [19/May/2015:01:17:05 +0000] "GET / HTTP/1.1" 200 415 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.135 Safari/537.36" "-"
So I decided to comment out the test location. So I have only the location = / { ... block. Nginx now 404s and logs the following error:
2015/05/19 01:24:12 [error] 3116#0: *6 open() "/etc/nginx/html/index.html" failed (2: No such file or directory), client: 192.168.50.1, server: *.example.dev, request: "GET / HTTP/1.1", host: "www.example.dev"
Which is the default root in the original nginx conf file? I guess this confirms my location = / pattern is not matching.
I added $uri to the access log and see that it is showing /index.html which I guess means the first location pattern is matching, but then it goes into the second location block? So now I just need to figure out how to serve my index.html from the / block, or just add another block like: location =/index.html
according to #Alexey Ten commented, in ngx_http_index doc:
It should be noted that using an index file causes an internal redirect, and the request can be processed in a different location. For example, with the following configuration:
location = / {
index index.html;
}
location / {
...
}
a “/” request will actually be processed in the second location as “/index.html”.
In your case, request to "/" will not get /www/src/index.html, but /www/test/index.html.

Duplicate Content-Type headers in Nginx for CSV files

I have some .csv files that I'm serving via Nginx straight from the filesystem.
It currently looks like this::
location ~ /static/csv_exports/ {
add_header Content-Type text/csv;
}
For some reason I had to do that because otherwise it would be served as text/plain. Here's what I get when I do a curl on it:
$ curl -v http://localhost/static/csv_exports/20110322_172651.csv >> /dev/null
...
< HTTP/1.1 200 OK
< Server: nginx/0.7.67
< Date: Tue, 22 Mar 2011 17:32:07 GMT
< Content-Type: text/plain
< Content-Length: 356623
< Last-Modified: Tue, 22 Mar 2011 17:26:52 GMT
< Connection: keep-alive
< Cache-Control: public
< Content-Type: text/csv
< Accept-Ranges: bytes
See! It's got two "Content-Type" headers. Opening it in the browser automatically opens Open Office and it works just fine but I suspect I haven't done it right.
location ~ /static/csv_exports/ {
- add_header Content-Type text/csv;
+ types {text/csv csv;}
}

Resources