The last & break mechanism in nginx rewrite - nginx

Am now dealing with nginx rewrite for my project but got something out of my expectation, just share with you to see if any reasonable advice on this.
The nginx server setting list below:
location /download/ {
rewrite ^(/download/.*)/media/(.*)\..*$ $1/mp3/$2.mp3;
rewrite ^(/download/.*)/movie/(.*)\..*$ $1/avi/$2.mp3 break;
rewrite ^(/download/.*)/avvvv/(.*)\..*$ $1/rmvb/$2.mp3;
}
From above section, you can see that I only add one download location with three rewrite rules.I open the debug log in nginx and start it to load this setting.
Now we enter url: localhost/download/123/movie/UBW.avi in our browser. From the log, we can see that the rules are hit, logs below:
2017/05/22 15:27:27 [notice] 1904#12520: *85 "^(/download/.*)/media/(.*)\..*$" does not match "/download/123/movie/UBW.avi", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:27:27 [notice] 1904#12520: *85 "^(/download/.*)/movie/(.*)\..*$" matches "/download/123/movie/UBW.avi", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:27:27 [notice] 1904#12520: *85 rewritten data: "/download/123/avi/UBW.mp3", args: "", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:27:27 [error] 1904#12520: *85 CreateFile() "D:\Soft\nginx-1.13.0/html/download/123/avi/UBW.mp3" failed (3: The system cannot find the path specified), client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
From above info, we can see that the second rule is hit and the section break due to the "break" keyword. This behavior is in our expectation.
But wen I changed the section like below and behave the same behavior as above:
location /download/ {
rewrite ^(/download/.*)/media/(.*)\..*$ $1/mp3/$2.mp3;
rewrite ^(/download/.*)/movie/(.*)\..*$ $1/avi/$2.mp3 last;
rewrite ^(/download/.*)/avvvv/(.*)\..*$ $1/rmvb/$2.mp3;
}
The log below:
2017/05/22 15:33:45 [notice] 11244#11544: *92 "^(/download/.*)/media/(.*)\..*$" does not match "/download/123/movie/UBW.avi", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:33:45 [notice] 11244#11544: *92 "^(/download/.*)/movie/(.*)\..*$" matches "/download/123/movie/UBW.avi", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:33:45 [notice] 11244#11544: *92 rewritten data: "/download/123/avi/UBW.mp3", args: "", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:33:45 [notice] 11244#11544: *92 "^(/download/.*)/media/(.*)\..*$" does not match "/download/123/avi/UBW.mp3", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:33:45 [notice] 11244#11544: *92 "^(/download/.*)/movie/(.*)\..*$" does not match "/download/123/avi/UBW.mp3", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:33:45 [notice] 11244#11544: *92 "^(/download/.*)/avvvv/(.*)\..*$" does not match "/download/123/avi/UBW.mp3", client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
2017/05/22 15:33:45 [error] 11244#11544: *92 CreateFile() "D:\Soft\nginx-1.13.0/html/download/123/avi/UBW.mp3" failed (3: The system cannot find the path specified), client: 127.0.0.1, server: localhost, request: "GET /download/123/movie/UBW.avi HTTP/1.1", host: "localhost"
From the second row of the log, we can see that the rule is hit, but then nginx start a new loop for the download section, the strange thing is that , it didn't hit the rule, why?
Also, if we use last keyword, then how many times will the nginx try to loop the section, in default it's two, right?

The ngx_http_rewrite_module module is used to change request URI using
PCRE regular expressions
Remember that when you rewrite the uri, the uri will change .
For last
When rewrite module meets last, it stops processing the current set
and the rewritten request is passed once again to find the appropriate
location (and the new set of rewriting rules).
In your configure
location /download/ {
rewrite ^(/download/.*)/media/(.*)\..*$ $1/mp3/$2.mp3; //a
rewrite ^(/download/.*)/movie/(.*)\..*$ $1/avi/$2.mp3 last; //b
rewrite ^(/download/.*)/avvvv/(.*)\..*$ $1/rmvb/$2.mp3; //c
}
your url localhost/download/123/movie/UBW.avi
your request uri is /download/123/movie/UBW.avi
first pass
the /download/123/movie/UBW.avi match b ,so the uri rewrite to /download/123/avi/UBW.mp3 ,the first pass finish.
second pass
And then, in the second pass ,the uri
is
/download/123/avi/UBW.mp3
So it do not match a b c, finally , it throw error in log.

Related

nginx 502 for 10s while upstream is restarting

I have a very simple config:
server {
listen 80;
server_name: example.fr;
location / {
proxy_pass http://localhost:8000;
}
}
However when I restart my backend (for example Node.JS restarted on change by nodemon), even if the backend starts like in 2 seconds, Nginx returns a 502 for 10s and this shows in the logs:
2022/05/10 16:10:48 [error] 2013398#2013398: *2241 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://127.0.0.1:8000/", host: "example.fr"
2022/05/10 16:10:48 [error] 2013398#2013398: *2241 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://[::1]:8000/", host: "example.fr"
2022/05/10 16:10:49 [error] 2013398#2013398: *2244 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:50 [error] 2013398#2013398: *2245 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:51 [error] 2013398#2013398: *2246 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:52 [error] 2013398#2013398: *2247 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:53 [error] 2013398#2013398: *2248 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:54 [error] 2013398#2013398: *2249 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:55 [error] 2013398#2013398: *2250 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:57 [error] 2013398#2013398: *2251 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
2022/05/10 16:10:58 [error] 2013398#2013398: *2252 no live upstreams while connecting to upstream, client: 127.0.0.1, server: example.fr, request: "HEAD / HTTP/2.0", upstream: "http://localhost/", host: "example.fr"
I've seen that for the upstream directive there are some controls (max_fails and fail_timeout (http://nginx.org/en/docs/http/ngx_http_upstream_module.html#max_fails)) but this only seems to apply to upstream and I cannot find equivalent options for plain proxy_pass.
Any idea?
Short answer: Replace localhost with 127.0.0.1.
Quoting user #rogerdpack for the long answer:
The really tricky part is that if you specify proxy_pass to
"localhost" and your box happens to also have ipv6 and ipv4 "versions
of localhost" on it at the same time (most boxes do by default), it
will count as if you had a "list" of multiple servers in your server
group, which means you can get into the situation above of having it
return "502 for 10s" even though you list only one server.
See here "If a domain name resolves to several addresses, all of them
will be used in a round-robin fashion." One workaround is to declare
it as proxy_pass http://127.0.0.1:8000; (its ipv4 address) to
avoid it being both ipv6 and ipv4. Then it counts as "only a
single server" behavior.
(answering my own question because the answers explaining the problem were buried or the question wasn't really explicit about the particular symptoms I had)

laravel valet-linux redirect error

I'm running the very useful valet-linux package for my local dev environment. I recently updated it (using composer global update), and now my local Joomla dev sites are not loading properly.
The error in ~/.valet/Log/nginx-error.log shows multiple occurrences of
2018/08/11 11:00:28 [error] 7825#7825: *11 rewrite or internal redirection cycle while processing "/home/admin/.config/composer/vendor/cpriego/valet-linux/server.php", client: 127.0.0.1, server: , request: "GET /media/plg_quickicon_extensionupdate/js/extensionupdatecheck.js?95ac38c67651723d445944ba719bf971 HTTP/1.1", upstream: "fastcgi://unix:/home/admin/.valet/valet.sock", host: "joomla310.test", referrer: "http://joomla310.test/administrator/index.php"
2018/08/11 11:00:28 [error] 7825#7825: *38 rewrite or internal redirection cycle while processing "/home/admin/.config/composer/vendor/cpriego/valet-linux/server.php", client: 127.0.0.1, server: , request: "GET /media/jui/js/bootstrap.min.js?95ac38c67651723d445944ba719bf971 HTTP/1.1", upstream: "fastcgi://unix:/home/admin/.valet/valet.sock", host: "joomla310.test", referrer: "http://joomla310.test/administrator/index.php"
2018/08/11 11:00:28 [error] 7825#7825: *40 rewrite or internal redirection cycle while processing "/home/admin/.config/composer/vendor/cpriego/valet-linux/server.php", client: 127.0.0.1, server: , request: "GET /media/mod_sampledata/js/sampledata-process.js HTTP/1.1", upstream: "fastcgi://unix:/home/admin/.valet/valet.sock", host: "joomla310.test", referrer: "http://joomla310.test/administrator/index.php"
2018/08/11 11:00:28 [error] 7825#7825: *42 rewrite or internal redirection cycle while processing "/home/admin/.config/composer/vendor/cpriego/valet-linux/server.php", client: 127.0.0.1, server: , request: "GET /media/system/js/core.js?95ac38c67651723d445944ba719bf971 HTTP/1.1", upstream: "fastcgi://unix:/home/admin/.valet/valet.sock", host: "joomla310.test", referrer: "http://joomla310.test/administrator/index.php"
2018/08/11 11:00:28 [error] 7825#7825: *44 rewrite or internal redirection cycle while processing "/home/admin/.config/composer/vendor/cpriego/valet-linux/server.php", client: 127.0.0.1, server: , request: "GET /administrator/templates/isis/js/template.js?95ac38c67651723d445944ba719bf971 HTTP/1.1", upstream: "fastcgi://unix:/home/admin/.valet/valet.sock", host: "joomla310.test", referrer: "http://joomla310.test/administrator/index.php"
2018/08/11 11:00:28 [error] 7825#7825: *62 rewrite or internal redirection cycle while processing "/home/admin/.config/composer/vendor/cpriego/valet-linux/server.php", client: 127.0.0.1, server: , request: "GET /administrator/templates/isis/images/logo.png HTTP/1.1", upstream: "fastcgi://unix:/home/admin/.valet/valet.sock", host: "joomla310.test", referrer: "http://joomla310.test/administrator/index.php"
I'm fairly experienced with Apache, but I don't know where to begin debugging the nginx errors. Has anyone run into this?
Uninstalling and re-installing valet-linux resolved the issue. Still not sure what had happened.

nginx error log - request from unknown hosts

I am getting request from unknown hosts in nginx error.log trying to request files or requests which don't exist in the system.
Receiving below list of errors in error log,
directory index is forbidden, client: 61.157.96.126, request: "GET http://www.haosou.com/?rands=_688651072032161501826712 HTTP/1.1", host: "www.haosou.com"
[error] 10006#0: *67753 open() /favicon.ico" failed (2: No such file or directory), client: 95.25.186.223, request: "POST http://t3.proxy-checks.com/favicon.ico HTTP/1.1", host: "t3.proxy-checks.com"
[error] 10006#0: *66849 directory index of "--" is forbidden, client: 120.132.95.89, request: "GET http://www.qunar.com/ HTTP/1.1", host: "www.qunar.com"
What could be the issue here? Do I need to make any configuration changes on my server to stop such requests? or Is it ok to receive such requests on server?

WordPress Crashing, nginx and fastcgi issue?

I currently have this problem with my wordpress website where it will work for about a day after a server restart, but then hit this set of errors:
2015/12/15 22:06:42 [crit] 12650#0: *28 connect() to unix:/var/run/php5-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: 46.166.139.20, server: example.com, request: "POST /xmlrpc.php HTTP/1.0", $
2015/12/15 22:08:19 [error] 3216#0: *18 FastCGI sent in stderr: "PHP message: PHP Warning: trim() expects parameter 1 to be string, array given in /var/www/html/wp-includes/option.php on line 30
PHP message: PHP Warning: trim() expects parameter 1 to be string, array given in /var/www/html/wp-includes/option.php on line 30" while reading response header from upstream, client: 104.33.64.70, server: example.com, request: "P$
2015/12/15 22:40:08 [error] 3216#0: *197 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.142, server: example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/$
2015/12/15 22:40:20 [error] 3216#0: *199 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.19, server: example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/v$
2015/12/15 23:08:27 [error] 3216#0: *201 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.143, server: example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/$
2015/12/15 23:08:39 [error] 3216#0: *203 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 180.76.15.12, server: example.com, request: "GET / HTTP/1.1", upstream: "fastcgi://unix:/v$
2015/12/15 23:20:20 [error] 3216#0: *205 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 104.33.64.70, server: example.com, request: "GET /wp-admin/upgrade.php?step=1&backto=%2Fwp$
2015/12/15 23:22:20 [error] 3216#0: *205 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 104.33.64.70, server: example.com, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcg$
2015/12/15 23:57:09 [error] 3216#0: *367 connect() to unix:/var/run/php5-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 46.166.139.20, server: example.com, request: "POST /xmlrpc.php HT$
2015/12/15 23:57:39 [error] 3216#0: *369 connect() to unix:/var/run/php5-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 46.166.139.20, server: example.com, request: "POST /xmlrpc.php HT$
2015/12/15 23:57:41 [error] 3216#0: *371 connect() to unix:/var/run/php5-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 46.166.139.20, server: example.com, request: "POST /xmlrpc.php HT$
2015/12/15 23:57:56 [error] 3216#0: *373 connect() to unix:/var/run/php5-fpm.sock failed (11: Resource temporarily unavailable) while connecting to upstream, client: 46.166.139.20, server: example.com, request: "POST /xmlrpc.php HT$
Afterwards, the error log just repeats that (11: Resource temporarily unavailable) code over and over. At this point trying to access the website itself just gives an nginx "an error has occurred" page, telling me to check error logs.
I don't know what exactly is causing the initial holdup, and it looks like after a few time outs the server just locks up entirely. Any advice? Thanks!
Someone from 46.166.139.20 tries to guess your password. If you don't use WP's XML-RPC you should disable it.

Kibana/nginx could not connect to elasticsearch

I have Elasticsearch and Kibana with nginx running on a EC2 instance but Kibana/nginx is not able to connect to Elasticsearch and I'm getting "Error Could not contact Elasticsearch at xxxxxxxx. Please ensure that Elasticsearch is reachable from your system".
When I see the nginx error logs, I get -
[error] 13067#0: *1 access forbidden by rule, client: xxxxx, server: 127.0.0.1, request: "GET /index.html HTTP/1.1", host: "xxxxx"
[error] 13283#0: *14 no user/password was provided for basic authentication, client: xxxxx, server: xxxxxx, request: "GET //cgi-bin/php5 HTTP/1.1", host: "xxxxx"
My conf file is - http://pastebin.com/JuxifP2n

Resources