Need some help in setting up nginx logs so that they are not duplicated.
My configuration is as following. What I would like to achieve is that all logs for say, http://example.com/app goes to file app.access.log and logs for rest of the site goes to file main.access.log
However, following configuration logs app logs to both, app.access.log and main.access.log.
server {
access_log /var/log/nginx/main.access.log;
location /app {
access_log /var/log/nginx/app.access.log;
}
}
Any idea how to fix this?
You could use a negation regexp to intercept all request NOT directed to app, and define there the access_log directive. Then define the other location for app
location ~ ^((?!app).)*$ {
access_log /var/log/nginx/not-an-app.access.log;
}
location /app {
access_log /var/log/nginx/app.access.log;
}
I think it's a bit of a stretch though, and i would test the hell out of this before putting it in production.
The access_log directive includes an if=condition which can be used to control logging.
For example:
map $request_uri $loggable {
~^/app 0;
default 1;
}
server {
access_log /var/log/nginx/main.access.log if=$loggable;
...
}
See this document for details.
The alternative is to log everything together and split it into two separate files later using grep.
Inspired by #Andrea's solution, you could also use this pattern:
server {
location / {
access_log /var/log/nginx/main.access.log;
location /foo { ... }
location /bar { ... }
...
}
location /app {
access_log /var/log/nginx/app.access.log;
}
}
So the top level has just two top level location blocks, and all other location blocks are nested within the default `location.
Related
I made a sample application that produces dynamic static files. I basically do not want to enter all the routes manually into the default file of Nginx. I saw some solutions here but the answers were not understandable for me in my simple case.
So basically my structure looks like this :
the main link of the application:
https://mysite.come/myproject/products
dynamic routes are numbers that also the link has query string
for example:
https://mysite.come/myproject/products/1?rand=something
https://mysite.come/myproject/products/2?rand=something
https://mysite.come/myproject/products/3?rand=something
https://mysite.come/myproject/products/4?rand=something
I could basically write :
location /myproject/products/1/ {
proxy_pass http://localhost:8000/1;
}
and need to repeat this for all the products which are none sense. Is there any way to do it automatically?
...
when I use dev mode ( npm run dev) I get
maybe using try_files? see the last part of this article
location /myproject/products/ {
proxy_pass http://localhost:8000;
}
server {
listen 8000;
server_name localhost;
location / {
root path_of_web_bundle;
index index.html index.htm;
try_files $uri $uri/ /index.html;
}
}
First of all, I have to deal with a project-related upgrade of a TYPO3 version and am also relatively new to the Nginx web server.
I have upgraded a TYPO3 installation from version 7.6-LTS to 8.7-LTS and transferred the site from an IIS (Windows) server to an Ubuntu 18.04 system with Nginx.
I have now discovered the following: the first click on an internal link on the website in the menu e.g. domain.com/prices works correctly. The URL domain.com/prices is called and also shown in the URL. Now when the page has been reloaded, the same menu item link now looks like this...
domain.com/index.php?id=8&L=1%20or%20%281%2C2%29%3D%28select%2Afrom%28select%20name_const%28CHAR%28111%2C108%2C111%2C108%2C111%2C115%2C104%2C101%2C114%29%2C1%29%2Cname_const%28CHAR%28111%2C108%2C111%2C108%2C111%2C108%2C111%2C115%2C104%2C101%2C114%29%2C1%29%29a%29%20- -%20and%201%3D1
...instead of the usual domain.com/prices. The oontent of the page is still correctly, so the correct page is being loaded.
The Typo3 installation uses the extension real url, which is exactly for this feature. It translates domain.com/index.php?id=8&L=1... into a speaking url like domain.com/prices. So why is it working on the first load, but then no longer?
I also thought, it could be an issue with the configuration of Nginx, but now I think it is a different topic here, because if it is a webserver issue, it won't work the first time, will it?
What can cause this behaviour?
Update
I actually saw, that the English version of the website is working well. So here, the links are displayed correctly: domain.com/prices, domain.com/modules and not even not first page load.
Maybe it is helpful. This is the Nginx config, I currently use:
server {
listen 80;
server_name domain.com;
root /var/www/domain.com/public;
index index.php index.html index.htm index.nginx-debian.html;
listen 443 ssl;
ssl_certificate /var/www/domain.com/ssl/domain.com-2020.crt;
ssl_certificate_key /var/www/domain.com/ssl/domain.com-2020.rsa;
location / {
try_files $uri $uri/ /index.php$is_args$args;
}
# Special root site case. prevent "try_files $uri/" + "index" from skipping the cache
# by accessing /index.php directly
location =/ {
recursive_error_pages on;
error_page 405 = #sfc;
return 405;
}
location #t3frontend {
# Using try_files for ease of configuration demonstration here,
# you can also fastcgi_pass directly to php here
try_files $uri /index.php$is_args$args;
}
location #sfc {
# Perform an internal redirect to TYPO3 if any of the required
# conditions for StaticFileCache don't match
error_page 405 = #t3frontend;
# Query String needs to be empty
if ($args != '') {
return 405;
}
# We can't serve static files for logged-in BE/FE users
if ($cookie_staticfilecache = 'fe_typo_user_logged_in') {
return 405;
}
if ($cookie_be_typo_user != '') {
return 405;
}
# Ensure we redirect to TYPO3 for non GET/HEAD requests
if ($request_method !~ ^(GET|HEAD)$ ) {
return 405;
}
charset utf-8;
try_files /typo3temp/tx_staticfilecache/${scheme}/${host}/${server_port}${uri}/index.html
/typo3temp/tx_staticfilecache/${scheme}/${host}/${server_port}${uri}
=405;
}
location /typo3temp/tx_staticfilecache {
deny all;
}
location ~ \.php$ {
include snippets/fastcgi-php.conf;
fastcgi_pass unix:/var/run/php/php7.2-fpm.sock;
}
location ~ /\.ht {
deny all;
}
# Prevent clients from accessing hidden files (starting with a dot)
# This is particularly important if you store .htpasswd files in the site hierarchy
# Access to `/.well-known/` is allowed.
# https://www.mnot.net/blog/2010/04/07/well-known
# https://tools.ietf.org/html/rfc5785
location ~* /\.(?!well-known\/) {
deny all;
}
# Prevent clients from accessing to backup/config/source files
location ~* (?:\.(?:bak|conf|dist|fla|in[ci]|log|psd|sh|sql|sw[op])|~)$ {
deny all;
}
# TYPO3 - Block access to composer files
location ~* composer\.(?:json|lock) {
deny all;
}
# TYPO3 - Block access to flexform files
location ~* flexform[^.]*\.xml {
deny all;
}
# TYPO3 - Block access to language files
location ~* locallang[^.]*\.(?:xml|xlf)$ {
deny all;
}
# TYPO3 - Block access to static typoscript files
location ~* ext_conf_template\.txt|ext_typoscript_constants\.(?:txt|typoscript)|ext_typoscript_setup\.(?:txt|typoscript) {
deny all;
}
# TYPO3 - Block access to miscellaneous protected files
location ~* /.*\.(?:bak|co?nf|cfg|ya?ml|ts|typoscript|dist|fla|in[ci]|log|sh|sql)$ {
deny all;
}
# TYPO3 - Block access to recycler and temporary directories
location ~ _(?:recycler|temp)_/ {
deny all;
}
# TYPO3 - Block access to configuration files stored in fileadmin
location ~ fileadmin/(?:templates)/.*\.(?:txt|ts|typoscript)$ {
deny all;
}
# TYPO3 - Block access to libaries, source and temporary compiled data
location ~ ^(?:vendor|typo3_src|typo3temp/var) {
deny all;
}
# TYPO3 - Block access to protected extension directories
location ~ (?:typo3conf/ext|typo3/sysext|typo3/ext)/[^/]+/(?:Configuration|Resources/Private|Tests?|Documentation|docs?)/ {
deny all;
}
}
server {
if ($host = domain.com) {
return 301 https://$host$request_uri;
}
listen 80;
server_name domain.com;
return 404;
}
And this is the config of realurl:
The template analysis does not show any error. This is the template analysis for realurl:
Update 2
I'm a step further in this issue. I've read the bug report https://github.com/dmitryd/typo3-realurl/issues/333 which is similar to my issue. I tried to switch the config.linkVars value from L to L(0,1). That helped in the URL issue. But now it switches always back to German language.
With the setting L in config.linkVars, it added the language in the url, if it was not German:
domain.com/en/prices
With the new value L(0,1) it removes the language in the URL
domain.com/prices
So it always shows the German version of the page. I even tried to set the value to L(0,2), because of German, English and French language is integrated (1=English, 2=French), but it does not help in this case. It switches always back to German because of the missing language code in the URL.
Ok. Now I found the solution.
As I wrote in the comments it was the config setting:
config.linkVars
which was set initially to
config.linkVars = L
After I found this bug report
https://github.com/dmitryd/typo3-realurl/issues/333
I saw that I need to limit the L-value for security reasons.
In our case 0 for german, 1 for english, 2 for french:
config.linkVars = L(0-2)
Then all worked fine.
I want to disable logging for a specific useragent. This is a part of my current conf-file.
if ($http_user_agent ~ (bingbot|AhrefsBot|DotBot|Exabot|Baiduspider|SemrushBot) ) {
return 403;
}
I've tried adding access_log off; but get the following error:
nginx: [emerg] "access_log" directive is not allowed here
I'm assuming this is because I only have a server block. I need a location block also. I've tried the following code:
location / {
if ($http_user_agent ~ (bingbot|AhrefsBot|DotBot|Exabot|Baiduspider|SemrushBot) ) {
return 403;
}
}
But I get the following error:
duplicate location "/"
In my conf-file I already have this code:
location / {
try_files $uri $uri/ =404;
}
Can I combine the two location snippets into one? Or how do I proceed?
As your question indicates, the access_log directive cannot be used within an if block unless enclosed within a location. However, the access_log directive does include an if=condition which can be controlled by a map. There is an example at the end of this section of the manual.
For example:
map $http_user_agent $goodagent {
default 1;
~(bingbot|AhrefsBot|DotBot|Exabot|Baiduspider|SemrushBot) 0;
}
server {
access_log ... if=$goodagent;
if ($goodagent = 0) { return 403; }
...
}
The map directive must be placed outside of the server block. The access_log statement can be placed inside or outside the server block depending on whether it applies to all server blocks or just one.
At the http level declare a map like so.
map $http_user_agent $ignore_status_checks {
default 0;
"~Pingdom.*" 1;
"~*\(StatusCake\)" 1;
"~*mod_pagespeed*" 1;
"~*NodePing*" 1;
}
Then in your server's location block add:
if ($ignore_status_checks) {
access_log off;
}
This will turn off the access_log for anything returns a 1 in the map. Of course, you can do want ever you want in the if.
I'm trying to deploy two different versions of same app on the same nginx-based server. If the URL starts with /v2, then "v2" should be used, otherwise use v1. Example:
http://example.com/v2/x/y/z * runs v2 app
http://example.com/anything/else * runs v1 app
The two different versions of the app are proxied through nginx, and that piece works well.
The issue is that I have two directories of static assets, /static and /cachedassets, that are common to both versions (and both rooted from /home/v1|2/www/public. So, even though a request to http://example.com/v2/x/y/z will initially use the right app, the page that loads will contain references to /static and /cachedassets, without the /v2 prefix, that will incorrectly load from /home/v1/www/public.
I know referer is an imperfect solution. As a temporary stopgap, until I have a chance to craft a more robust solution, I'm trying to use nginx's $http_referer to point to the correct location for these assets. Here's the nginx file:
server {
listen 1.2.3.4
server_name example.com
...
location /v2 {
root /home/v2/www/public;
try_files $uri #proxyv2;
access_log off;
expires max;
}
location ^/(static|cachedassets) {
root /home/v1/www/public;
if ($http_referer ~* "/v2/") {
root /home/v2/www/public;
}
}
location / {
root /home/v1/www/public;
try_files $uri #proxyv1;
access_log off;
expires max;
}
location #proxyv1 {
include uwsgi_params;
uwsgi_pass unix:///tmp/v1-www.sock;
uwsgi_modifier1 5;
}
location #proxyv2 {
include uwsgi_params;
uwsgi_pass unix:///tmp/v2-www.sock;
uwsgi_modifier1 5;
}
...
}
Any thoughts?
Bonus points for a solution where I can easily specify several "v2" prefixes. For example, here I might specify v2, versiontwo, and vtwo, and the following URLs would all invoke the v2 app:
http://www.example.com/v2/something
http://www.example.com/versiontwo/abc
http://www.example.com/vtwo/abc/def/ghi
and of course, http://www.example.com/somethingelse would run v1.
I'm also open to other ideas that don't use http_referer to accomplish this.
Thanks!
Rather than if blocks, use a map variable to set the root. The map can contain a number of arbitrarily complex regular expressions. See this document for more.
For example:
map $http_referer $root {
default "/home/v1/www/public";
~*/v2/ "/home/v2/www/public";
}
server {
...
location ~ ^/(static|cachedassets) {
root $root;
}
...
}
I'm trying to log only java-script files request in the nginx access_log.
I tried using the following code i found on this site:
location ~* ^.+.(jpg|jpeg|gif|css|png|html|htm|ico|xml|svg)$ {
access_log off;
}
the problem is it doesn't allow the get request at all and i get a 404 error when trying to run the html file that executes the js file in the browse.
I want everything to work just the same but for the access log to log only request for js files.
How do i do that?
Put it in the server block and make sure that the "root" is correctly set up. It does work
Working example:
location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
expires +60d;
access_log off;
}
I have this in the server block and not a location block.
Alternatively you can keep all requests within single location but use access_log with condidional if operator to disable images logging:
map $request_uri $is_loggable {
~* ^.+\.(jpg|jpeg|gif|css|png|html|htm|ico|xml|svg)$ 0;
default 1;
}
server {
location / {
access_log /path/to/log/file combined if=$is_loggable;
...
}
}
Here combined is a name of default log format.
You say that you want to log only java-script files, so actually you can use even simplier solution:
map $request_uri $is_loggable {
~* ^.+\.js$ 1;
default 0;
}