Switching from apache to nginx, and encountering something weird.
1) Say I have a file yo.txt in the document root of my site and it contains 'foo'.
curl http://localhost/yo.txt => 'foo'
2) then I alter the file to contain 'bar'
curl http://localhost/yo.txt => 'foo' (still!)
If I remove yo.txt, I get a 404. If I remove all the text, I correctly get an empty file when I curl the url.
I checked the last modified HTTP header after I modify the file, and it is correct, even though the contents of the file are stale.
I'm using the standard configuration from nginx after an apt-get install nginx.
what gives?
I'm using Vagrant. Setting sendfile to off in nginx.conf fixed the problem as found here, e.g."
sendfile off;
For me the following worked:
expires modified 10y;
According to the docs:
The time in the “Expires” field is computed as a sum of the current time and time specified in the directive. If the modified parameter is used (0.7.0, 0.6.32) then the time is computed as a sum of the file’s modification time and the time specified in the directive.
Related
We're got a pre-existing rsyslog config file which is working for papertrail e.g.
/etc/rsyslog.d/20-papertrail.conf which has
*.* #logs4.papertrailapp.com:44407
However we've got a couple of NGINX websites on the server so would like to have it also monitor their error logs.
The paths to them are:
/var/log/nginx/www.website-one.com-error.log
/var/log/nginx/www.website-two.com-error.log
/var/log/nginx/www.website-three.com-error.log
However this /var/log/nginx also contains a bunch of .log files which we do not want to monitor e.g.
/var/log/nginx/error.log
/var/log/nginx/access.log
/var/log/nginx/error.log1
/var/log/nginx/nginx.log
In my head we need to add something like...
/var/log/nginx/*-error.log
And make sure they pipe to the papertrail url as well.
However I'm struggling to decipher the rsyslog documentation to figure out how to do this.
Thanks!
In rsyslog documentation it seems that you can use wildcards in files.
File
The file being monitored. So far, this must be an absolute name (no macros or templates). Note that wildcards are supported at the file name level (see WildCards below for more details).
WildCards
Before Version: 8.25.0
Wildcards are only supported in the filename part, not in directory names.
/var/log/*.log works.
/var/log/*/syslog.log does not work.
Since Version: 8.25.0
Wildcards are supported in filename and paths which means these samples will work:
/var/log/*.log works.
/var/log/*/syslog.log works.
/var/log/*/*.log works.
All matching files in all matching subfolders will work. Note that this may decrease performance in imfile depending on how many directories and files are being watched dynamically.
If you want to forward your vhosts logs you can change configuration directly in NGINX vhosts configuration, you should change/add access_log and error_log policies as explained here or use custom facilities to forward your logs (using rsyslog).
HOW TO DO IT USING RSYSLOG?
Create a new custom file in /etc/rsyslog.d/nginx_custom.conf:
module(load="imfile" PollingInterval="1") #needs to be done just once
# File 1
input(type="imfile"
File="/var/log/nginx/www.website-*.com-error.log"
Tag="websites"
Facility="local0")
local0.* #logs4.papertrailapp.com:44407
#Just to test that logs are forwarded, comment the line once you've tested it
local0.* /var/log/test.log
And restart rsyslog service
NOTE: Line local0.* /var/log/test.log is just to test that you can see forwarded logs into your local server, comment this line after you've tested that everything works.
I spent days now in researching on how to add some headers to nginx. All I try to do is adding these lines:
location ~ ^/(assets)/ {
add_header Access-Control-Allow-Origin *;
}
What is the best way to put these lines into the nginx.conf?
Is there also a way to not overwrite the standard nginx.conf just in case beanstalk updates the settings so I wont miss it?
The default elastic beanstalk nginx.conf seems to have this line toward the end :
include /etc/nginx/conf.d/*.conf;
(Well, I can tell you that's what the file looks like for the docker solution stack versions 1.4.1 and 2.0.4, no idea if that's guaranteed across all solution stacks).
So I think one way would be to to drop a file named whatever.conf into the /etc/nginx/conf directory using the ebextensions mechanism .
I need to know about full downloading a resource from server. My server is configured with NginX web server, and I want to do something if and only if the resource downloaded completely by user.
If you are using Nginx to handle downloading your files (using XSendfile), you should add a specific access_log block to your download handling block in your Nginx configs (in your "server" block). It would be something like this:
location /download_music/ {
internal;
root /usr/share/nginx/MY_SITE/media/music;
access_log /var/log/nginx/download.MY_SITE.log download;
}
The word "download" at the end of the access_log line is actually a log format which you should add it to the nginx main config file (/etc/nginx/nginx.conf) in the "http" block. It could be something like this:
http {
...
log_format download '{ "remote_addr" : "$remote_addr", "time":"$time_local", "request":"$request", '
'"traffic":$body_bytes_sent, "x_forwarded_for":"$http_x_forwarded_for" }';
...
}
You can change this format to what format you want (you will use it in your script later). If you monitor this log file (using "tail -f /var/log/nginx/download.MY_SITE.log") you will see that any time a download is paused or finished, a line of log will be added to this file.
The next step is using rsyslog and the "imfile" and "omprog" modules. You should add these configs at the end of the config file of rsyslog (/etc/rsyslog.conf):
$ModLoad imfile
$InputFileName /var/log/nginx/download.MY_SITE.log
$InputFileTag nginx-access
$InputFileStateFile state-nginx-access
$InputFileSeverity info
$InputFileFacility local3
$InputFilePollInterval 1
$InputRunFileMonitor
$ModLoad omprog
$actionomprogbinary /usr/share/nginx/MY_SITE/scripts/download.py
$template LogZillaDbInsert,"%hostname:::lowercase%\9%pri%\9%programname%\9%msg%\n"
local3.* :omprog:;RSYSLOG_TraditionalFileFormat
Pay attention to this line:
/usr/share/nginx/MY_SITE/scripts/download.py
This is the address of the script which would be called every time a log entry is added to the log file and the whole log entry will be available in this script using (in Python code):
line = sys.stdin.readline()
Then you can parse the line and find whatever you have logged including the downloaded file size (on every pause/finish event). Now, you can simply include the file size in the download URL and retrieve it in this script and compare it with the downloaded bytes. If these two numbers are equal, it means that download has been finished successfully. Moreover, you can do any other thing you want in this script (for example, expire download link, increase download count on DB, ...)
I am trying to execute cgi script on Nginx. In nginx.conf file, I added a location directive such as below:
location /cgi-bin{
root cgi-bin;
index index.cgi;
}
When I try to connect to http://example.com/cgi-bin/index.cgi, it says file not found. In error.log, I see the request as "http://example.com/html/cgi-bin/index.cgi.
cgi-bin is not in html folder. The correct path is "http://example.com/cgi-bin/index.cgi
I tried many possibilities for location directive. Either it looks in html/cgi-bin or it does /cgi-bin/cgi-bin/index.cgi. I am not sure why it uses 'cgi-bin' twice
Any suggestions.. I have been trying for hours now!!!
I'm currently migrating from lighttpd to nginx.
I've got some weird files (don't ask why):
1. say a file named 'news', which actually should be more like news.txt
2. a file named '.html', which actually should be index.html
With lighttpd, simply rewrite those things would work.
Nginx would still locate those files with try_files or rewrite, but I've got no control of the content type returned. I mean if the file is named '.html', the content type is 'application/octet-stream'.
I know I can use more_set_headers to achieve that, but is there any other way to do that? I mean why does nginx think a file named '.html' not an html file?
I mean why does nginx think a file named '.html' not an html file?
A dot at the beginning in unix-like systems is usually used as the indicator of hidden files. In this case, a part after the dot isn't file extension.
I know I can use more_set_headers to achieve that, but is there any other way to do that?
You should use the default_type directive instead of 3-rd party modules.
For example:
location =/.html {
default_type text/html;
}