nginx under supervisor can't open conf file - nginx

This is very puzzling:
I'm on RHEL 7.5, trying to run Nginx 1.14.0 under Supervisor 3.3.4. The ultimate aim is to serve a Django site.
My "/etc/init.d/supervisord" looks like this:
#!/bin/sh
...
# Source init functions
. /etc/rc.d/init.d/functions
prog="supervisord"
prog_bin="/bin/supervisord -c /etc/supervisord.conf"
PIDFILE="/var/run/$prog.pid"
start()
{
echo -n $"Starting $prog: "
daemon $prog_bin --pidfile $PIDFILE
sleep 1
[ -f $PIDFILE ] && success $"$prog startup" || failure $"$prog startup"
echo
}
... # "stop", "restart" functions, etc.
"/etc/supervisord.conf" looks like this:
[unix_http_server]
file=/var/run//supervisor.sock
[supervisord]
logfile=/var/log/supervisor/supervisord.log
pidfile=/var/run/supervisord.pid
childlogdir=/var/log/supervisor
[rpcinterface:supervisor]
supervisor.rpcinterface_factory =
supervisor.rpcinterface:make_main_rpcinterface
[supervisorctl]
serverurl=unix:///var/run//supervisor.sock
[include]
files = /etc/supervisor/conf.d/*.conf
"/etc/supervisor/conf.d/" has just one file in it: nginx.conf:
[program:nginx]
user=root
command=/usr/sbin/nginx -c /path/to/site/etc/nginx.conf
autostart=true
autorestart=true
startretries=3
redirect_stderr=True
Invoking the above command directly with sudo /usr/sbin/nginx -c /path/to/site/etc/nginx.conf & is successful. It starts right up and I can see the nginx processes with ps -ef
But if I start supervisor d like this:
$ sudo /etc/init.d/supervisord restart
It fails to launch Nginx:
$ sudo cat /var/log/supervisor/nginx-stdout---supervisor-tqI97D.log
nginx: [emerg] open() "/path/to/site/etc/nginx.conf" failed (13: Permission denied)
Permissions to read that file are good all the way down. Of course, the path is not actually called "/path/to/site/etc/nginx.conf", but there's an "x" for all users on every directory and a "r" for all users on the conf file itself:
$ namei -om /path/to/site/etc/nginx.conf
f: /path/to/site/etc/nginx.conf
dr-xr-xr-x root root /
drwxr-xr-x root root path
drwxr-xr-x root root to
drwxrwxr-x user1 group1 site
drwxrwxr-x user1 group1 etc
-rw-r--r-- root group1 nginx.conf
How can there be an error on an "open()" operation for this file? I've tried changing the "user" to root in "/etc/supervisor/conf.d/nginx.conf" and/or in "/etc/supervisord.conf" but the result is always the same.
Could the fact that this is SELinux make a difference? It's currently activated.
$ getenforce
Enforcing
If it helps, the nginx.conf file that can't be opened looks like this:
user nginx;
daemon off;
error_log /path/to/site/var/log/nginx-error.log warn;
pid /path/to/site/var/run/nginx.pid;
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /path/to/site/var/log/nginx-access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
upstream app_server {
server unix:/path/to/site/var/run/my-django.socket fail_timeout=0;
}
server {
listen 8000;
server_name xxx.xxx.xxx.xxx;
charset utf-8;
location /media {
alias /path/to/site/htdocs/media;
}
location /static {
alias /path/to/site/htdocs/static;
}
location / {
uwsgi_pass app_server;
include /path/to/site/etc/uwsgi_params;
}
}
}
Does anyone have any ideas?

I suggest you check with SELinux to really grasp the whole concept before doing anything to a production machine.
The httpd_t context permits NGINX to listen on common web server ports, to access configuration files in /etc/nginx, and to access content in the standard docroot location (/usr/share/nginx). It does not permit many other operations, such as proxying to upstream locations or communicating with other processes through sockets.
Once you are a king at SELinux (hah) check out NGINX's - Modifying SELinux Settings
Let us know, good luck!

On SELinux enabled systems process and files have security labels. SELinux policy contains the allowed access patterns between these labels. Access is denied if there is not a rule allowing the access.
The security labels on your nginx files seem to be incorrect: the AVC error message tells that access was denied for a process in httpd_t domain trying to access a file with default_t label. To allow access, you need to assign suitable security contexts to your nginx files. Possible contexts are documented in httpd_selinux man page. For configuration files, an approriate context would be httpd_config_t and for user content perhaps httpd_user_content_t.
You can manually apply a new security context to a file using chcon tool. After you have decided the security contexts to use, you should save them in file context database using semanage. Otherwise automatic relabeling will label the files incorrectly.
I've written a more detailed answer on the topic on related Unix & Linux Stack exchange question Configure SELinux to allow daemons to use files in non-default locations.

Related

How to config nginx to tail log on web browser

My goal is to see the newest log messages on web page.
I know I could use tail -f <file> to trace the latest 10 line log messages on terminal.
But today, I want to config nginx, so that I could see the same result on web.
For example, when I access to http://192.168.1.200/nginx (My Nginx Host)
I could see the files under /var/log/nginx
Index of /nginx/
-------------------------------------------------
../
access.log 08-Aug-2019 16:43 20651
error.log 08-Aug-2019 16:43 17810
And when I access to http://192.168.1.200/nginx/access.log
I could see the same result as tail -f /var/log/nginx/access.log in terminal (and it is dynamic).
/etc/nginx/conf.d/log.conf
server {
listen 80;
root /var/log/nginx;
location /nginx {
root /var/log/;
default_type text/plain;
autoindex on;
}
}
This is my config, but there are 2 points that doesn't meet my requirements:
I want to access to log page by accessing /log/access.log not by /nginx/access.log
When I access to /log/access.log, this page is static.

nginx: create directory if it doesn't exist

I'm new to nginx and I have a given nginx config.
There is a mapping like:
map $http_host $my_customer {
default "default";
"~*cust1" "cust1";
"~*cust2" "cust2";
}
And there is the access_log line:
access_log /my/log/path/access.log
Now I want to have separate log-directories and log-files for each customer, so I changed the access_log line into:
access_log /my/log/path/$my_customer/access.log
This works fine if the $my_customer-directory already exists. But if it doesn't exist, then nginx does not log. I know how I can check if the directory exists:
if (!-d /my/log/path/$my_customer) {}
But how is it possible to create a directory inside the nginx config file?
In order to start nginx process all directories have to be created in advance.
The owner of dir should be the user used by worker processes defined in nginx configuration file (/etc/nginx/nginx.conf by default).
The user should have write permissions to this directory.
As #Alexey Ten noticed, it is a good practice to use default logs location:
/var/log/nginx/$my_customer.access.log
Otherwise, you have to do something like that:
mkdir -p -m 755 /my/log/path/$my_customer

Nginx gives an Internal Server Error 500 after I have configured basic auth

I am trying to do basic auth on Nginx. I have version 1.9.3 up and running on Ubuntu 14.04 and it works fine with a simple html file.
Here is the html file:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<title></title>
</head>
<body>
"Some shoddy text"
</body>
</html>
And here is my nginx.conf file:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
server {
listen 80;
server_name 192.168.1.30;
location / {
root /www;
index index.html;
auth_basic "Restricted";
auth_basic_user_file /etc/users;
}
}
}
I used htpasswd to create two users in the "users" file under /etc (username "calvin" password "Calvin", and username "hobbes" password "Hobbes"). It's encrypted by looks like this:
calvin:$apr1$Q8LGMfGw$RbO.cG4R1riIfERU/175q0
hobbes:$apr1$M9KoUUhh$ayGd8bqqlN989ghWdTP4r/
All files belong to root:root. The server IP address is 192.168.1.30 and I am referencing that directly in the conf file.
It all works fine if I comment out the two auth lines and restart nginx, but if I uncomment them, then I do indeed get the username and password prompts when I try to load the site, but immediately thereafter get an Error 500 Internal Server error which seems to persist and I have to restart nginx.
Anybody can see what I'm doing wrong here? I had the same behaviour on the standard Ubuntu 14.04 apt-get version of Nginx (1.4.something) so I don't think it's the nginx version.
Not really an answer to your question as you are using MD5. However as this thread pops up when searching for the error, I am attaching this to it.
Similar errors happen when bcrypt is used to generate passwords for auth_basic:
htpasswd -B <file> <user> <pass>
Since bcrypt is not supported within auth_basic ATM, mysterious 500 errors can be found in nginx error.log, (usually found at /var/log/nginx/error.log), they look something like this:
*1 crypt_r() failed (22: Invalid argument), ...
At present the solution is to generate a new password using md5, which is the default anyway.
Edited to address md5 issues as brought up by #EricWolf in the comments:
md5 has its problems for sure, some context can be found in the following threads
Is md5 considered insecure?
Is md5 still considered secure for single use authentications?
Of the two, speed issue can be mitigated by using fail2ban, by banning on failed basic auth you'll make online brute forcing impractical (guide). You can also use long passwords to try and fortify a bit as suggested here.
Other than that it seems this is as good as it gets with nginx...
I had goofed up when initially creating a user. As a result, the htpasswd file looked like:
user:
user:$apr1$passwdhashpasswdhashpasswdhash...
After deleting the blank user, everything worked fine.
I was running Nginx in a Docker environment and I had the same issue. The reason was that some of the passwords were generated using bcrypt. I resolved it by using nginx:alpine.
Do you want a MORE secure password hash with nginx basic_auth? Do this:
echo "username:"$(mkpasswd -m sha-512) >> .htpasswd
SHA-512 is not considered nearly as good as bcrypt, but it's the best nginx supports at the moment.
I will just stick the htpassword file under "/etc/nginx" myself.
Assuming it is named htcontrol, then ...
sudo htpasswd -c /etc/nginx/htcontrol calvin
Follow the prompt for the password and the file will be in the correct place.
location / {
...
auth_basic "Restricted";
auth_basic_user_file htcontrol;
}
or auth_basic_user_file /etc/nginx/htcontrol; but the first variant works for me
I just had the same problem - after checking log as suggested by #Drazen Urch I've discovered that the file had root:root permissions - after changing to forge:forge (I'm using Forge with Digital Ocean) - the problem went away.
Well, just use correct RFC 2307 syntax:
passwordvalue = schemeprefix encryptedpassword
schemeprefix = "{" scheme "}"
scheme = "crypt" / "md5" / "sha" / altscheme
altscheme = "x-" keystring
encryptedpassword = encrypted password
For example: sha1 for helloworld for admin will be
admin:{SHA}at+xg6SiyUovktq1redipHiJpaE=
I had same error cause i wrote {SHA1} what against RFC syntax. When i fixed it - all worked like a charm. {sha} will not work too. Only correct {SHA}.
First, check out your nginx error logs:
tail -f /var/log/nginx/error.log
In my case, I found the error:
[crit] 18901#18901: *6847 open() "/root/temp/.htpasswd" failed (13: Permission denied),
The /root/temp directory is one of my test directories, and cannot be read by nginx. After change it to /etc/apache2/ (follow the official guide https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/) everything works fine.
===
After executing the ps command we can see the nginx worker process maintained by the user www-data, I had tried to chown www-data:www-data /root/temp to make sure www-data can access this file, but it still not working. To be honest, I don't have a very deep understanding on Linux File Permissions, so I change it to /etc/apache2/ to fix this in the end. And after a test, you can put the .htpasswd file in other directories which in /etc (like /etc/nginx).
I too was facing the same problem while setting up authentication for kibana. Here is the error in my /var/log/nginx/error.log file
2020/04/13 13:43:08 [crit] 49662#49662: *152 crypt_r() failed (22:
Invalid argument), client: 157.42.72.240, server: 168.61.168.150,
request: “GET / HTTP/1.1”, host: “168.61.168.150”
I resolved this issue by adding authentication using this.
sudo sh -c "echo -n 'kibanaadmin:' >> /etc/nginx/htpasswd.users"
sudo sh -c "openssl passwd -apr1 >> /etc/nginx/htpasswd.users"
You can refer this post if you are trying to setup kibana and got this issue.
https://medium.com/#shubham.singh98/log-monitoring-with-elk-stack-c5de72f0a822?postPublishedType=repub
In my case, I was using plain text password by -p flag, and coincidentally my password start with $ character.
So I updated my password and thus the error was gone.
NB: Other people answer helped me a lot to figure out my problem. I am posting my solution here if anyone stuck in a rare case like me.
In my case, I had my auth_basic setup protecting an nginx location that was served by a proxy_pass configuration.
The configured proxy_pass location wasn't returning a successful HTTP200 response, which caused nginx to respond with an Internal Server Error after I had entered the correct username and password.
If you have a similar setup, ensure that the proxy_pass location protected by auth_basic is returning an HTTP200 response after you rule out username/password issues.

Nginx error: (13: Permission denied) while connecting to upstream

I am getting this error in my nginx-error.log file:
2014/02/17 03:42:20 [crit] 5455#0: *1 connect() to unix:/tmp/uwsgi.sock failed (13: Permission denied) while connecting to upstream, client: xx.xx.x.xxx, server: localhost, request: "GET /users HTTP/1.1", upstream: "uwsgi://unix:/tmp/uwsgi.sock:", host: "EC2.amazonaws.com"
The browser also shows a 502 Bad Gateway Error. The output of a curl is the same, Bad Gateway html
I've tried to fix it by changing permissions for /tmp/uwsgi.sock to 777. That didn't work. I also added myself to the www-data group (a couple questions that looked similar suggested that). Also, no dice.
Here is my nginx.conf file:
nginx.conf
worker_processes 1;
worker_rlimit_nofile 8192;
events {
worker_connections 3000;
}
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
I am running a Flask application with Nginsx and Uwsgi, just to be thorough in my explanation. If anyone has any ideas, I would really appreciate them.
EDIT
I have been asked to provide my uwsgi config file. So, I never personally wrote my nginx or my uwsgi file. I followed the guide here which sets everything up using ansible-playbook. The nginx.conf file was generated automatically, but there was nothing in /etc/uwsgi except a README file in both apps-enabled and apps-available folders. Do I need to create my own config file for uwsgi? I was under the impression that ansible took care of all of those things.
I believe that ansible-playbook figured out my uwsgi configuration since when I run this command
uwsgi -s /tmp/uwsgi.sock -w my_app:app
it starts up and outputs this:
*** Starting uWSGI 2.0.1 (64bit) on [Mon Feb 17 20:03:08 2014] ***
compiled with version: 4.7.3 on 10 February 2014 18:26:16
os: Linux-3.11.0-15-generic #25-Ubuntu SMP Thu Jan 30 17:22:01 UTC 2014
nodename: ip-10-9-xxx-xxx
machine: x86_64
clock source: unix
detected number of CPU cores: 1
current working directory: /home/username/Project
detected binary path: /usr/local/bin/uwsgi
!!! no internal routing support, rebuild with pcre support !!!
*** WARNING: you are running uWSGI without its master process manager ***
your processes number limit is 4548
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/uwsgi.sock fd 3
Python version: 2.7.5+ (default, Sep 19 2013, 13:52:09) [GCC 4.8.1]
*** Python threads support is disabled. You can enable it with --enable-threads ***
Python main interpreter initialized at 0x1f60260
your server socket listen backlog is limited to 100 connections
your mercy for graceful operations on workers is 60 seconds
mapped 72760 bytes (71 KB) for 1 cores
*** Operational MODE: single process ***
WSGI app 0 (mountpoint='') ready in 3 seconds on interpreter 0x1f60260 pid: 26790 (default app)
*** uWSGI is running in multiple interpreter mode ***
spawned uWSGI worker 1 (and the only) (pid: 26790, cores: 1)
The permission issue occurs because uwsgi resets the ownership and permissions of /tmp/uwsgi.sock to 755 and the user running uwsgi every time uwsgi starts.
The correct way to solve the problem is to make uwsgi change the ownership and/or permission of /tmp/uwsgi.sock such that nginx can write to this socket. Therefore, there are three possible solutions.
Run uwsgi as the www-data user so that this user owns the socket file created by it.
uwsgi -s /tmp/uwsgi.sock -w my_app:app --uid www-data --gid www-data
Change the ownership of the socket file so that www-data owns it.
uwsgi -s /tmp/uwsgi.sock -w my_app:app --chown-socket=www-data:www-data
Change the permissions of the socket file, so that www-data can write to it.
uwsgi -s /tmp/uwsgi.sock -w my_app:app --chmod-socket=666
I prefer the first approach because it does not leave uwsgi running as root.
The first two commands need to be run as root user. The third command does not need to be run as root user.
The first command leaves uwsgi running as www-data user. The second and third commands leave uwsgi running as the actual user that ran the command.
The first and second command allow only www-data user to write to the socket. The third command allows any user to write to the socket.
I prefer the first approach because it does not leave uwsgi running as root user and it does not make the socket file world-writeable .
While the accepted solution is true there might also SELinux be blocking the access. If you did set the permissions correctly and still get permission denied messages try:
sudo setenforce Permissive
If it works then SELinux was at fault - or rather was working as expected! To add the permissions needed to nginx do:
# to see what permissions are needed.
sudo grep nginx /var/log/audit/audit.log | audit2allow
# to create a nginx.pp policy file
sudo grep nginx /var/log/audit/audit.log | audit2allow -M nginx
# to apply the new policy
sudo semodule -i nginx.pp
After that reset the SELinux Policy to Enforcing with:
sudo setenforce Enforcing
Anyone who lands here from the Googles and is trying to run Flask on AWS using the default Ubuntu image after installing nginx and still can't figure out what the problem is:
Nginx runs as user "www-data" by default, but the most common Flask WSGI tutorial from Digital Ocean has you use the logged in user for the systemd service file. Change the user that nginx is running as from "www-data" (which is the default) to "ubuntu" in /etc/nginx/nginx.conf if your Flask/wsgi user is "ubuntu" and everything will start working. You can do this with one line in a script:
sudo sed -i 's/user www-data;/user ubuntu;/' /etc/nginx/nginx.conf
Trying to make Flask and uwsgi run as www-data did not work off the bat, but making nginx run as ubuntu worked just fine since all I'm running with this instance is Flask anyhow.
You have to set these permissions (chmod/chown) in uWSGI configuration.
It is the chmod-socket and the chown-socket.
http://uwsgi-docs.readthedocs.org/en/latest/Options.html#chmod-socket
http://uwsgi-docs.readthedocs.org/en/latest/Options.html#chown-socket
Nginx connect to .sock failed (13:Permission denied) - 502 bad gateway
change the name of the user on the first line in /etc/nginx/nginx.conf file.
the default user is www-data and change it to root or your username
I know it's too late, but it might helps to other. I'll suggest to follow Running flask with virtualenv, uwsgi, and nginx very simple and sweet documentation.
Must activate your environment if you run your project in virtualenv.
here is the yolo.py
from config import application
if __name__ == "__main__":
application.run(host='127.0.0.1')
And create uwsgi.sock file in /tmp/ directory and leave it blank.
As #susanpal answer said "The permission issue occurs because uwsgi resets the ownership and permissions of /tmp/uwsgi.sock to 755 and the user running uwsgi every time uwsgi starts." it is correct.
So you have to give permission to sock file whenever uwsgi starts.
so now follow the below command
uwsgi -s /tmp/uwsgi.sock -w yolo:application -H /var/www/yolo/env --chmod-socket=666
A little different command from #susanpal.
And for persist connection, simply add "&" end of command
uwsgi -s /tmp/uwsgi.sock -w yolo:app -H /var/www/yolo/env --chmod-socket=666 &
In my case changing some php permission do the trick
sudo chown user:group -R /run/php
I hope this helps someone.
You should post both nginx and uwsgi configuration file for your application (the ones in /etc/nginx/sites-enabled/ and /etc/uwsgi/ - or wherever you put them).
Typically check that you have a line similar to the following one in your nginx app configuration:
uwsgi_pass unix:///tmp/uwsgi.sock;
and the same socket name in your uwsgi config file:
socket=/tmp/uwsgi.sock

Nginx Password Protect 403 Forbidden

I am running CentOS 6 with nginx. It is currently running perfectly, I am trying to password protect my admin directory.I can successfully login. However, I get a 403 Forbbiden when I try to view the main index page (index.php) in the directory.
2013/04/18 02:10:17 [error] 17166#0: *24 directory index of "/usr/share/ngin/html /somedir/" is forbidden, client: XXX, server: mysite.com, request: "GET /somedir/ HTTP/1.1", host: "mysite.com"
I have double checked permissions on the ".htpasswd" file. It belongs to "root:root" with chmod 640. I have also tried setting owner ship to "nginx:nginx" and the error still persists.
This is how I am getting htpasswd working:
location ~ ^/([^/]*)/(.*) {
if (-f $document_root/$1/.htpasswd) {
error_page 599 = #auth;
return 599;
}
}
location #auth {
auth_basic "Password-protected";
auth_basic_user_file $document_root/$1/.htpasswd;
}
Though the question is pretty old, but I must put my solution here to help others. This very problem was like my pain in somewhere.
I probably have read out (and implemented/tried) almost all possible threads available online (till date) but none solved this "403 Forbidden" nginx issue all-together:
I will write down the steps from beginning: ( block my site access ):
1> We will create a hidden file called .htpasswd in the /etc/nginx
sudo sh -c "echo -n 'usernamee:' >> /etc/nginx/.htpasswd"
2> Now add an encrypted password to the given username
sudo sh -c "openssl passwd -apr1 >> /etc/nginx/.htpasswd"
This will ask you to enter a password and confirm it.
3> Now we need to setup nginx to check our newly created .htpasswd before serving any content.
location / {
try_files $uri $uri/ /index.php?$query_string; # as per my configuration
auth_basic "Authorized access only";
auth_basic_user_file .htpasswd;
}
4> Finally restart the server to take effect
sudo service nginx restart
Now browse the url:
Please note: I didnt do any alteration in permissions. By default the file permission for htpasswd will be set at the time of creation, which will look something like this:
-rw-r--r-- 1 root root 42 Feb 12 12:22 .htpasswd
Read the error carefully. You are missing an index.html or similar.

Resources