I could see lot more thread created for this issue. Tried all the solutions mentioned on those threads. But None of the options worked for me. So I'm creating this new thread.
OS : CentOS 7.2.1511
Nginx : 1.10.0
Error message:
Nov 2 23:21:36 localhost nginx: Starting nginx: nginx: [emerg] socket() x.x.x.x:80 failed (24: Too many open files)
Nov 2 23:21:36 localhost nginx: [FAILED]
Nov 2 23:21:36 localhost systemd: nginx.service: control process exited, code=exited status=1
Nov 2 23:21:36 localhost systemd: Failed to start SYSV: Nginx is an HTTP(S) server, HTTP(S) reverse proxy and IMAP/POP3 proxy server.
Nov 2 23:21:36 localhost systemd: Unit nginx.service entered failed state.
Nov 2 23:21:36 localhost systemd: nginx.service failed.
Ulimit Value:
[root#serv1 nginx]# ulimit -n
16000
[root#serv1 nginx]#
Sysctl.conf:
fs.file-max = 752415
nginx.conf:
worker_rlimit_nofile 30000;
nginx user Ulimit:
[nginx#serv1 ~]$ ulimit -Sn
16000
[nginx#serv1 ~]$ ulimit -Hn
16000
[nginx#serv1 ~]$
Nginx PID max Open files limit:
Max open files 1024 4096 files
Not sure, why the open files limit is not reflected to nginx process. when I checked the pid limit value from proc directory. It shows default value as mentioned above. Values should be 16000 for both soft & hard limits.
Also, I checked the openfiles limit on the server for nginx process. it does not go above 1042.
openfiles :
[root#serv1 ~]# ps -ef | grep nginx
root 4285 1 0 23:12 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/conf/nginx/nginx.conf
nginx 4286 4285 0 23:12 ? 00:00:00 nginx: worker process
root 4308 4290 0 23:12 pts/1 00:00:00 grep --color=auto nginx
[root#serv1 ~]# lsof -p 4285 | wc -l
1042
[root#serv1 ~]#
Someone please help me to fix this problem. Thanks.
Found the problem. its due to nginx.service file was not created as nginx is compiled manually. Once created nginx.service file and added limitonfile value in the file, fixed my problem.
Hope, this may help to people who had hit with this problem.
Related
I've a server with 2 services on it. Software are Ubuntu20.04 OS, Nginx and certbot.
So, then I check nginx status:
~$ sudo systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; disabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2021-12-13 10:48:55 CET; 35min ago
Docs: man:nginx(8)
....
des. 13 10:48:55 u nginx[248435]: nginx: [emerg] bind() to [::]:80 failed (98: Address already in use)
des. 13 10:48:55 u nginx[248435]: nginx: [emerg] still could not bind()
des. 13 10:48:55 u systemd[1]: nginx.service: Control process exited, code=exited, status=1/FAILURE
des. 13 10:48:55 u systemd[1]: nginx.service: Failed with result 'exit-code'.
des. 13 10:48:55 u systemd[1]: Failed to start A high performance web server and a reverse proxy server.
and checking of configuration:
~$ sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
But at the same time both services are up and working as they should. If I ask about nginxs' processes, i've next
~$ sudo ps -ax | grep nginx
245336 ? S 0:00 nginx: worker process
245337 ? S 0:00 nginx: worker process
245338 ? S 0:00 nginx: worker process
250704 pts/7 S+ 0:00 grep --color=auto nginx
and check one of them (it doesn't matter which one)
~$ sudo systemctl status 245336
● snap.certbot.certbot.***********.scope
Loaded: loaded (/run/systemd/transient/snap.certbot.certbot.***********.scope; transien>
Transient: yes
Active: active (running) since Mon 2021-12-13 09:39:30 CET; 1h 35min ago
I can't imagine what happens and why it works in this way. Before this problem, I updated certbot and deleted a few certifates (they were expired and unused).
Who can explain me how it's work? And why are the services up?
This is what it brings when you check nginx status
[root#ttproxyapp conf.d]# service nginx status
Redirecting to /bin/systemctl status nginx.service
● nginx.service - nginx - high performance web server
Loaded: loaded (/usr/lib/systemd/system/nginx.service; disabled; vendor preset: disabled)
Active: inactive (dead)
sudo systemctl stop apache2
sudo systemctl start nginx
should work since looks like the port is in use (with apache probably)
It realy looks like your NGINX ins't running. nginx -s reload is just working if there is a running instance and therfore a PID-file.
Please check the result of the following command.
[root#localhost conf.d]# sudo ps -elf | grep nginx
Should be something like
1 S root 88262 1 0 80 0 - 13143 sigsus 23:47 ? 00:00:00 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
5 S nginx 88263 88262 0 80 0 - 13257 ep_pol 23:47 ? 00:00:00 nginx: worker process
0 R root 88265 69227 0 80 0 - 28178 - 23:47 pts/1 00:00:00 grep --color=auto nginx
You need at least! the master process and the worker process! If there is no process start your instance by typing
sudo /bin/systemctl start nginx.service OR sudo service nginx start
Check your processlist after running the command.
After you have started the NGINX service there should be PID-file located at
/var/run/nginx.pid and sudo systemctl status nginx.service should printout something like
[root#localhost conf.d]# systemctl status nginx.service
● nginx.service - NGINX Plus - high performance web server
Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
Active: active (running) since Sat 2020-03-14 23:47:13 EDT; 5min ago
Docs: https://www.nginx.com/resources/
Process: 88232 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=0/SUCCESS)
Process: 88260 ExecStart=/usr/sbin/nginx -c /etc/nginx/nginx.conf (code=exited, status=0/SUCCESS)
Process: 88251 ExecStartPre=/usr/libexec/nginx-plus/check-subscription (code=exited, status=0/SUCCESS)
Main PID: 88262 (nginx)
Tasks: 2
Memory: 1.7M
CGroup: /system.slice/nginx.service
├─88262 nginx: master process /usr/sbin/nginx -c /etc/nginx/nginx.conf
└─88263 nginx: worker process
Mar 14 23:47:13 localhost.localdomain systemd[1]: Starting NGINX Plus - high performance web server...
Mar 14 23:47:13 localhost.localdomain systemd[1]: Can't open PID file /var/run/nginx.pid (yet?) after start: No such file or directory
Mar 14 23:47:13 localhost.localdomain systemd[1]: Started NGINX Plus - high performance web server.
I am running NGINX Plus but it doesn't matter in this case.
there is maybe an apache server running. you must stop that apache server order run the Nginx server.
sudo systemctl stop apache2
sudo systemctl start nginx
I try to start this service but i can´t, the error below occur:
root#zabbix:/home/appliance# systemctl status nginx.service
nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2018-07-25 18:33:26 UTC; 1min 27s ago
Process: 30040 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=1/FAILURE)
Process: 30037 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
Jul 25 18:33:25 zabbix nginx[30040]: nginx: [emerg] listen() to [::]:80, backlog 511 failed (98: Address already in use)
Jul 25 18:33:25 zabbix nginx[30040]: nginx: [emerg] listen() to 0.0.0.0:80, backlog 511 failed (98: Address already in use)
Jul 25 18:33:25 zabbix nginx[30040]: nginx: [emerg] listen() to [::]:80, backlog 511 failed (98: Address already in use)
Jul 25 18:33:26 zabbix nginx[30040]: nginx: [emerg] listen() to 0.0.0.0:80, backlog 511 failed (98: Address already in use)
Jul 25 18:33:26 zabbix nginx[30040]: nginx: [emerg] listen() to [::]:80, backlog 511 failed (98: Address already in use)
Jul 25 18:33:26 zabbix nginx[30040]: nginx: [emerg] still could not bind()
Jul 25 18:33:26 zabbix systemd[1]: nginx.service: Control process exited, code=exited status=1
Jul 25 18:33:26 zabbix systemd[1]: *******Failed to start A high performance web server*** and a reverse proxy server.****
Jul 25 18:33:26 zabbix systemd[1]: nginx.service: Unit entered failed state.
Jul 25 18:33:26 zabbix systemd[1]: nginx.service: Failed with result 'exit-code'.
You already have a process bound to the HTTP port 80. (Specially after upgrading systems! it will start apache2 by default)
So first try this:
sudo service apache2 stop
sudo systemctl restart nginx
If problem is not solved then run this command sudo lsof -i:80 to get a list of processes using the port and then stop or disable web server.
Try to stop the process which are using the port 80 using:
sudo fuser -k 80/tcp
sudo systemctl restart nginx
In some cases it may be some issues in the configuration file.
You can use nginx -t -c /etc/nginx/nginx.conf command to find any miss-configuration.
In some cases this error is caused by a default Nginx site already on port 80. Removing default config works if you don't need a default one!
sudo rm /etc/nginx/sites-enabled/default
sudo service nginx restart
For me this error was caused by a default nginx site already on port 80. Removing default site worked
sudo rm /etc/nginx/sites-enabled/default
sudo service nginx restart
You already have a process bound to the HTTP port 80.
You can run command sudo lsof -i:80 to get a list of processes using the port and then stop/disable web server.
Try to stop the process which are using the port 80:
sudo fuser -k 80/tcp
Try to stop the process which is using port 80:
sudo fuser -k 80/tcp
When you did restart using
sudo systemctl restart nginx
It may be some issues in the configuration file. You can use this
nginx -t -c /etc/nginx/nginx.confcommand to find any bugs in the configuration file. If you find the bugs resolve that run the sudo service nginx restart again. It will work.
Please check the reference here
In my experience, this error can be triggered in several different situations (which might have the same root, but are perceived as different scenarios).
Not only port 80
Depending on your nginx config, you should also try port 443:
sudo lsof -i:80
sudo lsof -i:443
You should be able to check your nginx listen ports in /etc/nginx/sites-enabled (under Debian)
Special case
In my case, there was an nginx instance running, that would block itself from restarting. It would not go down with sudo systemctl stop nginx. I had to use sudo killall nginx and could then use sudo systemctl start nginx.
Config error
I also experienced the exact same error message on a config error.
You can check your nginx config with /usr/sbin/nginx -c /etc/nginx/nginx.conf# (this will also test all virtual host files (/etc/nginx/sites-enabled`)
i had same issue, was because i have apache and nginx on same server !
so when i typed sudo reboot it didn't work because it started apache while am using nginx so i just run this two commands
sudo systemctl stop apache2
then
sudo systemctl start nginx
then sudo systemctl disable apache2
so next time i reboot not going to face same issue
help from aws services support
I had to use:
sudo killall nginx
And could then use;
sudo systemctl start nginx
For me stopping apache service solved the problem
sudo service apache2 stop
I had this problem too. I checked the /etc/nginx/site-available/default file and I have forgotten to close the bracket (syntax error).
Just restart the nginx server. It should resolve your problem
sudo systemctl restart nginx
I was using gunicorn with nginx, found that service was not active.
sudo systemctl enable gunicorn.socket
sudo systemctl start gunicorn.socket
sudo systemctl restart nginx
Worked for me.
if your gunicorn is active, please make sure it's working fine.
you can also go through the logs(errors) via nginx logs from
sudo tail -f /var/log/nginx/error.log
Certbot and nginx versions:
certbot installed using certbot.eff.org install guide.
Certbot version: 0.22.2
Nginx version: 1.10.3
Getting ssl certificates works fine:
certbot --nginx
But, in renewal of cerbot certificated
certbot renew --dry-run
nginx fails to start causing:
nginx: [error] open() "/run/nginx.pid" failed (2: No such file or directory)
I have tried changing post-hook and pre-hook in /etc/letsencrypt/renewal/*com.conf/
commenting installer=nginx
changing authenticator to nginx and standalone
Adding post and pre hooks in /etc/letsencrypt/renewal-hooks/pre/ and /etc/lestencrypt/renewal-hooks/post/ to stop and start nginx service.
Seems nginx is not starting properly or isn't stop properly.
after renewal completes nginx fails with (code=exited, status=1/FAILURE)
Nginx error log show:
Error while certbot renew:
Try to execute:
sudo service nginx restart
Then test your nginx configuration file(s) (until you see "nginx: configuration file /etc/nginx/nginx.conf test is successful")
sudo nginx -s reload -t
Pay attention on paths to certificates, and other stuff
and then reload configuration without -t option:
sudo nginx -s reload
It's not recommended to modify configuration files in /etc/letsencrypt/ but creating (if it doesn't exist) and modifying cli.ini file here is working for me. You can specify post-hook in this file once and it will work for all your certificates, see my current file:
# /etc/letsencrypt/cli.ini
max-log-backups = 0
authenticator = webroot
webroot-path = /var/www/html
post-hook = service nginx reload
text = True
I hope this will help future readers. Solution source is here (however the article is in Russian)
I had the same issue on Ubuntu 16.04
I've just removed post and pre hooks in /etc/letsencrypt/renewal/*.conf and changed authenticator to nginx - I had in two entries standalone.
And it is working now fine.
Edit:
Recommended way to update renewal config is to reissue new certificate using:
certbot -i nginx -d example.com -d www.example.com certonly
You can run this command line before run reload nginx.
sudo nginx -c /etc/nginx/nginx.conf
or
sudo nginx -c /usr/local/etc/nginx/nginx.conf
then you can start nginx nomaly
sudo nginx -s reload
Good luck.
I had the same error...
When I installed certbot, I followed the instructions and put in a cronjob (5 3 15 * *):
certbot renew --pre-hook "service nginx stop" --post-hook "service nginx start"
this morning nginx was dead, and the log showed
open() "/run/nginx.pid" failed (2: No such file or directory)
I did not connect the two, but do I understand that certbot triggers the nginx failure?
ps -ef | grep nginx, find all nginx process
sudo kill -9 xxx xxx xxx or sudo pkill nginx
sudo systemctl restart nginx
sudo nginx -t
I had this problem and followed a similar tack to those outlined here.
I had had certbot install a certificate, but it was in certonly --nginx mode, I supplied my own nginx serverblocks. certbot worked, but an nginx failure cast doubt on the accuracy of my provisioning.
This certbot call "restarts" nginx with a modified server block configuration, so it can answer the HTTP-01 challenges. I know this because when it fails, it will log, "nginx restart failed:" just before the bind() failures I'm about to show. My nginx server was down when provisioning succeeded.
I couldn't get systemctl or service to start it and systemd status nginx would only ever show "failed".
Whilst I could get nginx up, and serving, with nginx -s reload I wanted systemd to manage it for me.
No amount of systemctl {start|restart|stop|quit} nginx, would work. The status remained as failed and would show errors with bind():
Oct 07 10:04:13 HostXYZ systemd[1]: Starting A high performance web server and a reverse proxy server...
Oct 07 10:04:13 HostXYZ nginx[17096]: nginx: [emerg] bind() to 0.0.0.0:80 failed (98: Unknown error)
Oct 07 10:04:13 HostXYZ nginx[17096]: nginx: [emerg] bind() to [::]:80 failed (98: Unknown error)
Oct 07 10:04:13 HostXYZ nginx[17096]: nginx: [emerg] bind() to [::]:443 failed (98: Unknown error)
Oct 07 10:04:13 HostXYZ nginx[17096]: nginx: [emerg] bind() to 0.0.0.0:443 failed (98: Unknown error)
That would repeat in journalctl output, 4 or 5 times.
I checked the process and saw:
:~$ ps aux | grep nginx
root 12960 0.0 0.6 77216 9816 ? Ss Oct06 0:00 nginx: master process nginx -c /etc/nginx/nginx.conf
www-data 16944 0.0 0.5 77360 8604 ? S 08:43 0:00 nginx: worker process
That process, which appeared to be occupying the ports needed by my systemd service. My systemd service doesn't use that -c /etc/nginx/nginx.conf. It uses:
ExecStart=/usr/sbin/nginx -g 'daemon on; master_process on;'
nginx -s stop, and quit would not rid me of the rogue process. Instead they both gave the error the OP had:
:~$ sudo nginx -s stop
nginx: [error] open() "/run/nginx.pid" failed (2: No such file or directory)
Both my systemd service unit and /etc/nginx/nginx.conf gave /run/nginx.pid as the PIDFile/pid. For some reason, /etc/nginx/nginx.conf wasn't creating it.
What I needed to do:
sudo killall nginx
sudo systemctl start nginx
That knocked out the other nginx service (I think it came from nginx -s reload but I couldn't shut it down by the corollary command) Which looked like this:
:~$ sudo killall nginx
:~$ ps aux | grep nginx
john 17140 0.0 0.1 4008 2004 pts/0 S+ 10:10 0:00 grep --color=auto nginx
:~$ sudo systemctl start nginx
:~$ sudo systemctl status nginx
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: active (running) since Fri 2022-10-07 10:10:25 UTC; 1s ago
...
:~$ ps aux | grep nginx
root 11481 0.0 0.1 76484 2588 ? Ss 10:10 0:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 11482 0.0 0.2 76876 4284 ? S 10:10 0:00 nginx: worker process
:~$ cat /run/nginx.pid
11481
My NGINX server have stopped and it is generating error.
I am using Ubantu 16.04 and my app deploye at Digital Ocean Server.
When I run the following colde sudo systemctl start nginx It is giving following output:
Job for nginx.service failed because the control process exited with error code. See "systemctl status nginx.service" and "journalctl -xe" for details.
Output of $ nginx -t is:
nginx: [emerg] BIO_new_file("/root/bitradiology.chained.crt") failed (SSL: error:02001002:system library:fopen:No such file or directory:fopen('/root/bitradiology.chained.crt','r') error:2006D080:BIO routines:BIO_new_file:no such file)
nginx: configuration file /etc/nginx/nginx.conf test failed
I checked the status with $ sudo systemctl status nginx
output is:
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2017-08-21 11:01:51 UTC; 1min 50s ago
Process: 2085 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=2)
Process: 2420 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=1/FAILURE)
Main PID: 2462 (code=exited, status=0/SUCCESS)
Aug 21 11:01:51 ubuntu-512mb-nyc3-01 systemd[1]: Starting A high performance web server and a reverse proxy server...
Aug 21 11:01:51 ubuntu-512mb-nyc3-01 nginx[2420]: nginx: [emerg] BIO_new_file("/root/bitradiology.chained.crt") failed (SSL: error:02001002:system library:fopen:No such
Aug 21 11:01:51 ubuntu-512mb-nyc3-01 nginx[2420]: nginx: configuration file /etc/nginx/nginx.conf test failed
Aug 21 11:01:51 ubuntu-512mb-nyc3-01 systemd[1]: nginx.service: Control process exited, code=exited status=1
Aug 21 11:01:51 ubuntu-512mb-nyc3-01 systemd[1]: Failed to start A high performance web server and a reverse proxy server.
Aug 21 11:01:51 ubuntu-512mb-nyc3-01 systemd[1]: nginx.service: Unit entered failed state.
Aug 21 11:01:51 ubuntu-512mb-nyc3-01 systemd[1]: nginx.service: Failed with result 'exit-code'.
lines 1-14/14 (END)
How to fix this problem
I found this bitradiology.chained.crt file into the cd /usr directory.
Then go to the following directory cd /etc/nginx/sites-enabled and open the default file
sudo nano default
I edited this file and replaced /root/ by /usr/
/root/bitradiology.chained.crt by /usr/bitradiology.chained.crt
I run the nginx config test first:
nginx -t
after the test ran successful, I can restart the service
/etc/init.d/nginx restart
NGINX has been started and my app is running right now.
Thanks!!!
Check your ssl_certificate instructions in your config files.
Nginx is clearly saying it can't find one of the files specified:
fopen:No such file or
directory:fopen('/root/bitradiology.chained.crt','r')