My stack is nginx that runs python web.py fast-cgi scripts using spawn-fcgi. I am using runit to keep the process alive as a Daemon. I am using unix sockets fior the spawed-fcgi.
The below is my runit script called myserver in /etc/sv/myserver with the run file in /etc/sv/myserver/run.
exec spawn-fcgi -n -d /home/ubuntu/Servers/rtbTest/ -s /tmp/nginx9002.socket -u www-data -f /home/ubuntu/Servers/rtbTest/index.py >> /var/log/mylog.sys.log 2>&1
I need to push changes to the sripts to the production servers. I use paramiko to ssh into the box and update the index.py script.
My question is this, how do I gracefully reload the index.py using best practice to update to the new code.
Do I use:
sudo /etc/init.d/nginx reload
Do I restart the the runit script:
sudo sv start myserver
Or do I use both:
sudo /etc/init.d/nginx reload
sudo sv start myserver
Or none of the above?
Basically you have to re-start the process that's loaded your Python script. This is spawn-cgi and not nginx itself. nginx only communicates with spawn-cgi via the Unix socket and will happily re-connect if the connection is lost due to a restart of the spawn-cgi process.
Therefore I'd suggest a simple sudo sv restart myserver. No need to re-start/re-load nginx itself.
Related
I have tried following some tutorials and documentation on dockerizing my web server, but I am having trouble getting the service to run via the docker run command.
This is my Dockerfile:
FROM ubuntu:trusty
#Update and install stuff
RUN apt-get update
RUN apt-get install -y python-software-properties aptitude screen htop nano nmap nginx
#Add files
ADD src/main/resources/ /usr/share/nginx/html
EXPOSE 80
CMD service nginx start
I create my image:
docker build -t myImage .
And when I run it:
docker run -p 81:80 myImage
it seems to just stop:
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
90e54a254efa pms-gui:latest /bin/sh -c service n 3 seconds ago Exit 0 prickly_bohr
I would expect this to be running with port 81->80 but it is not. Running
docker start 90e
does not seem to do anything.
I also tried entering it directly
docker run -t -i -p 81:80 myImage /bin/bash
and from here I can start the service
service nginx start
and from another tab I can see it is working as intended (also in my browser):
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
408237a5e10b myImage:latest /bin/bash 12 seconds ago Up 11 seconds 0.0.0.0:81->80/tcp mad_turing
So I assume it is something I am doing wrong with my Dockerfile? Could anyone help me out with this, I am quite new to Docker. Thank you!
SOLUTION: Based on the answer from Ivant I found another way to start nginx in the foreground. My Dockerfile CMD now looks like:
CMD /usr/sbin/nginx -g "daemon off;"
As of now, the official nginx image uses this to run nginx (see the Dockerfile):
CMD ["nginx", "-g", "daemon off;"]
In my case, this was enough to get it to start properly. There are tutorials online suggesting more awkward ways of accomplishing this but the above seems quite clean.
Docker container runs as long as the command you specify with CMD, ENTRTYPOINT or through the command line is running. In your case the service command finishes right away and the whole container is shut down.
One way to fix this is to start nginx directly from the command line (make sure you don't run it as a daemon).
Another option is to create a small script which starts the service and then sleeps forever. Something like:
#!/bin/bash
service nginx start
while true; do sleep 1d; done
and run this instead of directly running the service command.
A third option would be to use something like runit or similar program, instead of the normal service.
Using docker-compose:
To follow the recommended solution, add to docker-compose.yml:
command: nginx -g "daemon off"
I also found I could simply add to nginx.conf:
daemon off;
...and continue to use in docker-compose.yml:
command: service nginx start
...although it would make the config file less portable outside docker.
Docker as a very nice index of offical and user images. When you want to do something, chances are someone already did it ;)
Just search for 'nginx' on index.docker.io, you will see, there is an official nginx image: https://registry.hub.docker.com/_/nginx/
There you have a full guide to help you start your webserver.
Feel free to take a look at other users nginx image to see variants :)
The idea is to start nginx in foreground mode.
If you run "service nginx start", it is a parent process which will start a child process of nginx. If you run "service nginx start" as CMD in a container, the Process ID 1 for the container will be "service nginx start" or ServiceManager (SystemD), while actual nginx would be running as a child process.
If you run "service nginx start", and then "ps -ef", you will get output as below. I have run it my host OS.
root#ip-172-31-85-74:/home/ubuntu# service nginx start
root#ip-172-31-85-74:/home/ubuntu#
root#ip-172-31-85-74:/home/ubuntu# ps -ef | grep nginx
root 18593 1 0 12:27 ? 00:00:00 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
www-data 18595 18593 0 12:27 ? 00:00:00 nginx: worker process
root 18599 17918 0 12:27 pts/0 00:00:00 grep --color=auto nginx
So, here the process ID 18593 is the child process which has parent process 1.
Container exits when their Process ID 1 exits. And in case of CMD "service nginx start", the PID 1 is the process manager, may be SystemD. It starts nginx as a child process, and exits itself, hence the container exits.
Similarly, if you run a shell script (for eg : start.sh) in CMD, as soon as the script ends, the container will exit. Even though the script starts some services (eg - nginx) in its execution, as soon as the script ends, the container will exit, because the PID 1 will be of the shell script. The parent process will be "./start.sh", and the services started by script will be child processes. In case you want to use a shell script in CMD, and want the container to run indefinitely, you need a command at last of the script which doesn't let it end. Something as shown below:
#!/bin/bash
service nginx start
while true; do sleep 1d; done
Below is by code for spawing a fcgi script for nginx.
spawn-fcgi -d /home/ubuntu/workspace -f /home/ubuntu/workspace/index.py -a 127.0.0.1 -p 9001
Now, lets I want to make changes to the index.py script and reload with out bring down the system. How do reload the spawned program so the next connections are using the updated program while the others finish? For now I am killing the spawned process and running command again. I am hoping for something more graceful.
I tried this by the way.
sudo kill -1 `sudo lsof -t -i:9001
I have recently made something similar for node.js.
The idea is to have index.py as a very simple bootstrap script (which doesnât actually change much over time). It should catch SIGHUP, and reload/reread the application files (which are expected to change frequently).
I am using supervisor to launch and manage a nginx process. So far this works perfectly. The problem I am having is shutting down the instance.
I have tried using "supervisorctl -c shutdown [all]" and this shuts down the daemon and in the supervisorctl interactive console it says nginx is stopped. However, if I do a ps -A | grep nginx command it still appears in the list.
My config for the nginx instance is as follows:
[program:nginx]
command=./bin/nginx
-p /home/me/sites/project.domain.com/
-c project/etc/nginx.conf
directory=/home/me/sites/project.domain.com
autostart=true
autorestart=true
redirect_stderr=true
exitcodes=0
stopsignal=TERM
Any suggestion why nginx could not be shutting down?
Have you made sure you are not starting up nginx in daemonized mode? It is important you start all your child-processes of supervisor in non-daemonized mode. I currently don't have nginx boot options at hand, but this might give you a start in the right direction.
I issued a nginx -s stop and after that I got this error when trying to reload it.
[error]: invalid PID number "" in "/var/run/nginx.pid"
That /var/run/nginx/pid file is empty atm.
What do I need to do to fix it?
nginx -s reload is only used to tell a running nginx process to reload its config. After a stop, you don't have a running nginx process to send a signal to. Just run nginx (possibly with a -c /path/to/config/file)
in my case I solved this by starting the service.
sudo /etc/init.d/nginx start
The command above will start the service in Debian/Ubuntu. It will issue an error if there is any problem (like Apache listening in the same port)
After that nginx -s reload will work like a charm
This will clear out the issue on ubuntu 16.04 and above
sudo service nginx stop
you may need to remove the pid file nginx.pid whose location may be defined in file /etc/nginx/nginx.conf look for line like
cat /etc/nginx/nginx.conf | grep pid # see if pid file is defined
this line may live in file /etc/nginx/nginx.conf
pid /run/nginx.pid; # in file /etc/nginx/nginx.conf
if pid file does exist then remove it now
ls -la /var/run/nginx/pid # this file may live elsewhere
ls -la /run/nginx.pid # on Ubuntu 16.04+
after the pid file has been removed lets launch nginx
sudo service nginx start
ps -eaf|grep nginx # confirm its running
sudo nginx -t && sudo nginx -s reload # confirm config is OK
# typical output
# nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
# nginx: configuration file /etc/nginx/nginx.conf test is successful
sudo service nginx stop # issue stop
ps -eaf|grep nginx # confirm it actually stopped
now sanity has been restored and you are free to launch at will
In the latest version(1.2.0) that I downloaded there is no "-s start" option, it will say
nginx: invalid option: "-s start"
You can start nginx by
sudo /etc/nginx/sbin/nginx
The server will be started and then there wont be any Invalid pid number errors.
To avoid downtime with restarting nginx,
ps aux | grep nginx
PID of nginx master process
echo PID > /var/run/nginx.pid
nginx -s reload
In my case nginx was stopped (crashed I assume). Solved the issue by:
service nginx status
nginx stop/waiting
service nginx start
nginx start/running, process 3535
Then nginx -s reload worked like a charm.
I am using nginx/1.8.0 on trusty.
This happens if the nginx process was stopped manually or was killed.
Check if the process is still running:
sudo lsof -nP -iTCP:<port> | grep LISTEN
I am on mac, and I reinstall the nginx with:
brew reinstall nginx
Then start the service using brew:
brew services start nginx
On CentOS 7 I done it with this:
sudo systemctl start nginx
#Then check all things are OK
sudo systemctl status -l nginx
For anyone who still has issues, in my case, there was an apache2 server that was running.
You can try debugging what went wrong in your nginx machine by executing this command -
systemctl status nginx
This gave me an insight that the port was already in us by apache2 server.
so you can do sudo service apache2 stop and then do sudo service nginx start.
Docker Alpine users should use
nginx
by using that nginx will be start there is no error by
nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
then reload it by
nginx -s reload
So I want to be able to cap:deploy without having to type any passwords. I have setup all private keys so I can get to the remote servers fine, and am now using svn over ssh, so no passwords there.
I have one last problem, I need to be able to restart nginx. Right now I have sudo /etc/init.d/nginx reload. That is a problem b/c it uses the capistrano password, the one I just removed b/c I am using keys. Any ideas on how to restart nginx w\out a password?
I just spent a good hour looking at sudoer wildcards and the like trying to solve this exact problem. In truth, all you really need is a root executable script that restarts nginx.
Add this to the /etc/sudoers file
username hostname ALL=NOPASSWD: /path/to/script
Write script as root
#! /bin/bash
/bin/kill -HUP `cat /var/run/nginx.pid`
Make the script executable
Test.
sudo /path/to/script
There is a better answer on Stack Overflow that does not involve writing a custom script:
The best practice is to use /etc/sudoers.d/myusername
The /etc/sudoers.d/ folder can contain multiple files that allow users
to call stuff using sudo without being root.
The file usually contains a user and a list of commands that the user
can run without having to specify a password.
Instructions:
In all commands, replace myusername with the name of your user that you want to use to restart nginx without sudo.
Open sudoers file for your user:
$ sudo visudo -f /etc/sudoers.d/myusername
Editor will open. There you paste the following line. This will allow that user to run nginx start, restart, and stop:
myusername ALL=(ALL) NOPASSWD: /usr/sbin/service nginx start,/usr/sbin/service nginx stop,/usr/sbin/service nginx restart
Save by hitting ctrl+o. It will ask where you want to save, simply press enter to confirm the default. Then exit out of the editor with ctrl+x.
Now you can restart (and start and stop) nginx without password. Let's try it.
Open new session (otherwise, you might simply not be asked for your sudo password because it has not timed out):
$ ssh myusername#myserver
Stop nginx
$ sudo /usr/sbin/service nginx stop
Confirm that nginx has stopped by checking your website or running ps aux | grep nginx
Start nginx
$ sudo /usr/sbin/service nginx start
Confirm that nginx has started by checking your website or running ps aux | grep nginx
PS: Make sure to use sudo /usr/sbin/service nginx start|restart|stop, and not sudo service nginx start|restart|stop.
Run sudo visudo
Append with below lines (in this example you can add multiple scripts and services after comma)
# Run scripts without asking for pass
<your-user> ALL=(root) NOPASSWD: /opt/fixdns.sh,/usr/sbin/service nginx *,/usr/sbin/service docker *
Save and exit with :wq
Create a rake task in Rails_App/lib/capistrano/tasks/nginx.rake and paste below code.
namespace :nginx do
%w(start stop restart reload).each do |command|
desc "#{command.capitalize} Nginx"
task command do
on roles(:app) do
execute :sudo, "service nginx #{command}"
end
end
end
end
Then ssh to your remote server and open file
sudo vi /etc/sudoers
and the paste this line (after line %sudo ALL=(ALL:ALL) ALL)
deploy ALL=(ALL:ALL) NOPASSWD: /usr/sbin/service nginx *
Or, as in your case,
deploy ALL=(ALL:ALL) NOPASSWD: /etc/init.d/nginx *
Here I am assuming your deployment user is deploy.
You can add here other commands too for which you dont require to enter password. For example
deploy ALL=(ALL:ALL) NOPASSWD: /usr/sbin/service nginx *, /etc/init.d/mysqld, /etc/init.d/apache2