How do you properly watch and reload Nginx conf? - nginx

I have two questions:
Is there a difference between: nginx -s reload and pkill -HUP -F nginx.pid
What's the simplest way to watch the Nginx conf file and upon changes test the conf file (nginx -t), and if it passes reload Nginx. Can that be done with runit or a process manager like Supervisor?

#!/bin/bash
# NGINX WATCH DAEMON
#
# Author: Devonte
#
# Place file in root of nginx folder: /etc/nginx
# This will test your nginx config on any change and
# if there are no problems it will reload your configuration
# USAGE: sh nginx-watch.sh
# Set NGINX directory
# tar command already has the leading /
dir='etc/nginx'
# Get initial checksum values
checksum_initial=$(tar --strip-components=2 -C / -cf - $dir | md5sum | awk '{print $1}')
checksum_now=$checksum_initial
# Start nginx
nginx
# Daemon that checks the md5 sum of the directory
# ff the sums are different ( a file changed / added / deleted)
# the nginx configuration is tested and reloaded on success
while true
do
checksum_now=$(tar --strip-components=2 -C / -cf - $dir | md5sum | awk '{print $1}')
if [ $checksum_initial != $checksum_now ]; then
echo '[ NGINX ] A configuration file changed. Reloading...'
nginx -t && nginx -s reload;
fi
checksum_initial=$checksum_now
sleep 2
done

At least on Unix, both "reload" action and HUP signal are treated as one thanks to the declaration code
ngx_signal_t signals[] = {
{ ngx_signal_value(NGX_RECONFIGURE_SIGNAL),
"SIG" ngx_value(NGX_RECONFIGURE_SIGNAL),
"reload",
ngx_signal_handler },
in src/os/unix/ngx_process.c. In ngx_signal_handler() the same comnmon code
case ngx_signal_value(NGX_RECONFIGURE_SIGNAL):
ngx_reconfigure = 1;
action = ", reconfiguring";
break;
is executed, that prepares for a common reconfiguration.
To trigger an action when a file is modified, you could either make a crontab and decide of a check-periodicity, or use inotifywait.
To determine if nginx -t is in error, check the return code in a bash file, $?
nginx -t
if [ $? -eq 0 ] then;
nginx -s reload
fi
Note: you may also use service nginx reload
(See return code check examples here)

Related

how to install Nginx on CentOs7 without internet connection with root permission?

I need to install Nginx on my target which there is no internet connection, how can I install Nginx with all dependencies in an offline mode?? thanks in advance for your answers.
I have recently gone through this procedure and this is what worked for me on centos7:
You need an online Linux server to download dependencies. You can use virtual machines or anything else.
On your online server create a .sh file and copy script below in it. (I named it download_dependencies)
#!/bin/bash
# This script is used to fetch external packages that are not available in standard Linux distribution
# Example: ./fetch-external-dependencies ubuntu18.04
# Script will create nms-dependencies-ubuntu18.04.tar.gz in local directory which can be copied
# into target machine and packages inside can be installed manually
set -eo pipefail
# current dir
PACKAGE_PATH="."
mkdir -p $PACKAGE_PATH
declare -A CLICKHOUSE_REPO
CLICKHOUSE_REPO['ubuntu18.04']="https://repo.clickhouse.tech/deb/lts/main"
CLICKHOUSE_REPO['ubuntu20.04']="https://repo.clickhouse.tech/deb/lts/main"
CLICKHOUSE_REPO['centos7']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['centos8']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['rhel7']="https://repo.clickhouse.tech/rpm/lts/x86_64"
CLICKHOUSE_REPO['rhel8']="https://repo.clickhouse.tech/rpm/lts/x86_64"
declare -A NGINX_REPO
NGINX_REPO['ubuntu18.04']="https://nginx.org/packages/mainline/ubuntu/pool/nginx/n/nginx/"
NGINX_REPO['ubuntu20.04']="https://nginx.org/packages/mainline/ubuntu/pool/nginx/n/nginx/"
NGINX_REPO['centos7']="https://nginx.org/packages/mainline/centos/7/x86_64/RPMS/"
NGINX_REPO['centos8']="https://nginx.org/packages/mainline/centos/8/x86_64/RPMS/"
NGINX_REPO['rhel7']="https://nginx.org/packages/mainline/rhel/7/x86_64/RPMS/"
NGINX_REPO['rhel8']="https://nginx.org/packages/mainline/rhel/8/x86_64/RPMS/"
CLICKHOUSE_KEY="https://repo.clickhouse.com/CLICKHOUSE-KEY.GPG"
NGINX_KEY="https://nginx.org/keys/nginx_signing.key"
declare -A CLICKHOUSE_PACKAGES
# for Clickhouse package names are static between distributions
# we use ubuntu/centos entries as placeholders
CLICKHOUSE_PACKAGES['ubuntu']="
clickhouse-server_21.3.10.1_all.deb
clickhouse-common-static_21.3.10.1_amd64.deb"
CLICKHOUSE_PACKAGES['centos']="
clickhouse-server-21.3.10.1-2.noarch.rpm
clickhouse-common-static-21.3.10.1-2.x86_64.rpm"
CLICKHOUSE_PACKAGES['ubuntu18.04']=${CLICKHOUSE_PACKAGES['ubuntu']}
CLICKHOUSE_PACKAGES['ubuntu20.04']=${CLICKHOUSE_PACKAGES['ubuntu']}
CLICKHOUSE_PACKAGES['centos7']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['centos8']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['rhel7']=${CLICKHOUSE_PACKAGES['centos']}
CLICKHOUSE_PACKAGES['rhel8']=${CLICKHOUSE_PACKAGES['centos']}
declare -A NGINX_PACKAGES
NGINX_PACKAGES['ubuntu18.04']="nginx_1.21.3-1~bionic_amd64.deb"
NGINX_PACKAGES['ubuntu20.04']="nginx_1.21.2-1~focal_amd64.deb"
NGINX_PACKAGES['centos7']="nginx-1.21.4-1.el7.ngx.x86_64.rpm"
NGINX_PACKAGES['centos8']="nginx-1.21.4-1.el8.ngx.x86_64.rpm"
NGINX_PACKAGES['rhel7']="nginx-1.21.4-1.el7.ngx.x86_64.rpm"
NGINX_PACKAGES['rhel8']="nginx-1.21.4-1.el8.ngx.x86_64.rpm"
download_packages() {
local target_distribution=$1
if [ -z $target_distribution ]; then
echo "$0 - no target distribution specified"
exit 1
fi
mkdir -p "${PACKAGE_PATH}/${target_distribution}"
# just in case delete all files in target dir
rm -f "${PACKAGE_PATH}/${target_distribution}/*"
readarray -t clickhouse_files <<<"${CLICKHOUSE_PACKAGES[${target_distribution}]}"
readarray -t nginx_files <<<"${NGINX_PACKAGES[${target_distribution}]}"
echo "Downloading Clickhouse signing keys"
curl -fs ${CLICKHOUSE_KEY} --output "${PACKAGE_PATH}/${target_distribution}/clickhouse-key.gpg"
echo "Downloading Nginx signing keys"
curl -fs ${NGINX_KEY} --output "${PACKAGE_PATH}/${target_distribution}/nginx-key.gpg"
for package_file in "${clickhouse_files[#]}"; do
if [ -z $package_file ]; then
continue
fi
file_url="${CLICKHOUSE_REPO[$target_distribution]}/$package_file"
save_file="${PACKAGE_PATH}/${target_distribution}/$package_file"
echo "Fetching $file_url"
curl -fs $file_url --output $save_file
done
for package_file in "${nginx_files[#]}"; do
if [ -z $package_file ]; then
continue
fi
file_url="${NGINX_REPO[$target_distribution]}/$package_file"
save_file="${PACKAGE_PATH}/${target_distribution}/$package_file"
echo "Fetching $file_url"
curl -fs $file_url --output $save_file
done
bundle_file="${PACKAGE_PATH}/nms-dependencies-${target_distribution}.tar.gz"
tar -zcf $bundle_file -C "${PACKAGE_PATH}/${target_distribution}" .
echo "Bundle file saved as $bundle_file"
}
target_distribution=$1
if [ -z $target_distribution ]; then
echo "Usage: $0 target_distribution"
echo "Supported target distributions: ${!CLICKHOUSE_REPO[#]}"
exit 1
fi
# check if target distribution is supported
if [ -z ${CLICKHOUSE_REPO[$target_distribution]} ]; then
echo "Target distribution is not supported."
echo "Supported distributions: ${!CLICKHOUSE_REPO[#]}"
exit 1
fi
download_packages "${target_distribution}"
Then on the same directory that contains download_dependencies.sh run command below:
download_dependencies.sh <your linux version>
In my case, I ran code below (leave it blank to see options):
download_dependencies.sh centos7
It should start to download and when it finished you should see nms-dependencies-rhel7.tar.gz in your directory.
Copy that file(.tar.gz) to your offline target.
Now on your target machine, go to directory which you copied your file and run the code below:
tar -zxvf nms-dependencies-rhel7.tar.gz
sudo yum install *.rpm
After installation you can start nginx using systemctl:
sudo systemctl start clickhouse-server
sudo systemctl start nginx
Your nginx service must be running now!
you can download tar file in another system and copy
did you try this link?
https://gist.github.com/taufiqibrahim/d7f697de6bb8b93ca348a5b94d6adbfc

Salt stack top file execution

This is a sample of how my top file looks
base:
'*':
- sls_file_1
- sls_file_2
'smtp*':
- sls_file_3
- sls_file_4
sls_file_1 & sls_file_2 are expected to run on all minions while sls_file_3 & sls_file_4 only on minions with hostname starting with smtp.
When I run the highstate on a host whose hostname starts with smtp,
Only '*' part of the top file is executed and not the 'smtp*' on the first try (First time after host is up and running). You might say maybe the hostname is not set at that point in time, thats why it is not executed, but I have a ExecStartPre set in the salt-minon service file which sets the hostname before the salt-minon starts up
~ ❯ cat /usr/lib/systemd/system/salt-minion.service
[Unit]
Description=The Salt Minion
Documentation=man:salt-minion(1) file:///usr/share/doc/salt/html/contents.html https://docs.saltstack.com/en/latest/contents.html
After=network.target salt-master.service
[Service]
KillMode=process
Type=notify
NotifyAccess=all
LimitNOFILE=8192
ExecStartPre=/etc/salt/add_minion_id.sh
ExecStart=/usr/bin/salt-minion
[Install]
WantedBy=multi-user.target
Contents of add_minion_id.sh (Sets the hostname)
~ ❯ cat /etc/salt/add_minion_id.sh
#!/usr/bin/env bash
udata=`curl -s http://169.254.169.254/latest/user-data`
if [[ ! $udata == \#* ]]
then
new_hostname=`echo $udata| cut -d , -f1| cut -d : -f2`
else
new_hostname=`cat /etc/salt/userdata | cut -d , -f1| cut -d : -f2`
fi
hostname $new_hostname
echo $new_hostname > /etc/salt/minion_id
echo $new_hostname > /etc/hostname
So my expectation is all the 4 files in the top file will be executed since the hostname is set at that point, but that is not the case. Is there something I am missing?

NordVPN setup on linux

NordVPN does not offer an automatic setup for linux, just VPN config files. What's the best way to implement this?
(my own implementation below, please feel free to comment or suggest improvements!)
EDIT: When I wrote this, I did not know that NordVPN did introduce a command line tool for linux recently.
I have written a little script that downloads the config files, renames them and enables automatic authentification. Insert your NordVPN login credentials in the generate authentification file part.
#!/bin/bash
# run as root!!!
# install openvpn. I'm running arch, this might be different on your system.
pacman -S openvpn
# go to openvpn config folder
cd /etc/openvpn
# download config files, extract and clean up
wget https://downloads.nordcdn.com/configs/archives/servers/ovpn.zip
unzip ovpn.zip
rm ovpn.zip
# rename tcp config files and put them in /etc/openvpn/client
cd ovpn_tcp
for file in *; do mv "${file}" "${file/.nordvpn.com.tcp.ovpn/}tcp.conf"; done
cp * ../client
# rename udp config files and put them in /etc/openvpn/client
cd ../ovpn_udp
for file in *; do mv "${file}" "${file/.nordvpn.com.udp.ovpn/}udp.conf"; done
cp * ../client
# generate authentification file
cd ../client
printf "<your email>\n<your password>" > auth.txt
# make all configs use authentification file
find . -name '*.conf' -exec sed -i -e 's/auth-user-pass/auth-user-pass\ auth.txt/g' {} \;
# clean up
cd ..
rm -r ovpn_tcp/
rm -r ovpn_udp
You can now start and stop vpn-connections via e.g.
systemctl start openvpn-client#de415tcp.service
and
systemctl stop openvpn-client#de415tcp.service
To automate this, and to connect to the server recommended by NordVPN, I have written two scripts. Make them executable and put them somewhere in your $PATH.
Pass a country code (like us, de or uk) as command line argument to start-vpn if you want to choose a specific country. It automatically chooses a tcp connection. You can change that to udp if you want.
start-vpn
#!/usr/bin/python
import sys
import requests
import os
import time
# you don't necessarily need the following. It's for monitoring via i3blocks.
def notify_i3blocks():
os.system('pkill -RTMIN+12 i3blocks')
def fork_and_continue_notifying_in_background():
newpid = os.fork()
if newpid == 0: # if this is the child process
for i in range(60):
notify_i3blocks()
time.sleep(1)
if __name__ == '__main__':
notify_i3blocks()
# below is what you do need.
suffix = ''
if len(sys.argv) > 1:
countries = requests.get('https://nordvpn.com/wp-admin/admin-ajax.php?action=servers_countries').json()
for country in countries:
if country["code"].lower() == sys.argv[1].lower():
suffix = '&filters={"country_id":' + str(country["id"]) + '}'
result = requests.get('https://nordvpn.com/wp-admin/admin-ajax.php?action=servers_recommendations' + suffix)
profile = result.json()[0]['subdomain'] + 'tcp'
command = 'systemctl start openvpn-client#' + profile + '.service'
os.system(command)
# the following is for i3blocks again.
fork_and_continue_notifying_in_background()
stop-vpn
#!/bin/bash
function service {
systemctl |
grep openvpn |
grep running |
head -n1 |
awk '{print $1;}'
}
while [[ $(service) ]]; do
systemctl stop $(service)
done
# notify i3blocks
pkill -RTMIN+12 i3blocks
For convenience, I have two aliases in my ~/.bashrc:
alias start-vpn='sudo start-vpn'
alias stop-vpn='sudo stop-vpn'
if you do want to monitor it via i3blocks, put this in your i3blocks config:
[vpn]
interval=once
signal=12
and this in your i3blocks-scripts-directory (with name vpn):
#!/bin/bash
function name {
systemctl |
grep openvpn |
grep running |
head -n1 |
awk '{print $1;}' |
cut -d # -f 2 |
cut -d . -f 1
}
starting=$(pgrep -f start-vpn) # this might not be the most accurate, but it works for me. Improvement suggestions are welcomed.
if [[ $(name) ]]; then
echo $(name)
echo && echo "#00FF00"
else
if [[ ${starting} ]]; then
echo starting vpn...
echo && echo "#FFFF00"
else
echo no vpn
echo && echo "#FF0000"
fi
fi
In order to automatically start and stop vpn when a network interface goes up/down, put the following in /etc/NetworkManager/dispatcher.d/10-openvpn. To activate the feature you need to enable and start the NetworkManager-dispatcher.service. More info here.
At my university, I connect to eduroam, which does not allow vpn. That's why I exclude that.
/etc/NetworkManager/dispatcher.d/10-openvpn
#!/bin/bash
case "$2" in
up)
if ! nmcli -t connection | grep eduroam | grep wlp3s0 ; then
start-vpn
fi
;;
down)
stop-vpn
;;
esac
I hope this helps other people who want to use NordVPN on linux. Again, feel free to comment and suggest improvements.
In particular, I am not sure how much of a security risk it is to have the NordVPN-password written out in plain text in a file.

I'm stuck on logrotate mystery

I have two logrotate files:
/etc/logrotate.d/nginx-size
/var/log/nginx/*.log
/var/log/www/nginx/50x.log
{
missingok
rotate 3
size 2G
dateext
compress
compresscmd /usr/bin/bzip2
compressoptions -6
compressext .bz2
uncompresscmd /usr/bin/bunzip2
notifempty
create 640 nginx nginx
sharedscripts
postrotate
[ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
endscript
}
and
/etc/logrotate.d/nginx-daily
/var/log/nginx/*.log
/var/log/www/nginx/50x.log
{
missingok
rotate 3
dateext
compress
compresscmd /usr/bin/bzip2
compressoptions -6
compressext .bz2
uncompresscmd /usr/bin/bunzip2
notifempty
create 640 nginx nginx
sharedscripts
postrotate
[ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
endscript
}
Command logrotate -d -v /etc/logrotate.d/nginx-sizeoutput:
reading config file /etc/logrotate.d/nginx-size
compress_prog is now /usr/bin/bzip2
compress_options is now -6
compress_ext is now .bz2
uncompress_prog is now /usr/bin/bunzip2
Handling 1 logs
rotating pattern: /var/log/nginx/*.log
/var/log/www/nginx/50x.log
2147483648 bytes (3 rotations)
empty log files are not rotated, old logs are removed
considering log /var/log/nginx/access.log
log does not need rotating
considering log /var/log/nginx/error.log
log does not need rotating
considering log /var/log/nginx/get.access.log
log does not need rotating
considering log /var/log/nginx/post.access.log
log needs rotating
considering log /var/log/www/nginx/50x.log
log does not need rotating
rotating log /var/log/nginx/post.access.log, log->rotateCount is 3
dateext suffix '-20141204'
glob pattern '-[0-9][0-9][0-9][0-9][0-9][0-9][0-9][0-9]'
glob finding old rotated logs failed
renaming /var/log/nginx/post.access.log to /var/log/nginx/post.access.log-20141204
creating new /var/log/nginx/post.access.log mode = 0640 uid = 497 gid = 497
running postrotate script
running script with arg /var/log/nginx/*.log
/var/log/www/nginx/50x.log
: "
[ -f /var/run/nginx.pid ] && kill -USR1 `cat /var/run/nginx.pid`
"
compressing log with: /usr/bin/bzip2
Same (normal) output on ngnix-daily..
If I run from root command
logrotate -f /etc/logrotate.d/nginx-size
manually, it do all the thing. BUT! It don't run it automatically!
contab:
*/5 5-23 * * * root logrotate -f -v /etc/logrotate.d/nginx-size 2>&1 > /tmp/logrotate_size
00 04 * * * root logrotate -f -v /etc/logrotate.d/nginx-daily 2>&1 > /tmp/logrotate_daily
Also, files /tmp/logrotate_daily & /tmp/logrotate_size are always empty..
Cron don't give me any errors in /var/log/cron
Dec 4 14:45:01 (root) CMD (logrotate -f -v /etc/logrotate.d/nginx-rz-size 2>&1 > /tmp/logrotate_size )
Dec 4 14:50:01 (root) CMD (logrotate -f -v /etc/logrotate.d/nginx-rz-size 2>&1 > /tmp/logrotate_size )
What's wrong with dat thing?.. Centos 6.5 x86_64, Logrotate version 3.8.7 (out of source) + logrotate version 3.7.8 (via rpm).
Thx in advance.
Your redirections are incorrect in those cron lines. They will not output error information to those files.
Redirection order matters. You want >/tmp/logrotate_size 2>&1 to get what you want.
The underlying issue here is one of the things covered by the "Debugging crontab" section of the cron info page.
Namely "Making assumptions about the environment".
Making assumptions about the environment
Graphical programs (X11 apps), java programs, ssh and sudo are notoriously problematic to run as cron jobs. This is because they rely on things from interactive environments that may not be present in cron's environment.
To more closely model cron's environment interactively, run
env -i sh -c 'yourcommand'
This will clear all environment variables and run sh which may be more meager in features that your current shell.
Common problems uncovered this way:
foo: Command not found or just foo: not found.
Most likely $PATH is set in your .bashrc or similar interactive init file. Try specifying all commands by full path (or put source ~/.bashrc at the start of the script you're trying to run).

sh script: no output when run in mounted filesystem

Need some help to understand what's wrong.
In short: I've written a bourne shell script, which creates links to contents of source directory in the target directory.
It worked fine on the host system but when targeted on directories on mounted fs (both from chroot and native system) it doesn't work and provides no output at all.
Details:
mounted fs: ext3, rw
host system: 3.2.0-48-generic #74-Ubuntu SMP GNU/Linux
To narrow the question, "/usr" was taken as an example.
permissions for "/usr" in the host system: drwxr-xr-x
permissions for "/usr" on mounted partition: drwxr-xr-x
Tried to use both bash and dash from host system. Same result - works for native file systems, does not work for the mounted.
script (cord.sh; run from root in my cases):
# !/bin/sh
SRCFOLDER=$2 # folder with package installation
DESTFOLDER=$3 # destination folder to install symlinks to ('/' - for base sys; '/usr' - userland)
TARGETS=$(ls $SRCFOLDER) # targets to handle
SRCFOLDER=${SRCFOLDER%/} # stripping slashes from the end, if they are present
DESTFOLDER=${DESTFOLDER%/} #
##
## LINKING
##
if [ "$1" = "-c" ];
then printf %s "$TARGETS" | while IFS= read -r line
do
current_target=$(file $SRCFOLDER/$line) # had an issue with different output in different systems
if [ "${current_target% }" = "$SRCFOLDER/$line: directory" ]; # stripping space helped
then
mkdir -v $DESTFOLDER/$line # if other package created it - it'll fail
/usr/local/bin/cord.sh -c $SRCFOLDER/$line $DESTFOLDER/$line # RECURSION
else
ln -sv $SRCFOLDER/$line $DESTFOLDER/$line # will fail, if exists
fi;
done
##
## REMOVING LINKS
##
elif [ "$1" = "-d" ];
then printf %s "$TARGETS" | while IFS= read -r line
do
current_target=$(file $SRCFOLDER/$line)
if [ "${current_target% }" = "$SRCFOLDER/$line: directory" ];
then
/usr/local/bin/cord.sh -d $SRCFOLDER/$line $DESTFOLDER/$line # RECURSION
else
rm -v $DESTFOLDER/$line
fi;
done
elif [ "$1" = "-h" ];
then
echo "Usage:"
echo "cord -c /path/to/pkgdir /path/to/linkdir - create simlinks for package contents"
echo "cord -d /path/to/pkgdir /path/to/linkdir - delete links for package"
echo "cord -h - displays this help note"
else
echo "Usage:"
echo "cord -c /path/to/pkgdir /path/to/linkdir - create simlinks for package contents"
echo "cord -d /path/to/pkgdir /path/to/linkdir - delete links for package"
echo "cord -h - displays this help note"
fi;
The most obvious thing to suggest, was some issue with permissions. Yet everything looks sane. Maybe I've missed something?
I don't know what your main problem might be (permissions or something else - you should include an example of how you run the script and how you prepare for it with the mounts and everything). But this script can be cleaned up.
First, if you want to test whether something is a directory, use
if [ -d "$something ]
That'll get rid of the clumsy file usage.
Second, don't go through the redundant steps of converting your $TARGETS array to a series of lines and then reading the lines with a loop. Just loop over the array directly.
for line in $TARGETS
Also, instead of using ls to populate an array of filenames, I'd use a glob. But instead of either of those, I'd use find so it can take care of recursion and eliminate the tree of processes you're creating by recursing with a call to the same script. And instead of writing a symlink-tree-maker script I'd use something like lndir which already exists for that purpose...

Resources