I'm using the latest nginx version (1.23.3) and latest rsyslog version (8.2212.0) in CentOS 7 and I'm trying to configure my nginx service for logging at the journal
journalctl -u nginx -f
I have already configure nginx.conf to write the logs in /var/log/nginx and it is working.
access_log /var/log/nginx/access.log main;
under this line I'm trying to configure access_log for service logging in my console via - journalctl -u nginx -f - but it does not working
access_log
syslog:server=unix:/dev/log,facility=local1,tag=MY_NGINX,nohostname,severity=info
combined;
It's weird because it was working, propable something with nginx or rsyslog version upate broke it. I have already check if its an issue with Selinux or read file permission.
Related
I have an OVH VPS with nginx server setup on it. I'm looking for a way to send nginx access and error logs to Google Cloud Logging service, but all info I could find was about sending logs from Google Cloud VMs. Is it even possible at this moment? I've tried also to find anything about sending syslog to GCP as a workaround but no luck too. Since my dotnet services succesfully send logs to GCP I suppose it should be possible. Any suggestions?
In GCP there is an integration with NGINX to collect connection metrics and access logs. There are some prerequisites that you need to accomplish before you start collecting logs from NGINX.
You must install Ops Agent in your instance . The Ops Agent collects logs and metrics on Compute Engine instances, sending your logs to Cloud Logging and your metrics to Cloud Monitoring. If you are using a single VM on a Linux SO, you can install the agent with the following command:
curl -sSO https://dl.google.com/cloudagents/add-google-cloud-ops-agent-repo.sh
sudo bash add-google-cloud-ops-agent-repo.sh --also-install
You can consult the details about the Ops Agent installation on this link
You will need to configure your NGINX instance enabling the stub_status module in the nginx configuration file to set up a locally reachable URL, like the following example:
http://www.example.com/status
If you don't have the stub_status module enabled, you can run the following command to enable it:
sudo tee /etc/nginx/conf.d/status.conf > /dev/null << EOF
server {
listen 80;
server_name 127.0.0.1;
location /status {
stub_status on;
access_log off;
allow 127.0.0.1;
deny all;
}
location / {
root /dev/null;
}
}
EOF
sudo service nginx reload
curl http://127.0.0.1:80/status
Please note that: 127.0.0.1 can be replaced with the real server name, for example, server_name mynginx.domain.com.
All these steps are detailed in the following link, it is a guide to setup all the prerequisites before you start collecting logs from your NGINX deployment. Also, there is an example to configure your deployment
I use Nginx on my server and want to serve my application on HTTPS using Let's Encrypt certs. I do the following on a fresh server before the application code gets deployed:
Install Nginx
Write the following nginx configuration file to sites-available, for certbot. Then symlink to sites-enabled and restart nginx
server {
listen 80;
server_name foo.bar.com;
allow all;
location ^~ /.well-known/acme-challenge/ {
proxy_pass http://0.0.0.0:22000;
}
}
Then run certbot
certbot certonly -m foo#bar.com --standalone --http-01-port 22000 --preferred-challenges http --cert-name bar.com -d foo.bar.com --agree-tos --non-interactive
All of the above work fine when run manually.
I use Chef to automate the above process. Certbot gets a 404 the first time I deploy. It works on subsequent deployments though.
Keep a note of the following detail:
The phenomenon happens only when I freshly install Nginx and then run my deploy script through Chef and disappears on subsequent deploys.
I use a custom LWRP to run the above steps in Chef expcept nginx installation. Nginx installation is taken care of by chef_nginx. I've pasted the snippet of the LWRP that runs the above steps.
vhost_file = "#{node['certbot']['sites_configuration_path']}/#{node['certbot']['sites_configuration_file']}"
template vhost_file do
cookbook 'certbot'
source 'nginx-letsencrypt.vhost.conf.erb'
owner 'root'
group 'root'
variables(
server_names: new_resource.sans,
certbot_port: node['certbot']['standalone_port'],
mode: node['certbot']['standalone_mode']
)
mode 00644
only_if "test -d #{node['certbot']['sites_configuration_path']}"
end
nginx_site node['certbot']['sites_configuration_file']
Using certbot in standalone mode on port 22000
How do I make things work even on the first deployment ?
I want to deploy my flask service in a server with centOS 7. So I followed this tutorial - https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uwsgi-and-nginx-on-centos-7 .
After runnning systemctl start nginx command, I got this error:
nginx: [emerg] bind() to 0.0.0.0:5000 failed (13: Permission denied)
My nginx.conf file:
server {
listen 5000;
server_name _;
location / {
include uwsgi_params;
uwsgi_pass unix:/root/fiproxy/fiproxyproject/fiproxy.sock;
}
}
Note: flask service and wsgi work ok. And I've tried to run nginx with superuser and the error remains.
After search a lot in Internet, I found a solution to my problem.
I ran this command to get all used ports in my machine: semanage port -l.
After that, I filtered the output with: semanage port -l | grep 5000.
I realized that this port 5000 is used by commplex_main_port_t, I searched in speedguide and I found: 5000 tcp,udp **UPnP**.
Conclusion, maybe my problem was bind a standard port.
To add your desired port use this command:
sudo semanage port -a -t http_port_t -p tcp [yourport]
Now run nginx with sudo:
sudo systemctl stop nginx
sudo systemctl start nginx
The Nginx master process needs root permission. Because it needs bind port.
You need start Nginx under root user.
Then you can define the user of child processes in nginx.conf.
I've an issue pushing my docker image to artifactory [Artifactory Pro Power Pack 3.5.2.1 (rev. 30160)] (which is used as a docker registry).
I have docker version:
$ sudo docker version
Client version: 1.5.0
Client API version: 1.17
Go version (client): go1.3.3
Git commit (client): a8a31ef/1.5.0
OS/Arch (client): linux/amd64
Server version: 1.5.0
Server API version: 1.17
Go version (server): go1.3.3
Git commit (server): a8a31ef/1.5.0
I've followed this link http://www.jfrog.com/confluence/display/RTF/Docker+Repositories and this one artifactory as docker registry
I create a docker registry in artifactory called docker-local and enable docker support for it.
My artifactory doesn't have an option where I can say docker v1 or v2 like in this document so I'm assuming it uses docker v1.
Artifactory generated these for me:
<distributionManagement>
<repository>
<id>sdpvvrwm812</id>
<name>sdpvvrwm812-releases</name>
<url>http://sdpvvrwm812.ib.tor.company.com:8081/artifactory/docker-local</url>
</repository>
<snapshotRepository>
<id>sdpvvrwm812</id>
<name>sdpvvrwm812-snapshots</name>
<url>http://sdpvvrwm812.ib.tor.company.com:8081/artifactory/docker-local</url>
</snapshotRepository>
</distributionManagement>
Though something's not working with these settings.
I installed the reverse proxy nginx and copied these settings into its /etc/nginx/nginx.conf:
http {
##
# Basic Settings
##
[...]
server {
listen 443;
server_name sdpvvrwm812.ib.tor.company.com;
ssl on;
ssl_certificate /etc/ssl/certs/sdpvvrwm812.ib.tor.company.com.crt;
ssl_certificate_key /etc/ssl/private/sdpvvrwm812.ib.tor.company.com.key;
access_log /var/log/nginx/sdpvvrwm812.ib.tor.company.com.access.log;
error_log /var/log/nginx/sdpvvrwm812.ib.tor.company.com.error.log;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Original-URI $request_uri;
proxy_read_timeout 900;
client_max_body_size 0; # disable any limits to avoid HTTP 413 for large image uploads
# required to avoid HTTP 411: see Issue #1486 (https://github.com/docker/docker/issues/1486)
chunked_transfer_encoding on;
location /v1 {
proxy_pass http://sdpvvrwm812.ib.tor.company.com:8081/artifactory/api/docker/docker-local/v1;
}
} }
I generated my ssl key as shown at http://www.akadia.com/services/ssh_test_certificate.html and placed in the 2 directories
/etc/ssl/certs/sdpvvrwm812.ib.tor.company.com.crt;
/etc/ssl/private/sdpvvrwm812.ib.tor.company.com.key;
I'm not sure how to ping the new docker registry, but doing
sudo docker login -u adrianus -p AT65UTJpXEFBHaXrzrdUdCS -e adrian#company.com http://sdpvvrwm812.ib.tor.company.com
gives this error:
FATA[0000] Error response from daemon: v1 ping attempt failed with
error: Get https://sdpvvrwm812.ib.tor.company.com/v1/_ping: dial tcp
172.25.10.44:443: connection refused. If this private registry supports only HTTP or HTTPS with an unknown CA certificate, please add
--insecure-registry sdpvvrwm812.ib.tor.company.com to the daemon's
arguments. In the case of HTTPS, if you have access to the registry's
CA certificate, no need for the flag; simply place the CA certificate
at /etc/docker/certs.d/sdpvvrwm812.ib.tor.company.com/ca.crt
BUT the certificate /etc/docker/certs.d/sdpvvrwm812.ib.tor.company.com/ca.crt exists so what's going on?
sudo curl -k -uadrianus:AP2pKojAeMSpXEFBHaXrzrdUdCS "https://sdpvvrwm812.ib.tor.company.com"
gives this error:
curl: (35) SSL connect error
I do start docker client with:
sudo docker -d --insecure-registry https://sdpvvrwm812.ib.tor.company.com
Could it be that since my docker registry is http://sdpvvrwm812.ib.tor.company.com:8081/artifactory/docker-local and docker and nginx are looking for http://sdpvvrwm812.ib.tor.company.com:8081/artifactory/docker-local/v1?
Any clues how to get docker to push images to artifactory?
The <distributionManagement/> part is for maven. It's a bit facepalm that Artifactory 3 shows maven snippet for Docker repos (fixed in Artifactory 4, you're welcome to upgrade), so please disregard it.
Generally with Docker you can't use /artifactory/repoName. It's Docker limitation, your registry must be hostname:port, without any additional path.
That's exactly why you have to configure the reverse proxy. What you are doing in your nginx config is forwarding all the requests to sdpvvrwm812.ib.tor.company.com:443/v1 to http://sdpvvrwm812.ib.tor.company.com:8081/artifactory/api/docker/docker-local/v1, which is correct thing to do.
Please note that the location for certificates should be /etc/docker/certs.d/sdpvvrwm812.ib.tor.company.com/, not /etc/ssl/certs/.
I installed Gitlab CE on a dedicated Ubuntu 14.04 server edition with Omnibus package.
Now I would want to install three other virtual hosts next to gitlab.
Two are node.js web applications launched by a non-root user running on two distinct ports > 1024, the third is a PHP web application that need a web server to be launched from.
There are:
a private bower registry running on 8081 (node.js)
a private npm registry running on 8082 (node.js)
a private composer registry (PHP)
But Omnibus listen 80 and doesn't seem to use neither Apache2 or Nginx, thus I can't use them to serve my PHP app and reverse-proxy my two other node apps.
What serving mechanics Gitlab Omnibus uses to listen 80 ?
How should I create the three other virtual hosts to be able to provide the following vHosts ?
gitlab.mycompany.com (:80) -- already in use
bower.mycompany.com (:80)
npm.mycompany.com (:80)
packagist.mycompany.com (:80)
About these
But Omnibus listen 80 and doesn't seem to use neither Apache2 or Nginx [, thus ...].
and #stdob comment :
Did omnibus not use nginx as a web server ??? –
Wich I responded
I guess not because nginx package isn't installed in the system ...
In facts
From Gitlab official docs :
By default, omnibus-gitlab installs GitLab with bundled Nginx.
So yes!
Omnibus package actually uses Nginx !
but it was bundled, explaining why it doesn't require to be installed as dependency from the host OS.
Thus YES! Nginx can, and should be used to serve my PHP app and reverse-proxy my two other node apps.
Then now
Omnibus-gitlab allows webserver access through user gitlab-www which resides
in the group with the same name. To allow an external webserver access to
GitLab, external webserver user needs to be added gitlab-www group.
To use another web server like Apache or an existing Nginx installation you will have to do
the following steps:
Disable bundled Nginx by specifying in /etc/gitlab/gitlab.rb
nginx['enable'] = false
# For GitLab CI, use the following:
ci_nginx['enable'] = false
Check the username of the non-bundled web-server user. By default, omnibus-gitlab has no default setting for external webserver user.
You have to specify the external webserver user username in the configuration!
Let's say for example that webserver user is www-data.
In /etc/gitlab/gitlab.rb set
web_server['external_users'] = ['www-data']
This setting is an array so you can specify more than one user to be added to gitlab-www group.
Run sudo gitlab-ctl reconfigure for the change to take effect.
Setting the NGINX listen address or addresses
By default NGINX will accept incoming connections on all local IPv4 addresses.
You can change the list of addresses in /etc/gitlab/gitlab.rb.
nginx['listen_addresses'] = ["0.0.0.0", "[::]"] # listen on all IPv4 and IPv6 addresses
For GitLab CI, use the ci_nginx['listen_addresses'] setting.
Setting the NGINX listen port
By default NGINX will listen on the port specified in external_url or
implicitly use the right port (80 for HTTP, 443 for HTTPS). If you are running
GitLab behind a reverse proxy, you may want to override the listen port to
something else. For example, to use port 8080:
nginx['listen_port'] = 8080
Similarly, for GitLab CI:
ci_nginx['listen_port'] = 8081
Supporting proxied SSL
By default NGINX will auto-detect whether to use SSL if external_url
contains https://. If you are running GitLab behind a reverse proxy, you
may wish to keep the external_url as an HTTPS address but communicate with
the GitLab NGINX internally over HTTP. To do this, you can disable HTTPS using
the listen_https option:
nginx['listen_https'] = false
Similarly, for GitLab CI:
ci_nginx['listen_https'] = false
Note that you may need to configure your reverse proxy to forward certain
headers (e.g. Host, X-Forwarded-Ssl, X-Forwarded-For, X-Forwarded-Port) to GitLab.
You may see improper redirections or errors (e.g. "422 Unprocessable Entity",
"Can't verify CSRF token authenticity") if you forget this step. For more
information, see:
What's the de facto standard for a Reverse Proxy to tell the backend SSL is used?
https://wiki.apache.org/couchdb/Nginx_As_a_Reverse_Proxy
To go further you can follow the official docs at https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/doc/settings/nginx.md#using-a-non-bundled-web-server
Configuring our gitlab virtual host
Installing Phusion Passenger
We need to install ruby (gitlab run in omnibus with a bundled ruby) globally in the OS
$ sudo apt-get update
$ sudo apt-get install ruby
$ sudo gem install passenger
Recompile nginx with the passenger module
Instead of Apache2 for example, nginx isn't able to be plugged with binary modules on-the-fly. It must be recompiled for each new plugin you want to add.
Phusion passenger developer team worked hard to provide saying, "a bundled nginx version of passenger" : nginx bins compiled with passenger plugin.
So, lets use it:
requirement: we need to open our TCP port 11371 (the APT key port).
$ sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 561F9B9CAC40B2F7
$ sudo apt-get install apt-transport-https ca-certificates
creating passenger.list
$ sudo nano /etc/apt/sources.list.d/passenger.list
with these lignes
# Ubuntu 14.04
deb https://oss-binaries.phusionpassenger.com/apt/passenger trusty main
use the right repo for your ubuntu version. For Ubuntu 15.04 for example:
deb https://oss-binaries.phusionpassenger.com/apt/passenger vivid main
Edit permissions:
$ sudo chown root: /etc/apt/sources.list.d/passenger.list
$ sudo chmod 600 /etc/apt/sources.list.d/passenger.list
Updating package list:
$ sudo apt-get update
Allowing it as unattended-upgrades
$ sudo nano /etc/apt/apt.conf.d/50unattended-upgrades
Find or create this config block on top of the file:
// Automatically upgrade packages from these (origin:archive) pairs
Unattended-Upgrade::Allowed-Origins {
// you may have some instructions here
};
Add the following:
// Automatically upgrade packages from these (origin:archive) pairs
Unattended-Upgrade::Allowed-Origins {
// you may have some instructions here
// To check "Origin:" and "Suite:", you could use e.g.:
// grep "Origin\|Suite" /var/lib/apt/lists/oss-binaries.phusionpassenger.com*
"Phusion:stable";
};
Now (re)install nginx-extra and passenger:
$ sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.bak_"$(date +%Y-%m-%d_%H:%M)"
$ sudo apt-get install nginx-extras passenger
configure it
Uncomment the passenger_root and passenger_ruby directives in the /etc/nginx/nginx.conf file:
$ sudo nano /etc/nginx/nginx.conf
... to obtain something like:
##
# Phusion Passenger config
##
# Uncomment it if you installed passenger or passenger-enterprise
##
passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini;
passenger_ruby /usr/bin/passenger_free_ruby;
create the nginx site configuration (the virtual host conf)
$ nano /etc/nginx/sites-available/gitlab.conf
server {
listen *:80;
server_name gitlab.mycompany.com;
server_tokens off;
root /opt/gitlab/embedded/service/gitlab-rails/public;
client_max_body_size 250m;
access_log /var/log/gitlab/nginx/gitlab_access.log;
error_log /var/log/gitlab/nginx/gitlab_error.log;
# Ensure Passenger uses the bundled Ruby version
passenger_ruby /opt/gitlab/embedded/bin/ruby;
# Correct the $PATH variable to included packaged executables
passenger_env_var PATH "/opt/gitlab/bin:/opt/gitlab/embedded/bin:/usr/local/bin:/usr/bin:/bin";
# Make sure Passenger runs as the correct user and group to
# prevent permission issues
passenger_user git;
passenger_group git;
# Enable Passenger & keep at least one instance running at all times
passenger_enabled on;
passenger_min_instances 1;
error_page 502 /502.html;
}
Now we can enable it:
$ sudo ln -s /etc/nginx/sites-available/gitlab.cong /etc/nginx/sites-enabled/
There is no a2ensite equivalent coming natively with nginx, so we use ln, but if you want, there is a project on github:
nginx_ensite:
nginx_ensite and nginx_dissite for quick virtual host enabling and disabling
This is a shell (Bash) script that replicates for nginx the Debian a2ensite and a2dissite for enabling and disabling sites as virtual hosts in Apache 2.2/2.4.
It' done :-). Finally, restart nginx
$ sudo service nginx restart
With this new configuration, you are able to run other virtual hosts next to gitlab to serve what you want
Just create new configs in /etc/nginx/sites-available.
In my case, I made running and serving this way on the same host :
gitlab.mycompany.com - the awesome git platform written in ruby
ci.mycompany.com - the gitlab continuous integration server written in ruby
npm.mycompany.com - a private npm registry written in node.js
bower.mycompany.com - a private bower registry written in node.js
packagist.mycompany.com - a private packagist for composer registry written in php
For example, to serve npm.mycompany.com :
Create a directory for logs:
$ sudo mkdir -p /var/log/private-npm/nginx/
And fill a new vhost config file:
$ sudo nano /etc/nginx/sites-available/npm.conf
With this config
server {
listen *:80;
server_name npm.mycompany.com
client_max_body_size 5m;
access_log /var/log/private-npm/nginx/npm_access.log;
error_log /var/log/private-npm/nginx/npm_error.log;
location / {
proxy_pass http://localhost:8082;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
}
}
Then enable it and restart it:
$ sudo ln -s /etc/nginx/sites-available/npm.conf /etc/nginx/sites-enabled/
$ sudo service nginx restart
As I would not like to change the nginx server for gitlab (with some other integrations), the safest way would be below solution.
also as per
Gitlab:Ningx =>Inserting custom settings into the NGINX config
edit the /etc/gitlab/gitlab.rb of your gitlab:
nano /etc/gitlab/gitlab.rb
and sroll to nginx['custom_nginx_config'] and modify as below make sure to uncomment
# Example: include a directory to scan for additional config files
nginx['custom_nginx_config'] = "include /etc/nginx/conf.d/*.conf;"
create the new config dir:
mkdir -p /etc/nginx/conf.d/
nano /etc/nginx/conf.d/new_app.conf
and add content to your new config
# my new app config : /etc/nginx/conf.d/new_app.conf
# set location of new app
upstream new_app {
server localhost:1234; # wherever it might be
}
# set the new app server
server {
listen *:80;
server_name new_app.mycompany.com;
server_tokens off;
access_log /var/log/new_app_access.log;
error_log /var/log/new_app_error.log;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
location / { proxy_pass http://new_app; }
}
and reconfigure gitlab to get the new settings inserted
gitlab-ctl reconfigure
to restart nginx
gitlab-ctl restart nginx
to check nginx error log:
tail -f /var/log/gitlab/nginx/error.log