I am trying to install geoip module for nginx though dockerfile by adding to my dockerfile the following:
RUN apk add --no-cache libmaxminddb nginx-mod-http-geoip
RUN cd /var/lib; \
mkdir -p nginx; \
wget -q -O- https://dl.miyuru.lk/geoip/maxmind/country/maxmind.dat.gz | gunzip -c > nginx/maxmind-country.dat; \
wget -q -O- https://dl.miyuru.lk/geoip/maxmind/city/maxmind.dat.gz | gunzip -c > nginx/maxmind-city.dat; \
chown -R nginx. nginx
COPY nginx.conf /etc/nginx/nginx.conf
The nginx.config is the following:
load_module "modules/ngx_http_geoip_module.so";
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events{worker_connections 1024;
}
# See blow link for Creating NGINX Plus and NGINX Configuration Files
# https://docs.nginx.com/nginx/admin-guide/basic-functionality/managing-configuration-files/
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format kv 'site="$server_name" server="$host" dest_port="$server_port" dest_ip="$server_addr" '
'src="$remote_addr" src_ip="$realip_remote_addr" user="$remote_user" '
'time_local="$time_local" protocol="$server_protocol" status="$status" '
'bytes_out="$bytes_sent" bytes_in="$upstream_bytes_received" '
'http_referer="$http_referer" http_user_agent="$http_user_agent" '
'nginx_version="$nginx_version" http_x_forwarded_for="$http_x_forwarded_for" '
'http_x_header="$http_x_header" uri_query="$query_string" uri_path="$uri" '
'http_method="$request_method" response_time="$upstream_response_time" '
'cookie="$http_cookie" request_time="$request_time" category="$sent_http_content_type" https="$https"'
'geoip_country_name="$geoip_country_name"';
access_log /var/log/nginx/access.log kv;
sendfile on;
keepalive_timeout 65;
geoip_country /var/lib/nginx/maxmind-country.dat;
geoip_city /var/lib/nginx/maxmind-city.dat;
include /etc/nginx/conf.d/*.conf;
# The identifier Backend is internal to nginx, and used to name this specific upstream
upstream backend {
# dashboard is the internal DNS name used by the backend Service inside Kubernetes
server localhost:5005;
}
server {
listen 80;
root /usr/share/nginx/html;
index index.html;
location / {
try_files $uri $uri/ /index.html;
}
location /api/ {
resolver 127.0.0.11; #nginx will not crash if host is not found
# The following statement will proxy traffic to the upstream
proxy_pass http://backend;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
However, when I am inspecting the logs I am getting
geoip_country_name = "-"
Any idea of what is going wrong here? Could it be that I am running this locally?
The "-" is what the logfile uses when the value is empty. GeoIP uses the $remote_addr to calculate the source of the request.
172.17.0.1 is not a public IP address, it is an internal address of one of your proxy servers. Check the $http_x_forwarded_for header value for the real remote address (assuming your reverse proxy servers are configured correctly.
The Geoip module provides the geoip_proxy directive to ignore $remote_addr and use $http_x_forwarded_for instead.
For example (added to your other geoip_ directives):
geoip_proxy 172.17.0.1;
We were experiencing a similar problem.
It essentially came back to the points made by #RichardSmith, however in our case the following configuration resolved the problem:
geoip_proxy 0.0.0.0/0;
Related
I have reconfigured nginx but i can't get it to restart using the following config:
conf:
server {
listen 80;
server_name www.example.com;
return 301 $scheme://example.com$request_uri;
}
server {
listen 80;
server_name example.com;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
location /robots.txt {
alias /path/to/robots.txt;
access_log off;
log_not_found off;
}
location = /favicon.ico { access_log off; log_not_found off; }
location / {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 30;
proxy_read_timeout 30;
proxy_pass http://127.0.0.1:8000;
}
location /static {
expires 1M;
alias /path/to/staticfiles;
}
}
after running sudo nginx -c conf -t to test the configuration the following error is returned i can't figure out what is really the problem
nginx: [emerg] "server" directive is not allowed here in /etc/nginx/sites-available/config:1
nginx: configuration file /etc/nginx/sites-available/config test failed
That is not an nginx configuration file. It is part of an nginx configuration file.
The nginx configuration file (usually called nginx.conf) will look like:
events {
...
}
http {
...
server {
...
}
}
The server block is enclosed within an http block.
Often the configuration is distributed across multiple files, by using the include directives to pull in additional fragments (for example from the sites-enabled directory).
Use sudo nginx -t to test the complete configuration file, which starts at nginx.conf and pulls in additional fragments using the include directive. See this document for more.
Example valid nginx.conf for reverse proxy; In case someone is stuck like me.
where 10.x.x.x is the server where you are running the nginx proxy server and to which you are connecting to with the browser, and 10.y.y.y is where your real web server is running
events {
worker_connections 4096; ## Default: 1024
}
http {
server {
listen 80;
listen [::]:80;
server_name 10.x.x.x;
location / {
proxy_pass http://10.y.y.y:80/;
proxy_set_header Host $host;
}
}
}
Here is the snippet if you want to do SSL pass through. That is if 10.y.y.y is running a HTTPS webserver. Here 10.x.x.x, or where the nignx runs is listening to port 443, and all traffic to 443 is directed to your target web server
events {
worker_connections 4096; ## Default: 1024
}
stream {
server {
listen 443;
proxy_pass 10.y.y.y:443;
}
}
and you can serve it up in docker too
docker run --name nginx-container --rm --net=host -v /home/core/nginx/nginx.conf:/etc/nginx/nginx.conf nginx
The path to the nginx.conf file which is the primary Configuration file for Nginx - which is also the file which shall INCLUDE the Path for other Nginx Config files as and when required is /etc/nginx/nginx.conf.
You may access and edit this file by typing this at the terminal
cd /etc/nginx
/etc/nginx$ sudo nano nginx.conf
Further in this file you may Include other files - which can have a SERVER directive as an independent SERVER BLOCK - which need not be within the HTTP or HTTPS blocks, as is clarified in the accepted answer above.
I repeat - if you need a SERVER BLOCK to be defined within the PRIMARY Config file itself than that SERVER BLOCK will have to be defined within an enclosing HTTP or HTTPS block in the /etc/nginx/nginx.conf file which is the primary Configuration file for Nginx.
Also note -its OK if you define , a SERVER BLOCK directly not enclosing it within a HTTP or HTTPS block , in a file located at path /etc/nginx/conf.d . Also to make this work you will need to include the path of this file in the PRIMARY Config file as seen below :-
http{
include /etc/nginx/conf.d/*.conf; #includes all files of file type.conf
}
Further to this you may comment out from the PRIMARY Config file , the line
http{
#include /etc/nginx/sites-available/some_file.conf; # Comment Out
include /etc/nginx/conf.d/*.conf; #includes all files of file type.conf
}
and need not keep any Config Files in /etc/nginx/sites-available/ and also no need to SYMBOLIC Link them to /etc/nginx/sites-enabled/ , kindly note this works for me - in case anyone think it doesnt for them or this kind of config is illegal etc etc , pls do leave a comment so that i may correct myself - thanks .
EDIT :- According to the latest version of the Official Nginx CookBook , we need not create any Configs within - /etc/nginx/sites-enabled/ , this was the older practice and is DEPRECIATED now .
Thus No need for the INCLUDE DIRECTIVE include /etc/nginx/sites-available/some_file.conf; .
Quote from Nginx CookBook page - 5 .
"In some package repositories, this folder is named sites-enabled, and
configuration files are linked from a folder named site-available;
this convention is depre‐ cated."
There might be just a typo anywhere inside a file imported by the config. For example, I made a typo deep inside my config file:
loccation /sense/movies/ {
mp4;
}
(loccation instead of location), and this causes the error:
nginx: [emerg] "server" directive is not allowed here in /etc/nginx/sites-enabled/xxx.xx:1
Replace include /etc/nginx/conf.d/*.conf; in nginx.conf with include /etc/nginx/conf.d/includes-optional/cpanel-proxy-vendors/*.conf; or /etc/nginx/conf.d/includes-optional/site-available/*.conf;
I have reconfigured nginx but i can't get it to restart using the following config:
conf:
server {
listen 80;
server_name www.example.com;
return 301 $scheme://example.com$request_uri;
}
server {
listen 80;
server_name example.com;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
location /robots.txt {
alias /path/to/robots.txt;
access_log off;
log_not_found off;
}
location = /favicon.ico { access_log off; log_not_found off; }
location / {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_connect_timeout 30;
proxy_read_timeout 30;
proxy_pass http://127.0.0.1:8000;
}
location /static {
expires 1M;
alias /path/to/staticfiles;
}
}
after running sudo nginx -c conf -t to test the configuration the following error is returned i can't figure out what is really the problem
nginx: [emerg] "server" directive is not allowed here in /etc/nginx/sites-available/config:1
nginx: configuration file /etc/nginx/sites-available/config test failed
That is not an nginx configuration file. It is part of an nginx configuration file.
The nginx configuration file (usually called nginx.conf) will look like:
events {
...
}
http {
...
server {
...
}
}
The server block is enclosed within an http block.
Often the configuration is distributed across multiple files, by using the include directives to pull in additional fragments (for example from the sites-enabled directory).
Use sudo nginx -t to test the complete configuration file, which starts at nginx.conf and pulls in additional fragments using the include directive. See this document for more.
Example valid nginx.conf for reverse proxy; In case someone is stuck like me.
where 10.x.x.x is the server where you are running the nginx proxy server and to which you are connecting to with the browser, and 10.y.y.y is where your real web server is running
events {
worker_connections 4096; ## Default: 1024
}
http {
server {
listen 80;
listen [::]:80;
server_name 10.x.x.x;
location / {
proxy_pass http://10.y.y.y:80/;
proxy_set_header Host $host;
}
}
}
Here is the snippet if you want to do SSL pass through. That is if 10.y.y.y is running a HTTPS webserver. Here 10.x.x.x, or where the nignx runs is listening to port 443, and all traffic to 443 is directed to your target web server
events {
worker_connections 4096; ## Default: 1024
}
stream {
server {
listen 443;
proxy_pass 10.y.y.y:443;
}
}
and you can serve it up in docker too
docker run --name nginx-container --rm --net=host -v /home/core/nginx/nginx.conf:/etc/nginx/nginx.conf nginx
The path to the nginx.conf file which is the primary Configuration file for Nginx - which is also the file which shall INCLUDE the Path for other Nginx Config files as and when required is /etc/nginx/nginx.conf.
You may access and edit this file by typing this at the terminal
cd /etc/nginx
/etc/nginx$ sudo nano nginx.conf
Further in this file you may Include other files - which can have a SERVER directive as an independent SERVER BLOCK - which need not be within the HTTP or HTTPS blocks, as is clarified in the accepted answer above.
I repeat - if you need a SERVER BLOCK to be defined within the PRIMARY Config file itself than that SERVER BLOCK will have to be defined within an enclosing HTTP or HTTPS block in the /etc/nginx/nginx.conf file which is the primary Configuration file for Nginx.
Also note -its OK if you define , a SERVER BLOCK directly not enclosing it within a HTTP or HTTPS block , in a file located at path /etc/nginx/conf.d . Also to make this work you will need to include the path of this file in the PRIMARY Config file as seen below :-
http{
include /etc/nginx/conf.d/*.conf; #includes all files of file type.conf
}
Further to this you may comment out from the PRIMARY Config file , the line
http{
#include /etc/nginx/sites-available/some_file.conf; # Comment Out
include /etc/nginx/conf.d/*.conf; #includes all files of file type.conf
}
and need not keep any Config Files in /etc/nginx/sites-available/ and also no need to SYMBOLIC Link them to /etc/nginx/sites-enabled/ , kindly note this works for me - in case anyone think it doesnt for them or this kind of config is illegal etc etc , pls do leave a comment so that i may correct myself - thanks .
EDIT :- According to the latest version of the Official Nginx CookBook , we need not create any Configs within - /etc/nginx/sites-enabled/ , this was the older practice and is DEPRECIATED now .
Thus No need for the INCLUDE DIRECTIVE include /etc/nginx/sites-available/some_file.conf; .
Quote from Nginx CookBook page - 5 .
"In some package repositories, this folder is named sites-enabled, and
configuration files are linked from a folder named site-available;
this convention is depre‐ cated."
There might be just a typo anywhere inside a file imported by the config. For example, I made a typo deep inside my config file:
loccation /sense/movies/ {
mp4;
}
(loccation instead of location), and this causes the error:
nginx: [emerg] "server" directive is not allowed here in /etc/nginx/sites-enabled/xxx.xx:1
Replace include /etc/nginx/conf.d/*.conf; in nginx.conf with include /etc/nginx/conf.d/includes-optional/cpanel-proxy-vendors/*.conf; or /etc/nginx/conf.d/includes-optional/site-available/*.conf;
There are 3 ingredients to this issue:
Docker container: I have a Docker container that is deployed on an EC2 instance. More specifically, I have the rocker/shiny image, which I have run using:
sudo docker run -d -v /home/ubuntu/projects/shiny_example:/srv/shiny-server -p 3838:3838 rocker/shiny
Shiny server: The standard Shiny server configuration file is untouched, and is set up to serve everything in the /srv/shiny-server folder on port 3838, and the contents of my local ~/projects/shiny_example are mapped to the container's /srv/shiny-server/.
In my local ~/projects/shiny_example, I have cloned a random Shiny app:
git clone https://github.com/rstudio/shiny_example
nginx: I have set up nginx as a reverse proxy and here are the contents of the /etc/nginx/nginx.conf in its entirety.
The issue is that with this setup, when I try to retrieve http://<ip-address>/shiny/shiny_example, I get a 404. The main clue I have as to what might be wrong is that when I do a:
wget http://localhost:3838/shiny_example
from the command line on my EC2 instance, I get:
--2016-06-13 11:05:08-- http://localhost:3838/shiny_example
Resolving localhost (localhost)... 127.0.0.1
Connecting to localhost (localhost)|127.0.0.1|:3838... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: /shiny_example/ [following]
--2016-06-13 11:05:08-- http://localhost:3838/shiny_example/
Reusing existing connection to localhost:3838.
HTTP request sent, awaiting response... 200 OK
Length: 3136 (3.1K) [text/html]
Saving to: ‘shiny_example.3’
100%[==============================================================================================================================>] 3,136 --.-K/s in 0.04s
2016-06-13 11:05:09 (79.6 KB/s) - ‘shiny_example.3’ saved [3136/3136]
where the emphasis is mine.
I think that my nginx configuration does not account for the fact that when requesting a Docker mapped port, there is a 301 redirect. I think that the solution involves proxy_next_upstream, but I would appreciate some help in trying to set this up in my context.
I also think that this question can be shorn of the Docker context, but it would be nice to understand how to prevent a 301 redirect when requesting a resource from Shiny server that is in a Docker container, and whether this behavior is expected.
I can't be sure without more output, but suspect your error is in your proxy_redirect line:
location /shiny/ {
rewrite ^/shiny/(.*)$ /$1 break;
proxy_pass http://localhost:3838;
proxy_redirect http://localhost:3838/ $scheme://$host/shiny_example;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_read_timeout 20d;
}
Try changing that to:
location /shiny/ {
rewrite ^/shiny/(.*)$ /$1 break;
proxy_pass http://localhost:3838;
proxy_redirect http://localhost:3838/ $scheme://$host/shiny/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_read_timeout 20d;
}
The reason for that is when the 301 header comes back from "http://localhost:3838" to add the trailing slash, it gets rewritten to "http://localhost/shiny_example" which doesn't exist in your nginx config, plus it may also remove a slash from the path. This means the 301 from "http://localhost:3838/shiny_example" to "http://localhost:3838/shiny_example/" would get rewritten to to "http://localhost/shiny_exampleshiny_example/", at which point you get a 404.
There was nothing wrong with anything. Basically, one of the lines in /etc/nginx/nginx.conf was include /etc/nginx/sites-enabled/*, which was pulling in the default file for enabled sites, which has the following lines:
server {
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
root /usr/share/nginx/html;
index index.html index.htm;
# Make site accessible from http://localhost/
server_name localhost;
location / {
# First attempt to serve request as file, then
# as directory, then fall back to displaying a 404.
try_files $uri $uri/ =404;
# Uncomment to enable naxsi on this location
# include /etc/nginx/naxsi.rules
}
which was overwriting my listen directives for port 80 and for location /. Commenting out the include directive for the default conf file for enabled sites in the /etc/nginx/nginx.conf file resolved all issues for me.
Not sure if this is still relevant but I have a minimal example here: https://github.com/mRcSchwering/butterbirne
A service shinyserver (which is based on rocker/shiny) is started with a service webserver (based on nginx:latest):
version: '2'
services:
shinyserver:
build: shinyserver/
webserver:
build: webserver/
ports:
- 80:80
I configured the ngin, so that it would redirect directly to the shiny server root. In my case I added the app (called myapp here) as the root of shinyserver (so no /myapp is needed). This is the whole nginx.conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
# apparently this is needed for shiny server
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
# proxy shinyserver
server {
listen 80;
location / {
proxy_pass http://shinyserver:3838;
proxy_redirect http://shinyserver:3838/ $scheme://$host/;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_read_timeout 20d;
proxy_buffering off;
}
}
}
I have 3 computers on same network(LAN). And I want to configure one computer as Nginx Web-Server, and another as Varnish Cache server and one client . I succesfully installed one(let's say A) Nginx ( 192.168.0.15 ) and B Varnish( 192.168.0.20 ). I configured A as a webserver and I can browse the index.html from other computers. But I couldn't connect it with B.
I messed up with "nginx.conf" and "/sites-available/server.com" and Varnish's "default.vcl"
Could you give me the basic configurations which suit my environment ?
If you want to take a look
My nginx.conf :
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
upstream dynamic_node {
server 1.1.1.1:80; # 1.1.1.1 is the IP of the Dynamic Node
}
server {
listen 81;
server_name myserver.myhome.com;
location / {
#root /var/www/server.com/public_html;
#index index.html index.htm;
# pass the request on to Varnish
proxy_pass http://192.168.0.20;
# Pass a bunch of headers to the downstream server.
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_redirect off;
}
}
}
/sites-available/server.com :
server {
listen 80;
server_name myserver.myhome.com;
access_log /var/www/server.com/access.log;
error_log /var/www/server.com/error.log;
}
And default.vcl like this :
backend web1 {
.host = "192.168.0.15";
.port = "8080";
}
sub vcl_recv {
if (req.http.host == "192.168.0.15") {
#set req.http.host = "myserver.myhome.com";
set req.backend = web1;
}
}
Lastly /etc/default/varnish :
DAEMON_OPTS="-a :6081 \
-T localhost:6082 \
-f /etc/varnish/default.vcl \
-S /etc/varnish/secret \
-s malloc,256m"
Thanks in advance :)
right now, your varnish instance is listening on port 6081. This needs to be specified in the proxy_pass for nginx e.g.
proxy_pass http://192.168.0.20:6081
I am assuming that the ip addresses you mentioned are correct and network connection between the computers is not restricted.
Update
Please bear in mind that you can use nginx in front of varnish or the other way around. Both nginx and varnish can serve as proxies to back end services.
Your current implementation is using nginx as the proxy. This means that you can rely on proxy_pass or use upstream module in nginx (in case you wish to load balance behind with multiple varnish instances with just one nginx in front). Essentially, whichever is the proxy, the ip address and port number for the backend specified in the proxy (nginx in your case) must match the ip address and port number for the backend service (varnish in your case). The backend in varnish would need to match the ip address and port number for whichever application server/service you are using (tomcat/netty/django/ror etc.).
I am currently wanting to use NGINX in my Rails setup. I have placed the configuration files in the directory RAILS_ROOT/config/nginx. Here is my config-file placed named development.conf and the mime.types-file.
I am wanting to place my logs in the RAILS_ROOT/log-directory.
This is my development.conf:
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application/octet-stream;
# main access log
access_log log/nginx.access.log;
# main error log
error_log log/nginx.error.log debug;
sendfile on;
keepalive_timeout 65;
server {
listen 9001; #omg!
server_name local.woman.dk;
rewrite ^/(.*)/$ /$1 last;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_max_temp_file_size 0;
location / {
ssi on;
proxy_pass http://127.0.0.1:3000;
}
}
}
I am starting NGINX from my RAILS_ROOT with this command:
nginx -p . -c config/nginx/development.conf
And I get the following error:
nginx: [alert] could not open error log file: open() "./logs/error.log" failed (2: No such file or directory)
My version is this:
[(master)]=> nginx -v
nginx version: nginx/1.2.4
Am I doing anything wrong?
http://nginx.org/en/docs/ngx_core_module.html#error_log indicates that:
the default value is error_log logs/error.log error;
that for debug logging to work, nginx needs to be built with --with-debug.`
what's happening is that you're falling through to the default value, I'm not spotting any syntax errors so my guess is that your nginx is not compiled with --with-debug.
you can check that with the nginx -V (note: that's capital V)
On MAC it was in
/usr/local/logs/error.log
i found this after running:
nginx -V
configure arguments: --prefix=/usr/local --with-cc-opt=-Wno-deprecated-declarations --with-http_ssl_module --add-module=../nginx-rtmp-module/