i'm setting up basic authentication(credential) for loki and promtail using nginx in docker-compose. i have created htpasswd to set the password for loki and promtail andcreated seperate config file for loki and promtail and passing it through volumes.
however its not triggering authentication for loki and promtail
docker-compose.yaml
version: "2"
services:
my-nginx-service:
image: nginx
ports:
- "8098:80"
container_name: nginx
volumes:
- ./config/sites-enabled/loki:/etc/nginx/sites-enabled/loki
- ./config/conf.d/loki.conf:/etc/nginx/conf.d/loki.conf
- ./config/conf.d/loki.conf:/etc/nginx/conf.d/promtail.conf
- ./config/sites-available/default:/etc/nginx/sites-available/default
- ./config/htpasswd/.htloki:/etc/nginx/.htloki
- ./config/htpasswd/.htloki:/etc/nginx/.htpromtail
loki:
image: grafana/loki:2.0.0
container_name: loki
volumes:
- ./config/loki.yaml:/etc/config/loki.yaml
entrypoint:
- /usr/bin/loki
- -config.file=/etc/config/loki.yaml
ports:
- "3100:3100"
promtail:
image: grafana/promtail:2.0.1
container_name: promtail
user: root
volumes:
- ./log:/var/log/test
- /var/log/system.log:/var/log/root/system.log
- ./config/promtail-local-config.yaml:/etc/config/promtail-local-config.yaml
entrypoint:
- /usr/bin/promtail
- -config.file=/etc/config/promtail-local-config.yaml
ports:
- "9080:9080"
loki.conf
server {
listen 443;
location / {
auth_basic "Protected Area";
auth_basic_user_file /etc/nginx/.htloki;
proxy_pass http://loki:3100;
}
}
promtail.conf
server {
listen 442;
location / {
auth_basic "Protected Area";
auth_basic_user_file /etc/nginx/.htpromtail;
proxy_pass http://promtail:9080;
}
}
has anybody faced this issue?
Apparently, I forgot to rebuild the container images after including the .htpasswd files. Rebuilding fixed this
docker compose up --build
Related
I've been scratching my head trying to figure out what I'm doing wrong. I have a project running on a DigitalOcean droplet - it has 3 containers: an Nginx server and a frontend and api server. It was all working fine, but I've now tried to add an SSL cert to the Nginx server and the browser just gives back 'this site can't be reached'.
My docker-compose is as follows:
services:
clarbeast-frontend:
container_name: clarbeast-frontend
build:
context: ./frontend
dockerfile: Dockerfile
image: clarbeast-frontend
clarbeast-api:
container_name: clarbeast-api
build:
context: ./api
dockerfile: Dockerfile
image: clarbeast-api
clarbeast-nginx:
container_name: clarbeast-nginx
build:
context: ./nginx
dockerfile: Dockerfile
image: clarbeast-nginx
ports:
- "80:80"
depends_on:
- clarbeast-frontend
- clarbeast-api
volumes:
- ./nginx/nginx.conf:/etc/nginx/nginx.conf
- /root/nginx_bundle_5a52b057ab94.crt:/root/nginx_bundle_5a52b057ab94.crt
- /root/example.com.key:/root/example.com.key
and my nginx.conf is as follows:
events {}
http {
server {
listen 80;
server_name example.com www.example.com;
return 301 https://example.com$request_uri;
}
server {
listen 443 ssl;
server_name example.com www.example.com;
ssl_certificate /root/nginx_bundle_5a52b057ab94.crt;
ssl_certificate_key /root/example.com.key;
location / {
proxy_pass http://clarbeast-frontend:3000;
}
location /api {
proxy_pass http://clarbeast-api:9874;
}
}
}
my docker container for nginx is up and running fine according to the logs - when I've had the path to the certs wrong, it's complained. Can anyone see what the issue might be?
I've tried double checking all the paths
I got it working - there were a few issues I had to address.
I wasn't exposing port 443 - so I needed to change the docker-compose to
ports:
- "80:80"
I also needed to add proxy_ssl_server_name on to the location blocks for the frontend and api in the nginx.conf.
I developed a web app with vue and django however, I'm having problems deploying it.
I added another container to serve as reverse proxy so only port 80 would be exposed and when I finish struggling with this also port 443. I could not find exact anwser how to do it so I hope someone here will be kind enoug to give me some clues.
Here is the conf for the nginx.
The error I'm getting is on ui container.
2022/07/14 09:09:00 [emerg] 1#1: bind() to 0.0.0.0:8080 failed (98: Address already in use)
I looked it up of course, but it was always some different scenario.
BR and thanks in advance
server {
listen 0.0.0.0:80;
listen [::]:80;
location / {
proxy_pass http://0.0.0.0:3000;
}
location /predict {
proxy_pass http://0.0.0.0:5000/predict;
}
location /aggregate {
proxy_pass http://0.0.0.0:5000/aggregate;
}
location /media/pictures {
proxy_pass http://0.0.0.0:5000/media/pictures;
}
access_log /opt/bitnami/nginx/logs/anomaly_access.log;
error_log /opt/bitnami/nginx/logs/anomaly_error.log;
}
My docker-compose looks as follows.
version: '3.2'
services:
se-kpi-sim:
image: test-app:0.0.1
network_mode: "host"
restart: unless-stopped
environment:
MODEL_NAME: "model_final.pickle.dat"
se-kpi-sim-ui:
image: test-ui:0.0.3
network_mode: "host"
restart: unless-stopped
reverse-proxy:
image: test-proxy:0.0.7
network_mode: "host"
restart: unless-stopped
database:
image: postgres
environment:
POSTGRES_PASSWORD: password
POSTGRES_USER: kpi_sim_user
POSTGRES_DB: kpi_sim
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
volumes:
- database:/var/lib/postgresql/data
restart: unless-stopped
volumes:
database:
You can run containers on docker internal network and docker-compose by default creates an network for inter communication of containers. One can modify the port to expose the application to host. while you are trying to run most of the app on host network, there might be two application trying to use the same port (like port 8080 [in this case]), one port can only be used by one application in an OS . Please look at the below snippet for more information to solve this issue.
[port mapping <port on HOST>:<container port where app is exposed inside container>]
version: '3.2'
services:
se-kpi-sim:
image: test-app:0.0.1
ports:
- 5000:8080
restart: unless-stopped
environment:
MODEL_NAME: "model_final.pickle.dat"
se-kpi-sim-ui:
image: test-ui:0.0.3
ports:
- 3000:8080
restart: unless-stopped
reverse-proxy:
image: test-proxy:0.0.7
ports:
- 80:80
# this volume mount if you are using bitnami/nginx image
volumes:
- /path/to/my_server_block.conf:/opt/bitnami/nginx/conf/server_blocks/my_server_block.conf:ro
restart: unless-stopped
database:
image: postgres
environment:
POSTGRES_PASSWORD: password
POSTGRES_USER: kpi_sim_user
POSTGRES_DB: kpi_sim
POSTGRES_HOST_AUTH_METHOD: trust
ports:
- 5432:5432
volumes:
- database:/var/lib/postgresql/data
restart: unless-stopped
volumes:
database:
One have to specify either the IP address or DNS name of the application , in order to forward the traffic to specific application. docker-compose create Domain name for all the services defined in docker-compose.yaml file.
server {
listen 0.0.0.0:80;
listen [::]:80;
location / {
proxy_pass http://se-kpi-sim-ui:8080;
}
location /predict {
proxy_pass http://se-kpi-sim:8080/predict;
}
location /aggregate {
proxy_pass http://se-kpi-sim:8080/aggregate;
}
location /media/pictures {
proxy_pass http://se-kpi-sim:8080/media/pictures;
}
access_log /opt/bitnami/nginx/logs/anomaly_access.log;
error_log /opt/bitnami/nginx/logs/anomaly_error.log;
}
One can mount the nginx.conf like:[in bitnami/nginx image]
...
volumes:
- /path/to/my_server_block.conf:/opt/bitnami/nginx/conf/server_blocks/my_server_block.conf:ro
...
Note: all the above is an example for reference to solve the problem. entrypoint for containers might change according to one's requirements.
I have this docker-compose file that deploys my 3 services
version: '3'
services:
db:
image: mariadb:10.3.9
volumes:
- data:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=secret
- MYSQL_DATABASE=wordpress
- MYSQL_USER=manager
- MYSQL_PASSWORD=secret
wp:
image: wordpress:4.9.8
depends_on:
- db
volumes:
- ./target:/var/www/html
environment:
- WORDPRESS_DB_USER=manager
- WORDPRESS_DB_PASSWORD=secret
- WORDPRESS_DB_HOST=db
ports:
- "80"
#deploy:
#mode: replicated
#replicas: 3
lb:
image: nginx:latest
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
depends_on:
- wp
ports:
- "8080:80"
volumes:
data:
You can escalate to more than 1 wordpress any time you want, just uncommenting that. The idea is to have 3 exact same wp and nginx as load balancer.
My nginx.conf is like this
user nginx;
events {
worker_connections 1000;
}
http {
server {
listen 80;
location / {
proxy_pass http://wp:80;
}
}
Now if I go to localhost:8080 the url resolves to http://wp/wp-admin/install.php but the web is blank and I can't connect. What did I do wrong?
I've attempted to migrate my stack to use version 2 docker-compose.yml and have run into a problem with network hostnames not being resolved by nginx.
My stack involves an nginx reverse proxy (on debian:wheezy) that serves secure content via several other software components of which I won't go into detail (see config below).
In the version 1 yaml, I used environment variables from docker links alongside with LUA script to insert them into the nginx.conf (using nginx-extras). This worked perfectly as a reverse proxy in front of the docker containers.
In the version 2 yaml I am using the hostnames as generated by docker networking. I am able to successfully ping these hostnames from within the container, however nginx is unable to resolve them.
2016/05/04 01:23:44 [error] 5#0: *3 no resolver defined to resolve ui, client: 10.0.2.2, server: , request: "GET / HTTP/1.1", host: "localhost"
Here is my current config:
docker-compose.yml:
version: '2'
services:
# back-end
api:
build: .
depends_on:
- db
- redis
- worker
environment:
RAILS_ENV: development
ports:
- "3000:3000"
volumes:
- ./:/mmaps
- /var/log/mmaps/api:/mmaps/log
volumes_from:
- apidata
command: sh -c 'rm -rf /mmaps/tmp/pids/server.pid; rails server thin -b 0.0.0.0 -p 3000'
# background process workers
worker:
build: .
environment:
RAILS_ENV: development
QUEUE: "*"
TERM_CHILD: "1"
volumes:
- ./:/mmaps
- /var/log/mmaps/worker:/mmaps/log
volumes_from:
- apidata
command: rake resque:work
# front-end
ui:
image: magiandev/mmaps-ui:develop
depends_on:
- api
ports:
- "8080:80"
volumes:
- /var/log/mmaps/ui:/var/log/nginx
# database
db:
image: mysql:5.7
environment:
MYSQL_ROOT_PASSWORD: pewpewpew
volumes_from:
- mysqldata
volumes:
- /var/log/mmaps/db:/var/log/mysql
# key store
redis:
image: redis:2.8.13
user: root
command: ["redis-server", "--appendonly yes"]
volumes_from:
- redisdata
volumes:
- /var/log/mmaps/redis:/var/log/redis
# websocket server
monitor:
image: magiandev/mmaps-monitor:develop
depends_on:
- api
environment:
NODE_ENV: development
ports:
- "8888:8888"
# media server
media:
image: nginx:1.7.1
volumes_from:
- apidata
ports:
- "3080:80"
volumes:
- ./docker/media/nginx.conf:/etc/nginx/nginx.conf:ro
- /srv/mmaps/public:/usr/local/nginx/html:ro
- /var/log/mmaps/mediapool:/usr/local/nginx/logs
# reverse proxy
proxy:
build: docker/proxy
ports:
- "80:80"
- "443:443"
volumes:
- /var/log/mmaps/proxy:/var/log/nginx
apidata:
image: busybox:ubuntu-14.04
volumes:
- /srv/mmaps/public:/mmaps/public
command: echo api data
mysqldata:
image: busybox:ubuntu-14.04
volumes:
- /srv/mmaps/db:/var/lib/mysql
command: echo mysql data
redisdata:
image: busybox:ubuntu-14.04
volumes:
- /srv/mmaps/redis:/data
command: echo redis data
# master data
# convenience container for backups
data:
image: busybox:ubuntu-14.04
volumes_from:
- apidata
- mysqldata
- redisdata
command: echo mmaps data
nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
# permanent redirect to https
server {
listen 80;
rewrite ^ https://$host$request_uri? permanent;
}
server {
listen 443 ssl;
ssl on;
ssl_certificate /etc/nginx/ssl/server.crt;
ssl_certificate_key /etc/nginx/ssl/server.key;
location / {
proxy_pass http://ui:80$request_uri;
}
location /monitor/ {
proxy_pass http://monitor:8888$request_uri;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
location /api/ {
client_max_body_size 0;
proxy_pass http://api:3000$request_uri;
}
location /files/ {
client_max_body_size 0;
proxy_pass http://media:80$request_uri;
}
location /mediapool/ {
proxy_pass http://media:80$request_uri;
add_header X-Upstream $upstream_addr;
if ($request_uri ~ "^.*\/(.*\..*)\?download=true.*$"){
set $fname $1;
add_header Content-Disposition 'attachment; filename="$fname"';
}
proxy_pass_request_headers on;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /var/www;
}
}
}
# stay in the foreground so Docker has a process to track
daemon off;
After some reading I have tried to use 'dnsmasq' and set resolver 127.0.0.1 within the nginx.conf but I cannot get this to work:
2016/05/04 01:54:26 [error] 6#0: recv() failed (111: Connection refused) while resolving, resolver: 127.0.0.1:53
Is there a better way to configure nginx to proxy pass to my containers that works with V2?
You can rename your containers and resolving by names.
I've been trying to set up an environment in docker-compose where there are several containers:
Django
Nginx
Postgres
DbData
Storage
I've used the following configuration:
app:
restart: always
build: src
expose:
- "8000"
links:
- postgres:postgres
volumes_from:
- storage_files_1
env_file: .env
command: gunicorn barbell.wsgi:application \
-b 0.0.0.0:8000 -w 4
nginx:
restart: always
build: nginx
ports:
- "80:80"
- "443:443"
volumes_from:
- storage_files_1
links:
- app:app
postgres:
restart: always
image: postgres:latest
volumes_from:
- storage_data_1
ports:
- "5432:5432"
My nginx sites-enabled config file looked like this:
server {
listen 80;
server_name localhost;
error_log /var/log/nginx/error.log;
access_log /var/log/nginx/access.log;
location /static {
alias /static/;
autoindex on;
}
location / {
proxy_pass http://app:8000;
proxy_set_header X-Forwarded-Host $server_name;
proxy_set_header X-Real-IP $remote_addr;
add_header P3P 'CP="ALL DSP COR PSAa PSDa OUR NOR ONL UNI COM NAV"';
}
}
And it doesn't work - nginx always returns 502, but serves static files perfectly. I also tried the same setup with uwsgi, no luck. However, when I combine the Django with nginx and serve everything from the same container, everything works (again, both on uwsgi and gunicorn).
Any idea what am I missing?
Update
Here are the nginx logs:
*1 connect() failed (111: Connection refused) while connecting to upstream,
client: 172.17.42.1, server: 0.0.0.0, request: "GET / HTTP/1.1", upstream:
"http://172.17.1.75:8000/", host: "localhost"
It turned out that Gunicorn was the culprit. Putting its configuration into a file resolved the issue.
gunicorn_config.py put in the same folder as manage.py:
bind = "0.0.0.0:8000"
loglevel = "INFO"
workers = "4"
reload = True
errorlog = "/var/log/gunicorn/error.log"
accesslog = "/var/log/gunicorn/access.log"
And some changes in docker-compose.yml:
app:
restart: always
build: src
expose:
- "8000"
links:
- postgres:postgres
volumes_from:
- storage_files_1
env_file: .env
command: gunicorn --config=gunicorn_config.py barbell.wsgi:application
Now it works as it should.
So I haven't seen any further feedback from you regarding error logs and what may or may not be hapenning for you; however I've stripped your example down to it's simplest as a demonstration of Docker+Django+NGINX working:
See: docker-django-test
NB: This is running on some infrastructure of mine that uses autodock If you want to replicate this you'll need this snippet of docker-compose.yml:
autodock:
image: prologic/autodock
ports:
- "1338:1338/udp"
- "1338:1338/tcp"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
autodockhipache:
image: prologic/autodock-hipache
links:
- autodock
- hipache:redis
hipache:
image: hipache
ports:
- 80:80
- 443:443
See: A Docker-based mini-PaaS