Dokku postgres: Expose command bug: `The container name "/dokku.postgres.APP_NAME.ambassador" is already in use by container` - dokku

$ dokku postgres:expose wiki-fashion-hasura
docker: Error response from daemon: Conflict. The container name "/dokku.postgres.wiki-fashion-hasura.ambassador" is already in use by container "05ac13c5682af1b1334ffda6d9142c2e577c81f0776c9a0449516d5ca6d55c8d". You have to remove (or rename) that container to be able to reuse that name.
See 'docker run --help'.
I checked docker ps and there is no container 05ac13c5682af1b1334ffda6d9142c2e577c81f0776c9a0449516d5ca6d55c8d
Then when trying to expose again:
$ dokku postgres:expose wiki-fashion-hasura
! Service wiki-fashion-hasura already exposed on port(s) 729
$ dokku postgres:info wiki-fashion-hasura
=====> wiki-fashion-hasura postgres service information
Config dir: /var/lib/dokku/services/postgres/wiki-fashion-hasura/data
Data dir: /var/lib/dokku/services/postgres/wiki-fashion-hasura/data
Dsn: postgres://postgres:03baa499ae71ae371a9276536df5fa56#dokku-postgres-wiki-fashion-hasura:5432/wiki_fashion_hasura
Exposed ports: 5432->729
Id: 89aa118cd1a41fc28170f6de3ed236171d3f3e2d8c019c62f74b2381282284f9
Internal ip: 172.17.0.8
Links: wiki-fashion-hasura
Service root: /var/lib/dokku/services/postgres/wiki-fashion-hasura
Status: running
Version: postgres:12
But
telnet <HOST> 729
telnet: connect to address <HOST>: Connection refused
It isn't exposed. (other ports with this same IP are resolving)
How can I debug this further?

Related

Laravel Homestead Mailhog Error listening on socket: listen tcp 0.0.0.0:1025: bind: address already in use

I am struggling to make MailHog work in my Laravel Homestead VM.
If I browse http://localhost:8025/, the web browser says it can't find it.
Then, in the command line, when executing $ mailhog, I get the following error:
Error listening on socket: listen tcp 0.0.0.0:1025: bind: address
already in use
Moreover, if I execute $ mailhog --invite-jim then I get:
2021/09/13 22:49:22 Using in-memory storage 2021/09/13 22:49:22 [SMTP]
Binding to address: 0.0.0.0:1025 [HTTP] Binding to address:
0.0.0.0:8025 2021/09/13 22:49:22 Serving under http://0.0.0.0:8025/ 2021/09/13 22:49:22 [SMTP] Error listening on socket: listen tcp
0.0.0.0:1025: bind: address already in use
Here is my .env config file
.env
MAIL_MAILER=smtp
MAIL_HOST=localhost
MAIL_PORT=1025
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null
MAIL_FROM_ADDRESS=reply#app.io
MAIL_FROM_NAME="${APP_NAME}"
Does anyone know or can shed some light how to make MailHog work?
Mailhog web interface is clearly listening at the 8025 port. The localhost or 127.0.0.1 address may not work by default.
Take a look at both the /etc/hosts and Homestead.yml file
$ sudo vim /etc/hosts
$ vim Homestead.yml
Look for the address that Homestead is working on. In this case it's 192.168.10.10 or 192.168.56.56
So now try
http://192.168.10.10:8025 or http://192.168.56.56:8025
And it should be working now

Mailcow setup behind Traefik Proxy causes https certificate error

I am trying to setup the mailcow installation behind Traefik proxy. Apparently, Traefik proxy is not able to recognize the nginx-mailcow container in its network and hence does not create a certificate for https connection. so when I bring up the mailcow service using docker-compose up, I can access the mailcow services but on insecure connection (http) and browser warns that connection is not secure.
When I check my acme.json file from Traefik: I can not find any certificate related to mailcow domain i.e., mail.tld.com there.
I have the following setup:
Logs of affected containers:
Traefik Container Logs:
time="2020-04-18T13:40:35+02:00" level=error msg="accept tcp [::]:80: use of closed network connection" entryPointName=http
time="2020-04-18T13:40:35+02:00" level=error msg="accept tcp [::]:443: use of closed network connection" entryPointName=https
time="2020-04-18T13:40:35+02:00" level=error msg="close tcp [::]:80: use of closed network connection" entryPointName=http
time="2020-04-18T13:40:35+02:00" level=error msg="close tcp [::]:443: use of closed network connection" entryPointName=https
time="2020-04-18T13:40:35+02:00" level=error msg="Cannot connect to docker server context canceled" providerName=docker
time="2020-04-18T13:40:37+02:00" level=info msg="Configuration loaded from file: /traefik.yml"
time="2020-04-19T00:27:31+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" container=nginx-mailcow-mailcowdockerized-5f3a25b43c42fd85df675d2d9682b6053501844c2cfe15b7802cf918df138025 providerName=docker
time="2020-04-19T00:33:32+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" providerName=docker container=nginx-mailcow-mailcowdockerized-f4d41ee79e382b413e04b039b5fc91e1c6217c78740245c8666373fe2d6a9b23
2020/04/19 00:39:44 reverseproxy.go:445: httputil: ReverseProxy read error during body copy: unexpected EOF
time="2020-04-19T00:50:32+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" providerName=docker container=nginx-mailcow-mailcowdockerized-915f80e492c2c22917d0af81add1dde15577173c82cc928b0b6101c8a260adc5
time="2020-04-19T00:58:43+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" container=nginx-mailcow-mailcowdockerized-852985c4efc48559ca3568b1829e31b46eb9f968fc328a8566e3dc6ab6f1af21 providerName=docker
time="2020-04-19T02:02:39+02:00" level=error msg="Error while Peeking first byte: read tcp 172.21.0.2:80->208.91.109.90:55153: read: connection reset by peer"
time="2020-04-19T08:11:32+02:00" level=error msg="service \"nginx-mailcow\" error: unable to find the IP address for the container \"/mailcowdockerized_nginx-mailcow_1\": the server is ignored" providerName=docker container=nginx-mailcow-mailcowdockerized-840ef4db0ccc9fa84038dc7a52133779926dba4c51554516c17404ede80a2c01
The contents of Traefik docker-compose.yml:
version: '3'
services:
traefik:
image: traefik:v2.1
container_name: traefik
restart: unless-stopped
security_opt:
- no-new-privileges:true
networks:
- proxy
ports:
- 80:80
- 443:443
volumes:
- /etc/localtime:/etc/localtime:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./data/traefik.yml:/traefik.yml:ro
- ./data/acme.json:/acme.json
labels:
- "traefik.enable=true"
- "traefik.http.routers.traefik.entrypoints=http"
- "traefik.http.routers.traefik.rule=Host(`traefik.tld.com`)"
- "traefik.http.middlewares.traefik-auth.basicauth.users=admin:pass"
- "traefik.http.middlewares.traefik-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.traefik.middlewares=traefik-https-redirect"
- "traefik.http.routers.traefik-secure.entrypoints=https"
- "traefik.http.routers.traefik-secure.rule=Host(`traefik.tld.com`)"
- "traefik.http.routers.traefik-secure.middlewares=traefik-auth"
- "traefik.http.routers.traefik-secure.tls=true"
- "traefik.http.routers.traefik-secure.tls.certresolver=http"
- "traefik.http.routers.traefik-secure.service=api#internal"
networks:
proxy:
external: true
Contents of traefik.yml (I used .yml instead of .toml)
api:
dashboard: true
entryPoints:
http:
address: ":80"
https:
address: ":443"
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
certificatesResolvers:
http:
acme:
email: myemail#tld.com
storage: acme.json
httpChallenge:
entryPoint: http
Just to point out, with this setup of Traefik, certificates are generated automatically for other services like gitlab. For that, I just correctly labelled the gitlab service and assigned the Traefik network to it and Traefik service would recognize the gitlab service and generates the certificate in acme.json but sadly not for nginx-mailcow.
The contents of my docker-compose.override.yml for mailcow:
version: '2.1'
services:
nginx-mailcow:
labels:
- "traefik.enable=true"
- "traefik.http.routers.nginx-mailcow.entrypoints=http"
- "traefik.http.routers.nginx-mailcow.rule=HostRegexp(`{host:(autodiscover|autoconfig|webmail|mail|email).+}`)"
- "traefik.http.middlewares.nginx-mailcow-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.nginx-mailcow.middlewares=nginx-mailcow-https-redirect"
- "traefik.http.routers.nginx-mailcow-secure.entrypoints=https"
- "traefik.http.routers.nginx-mailcow-secure.rule=Host(`mail.tld.com`)"
- "traefik.http.routers.nginx-mailcow-secure.tls=true"
- "traefik.http.routers.nginx-mailcow-secure.service=nginx-mailcow"
- "traefik.http.services.nginx-mailcow.loadbalancer.server.port=80"
- "traefik.docker.network=proxy"
networks:
proxy:
certdumper:
image: humenius/traefik-certs-dumper
container_name: traefik_certdumper
network_mode: none
command: --restart-containers mailcowdockerized_postfix-mailcow_1,mailcowdockerized_dovecot-mailcow_1
volumes:
- /opt/containers/traefik/data:/traefik:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./data/assets/ssl:/output:rw
environment:
- DOMAIN=tld.com
networks:
proxy:
external: true
The contents of my nginx-mailcow service in docker-compose.yml
version: '2.1'
services:
...
nginx-mailcow:
depends_on:
- sogo-mailcow
- php-fpm-mailcow
- redis-mailcow
image: nginx:mainline-alpine
dns:
- ${IPV4_NETWORK:-172.22.1}.254
command: /bin/sh -c "envsubst < /etc/nginx/conf.d/templates/listen_plain.template > /etc/nginx/conf.d/listen_plain.active &&
envsubst < /etc/nginx/conf.d/templates/listen_ssl.template > /etc/nginx/conf.d/listen_ssl.active &&
envsubst < /etc/nginx/conf.d/templates/server_name.template > /etc/nginx/conf.d/server_name.active &&
envsubst < /etc/nginx/conf.d/templates/sogo.template > /etc/nginx/conf.d/sogo.active &&
envsubst < /etc/nginx/conf.d/templates/sogo_eas.template > /etc/nginx/conf.d/sogo_eas.active &&
. /etc/nginx/conf.d/templates/sogo.auth_request.template.sh > /etc/nginx/conf.d/sogo_proxy_auth.active &&
. /etc/nginx/conf.d/templates/sites.template.sh > /etc/nginx/conf.d/sites.active &&
nginx -qt &&
until ping phpfpm -c1 > /dev/null; do sleep 1; done &&
until ping sogo -c1 > /dev/null; do sleep 1; done &&
until ping redis -c1 > /dev/null; do sleep 1; done &&
until ping rspamd -c1 > /dev/null; do sleep 1; done &&
exec nginx -g 'daemon off;'"
environment:
- HTTPS_PORT=${HTTPS_PORT:-443}
- HTTP_PORT=${HTTP_PORT:-80}
- MAILCOW_HOSTNAME=${MAILCOW_HOSTNAME}
- IPV4_NETWORK=${IPV4_NETWORK:-172.22.1}
- TZ=${TZ}
- ALLOW_ADMIN_EMAIL_LOGIN=${ALLOW_ADMIN_EMAIL_LOGIN:-n}
volumes:
- ./data/web:/web:ro
- ./data/conf/rspamd/dynmaps:/dynmaps:ro
- ./data/assets/ssl/:/etc/ssl/mail/:ro
- ./data/conf/nginx/:/etc/nginx/conf.d/:rw
- ./data/conf/rspamd/meta_exporter:/meta_exporter:ro
- sogo-web-vol-1:/usr/lib/GNUstep/SOGo/
ports:
- "${HTTPS_BIND:-0.0.0.0}:${HTTPS_PORT:-443}:${HTTPS_PORT:-443}"
- "${HTTP_BIND:-0.0.0.0}:${HTTP_PORT:-80}:${HTTP_PORT:-80}"
restart: always
networks:
mailcow-network:
aliases:
- nginx
....
I have also tried comment out ports in nginx-mailcow service but the problem persists. My current mailcow.conf changes:
HTTP_BIND=127.0.0.1
HTTP_PORT=8080
HTTPS_BIND=127.0.0.1
HTTPS_PORT=8443
SKIP_LETS_ENCRYPT=y
SKIP_CLAMD=y
Reproduction of said bug:
I setup the traefik proxy first (see contents above). Once the Traefik is up and running (I also tested for other services and it works fine in generating a certificate). Now first I cloned the mailcow repository. Then I run ./generate_config.sh to generate mailcow.conf file. As input to generate_config.sh I provide my domain name i.e., mail.tld.com
Then I comment out the ports in docker-compose.yml file because I do not want to use port 80 and 443 for nginx-mailcow as these ports are already being used by Traefik.
Then I create a docker-compose.override.yml (see contents above) to add additional configs to nginx-mailcow service (traefik labels, traefik network). The override file also contain the certdumper service which would copy https certificate from acme.json to mailcow services.
Then, I change the following two variables in mailcow.conf:
SKIP_LETS_ENCRYPT=y
SKIP_CLAMD=y
Finally, I run the mailcow using docker-compose up -d. In browser, if check https://mail.tld.com => It warns that connection is insecure. If I check acme.json. I find no certificate for mail.tld.com.
System information:
+-------------------------------------------------+---------------------------------+
| Question | Answer |
+-------------------------------------------------+---------------------------------+
| My operating system | linux x86_64 Ubuntu 18.04.1 LTS |
| Is Apparmor, SELinux or similar active? | No |
| Virtualization technlogy | KVM |
| Server/VM specifications (Memory, CPU Cores) | 16GB, 6 cores |
| Docker Version (docker version) | 19.03.8 |
| Docker-Compose Version (docker-compose version) | 1.25.4, build 8d51620a |
| Reverse proxy (custom solution) | Traefik |
+-------------------------------------------------+---------------------------------+
If you need more information, I would be happy to provide. Any help will be much appreciated. Thank you.
Finally I was able to solve the problem after investing many hours in reading the Traefik Documentation. I made tiny mistake in assigning proxy labels to the nginx-mailcow service. The solution is below.
I forgot to mention certificate resolver and I had to expose the port which I now added as follows:
services:
nginx-mailcow:
expose:
- "8080"
labels:
- "traefik.enable=true"
- "traefik.http.routers.nginx-mailcow.entrypoints=http"
- "traefik.http.routers.nginx-mailcow.rule=HostRegexp(`{host:(autodiscover|autoconfig|webmail|mail|email).+}`)"
- "traefik.http.middlewares.nginx-mailcow-https-redirect.redirectscheme.scheme=https"
- "traefik.http.routers.nginx-mailcow.middlewares=nginx-mailcow-https-redirect"
- "traefik.http.routers.nginx-mailcow-secure.entrypoints=https"
- "traefik.http.routers.nginx-mailcow-secure.rule=Host(`mail.example.com`)"
- "traefik.http.routers.nginx-mailcow-secure.tls=true"
- "traefik.http.routers.nginx-mailcow-secure.certresolver=http"
- "traefik.http.routers.nginx-mailcow-secure.service=nginx-mailcow"
- "traefik.http.services.nginx-mailcow.loadbalancer.server.port=8080"
- "traefik.docker.network=proxy"
networks:
proxy:
certdumper:
image: humenius/traefik-certs-dumper
container_name: traefik_certdumper
network_mode: none
command: --restart-containers mailcowdockerized_postfix-mailcow_1,mailcowdockerized_dovecot-mailcow_1
volumes:
- <path_to_acme.json_file_dir>:/traefik:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./data/assets/ssl:/output:rw
environment:
- DOMAIN=example.com
For people who are setting up for the first time, I had to make some additional changes beforehand.
Firstly, when you run generate.sh file then in mailcow.conf file you need to make following changes:
HTTP_PORT=8080
HTTP_BIND=127.0.0.1
HTTPS_PORT=8443
HTTPS_BIND=127.0.0.1
SKIP_LETS_ENCRYPT=y
SKIP_CLAMD=y
We make these changes as we can not run mailcow nginx on the same ports as traefik.
Now as nginx-mailcow will be running on 8080 or 8443 so we need to expose one of these ports so traefik can talk to mailcow-nginx service. I already exposed port 8080 in the override compose file)
You also need to also adapt your loadbalancer port from 80 to 8080. (As I configured above)
You need to also tell which certificate resolver should it use. So you need to add this line in labels (I made this as well above in override config)
You have to make sure that your acme.json file (certificate file is accessible by certdumper service). So replace to actual path of acme.json directory path
I hope this helps.

Kubernetes (GKE) internal lookups cannot be resolved

I am having an issue with Kubernetes on GKE. I am unable to resolve services by name. I got an drone-server service running which is connected to a single pod. The ingress connected to the service is successfully connecting but when trying to do for example a nslookup from a busybox pod is it unable to resolve the hostname.
Services:
$ k get services -n drone
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
drone-server ClusterIP 10.39.242.23 <none> 80/TCP 2d
drone-vault ClusterIP 10.39.248.166 <none> 80/TCP 40m
Busybox nslookup:
$ kubectl exec -ti busybox -- nslookup drone-server
Server: 10.39.240.10
Address 1: 10.39.240.10 kube-dns.kube-system.svc.cluster.local
nslookup: can't resolve 'drone-server'
When i try to lookup kubernetes.default am I getting a local address back:
$ kubectl exec -ti busybox -- nslookup kubernetes.default
Server: 10.39.240.10
Address 1: 10.39.240.10 kube-dns.kube-system.svc.cluster.local
Name: kubernetes.default
Address 1: 10.39.240.1 kubernetes.default.svc.cluster.local
Resolv config:
/etc/resolv.conf seems to be configured correctly (the nameserver is matching the kube-dns service cluster ip).
$ kubectl exec -ti busybox -- cat /etc/resolv.conf
nameserver 10.39.240.10
search default.svc.cluster.local svc.cluster.local cluster.local europe-west3-a.c.cluster-a8e6d9e252b63e03.internal c.cluster-a8e6d9e252b63e03.internal google.internal
options ndots:5
Your drone-server service is in the drone namespace and you're trying to nslookup from default namespace. You need to provide the namespace also in command as follows:
kubectl exec -ti busybox -- nslookup drone-server.drone
This is because your busybox in the default namespace and it tries to look drone-server in same namespace.

Docker - port prevents listening

I am trying to setup xdebug integration on my docker-based setup.
I am using Docker for Mac 1.12.0-rc2-beta17 with the "native" docker machine
I have a container, with xdebug installed, exposing port 9000 and mapping it to the port 9000:
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6950c2a2b05d app "/usr/bin/supervisord" 9 minutes ago Up 9 minutes 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:9000->9000/tcp, 0.0.0.0:2222->22/tcp app_1
When I'm trying to use PhpStorm to listen to the port 9000 for debug connections, I'm getting an error "Cannot listen: port 9000 is busy".
I must precise that I'm a newbie in networks..
It dependent how you want to connect via Xdebug
xdebug.remote_connect_back=1 said that PHP will wait until a HTTP request with GET parameter XDEBUG_SESSION_START=<IDE_key>. Then will PHP within the server try to connect back via port 9000 where your PHPStorm is listing. Classic don't call us, we will call you situation.
Now your situation with docker say simple, your container is responsible for port 9000. So PHP will get a loopback and PHPStorm isn't able to use port 9000 because its already used by your docker container.
So skip the assignment of port 9000 to docker, that will fix this situation.
You must bind 9000 port with --expose option.
This is the reference
if you are using docker compose sample docker-compose.yml file is here:
version: '2'
services:
your_app:
ports:
- "80:80"
expose:
- "9000"
image: "your-image:tag"
Firstly check your container logs to debug:
docker logs 6950c2a2b05d
or
docker logs app_1
Add -f flags for tail-like behavior:
docker logs -f app_1
Two things I discovered:
There is no need to expose the port 9000 on a container with xdebug (that seems rather counter-intuitive for me, as I do not exactly understand how my IDE connects to xdebug then).
I was able to use xdebug using the workaround described in https://forums.docker.com/t/ip-address-for-xdebug/10460/4.

I am not able to Launch a VM/instance in proxmox from Salt-master

I have installed Saltsatck(salt-master) on my virtual machine and also installed Proxmox(Cloud) on another virtual machine.
They both are on same network.
Salt-master and proxmox are running successfully.
Whenever I run the below command :-
# salt-cloud -p my-proxmox-config mytest
I get the following output :-
[INFO ] salt-cloud starting
[INFO ] Starting new HTTPS connection (1): 192.168.2.245
[INFO ] Creating Cloud VM mytest
[ERROR ] Error creating mytest on PROXMOX
The following exception was thrown when trying to run the initial deployment:
Error: There was a profile error: Failed to deploy VM
Please look at the below config files :-
1. /etc/salt/cloud.providers.d/proxmox.conf
proxmox-config:
user: root#pam or root#pve
password: oodles
url: 192.168.2.245
driver: proxmox
verify_ssl: False
minion:
master: 192.168.2.228
2. /etc/salt/cloud.profiles.d/proxmox.conf
my-proxmox-config :
provider: proxmox-config
image: /root/ISO/ubuntu-14.04-server-amd64.iso
technology: kvm / Openvz
host: cloud
ip_address: 192.168.2.245
ssh_username: root
password: oodles
cpus: 1
memory: 512
swap: 512
disk: 2
nameserver: 8.8.8.8 8.8.4.4
Please suggest/advice me what to correct from my configurations file .
Thanks
The error you're getting is saying that something is wrong with your profile config. We just need to troubleshoot what's going on with it.
I haven't used the proxmox provider, but according to https://docs.saltstack.com/en/latest/topics/cloud/proxmox.html it looks like for the image option you might have to use local:/root/ISO/ubuntu-14.04-server-amd64.iso.
Also, have you tried just technology: openvz?
I am able to solve the above issue i.e. now I am able to launch a VM/instance in proxmox from salt-master by doing the below configurations :-
1. /etc/salt/cloud.providers.d/proxmox.conf
proxmox-config:
minion:
master_type: standard
master: '192.x.x.x'
user: 'root#pam'
password: "your password"
url: '192.168.x.x'
port: '8006'
driver: proxmox
verify_ssl: False
2. /etc/salt/cloud.profiles.d/proxmox.conf
my-proxmox-config :
provider: proxmox-config
image: local:vztmpl/ubuntu-12.04-standard_12.04-1_i386.tar.gz
technology: openvz
host: cloud
ip_address: 192.168.x.x
ssh_username: root
password: "your password"
cpus: 1
memory: 512
swap: 512
disk: 2
nameserver: 8.8.8.8 8.8.4.4
In the above file, the image option will only work if you have downloaded the desired operating ISO in templates option available in PROXMOX GUI.
Now , you can easily launch a Instance by using below command :-
# salt-cloud -p my-proxmox-config mytest
Thanks

Resources