Keycloack in prod is stuck on loading admin page - nginx

I have a keycloak server deployed with docker behind a nginx reverse proxy.
In dev mode all is working fine, but since I am trying to deploy it in prod I can acces the app but when I want to config my server from the admin console I'm stuck in this page :
Docker-compose
version: '3'
services:
postgresql:
image: postgres:latest
command: postgres -c 'max_connections=200'
networks:
- keycloak-net
- postgresql-net
restart: always
environment:
- POSTGRES_USER=keycloak
- POSTGRES_PASSWORD=password
- POSTGRES_DB=keycloak
volumes:
- ./keycloak/postgresql:/var/lib/postgresql/data
keycloak:
image: quay.io/keycloak/keycloak:20.0.3
networks:
- proxy-net
- keycloak-net
restart: always
command: start --spi-truststore-file-file /etc/x509/https/truststore.jks --spi-truststore-file-password password --spi-truststore-file-hostname-verification-policy ANY
environment:
- KC_HOSTNAME=keycloak
- KC_HOSTNAME_STRICT=false
- KC_PROXY=edge
- KC_HOSTNAME_STRICT_HTTPS=false
- KC_DB=postgres
- KC_DB_URL_HOST=postgresql
- KC_DB_URL_DATABASE=keycloak
- KC_DB_PASSWORD=password
- KC_DB_USERNAME=keycloak
- KC_DB_SCHEMA=public
- KEYCLOAK_ADMIN=admin
- KEYCLOAK_ADMIN_PASSWORD=password
depends_on:
- postgresql
volumes:
- ./keycloak/stores:/etc/x509/https
networks:
proxy-net:
external: true
keycloak-net:
driver: bridge
postgresql-net:
driver: bridge
Keycloak logs on startup
keycloak_1 | Next time you run the server, just run:
keycloak_1 |
keycloak_1 | kc.sh start --optimized --spi-truststore-file-file=/etc/x509/https/truststore.jks --spi-truststore-file-password=password --spi-truststore-file-hostname-verification-policy=ANY
keycloak_1 |
keycloak_1 | 2023-01-19 18:15:53,562 INFO [org.keycloak.quarkus.runtime.hostname.DefaultHostnameProvider] (main) Hostname settings: Base URL: <unset>, Hostname: keycloak, Strict HTTPS: false, Path: <request>, Strict BackChannel: false, Admin URL: <unset>, Admin: <request>, Port: -1, Proxied: true
keycloak_1 | 2023-01-19 18:15:57,778 WARN [io.quarkus.agroal.runtime.DataSources] (main) Datasource <default> enables XA but transaction recovery is not enabled. Please enable transaction recovery by setting quarkus.transaction-manager.enable-recovery=true, otherwise data may be lost if the application is terminated abruptly
keycloak_1 | 2023-01-19 18:16:00,549 INFO [org.keycloak.broker.provider.AbstractIdentityProviderMapper] (main) Registering class org.keycloak.broker.provider.mappersync.ConfigSyncEventListener
keycloak_1 | 2023-01-19 18:16:00,555 WARN [org.infinispan.PERSISTENCE] (keycloak-cache-init) ISPN000554: jboss-marshalling is deprecated and planned for removal
keycloak_1 | 2023-01-19 18:16:00,658 WARN [org.infinispan.CONFIG] (keycloak-cache-init) ISPN000569: Unable to persist Infinispan internal caches as no global state enabled
keycloak_1 | 2023-01-19 18:16:00,742 INFO [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000556: Starting user marshaller 'org.infinispan.jboss.marshalling.core.JBossUserMarshaller'
keycloak_1 | 2023-01-19 18:16:01,352 INFO [org.infinispan.CONTAINER] (keycloak-cache-init) ISPN000128: Infinispan version: Infinispan 'Triskaidekaphobia' 13.0.10.Final
keycloak_1 | 2023-01-19 18:16:01,673 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000078: Starting JGroups channel `ISPN`
keycloak_1 | 2023-01-19 18:16:01,675 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000088: Unable to use any JGroups configuration mechanisms provided in properties {}. Using default JGroups configuration!
keycloak_1 | 2023-01-19 18:16:01,866 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
keycloak_1 | 2023-01-19 18:16:01,869 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 20.00MB, but the OS only allocated 212.99KB
keycloak_1 | 2023-01-19 18:16:01,870 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the send buffer of socket MulticastSocket was set to 1.00MB, but the OS only allocated 212.99KB
keycloak_1 | 2023-01-19 18:16:01,872 WARN [org.jgroups.protocols.UDP] (keycloak-cache-init) JGRP000015: the receive buffer of socket MulticastSocket was set to 25.00MB, but the OS only allocated 212.99KB
keycloak_1 | 2023-01-19 18:16:03,900 INFO [org.jgroups.protocols.pbcast.GMS] (keycloak-cache-init) 4b41e052f453-15754: no members discovered after 2003 ms: creating cluster as coordinator
keycloak_1 | 2023-01-19 18:16:03,933 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000094: Received new cluster view for channel ISPN: [4b41e052f453-15754|0] (1) [4b41e052f453-15754]
keycloak_1 | 2023-01-19 18:16:03,947 INFO [org.infinispan.CLUSTER] (keycloak-cache-init) ISPN000079: Channel `ISPN` local address is `4b41e052f453-15754`, physical addresses are `[192.168.16.3:54114]`
keycloak_1 | 2023-01-19 18:16:05,038 INFO [org.keycloak.connections.infinispan.DefaultInfinispanConnectionProviderFactory] (main) Node name: 4b41e052f453-15754, Site name: null
keycloak_1 | 2023-01-19 18:16:06,893 INFO [io.quarkus] (main) Keycloak 20.0.3 on JVM (powered by Quarkus 2.13.6.Final) started in 17.483s. Listening on: http://0.0.0.0:8080
keycloak_1 | 2023-01-19 18:16:06,897 INFO [io.quarkus] (main) Profile prod activated.
keycloak_1 | 2023-01-19 18:16:06,899 INFO [io.quarkus] (main) Installed features: [agroal, cdi, hibernate-orm, jdbc-h2, jdbc-mariadb, jdbc-mssql, jdbc-mysql, jdbc-oracle, jdbc-postgresql, keycloak, logging-gelf, narayana-jta, reactive-routes, resteasy, resteasy-jackson, smallrye-context-propagation, smallrye-health, smallrye-metrics, vault, vertx]
keycloak_1 | 2023-01-19 18:16:07,033 ERROR [org.keycloak.services] (main) KC-SERVICES0010: Failed to add user 'admin' to realm 'master': user with username exists
Note that the error at the end append beacause my db has already an admin.
I have the same issue when strating from an empty db.
nginx.conf
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name myserver.fr;
ssl_certificate /etc/nginx/ssl/live/myserver.fr/fullchain.pem;
ssl_certificate_key /etc/nginx/ssl/live/myserver.fr/privkey.pem;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host;
location / {
proxy_pass http://keycloak:8080;
}
}
Network console web browser
Thanks in advance for your help !
I have tried a lot of configuration, I saw a lot of time that adding KC_HOSTNAME_STRICT_HTTPS=false on env variable is the solution. I tried but it does not work..

With the help of #stdunbar I was able to solve this.
I had set KC_HOSTNAME to keycloak which is the hostname of the container in my docker network.
To solve my problem I set the KC_HOSTNAME to the real hostname accessible from browser : keycloak.mydomain.fr

Related

flyway cli failing to connect to mysql database - getDatabaseTypeForUrl

I've been trying without success to connect to a local mySQL database, it keeps failing with the errors listed below.
[root#stargate drivers]# flyway info -X -url='"jdbc:mysql://localhost:3306/sms_database"'
DEBUG: Loading config file: /home/development/wip/flyway-9.1.3/conf/flyway.conf
DEBUG: Loading config file: /root/flyway.conf
DEBUG: Unable to load config file: /home/development/wip/flyway-9.1.3/drivers/flyway.conf
DEBUG: Using configuration:
DEBUG: flyway.jarDirs -> /home/development/wip/flyway-9.1.3/jars
DEBUG: flyway.locations -> service-sms/service-sms
DEBUG: flyway.password -> ********
DEBUG: flyway.url -> "jdbc:mysql://localhost:3306/sms_database"
DEBUG: flyway.user -> root
ERROR: Unexpected error
org.flywaydb.core.api.FlywayException: No database found to handle "jdbc:mysql://localhost:3306/sms_database"
at org.flywaydb.core.internal.database.DatabaseTypeRegister.getDatabaseTypeForUrl(DatabaseTypeRegister.java:54)
at org.flywaydb.core.internal.jdbc.DriverDataSource.<init>(DriverDataSource.java:109)
at org.flywaydb.core.internal.jdbc.DriverDataSource.<init>(DriverDataSource.java:89)
at org.flywaydb.core.api.configuration.ClassicConfiguration.configure(ClassicConfiguration.java:1528)
at org.flywaydb.core.api.configuration.FluentConfiguration.configuration(FluentConfiguration.java:943)
at org.flywaydb.commandline.Main.main(Main.java:157)
I've confirmed connectivity:
[root#stargate drivers]# mysql -h 127.0.0.1 -u root -ppassword
mysql: [Warning] Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 26
Server version: 8.0.23-vitess
Copyright (c) 2000, 2021, Oracle and/or its affiliates.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> status;
--------------
mysql Ver 8.0.26 for Linux on x86_64 (Source distribution)
Connection id: 26
Current database:
Current user: vt_app#localhost
SSL: Not in use
Current pager: stdout
Using outfile: ''
Using delimiter: ;
Server version: 8.0.23-vitess
Protocol version: 10
Connection: 127.0.0.1 via TCP/IP
Server characterset: utf8mb4
Db characterset: utf8mb4
Client characterset: utf8mb4
Conn. characterset: utf8mb4
TCP port: 3306
Binary data as: Hexadecimal
--------------
I've tried both the mariadb-java-client-2.7.3.jar and mysql-connector-java-8.0.30.jar drivers and error is the same. Any advice ?
This is fixed by removing the extra '' characters from the URL string.

Problems using MariaDB in Docker Swarm with NFS

I have problems using MariaDB within a Docker Swarm using an nfs share. The database suddenly stops accepting new connections after fdatasync() failed. This happens randomly. Aftera few hours or after a few days. If I remove the service and start it again, everything ist running fine. The service seems not to repair itself. But I think this error should not even occur, even if the service should heal itself. I run the database as a persistence layer for the nextcloud app.
This is my docker-compose file:
version: '3.3'
services:
nextcloud_db:
image: mariadb:10.7.4
#container_name: nextcloud-db
command:
- "--transaction-isolation=READ-COMMITTED"
- "--log-bin=ROW"
- "--innodb_read_only_compressed=OFF"
- "--character-set-server=utf8mb4"
- "--collation-server=utf8mb4_unicode_ci"
#- "--innodb-rollback-on-timeout=ON" # Tested this but did not help
deploy:
replicas: 1
update_config:
parallelism: 1
delay: 10s
restart_policy:
condition: on-failure
labels:
- traefik.enable=false
volumes:
- /etc/localtime:/etc/localtime:ro
- /etc/timezone:/etc/timezone:ro
- db:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=myrootpassword
- MYSQL_PASSWORD=mymysqlpassword
- MYSQL_DATABASE=nextcloud
- MYSQL_USER=nextcloud
- MYSQL_INITDB_SKIP_TZINFO=1
networks:
- nextcloud
### other services for running nextcloud ###
volumes:
db:
driver_opts:
type: "nfs"
o: "addr=<storage-server-ip>,nolock,soft,rw"
device: ":/mnt/storage/nextcloud/db"
networks:
traefik-public:
external: true
nextcloud:
driver: overlay
# driver_opts:
# encrypted: "true"
These are the logs from the moment the db died:
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-06-29 19:51:17 4671 [ERROR] [FATAL] InnoDB: fdatasync() returned 5
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 220629 19:51:17 [ERROR] mysqld got signal 6 ;
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | This could be because you hit a bug. It is also possible that this binary
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | or one of the libraries it was linked against is corrupt, improperly built,
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | or misconfigured. This error can also be caused by malfunctioning hardware.
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 |
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | To report this bug, see https://mariadb.com/kb/en/reporting-bugs
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 |
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | We will try our best to scrape up some info that will hopefully help
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | diagnose the problem, but since we have already crashed,
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | something is definitely wrong and this may fail.
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 |
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | Server version: 10.7.4-MariaDB-1:10.7.4+maria~focal-log
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | key_buffer_size=134217728
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | read_buffer_size=131072
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | max_used_connections=10
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | max_threads=153
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | thread_count=11
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | It is possible that mysqld could use up to
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 467995 K bytes of memory
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | Hope that's ok; if not, decrease some variables in the equation.
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 |
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | Thread pointer: 0x55d81db99108
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | Attempting backtrace. You can use the following information to find out
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | where mysqld died. If you see no messages after this, something went
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | terribly wrong...
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | stack_bottom = 0x7fcf10137d98 thread_stack 0x49000
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | mariadbd(my_print_stacktrace+0x32)[0x55d81b24de52]
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | mariadbd(handle_fatal_signal+0x485)[0x55d81ad282b5]
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-06-29 21:49:49 4673 [Warning] Aborted connection 4673 to db: 'nextcloud' user: 'nextcloud' host: '10.0.7.189' (Got an error reading communication packets)
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-06-29 21:49:49 4672 [Warning] Aborted connection 4672 to db: 'nextcloud' user: 'nextcloud' host: '10.0.7.189' (Got an error reading communication packets)
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-06-29 21:49:49 4674 [Warning] Aborted connection 4674 to db: 'nextcloud' user: 'nextcloud' host: '10.0.7.189' (Got an error reading communication packets)
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-06-29 22:16:02 4676 [Warning] Aborted connection 4676 to db: 'nextcloud' user: 'nextcloud' host: '10.0.7.189' (Got an error reading communication packets)
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-06-29 22:18:13 4678 [Warning] Aborted connection 4678 to db: 'nextcloud' user: 'nextcloud' host: '10.0.7.189' (Got an error reading communication packets)
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-06-29 22:24:46 4679 [Warning] Aborted connection 4679 to db: 'nextcloud' user: 'nextcloud' host: '10.0.7.189' (Got an error reading communication packets)
nc_nextcloud_db.1.1mfx9xkwd1sd#v220210169548138574 | 2022-07-01 21:49:02 7148 [Warning] Aborted connection 7148 to db: 'nextcloud' user: 'nextcloud' host: '10.0.7.189' (Got an error reading communication packets)
I found no other logs related to the isse.
Anyone has a clue what's going on here?
Maybe the NFS share is unavailable for a few seconds and so the database has problems reading/writing? Is it possible to self-heal the mariadb service after this error occurs? There are no other problems as long as the database service is running. I can upload and delete files etc. So it is not a permissions issue on the nfs share.
Further MariaDB metrics:
https://jpst.it/2TX-F
Host system info:
Docker node VM with Ubuntu:
Ubuntu 20.04.4 LTS
2 vCPUs
8 GB RAM
160 GB SSD System-Storage (Raid 10)
MySQL does not support mounting NFS to initialize data

Trying to Configure NGINX as Reverse Proxy for RocketChat

I am trying to configure NGINX as reverse proxy to use the RocketChat Service but the service is not loading on the desired location. The in port is 2357 at which the service(RocketChat) is running and the out port on which it should run is port 3000. Can anyone help me out with this?
Code for the location on which RocketChat should load
I am also adding the docker-compose image infrastructure for reference.
Docker Image Details
nginx.conf file :
location ^~ /chat {
client_max_body_size 200m;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Nginx-Proxy true;
proxy_redirect off;
proxy_pass http://rocketchat:3000/;
}
docker-compose-org.yml:
# Image--infrastructure:rocketchat
rocketchat:
image: rocketchat/rocket.chat:latest
hostname: rocketchat
command: >
bash -c
"for i in `seq 1 30`; do
node main.js &&
s=$$? && break || s=$$?;
echo \"Tried $$i times. Waiting 5 secs...\";
sleep 5;
done; (exit $$s)"
restart: unless-stopped
networks:
- nodelocal
- nodelocal-private
ports:
- 2357:3000
volumes:
- ./data/rocketchat/app/uploads:/app/uploads
environment:
PORT: 3000
ROOT_URL: http://localhost:3000
MONGO_URL: mongodb://mongo:27017/rocketchat
MONGO_OPLOG_URL: mongodb://mongo:27017/local
MAIL_URL: smtp://smtp.email
# HTTP_PROXY: 'http://rocketchat.cdli.com'
# HTTPS_PROXY: 'http://rocketchat.cdli.com'
labels:
- "traefik.backend=rocketchat"
- "traefik.frontend.rule=Host: rocketchat.cdli.com"
depends_on:
- mongo
RocketChat Container Logs:
LocalStore: store created at
LocalStore: store created at
LocalStore: store created at
{"level":40,"time":"2022-04-27T05:00:03.411Z","pid":10,"hostname":"rocketchat","name":"","name":"VoIPService","msg":"Voip is not enabled. Cant start the service"}
{"level":51,"time":"2022-04-27T05:00:03.855Z","pid":10,"hostname":"rocketchat","name":"","name":"Migrations","msg":"Not migrating, already at version 256"}
ufs: temp directory created at "/tmp/ufs"
Loaded the Apps Framework and loaded a total of 0 Apps!
+----------------------------------------------+
| SERVER RUNNING |
+----------------------------------------------+
| |
| Rocket.Chat Version: 4.5.0 |
| NodeJS Version: 14.18.3 - x64 |
| MongoDB Version: 4.0.28 |
| MongoDB Engine: mmapv1 |
| Platform: linux |
| Process Port: 3000 |
| Site URL: http://localhost:3000 |
| ReplicaSet OpLog: Enabled |
| Commit Hash: 3901054684 |
| Commit Branch: HEAD |
| |
+----------------------------------------------+
+----------------------------------------------------------------------+
| DEPRECATION |
+----------------------------------------------------------------------+
| |
| YOUR CURRENT MONGODB VERSION (4.0.28) IS DEPRECATED. |
| IT WILL NOT BE SUPPORTED ON ROCKET.CHAT VERSION 5.0.0 AND GREATER, |
| PLEASE UPGRADE MONGODB TO VERSION 4.2 OR GREATER |
| |
+----------------------------------------------------------------------+
LocalStore: store created at
LocalStore: store created at
LocalStore: store created at
{"level":40,"time":"2022-05-01T09:58:08.360Z","pid":10,"hostname":"rocketchat","name":"","name":"VoIPService","msg":"Voip is not enabled. Cant start the service"}
{"level":51,"time":"2022-05-01T09:58:08.797Z","pid":10,"hostname":"rocketchat","name":"","name":"Migrations","msg":"Not migrating, already at version 256"}
Loaded the Apps Framework and loaded a total of 0 Apps!
+----------------------------------------------+
| SERVER RUNNING |
+----------------------------------------------+
| |
| Rocket.Chat Version: 4.5.0 |
| NodeJS Version: 14.18.3 - x64 |
| MongoDB Version: 4.0.28 |
| MongoDB Engine: mmapv1 |
| Platform: linux |
| Process Port: 3000 |
| Site URL: http://localhost:3000 |
| ReplicaSet OpLog: Enabled |
| Commit Hash: 3901054684 |
| Commit Branch: HEAD |
| |
+----------------------------------------------+
+----------------------------------------------------------------------+
| DEPRECATION |
+----------------------------------------------------------------------+
| |
| YOUR CURRENT MONGODB VERSION (4.0.28) IS DEPRECATED. |
| IT WILL NOT BE SUPPORTED ON ROCKET.CHAT VERSION 5.0.0 AND GREATER, |
| PLEASE UPGRADE MONGODB TO VERSION 4.2 OR GREATER |
| |
+----------------------------------------------------------------------+
NGINX is receiving http/https requests and the proxy configuration should pass those requests to the host and port on which rocketchat is listening.
In your docker-compose.yaml file, you specify the rocketchat service and map port 2357 on the host machine to port 3000 in the container.
So which that said, if NGINX is running on the host machine as a linux service (not as part of the docker-compose file) then you need to edit the last line of the NGINX configuration for the location to point to localhost:2357:
proxy_pass http://localhost:2357
look at my blog posts regarding hosting and managing Rocket.Chat using docker-compose... here:
https://blog.jarrousse.org/2022/04/09/an-elegant-way-to-use-docker-compose-to-obtain-and-renew-a-lets-encrypt-ssl-certificate-with-certbot-and-configure-the-nginx-service-to-use-it/
here:
https://blog.jarrousse.org/2022/04/26/using-docker-compose-in-to-deploy-rocket-chat/
and here:
https://blog.jarrousse.org/2022/08/01/migrate-a-self-hosted-manual-deployment-of-rocket-chat-to-a-deployment-based-on-docker-compose-without-losing-data/

Problem with deployment bitnami/wordpress on kubernates [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 1 year ago.
Improve this question
Welcome,
Im trying to install Wordpress on kubernetes. I installed chart and I typed:
"helm install projectname-wordpress bitnami/wordpress --set allowOverrideNone=true"
but even it giving output its not working and I cant login into.
When I typed "kubectl describe pods"
I get output like this:
Name: projectname-wordpress-785d4c4c84-xzt6m
Namespace: default
Priority: 0
Node: skalowalne-node-73a107/59.813.226.646
Start Time: Fri, 28 May 2021 02:00:35 +0200
Labels: app.kubernetes.io/instance=projectname-wordpress
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=wordpress
helm.sh/chart=wordpress-11.0.10
pod-template-hash=785d4c4c84
Annotations: <none>
Status: Pending
IP:
IPs: <none>
Controlled By: ReplicaSet/projectname-wordpress-785d4c4c84
Containers:
wordpress:
Container ID:
Image: docker.io/bitnami/wordpress:5.7.2-debian-10-r9
Image ID:
Ports: 8080/TCP, 8443/TCP
Host Ports: 0/TCP, 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Requests:
cpu: 300m
memory: 512Mi
Liveness: http-get http://:http/wp-admin/install.php delay=120s timeout=5s period=10s #success=1 #failure=6
Readiness: http-get http://:http/wp-login.php delay=30s timeout=5s period=10s #success=1 #failure=6
Environment:
ALLOW_EMPTY_PASSWORD: yes
MARIADB_HOST: projectname-wordpress-mariadb
MARIADB_PORT_NUMBER: 3306
WORDPRESS_DATABASE_NAME: bitnami_wordpress
WORDPRESS_DATABASE_USER: bn_wordpress
WORDPRESS_DATABASE_PASSWORD: <set to the key 'mariadb-password' in secret 'projectname-wordpress-mariadb'> Optional: false
WORDPRESS_USERNAME: user
WORDPRESS_PASSWORD: <set to the key 'wordpress-password' in secret 'projectname-wordpress'> Optional: false
WORDPRESS_EMAIL: user#example.com
WORDPRESS_FIRST_NAME: FirstName
WORDPRESS_LAST_NAME: LastName
WORDPRESS_HTACCESS_OVERRIDE_NONE: no
WORDPRESS_ENABLE_HTACCESS_PERSISTENCE: no
WORDPRESS_BLOG_NAME: User's Blog!
WORDPRESS_SKIP_BOOTSTRAP: no
WORDPRESS_TABLE_PREFIX: wp_
WORDPRESS_SCHEME: http
WORDPRESS_EXTRA_WP_CONFIG_CONTENT:
WORDPRESS_AUTO_UPDATE_LEVEL: none
WORDPRESS_PLUGINS: none
Mounts:
/bitnami/wordpress from wordpress-data (rw,path="wordpress")
/var/run/secrets/kubernetes.io/serviceaccount from default-token-mxtw7 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
wordpress-data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: projectname-wordpress
ReadOnly: false
default-token-mxtw7:
Type: Secret (a volume populated by a Secret)
SecretName: default-token-mxtw7
Optional: false
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount 15m (x181 over 9h) kubelet Unable to attach or mount volumes: unmounted volumes=[wordpress-data], unattached volumes=[wordpress-data default-token-mxtw7]: timed out waiting for the condition
Warning FailedMount 3m49s (x58 over 8h) kubelet Unable to attach or mount volumes: unmounted volumes=[wordpress-data], unattached volumes=[default-token-mxtw7 wordpress-data]: timed out waiting for the condition
Warning FailedAttachVolume 2m40s (x139 over 9h) attachdetach-controller AttachVolume.Attach failed for volume "ovh-managed-kubernetes-do2ymc-pvc-80079ec2-e6f9-4210-852e-04fa286f714c" : attachdetachment timeout for volume 3b160677-40e8-4170-9cc3-cdd58e230942
Name: projectname-wordpress-mariadb-0
Namespace: default
Priority: 0
Node: skalowalne-node-f1da93/59.83.226.180
Start Time: Fri, 28 May 2021 02:00:27 +0200
Labels: app.kubernetes.io/component=primary
app.kubernetes.io/instance=projectname-wordpress
app.kubernetes.io/managed-by=Helm
app.kubernetes.io/name=mariadb
controller-revision-hash=projectname-wordpress-mariadb-85d4cb8f7
helm.sh/chart=mariadb-9.3.11
statefulset.kubernetes.io/pod-name=projectname-wordpress-mariadb-0
Annotations: checksum/configuration: 878384c0d68b5abc46d5d5d719a9e83aa911941710552c3dfcebd48203ce5d9f
Status: Pending
IP:
IPs: <none>
Controlled By: StatefulSet/projectname-wordpress-mariadb
Containers:
mariadb:
Container ID:
Image: docker.io/bitnami/mariadb:10.5.10-debian-10-r0
Image ID:
Port: 3306/TCP
Host Port: 0/TCP
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Liveness: exec [/bin/bash -ec password_aux="${MARIADB_ROOT_PASSWORD:-}"
if [[ -f "${MARIADB_ROOT_PASSWORD_FILE:-}" ]]; then
password_aux=$(cat "$MARIADB_ROOT_PASSWORD_FILE")
fi
mysqladmin status -uroot -p"${password_aux}"
] delay=120s timeout=1s period=10s #success=1 #failure=3
Readiness: exec [/bin/bash -ec password_aux="${MARIADB_ROOT_PASSWORD:-}"
if [[ -f "${MARIADB_ROOT_PASSWORD_FILE:-}" ]]; then
password_aux=$(cat "$MARIADB_ROOT_PASSWORD_FILE")
fi
mysqladmin status -uroot -p"${password_aux}"
] delay=30s timeout=1s period=10s #success=1 #failure=3
Environment:
BITNAMI_DEBUG: false
MARIADB_ROOT_PASSWORD: <set to the key 'mariadb-root-password' in secret 'projectname-wordpress-mariadb'> Optional: false
MARIADB_USER: bn_wordpress
MARIADB_PASSWORD: <set to the key 'mariadb-password' in secret 'projectname-wordpress-mariadb'> Optional: false
MARIADB_DATABASE: bitnami_wordpress
Mounts:
/bitnami/mariadb from data (rw)
/opt/bitnami/mariadb/conf/my.cnf from config (rw,path="my.cnf")
/var/run/secrets/kubernetes.io/serviceaccount from projectname-wordpress-mariadb-token-92mm2 (ro)
Conditions:
Type Status
Initialized True
Ready False
ContainersReady False
PodScheduled True
Volumes:
data:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: data-projectname-wordpress-mariadb-0
ReadOnly: false
config:
Type: ConfigMap (a volume populated by a ConfigMap)
Name: projectname-wordpress-mariadb
Optional: false
projectname-wordpress-mariadb-token-92mm2:
Type: Secret (a volume populated by a Secret)
SecretName: projectname-wordpress-mariadb-token-92mm2
Optional: false
QoS Class: BestEffort
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedMount 19m (x41 over 8h) kubelet Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[config projectname-wordpress-mariadb-token-92mm2 data]: timed out waiting for the condition
Warning FailedMount 9m51s (x36 over 8h) kubelet Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[projectname-wordpress-mariadb-token-92mm2 data config]: timed out waiting for the condition
Warning FailedMount 5m21s (x161 over 9h) kubelet Unable to attach or mount volumes: unmounted volumes=[data], unattached volumes=[data config projectname-wordpress-mariadb-token-92mm2]: timed out waiting for the condition
Warning FailedAttachVolume 2m48s (x139 over 9h) attachdetach-controller AttachVolume.Attach failed for volume "ovh-managed-kubernetes-do2ymc-pvc-fad9b535-f6d5-4e71-9e47-3a555936c546" : attachdetachment timeout for volume d96dbb2d-2200-48bd-940d-74dc0c3b5128
UPDATE: I dont have firewall enabled on cloud machine. Im using ovh services.
What should I do to make it working?
Failed events looks exacly like this just after try to deploy wordpress:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 5m19s default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.
Warning FailedScheduling 5m19s default-scheduler 0/3 nodes are available: 3 pod has unbound immediate PersistentVolumeClaims.
Normal Scheduled 5m12s default-scheduler Successfully assigned default/projectname-wordpress-5466b7b45c-rzx9h to standard-node-fe7236
Warning FailedMount 3m10s kubelet Unable to attach or mount volumes: unmounted volumes=[wordpress-data], unattached volumes=[default-token-mxtw7 wordpress-data]: timed out waiting for the condition
Warning FailedAttachVolume 72s (x2 over 3m13s) attachdetach-controller AttachVolume.Attach failed for volume "ovh-managed-kubernetes-do2ymc-pvc-3e3686eb-6cf5-4697-99b0-0689bbd7d0a9" : attachdetachment timeout for volume f8b78a8d-f0d8-4dcb-bcae-ec84fb7d82e4
Warning FailedMount 56s kubelet Unable to attach or mount volumes: unmounted volumes=[wordpress-data], unattached volumes=[wordpress-data default-token-mxtw7]: timed out waiting for the condition
Logs from first pod
mariadb 16:49:02.01 mariadb 16:49:02.01 Welcome to the Bitnami mariadb
container mariadb 16:49:02.01 Subscribe to project updates by watching
https://github.com/bitnami/bitnami-docker-mariadb mariadb 16:49:02.02
Submit issues and feature requests at
https://github.com/bitnami/bitnami-docker-mariadb/issues mariadb
16:49:02.02 Send us your feedback at containers#bitnami.com mariadb
16:49:02.02 mariadb 16:49:02.02 INFO ==> ** Starting MariaDB setup **
mariadb 16:49:02.07 INFO ==> Validating settings in MYSQL_/MARIADB_
env vars mariadb 16:49:02.07 INFO ==> Initializing mariadb database
mariadb 16:49:02.09 INFO ==> Using persisted data mariadb 16:49:02.10
INFO ==> Running mysql_upgrade mariadb 16:49:02.10 INFO ==> Starting
mariadb in background mariadb 16:49:03.14 INFO ==> Stopping mariadb
Logs from second pod
Welcome to the Bitnami wordpress container Subscribe to project
updates by watching
https://github.com/bitnami/bitnami-docker-wordpress Submit issues and
feature requests at
https://github.com/bitnami/bitnami-docker-wordpress/issues Send us
your feedback at containers#bitnami.com
WARN ==> You set the environment variable ALLOW_EMPTY_PASSWORD=yes.
For safety reasons, do not use this flag in a production environment.
nami INFO Initializing apache nami INFO apache successfully
initialized nami INFO Initializing php nami INFO php
successfully initialized nami INFO Initializing mysql-client nami
INFO mysql-client successfully initialized nami INFO Initializing
wordpress wordpre INFO ==> Preparing Varnish environment wordpre INFO
==> Preparing Apache environment wordpre INFO ==> Preparing PHP environment mysql-c INFO Trying to connect to MySQL server Error
executing 'postInstallation': Failed to connect to
student-mariadb:3306 after 36 tries
I reset kubernates and same code started to work. I made everything same so I dont know how. Thank you guys for support.

Maxscale is writing on slave with router_options=master (slave/master replication) and listeners stopped

I've configured on 2 servers(srv50/51),
one of them is Master and the second one is slave,
Here the configuration of my configuration file /etc/maxscale.cnf :
[Read-Only Service]
type=service
router=readconnroute
servers=server50, server51
user=YYYYYYYYYYYYY
passwd=XXXXXXXXXXXXXX
router_options=slave
[Write-Only Service]
type=service
router=readconnroute
servers=server50, server51
user=YYYYYYYYYYYYY
passwd=XXXXXXXXXXXXXX
router_options=master
[Read-Only Listener]
type=listener
service=Read-Only Service
protocol=MySQLClient
port=4008
[Write-Only Listener]
type=listener
service=Write-Only Service
protocol=MySQLClient
port=4009
As i understool the router_options look who is the master and send the writing query to the master
Maxscale (via maxadmin) seems to discover the 2 serveur and understand witch one is the Master :
MaxScale> list servers
Servers.
-------------------+-----------------+-------+-------------+--------------------
Server | Address | Port | Connections | Status
-------------------+-----------------+-------+-------------+--------------------
server51 | 192.168.0.51 | 3306 | 0 | Slave, Running
server50 | 192.168.0.50 | 3306 | 0 | Master, Running
-------------------+-----------------+-------+-------------+--------------------
But even if I connect in Mysql in local on my Maxscale Write-Only Listener port (4009), Listener are in Stopped mode, is it normal ?
MaxScale> list listeners
Listeners.
---------------------+--------------------+-----------------+-------+--------
Service Name | Protocol Module | Address | Port | State
---------------------+--------------------+-----------------+-------+--------
Read-Only Service | MySQLClient | * | 4008 | Stopped
Write-Only Service | MySQLClient | * | 4009 | Stopped
MaxAdmin Service | maxscaled | * | 6603 | Running
---------------------+--------------------+-----------------+-------+--------
I've try to create a database in srv51 (slave), and it was created only on srv51, not in srv50.
Is something wrong in my configuration ? It's strange because it's not my first cluster, and on other cluster all write go to the master (but listeners are Running). Do i don't understand well the meaning of "router_options=master" ? How to start listeners ? I prefere to keep the 51 in Write list to detect topology change
===== UPDATE =====
After watching Log file /var/log/maxscale/maxscale1.log
I found that my monitor user didn't have the correct password :
[MySQL Monitor]
type=monitor
module=mysqlmon
servers=server50, server51
user=MONITOR
passwd=MONITOR_PASS
monitor_interval=10000
I corrected password for user and restarted maxscale, Now everything is running :
MaxScale> list listeners
Listeners.
---------------------+--------------------+-----------------+-------+--------
Service Name | Protocol Module | Address | Port | State
---------------------+--------------------+-----------------+-------+--------
Read-Only Service | MySQLClient | * | 4008 | Running
Write-Only Service | MySQLClient | * | 4009 | Running
MaxAdmin Service | maxscaled | * | 6603 | Running
---------------------+--------------------+-----------------+-------+--------
But write query are still done on Slave and not on Master
Thanks to MariaDb support, I was trying to connect like this :
mysql -h localhost --port=4009 -u USER -p
But Maxscale & Mysql were installed in the same server, even if Mysql bind port 3306, when you specify 'localhost', the connection is done on Mysql port 3306 and not in Maxscale port 4009, the port is ignore !!
The solution is to connect like this :
mysql -h 127.0.0.1 --port=4009 -u USER -p
or like this :
mysql -h localhost --protocol=tcp --port=4009 -u USER -p
I've try both solution and they works.
The solution about the listener not Running is on update of the question.
If writes are done on the slaves, the simplest explanation would be that you're executing writes on the wrong port or your configuration is wrong. To diagnose these problems, enable the info log level by adding log_info=true under the [maxscale] section.
If enabling the info log and inspecting the log files does not provide any clues, I'd suggest opening a bug report on the Maxscale Jira.

Resources