Is there any way (and documentation) for running Azure Cosmos DB emulator on Linux with Cassandra API available?
I have been trying running it with docker-compose.yaml
version: '3.4'
services:
db:
container_name: cosmosdb
image: "mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator"
tty: true
restart: always
mem_limit: 2G
cpu_count: 2
environment:
- AZURE_COSMOS_EMULATOR_PARTITION_COUNT=10
- AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE=true
- AZURE_COSMOS_EMULATOR_CASSANDRA_ENDPOINT=true
- AZURE_COSMOS_EMULATOR_GREMLIN_ENDPOINT=true
ports:
- "8081:8081"
- "8900:8900"
- "8901:8901"
- "8979:8979"
- "10250:10250"
- "10251:10251"
- "10252:10252"
- "10253:10253"
- "10254:10254"
- "10255:10255"
- "10256:10256"
- "10350:10350"
volumes:
- vol_cosmos:/data/db
volumes:
vol_cosmos:
But port 10350 is not used in container and nothing is listening there.
Is there any documented way how to run Cassandra API (and MongoDb as well) with linux container?
Thanks
Related
I was using Keycloak 16. Now that I want to upgrade to Keycloak 20, I see that they have changed a lot.
This is my docker-compose.yml file from 16:
version: "3.9"
services:
accounts:
image: jboss/keycloak:latest
container_name: Accounts
ports:
- 8080:8080
environment:
- KEYCLOAK_FRONTEND_URL=https://accounts.example.local/auth
- PROXY_ADDRESS_FORWARDING=true
- KEYCLOAK_USER=user
- KEYCLOAK_PASSWORD=pass
- DB_VENDOR=mariadb
- DB_ADDR=database
- DB_DATABASE=accounts
- DB_USER=db_user
- DB_PASSWORD=db_pass
logging:
driver: none
restart: always
database:
image: mariadb
container_name: AccountsDatabase
ports:
- 3306:3306
environment:
- MARIADB_ROOT_PASSWORD=root_pass
- MYSQL_DATABASE=accounts
- MYSQL_USER=db_user
- MYSQL_PASSWORD=db_pass
volumes:
- /Temp/AccountsDatabases:/var/lib/mysql
logging:
driver: none
restart: always
admin:
image: adminer
container_name: AccountsAdminer
restart: always
logging:
driver: none
ports:
- 8080:8080
environment:
- ADMINER_DEFAULT_SERVER=database
Now it seems that Keycloak needs a database URL.
I can't find out how can I connect MariaDB to Keycloak. I can't find out the URL of my MariaDB URL and the Keycloak blog says that they won't provide examples for any database other than their first class PostreSQL.
I'm stuck at this point. Any help is appreciated.
Their documents show KC_DB_URL is a JDBC URL.
So the simple form of jdbc:mariadb://host/database seems used in their tests, so for you:
environment:
- KEYCLOAK_FRONTEND_URL=https://accounts.example.local/auth
- PROXY_ADDRESS_FORWARDING=true
- KEYCLOAK_USER=user
- KEYCLOAK_PASSWORD=pass
- KB_DB_URL=jdbc:mariadb://database/accounts
- KB_DB_USER=db_user
- KB_DB_PASSWORD=db_pass
note: I'm hoping/assuming the JDBC driver for MariaDB is in their container which it may not be.
Need help connecting the certificate. For example, the image is ready docker wordpress bitnami. How to configure ssl for apache in container using certbot?
Docker-compose:
version: '2'
services:
mariadb:
image: docker.io/bitnami/mariadb:10.6
volumes:
- 'mariadb_data:/bitnami/mariadb'
environment:
# ALLOW_EMPTY_PASSWORD is recommended only for development.
- ALLOW_EMPTY_PASSWORD=yes
- MARIADB_USER=bn_wordpress
- MARIADB_DATABASE=bitnami_wordpress
wordpress:
image: docker.io/bitnami/wordpress:6
ports:
- '80:8080'
- '443:8443'
volumes:
- 'wordpress_data:/bitnami/wordpress'
depends_on:
- mariadb
environment:
# ALLOW_EMPTY_PASSWORD is recommended only for development.
- ALLOW_EMPTY_PASSWORD=yes
- WORDPRESS_DATABASE_HOST=mariadb
- WORDPRESS_DATABASE_PORT_NUMBER=3306
- WORDPRESS_DATABASE_USER=bn_wordpress
- WORDPRESS_DATABASE_NAME=bitnami_wordpress
I'm having an issue with my php container not connecting to my database container.
My docker-compose.yml :
version: "2"
volumes:
# this is the mysql data volume we are going to host the data inside
dev_mysql_data:
# This volume is used for elasticsearch
dev_elastic_search:
networks:
mp_pixel:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.20.0.0/16
services:
# database container for local development purposes
dev_database:
image: mysql:5.6
networks:
mp_pixel:
aliases:
- database
ports:
# port 3304 (external) is for use on your desktop sql client
# port 3306 (internal) is for use inside your application code
- 3304:3306
volumes:
# mount the mysql_data docker volume to host the local development database
- dev_mysql_data:/var/lib/mysql
# the provision file helps when trying to use the provision script to clone databases
- ./provision.cnf:/provision.cnf
environment:
MYSQL_ROOT_PASSWORD: pixel
# This is the local development version of the nginx container
dev_nginx:
image: mp-pixel-nginx:latest
build: ./nginx
ports:
- '80:80'
- '443:443'
networks:
mp_pixel:
aliases:
- nginx
depends_on:
- dev_phpfpm
volumes_from:
- dev_phpfpm
environment:
- VIRTUAL_HOST=~^(mp-pixel|mp-location|mp-feedback|mp-user|mp-phone|mp-loancalculator|mp-seo|mp-media|mp-listing|mp-development|mp-kpi|mp-newsletter|mp-auth|mp-worker|mp-search)-ph-dev.pixel.local
# This is the local development version of the phpfpm container
dev_phpfpm:
image: mp-pixel-phpfpm:latest
build:
context: ./
args:
# this build might fail, if so, run in a terminal: export SSH_KEY=$(cat ~/.ssh/id_rsa)
- SSH_KEY=$SSH_KEY
networks:
mp_pixel:
aliases:
- phpfpm
depends_on:
- dev_database
volumes:
# we override the images /www directory with the code from the live machine
- ./:/www
env_file:
# inside this file, are the shared database secrets such as username/password
- ./env/common
- ./env/dev
dev_elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.3.3
networks:
mp_pixel:
aliases:
- elasticsearch
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
mem_limit: 1g
cap_add:
- IPC_LOCK
volumes:
- dev_elastic_search:/usr/share/elasticsearch/data
ports:
- 9200:9200
environment:
- cluster.name=dev-elasticsearch-pixel
- bootstrap.memory_lock=true
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- "xpack.security.enabled=false"
I run it with docker-compose up and the php logs show
An exception occured in driver: SQLSTATE[HY000] [2002] Connection timed out
I try to access the database container with docker exec, and I can confirm that I have the right credentials.
What could be the problem?
When your containers are up, did you already try to connect to the database with a tool like Sequel Pro? Maybe the database is just not initialized and because of this, the connection from the php container can't be established? You tried to access the db container but not the database itself.
Additionally you could add some more environment variables to the database section of your docker-compose.yml
environment:
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
- MYSQL_DATABASE=databasename
- MYSQL_USER=databaseuser
- MYSQL_PASSWORD=databasepassword
Hope that helps
Description
I am trying to build an equal configuration in my local docker-environment like on our production system. After spending some time investigating and rebuilding the docker container setup, still can't get it to work and Graylog is not receiving any data.
Overview and interim results
web, php and db container are in use for the symfony based application
symfony runs properly on localhost in php-container and generates logfiles
symfony-logfiles are located here: /var/www/html/var/logs/*.log
symfony-logfiles format is json / gelf
all other containers are also up and running when starting the complete composition
filebeat configuration is based on first link below
filebeat.yml seems to retrieve any logfile found in any container
filebeat configured to transfer data directly to elasticsearch
elasticsearch persists data in mongodb
all graylog related data in persisted in named volumes in docker
additionally I am working with docker-sync on a Mac
The docker-compose.yml is based on the following resources:
https://github.com/jochenchrist/docker-logging-elasticsearch
http://docs.graylog.org/en/2.4/pages/installation/docker.html?highlight=docker
https://www.elastic.co/guide/en/beats/filebeat/6.3/running-on-docker.html
https://www.elastic.co/guide/en/beats/filebeat/6.3/filebeat-reference-yml.html
config.yml
# Monolog Configuration
monolog:
channels: [graylog]
handlers:
graylog:
type: stream
formatter: line_formatter
path: "%kernel.logs_dir%/graylog.log"
channels: [graylog]
docker-compose.yml
version: "3"
services:
web:
image: nginx
ports:
- "80:80"
- "443:443"
links:
- php
volumes:
- ./docker-config/nginx.conf:/etc/nginx/conf.d/default.conf
- project-app-sync:/var/www/html
- ./docker-config/localhost.crt:/etc/nginx/ssl/localhost.crt
- ./docker-config/localhost.key:/etc/nginx/ssl/localhost.key
php:
build:
context: .
dockerfile: ./docker-config/Dockerfile-php
links:
- graylog
volumes:
- project-app-sync:/var/www/html
- ./docker-config/php.ini:/usr/local/etc/php/php.ini
- ./docker-config/www.conf:/usr/local/etc/php-fpm.d/www.conf
db:
image: mysql
ports:
- "3306:3306"
environment:
- MYSQL_ALLOW_EMPTY_PASSWORD=yes
- MYSQL_DATABASE=project
- MYSQL_USER=project
- MYSQL_PASSWORD=password
volumes:
- ./docker-config/mysql.cnf:/etc/mysql/conf.d/mysql.cnf
- project-mysql-sync:/var/lib/mysql
# Graylog / Filebeat
filebeat:
build: ./docker-config/filebeat
volumes:
- /var/lib/docker/containers:/var/lib/docker/containers:ro
- /var/run/docker.sock:/var/run/docker.sock
networks:
- graylog-network
depends_on:
- graylog-elasticsearch
graylog:
image: graylog/graylog:2.4
volumes:
- graylog-journal:/usr/share/graylog/data/journal
networks:
- graylog-network
environment:
- GRAYLOG_PASSWORD_SECRET=somepasswordpepper
- GRAYLOG_ROOT_PASSWORD_SHA2=8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
- GRAYLOG_WEB_ENDPOINT_URI=http://127.0.0.1:9000/api
links:
- graylog-mongo:mongo
- graylog-elasticsearch:elasticsearch
depends_on:
- graylog-mongo
- graylog-elasticsearch
ports:
# Graylog web interface and REST API
- 9000:9000
graylog-mongo:
image: mongo:3
volumes:
- graylog-mongo-data:/data/db
networks:
- graylog-network
graylog-elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:5.6.10
ports:
- "9200:9200"
volumes:
- graylog-elasticsearch-data:/usr/share/elasticsearch/data
networks:
- graylog-network
environment:
- cluster.name=graylog
- "discovery.zen.minimum_master_nodes=1"
- "discovery.type=single-node"
- http.host=0.0.0.0
- transport.host=localhost
- network.host=0.0.0.0
# Disable X-Pack security: https://www.elastic.co/guide/en/elasticsearch/reference/5.6/security-settings.html#general-security-settings
- xpack.security.enabled=false
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
ulimits:
memlock:
soft: -1
hard: -1
volumes:
project-app-sync:
external: true
project-mysql-sync: ~
graylog-mongo-data:
driver: local
graylog-elasticsearch-data:
driver: local
graylog-journal:
driver: local
networks:
graylog-network: ~
Dockerfile of filebeat container
FROM docker.elastic.co/beats/filebeat:6.3.1
COPY filebeat.yml /usr/share/filebeat/filebeat.yml
# must run as root to access /var/lib/docker and /var/run/docker.sock
USER root
RUN chown root /usr/share/filebeat/filebeat.yml
# dont run with -e, to disable output to stderr
CMD [""]
filebeat.yml
filebeat.prospectors:
- type: docker
paths:
- '/var/lib/docker/containers/*/*.log'
# path to symfony based logs
- '/var/www/html/var/logs/*.log'
containers.ids: '*'
processors:
- decode_json_fields:
fields: ["host","application","short_message"]
target: ""
overwrite_keys: true
- add_docker_metadata: ~
output.elasticsearch:
# transfer data to elasticsearch container?
hosts: ["localhost:9200"]
logging.to_files: true
logging.to_syslog: false
Graylog backend
After setting up this docker composition I started the Graylog web-view and set up a collector and input as described here:
http://docs.graylog.org/en/2.4/pages/collector_sidecar.html#step-by-step-guide
Maybe I have totally misunderstood how this could work. I am not totally sure if Beats from Elastic is the same as the filebeats container and if the sidecar collector is something extra I forgot to add. Maybe I misconfigured the collector and input in graylog?!
I would be thankful to any help or working example according to my problem ...
Graylog seems to be running on http://127.0.0.1:9000/api which is in the container. You might want to run it as http://graylog:9000/api or as http://0.0.0.0:9000/api
Accessing the other images from within any of the other images will have be done with the same name as the service name, as defined in the docker-compose.yml files. The url to the graylog-elasticsearch would be something like: http://graylog-elasticsearch/.... if you would post to localhost it would stay inside its own image.
Hope this will help you along in finding the solution.
I am running two docker containers, using the following docker-composer script:
db:
image: cofoundry/mysql
environment:
- MYSQL_USER=wordpress
- MYSQL_PASS=wordpress
- MYSQL_DB=wordpress
ports:
- "3306:3306"
privileged: true
web:
image: cofoundry/nginx-phpfpm
volumes:
- ../bedrock:/app
environment:
- DOCROOT=/app/web
links:
- db
ports:
- "80:80"
privileged: true
But I'm not sure how can I go about running multiple nginx-phpfpm web containers with different sites, like web_1, web_2 etc. ?
Or, alternatively, how to propagate custom nginx.conf while running docker-composer up, so I can configure virtualhosts there.
Thanks!
So the way to do it (as I mentioend in my comment) is to use something like autodock:
Example: (based on your docker-compose.yml):
db:
image: cofoundry/mysql
environment:
- MYSQL_USER=wordpress
- MYSQL_PASS=wordpress
- MYSQL_DB=wordpress
ports:
- "3306:3306"
privileged: true
web:
image: cofoundry/nginx-phpfpm
volumes:
- ../bedrock:/app
environment:
- DOCROOT=/app/web
- VIRTUALHOST=myapp.example.com
links:
- db
privileged: true
autodock:
image: prologic/autodock
ports:
- "1338:1338/udp"
- "1338:1338/tcp"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
autodockhipache:
image: prologic/autodock-hipache
links:
- autodock
- hipache:redis
hipache:
image: hipache
ports:
- 80:80
- 443:443
The only requirement is that you setup a wildcard DNS entry for *.example.com to point to your "Docker Host". e.g:
*.example.com A 1.1.1.1
You can read more about how this all works in the article:
A Docker-based mini PaaS
This will scale and work for as many websites as you want to run on your "Docker Host".
Edit: The nice thing about this setup is that it's simple, it's automated and it only requires you add a single extra environment variable to the container you want to expose to the web with only one initial setup requirements (wildcard dns entry).