I'm trying to configure Gitlab-ci with a Symfony project, and after reading the documentation and some examples in external blogs I'm unable to complete the setup. Those are my files:
.gitlab-ci.yml:
# Select image from https://hub.docker.com/_/php/
image: php:5.6
# Select what we should cache
cache:
paths:
- vendor/
before_script:
# Install git, the php image doesn't have installed
- apt-get update -yqq
- apt-get install git -yqq
- apt-get install wget -yqq
- apt-get install zip unzip zlib1g-dev -yqq
# Install mysql driver & zip
- docker-php-ext-install pdo_mysql
- docker-php-ext-install zip
- docker-php-ext-install mbstring
# Install composer
- curl -sS https://getcomposer.org/installer | php
# Install all project dependencies
- mv app/config/parameters.gitlab.yml app/config/parameters.yml.dist
- ping -c 3 mysql
- php -v
- php composer.phar clear-cache
- php composer.phar install
- php bin/console doctrine:schema:create
services:
- mysql:latest
variables:
# Configure mysql service (https://hub.docker.com/_/mysql/)
MYSQL_DATABASE: symfony
MYSQL_ROOT_PASSWORD: password
# We test PHP5.6 (the default) with MySQL
test:mysql:
script:
- vendor/bin/phpunit --configuration phpunit.xml --coverage-text
parameters.gitlab.yml (which becomes parameters.yml.dist)
# This file is auto-generated during the composer install
parameters:
database_driver: pdo_mysql
database_host: mysql
database_port: 3306
database_name: symfony
database_user: root
database_password: password
I have tried mysql user root, and another user using 'MYSQL_USER' variable. The result is always the same:
$ mv app/config/parameters.gitlab.yml app/config/parameters.yml.dist
$ ping -c 3 mysql
PING mysql (172.17.0.2): 56 data bytes
64 bytes from 172.17.0.2: icmp_seq=0 ttl=64 time=0.282 ms
64 bytes from 172.17.0.2: icmp_seq=1 ttl=64 time=0.140 ms
64 bytes from 172.17.0.2: icmp_seq=2 ttl=64 time=0.151 ms
--- mysql ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.140/0.191/0.282/0.065 ms
$ php -v
PHP 5.6.26 (cli) (built: Sep 23 2016 21:22:39)
Copyright (c) 1997-2016 The PHP Group
Zend Engine v2.6.0, Copyright (c) 1998-2016 Zend Technologies
$ php composer.phar clear-cache
Do not run Composer as root/super user! See https://getcomposer.org/root for details
Clearing cache (cache-dir): /root/.composer/cache
Clearing cache (cache-files-dir): /root/.composer/cache/files
Clearing cache (cache-repo-dir): /root/.composer/cache/repo
Cache directory does not exist (cache-vcs-dir):
All caches cleared.
$ php composer.phar install
......
> Incenteev\ParameterHandler\ScriptHandler::buildParameters
Updating the "app/config/parameters.yml" file
> Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::buildBootstrap
> Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache
[Doctrine\DBAL\Exception\ConnectionException]
An exception occured in driver: SQLSTATE[HY000] [2002] Connection refused
[Doctrine\DBAL\Driver\PDOException]
SQLSTATE[HY000] [2002] Connection refused
[PDOException]
SQLSTATE[HY000] [2002] Connection refused
Script Sensio\Bundle\DistributionBundle\Composer\ScriptHandler::clearCache handling the post-install-cmd event terminated with an exception
What am I doing wrong or missing?
The problem was in dev environment, as tackerm told me in Gitlab forum. I was using a parameters_dev.yml file on dev environment, and composer install was reading that file instead of parameters.yml as I thought. Changing database settings in parameters_dev.yml solves the error and now Gitlab is connecting and executing tests.
Related
I am running my flask project from uwsgi on nginx. But my nginx is not routing the request to uwsgi when i hit localhost:80/
My nginx.conf looks like this
server {
listen 80;
server_name <your machine ip/domain>;(if on local it would be localhost but I was running on WSL so I put it IP)
location / {
include uwsgi_params;
uwsgi_pass web_app:5000; (you might see suggestion of .sock files or suffixing http:// or unix: but none work for me plain simple your python server's service name which you would provide in docker-compose)
}
}
docker-compose looks like this
version: '3.7'
services:
web_app:
build: .
container_name: kpi-dashboard
ports:
- 5000:5000
depends_on:
- db
nginx:
build: ./nginx
container_name: nginx
restart: always
ports:
- "80:80"
depends_on:
- web_app
db:
image: postgres:13-alpine
container_name: postgresql
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=postgres
- POSTGRES_DB=postgres
ports:
- 5432:5432
volumes:
postgres_data:
nginx dockerfile
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf (it is important to remove the default conf as it would not take your custom conf no matter where you copy it)
COPY nginx.conf /etc/nginx/conf.d/
(there are answers online to copy it no other places but this only works)
EXPOSE 80
web app dockerfile
FROM python:3.8.16-slim-buster
RUN apt-get update
RUN apt-get install gcc -y && apt-get install python3-dev -y && apt-get install libpq-dev -y
ENV PYTHONPATH=${PYTHONPATH}:${PWD}
RUN pip install poetry
WORKDIR /app
COPY pyproject.toml /app/
COPY . /app/
RUN poetry config virtualenvs.create false
RUN poetry install --no-dev
EXPOSE 5000
CMD ["uwsgi", "--ini", "wsgi.ini"]
wsgi.ini file
[uwsgi]
module = app (this is when you are writing you project entrypoint in app.py. if you are writing in wsgi.py then this would become wsgi:app)
socket = 0.0.0.0:5000
callable = app (this is important as wsgi by default considers your app instance as application either handle it in your main file or just add this configuration)
processes = 1
threads = 1
master = true
vacuum = true
die-on-term = true
This is what the nginx container output looks like
Editing question as the 404 issue was solved. But nginx is still not routing to wsgi.
The solution
changed the location of copying the nginx.conf file in nginx dockerfile
COPY nginx.conf /etc/nginx/nginx.config
Editing question again as nginx routing to wsgi issue also resolved.
The solution
updated files as mentioned above
Yes so this worked for me. There are n number of configurations available online and almost all are same yet a slight difference causes the issue.
I am updating my question to change files with the content that worked. Hope it helps someone.
In my first attempt at running a more complex application through Docker, I selected the Symfony Demo app and assembled a docker build structure to accommodate it.
The first image is httpd: it runs as root (dropping to www-data afterwards) and talks through the 'server' custom network.
The second image is php (fpm): it runs as root (dropping to www-data afterwards) and also talks through the 'server' custom network.
The third image is composer: it runs as UID and GID 1000. Its entrypoint command is composer create-project symfony/symfony-demo symfony-demo
All containers share the same bind mount, where the symfony-demo app is located.
Then I go to localhost:8080 in the browser just to end up with a Symfony error:
The stream or file "/usr/local/apache2/htdocs/symfony-demo/var/log/dev.log" could not be opened: failed to open stream: Permission denied
The thing is... this file mentioned doesn't even exist at /var/log/. That folder is empty.
All files in the bind mount have permissions 1000:1000 (my user UID/GID) and are configured like this: -rw-r--r--.
I've tried running httpd and php as: UID 33 (www-data) and GID 33; UID 0 (root) and GID 33 (and vice-versa); and also as 1000:1000 or 1000:33, but all these combinations (when they successfully get httpd/php to start up) result in the same error.
docker-compose.yml:
version: "3"
services:
httpd:
build: "./httpd/"
container_name: "webserver"
depends_on:
- php
ports:
- "8080:80"
networks:
- server
volumes:
- ../app:/usr/local/apache2/htdocs/
php:
build: "./php/"
depends_on:
- composer
container_name: "php"
networks:
- server
volumes:
- ../app:/usr/local/apache2/htdocs/
composer:
build: "./composer/"
container_name: "composer"
user: "1000:1000"
volumes:
- ../app:/usr/local/apache2/htdocs/
networks:
server:
driver: bridge
composer Dockerfile:
FROM composer:1.8
WORKDIR /usr/local/apache2/htdocs/
CMD ["composer", "create-project", "symfony/symfony-demo", "symfony-demo"]
httpd Dockerfile:
FROM httpd:2.4
COPY ./config/httpd.conf /usr/local/apache2/conf/httpd.conf
COPY ./config/httpd-vhosts.conf /usr/local/apache2/conf/extra/httpd-vhosts.conf
COPY ./config/php-fpm.conf /usr/local/apache2/conf/extra/php-fpm.conf
WORKDIR /usr/local/apache2/htdocs
php Dockerfile:
FROM php:7.3-fpm
RUN cp "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini"
COPY ./config/timezone.ini $PHP_INI_DIR/conf.d/
COPY ./config/www.conf /usr/local/etc/php-fpm.d/www.conf
RUN apt-get update && \
apt-get install -y libicu-dev
RUN docker-php-ext-install intl
WORKDIR /usr/local/apache2/htdocs
just give the write permission
chmod -R 777 /usr/local/apache2/htdocs/symfony-demo/var/log/dev.log
here symfony doc for file permission: https://symfony.com/doc/current/setup/file_permissions.html
On second thoughts: my previous solution (as is) doesn't work in RHEL/Fedora/CentOS, because www-data does not exist there by default, causing Docker to fail to start.
My new solution - distro agnostic
For simplicity, I've decided to simply write composer's entrypoint script to set -rw-rw---- permissions at /app. That way, I can run composer as user 1000 and the same group PHP runs as (a new user and group was created just for that). Now PHP can write to SQLite3 database files inside the project and composer writes as user 1000, which I can edit.
It's basically what #habibun said, but I only need to give group write permissions, not full write permissions.
Be aware that SELinux will deny composer write access to your bind mount. You must configure SELinux to allow this operation.
This is my repository where this project is stored, if you're looking for a reference: https://github.com/o-alquimista/symfony-demo-docker/
User namespace solution - works fine for Debian/Ubuntu hosts
Composer should write to /app as user 33 (www-data), and so should php and httpd after they drop privileges. I was able to keep present permission settings (only owner can write) by making use of user namespaces. The user www-data is now mapped to the range 967 and beyond, which will result in user 33 being = me (user 1000).
Now all containers can write where they need to, and I can edit the project files as an unprivileged user.
I'm looking how to access database with something like phpmyadmin using Ubuntu and Symfony 3.4 (I will have to work with this kind of symfony version soon).
php bin/console server:start
[OK] Server listening on http://127.0.0.1:8000
But when I try localhost:8000/phpmyadmin it's not working. I think there is a specific url when we are using this light servor, but I can't find the symfony documentation about it.
Thanks for your help.
EDIT: Can't create the database.
MySql is installed
dpkg --get-selections | grep mysql
dbconfig-mysql install
libmysqlclient20:amd64 install
libqt4-sql-mysql:amd64 install
mysql-client-5.7 install
mysql-client-core-5.7 install
mysql-common install
mysql- server install
mysql-server-5.7 install
mysql-server-core-5.7 install
php-mysql install
php7.0-mysql install
I can connect to this page: http://127.0.0.1:80/phpmyadmin/
I can connect to the symfony app on this url: http://localhost:8000/
My parameters file
# This file is auto-generated during the composer install
parameters:
database_host: 127.0.0.1
database_port: 80
database_name: symfonyTest
database_user: root
database_password: MyPassword
mailer_transport: smtp
mailer_host: 127.0.0.1
mailer_user: null
mailer_password: null
secret: b3b09e3360a0e09dfbb92723c767ec81d87235ff
But still php bin/console doctrine:database:create is not working
It won't work, It is an internal Symfony server, you should install PHPMyAdmin separately and set up a web server on your computer. also, you can use mysql workbench to have something like PHPMyAdmin in ubuntu.
On symfony 4, I unzipped phpmyadmin into my symfony's project's public/ folder
I can then access it using symfony's server via
localhost:8000/phpmyadmin/index.php
Issue
My main goal is to create a wordpress container on my linux machine for development on that container.
Therefore, I'm creating new volumes for the plugins and themes folders and even add read & write permissions, with the :rw option.
However, when I'm trying to create a new directory or file in those "volumes", I get an error message (especially in VS Code), that tells that I don't have the permission to add these volumes.
Moreover, the permissions based on the ll command shows that the owner is the only one with the w permissions. It means that I cannot use the group www-data and add it to my user in order to edit those volumes.
When I'm trying to run chmod 766 themes plugins, the volumes are not bind anymore, so this solution doesn't work.
I've searched the whole web (including stack overflow), but none of the answers didn't work for me, so I'm lost :(
Here are some details that can help you with finding solution, including the docker-compose.yml file.
Details
Error on VS Code when trying to add a directory named hello
A system error occurred (EACCES: permission denied, mkdir '~/Workspace/WordpressProject/themes/hello')
Run mkdir hello in terminal, returns this error
mkdir: cannot create directory ‘hello’: Permission denied
ll command result for volumes
drwxr-xr-x 2 www-data www-data 4096 Jun 14 23:54 plugins/
drwxr-xr-x 5 www-data www-data 4096 May 17 22:00 themes/
docker-compose.yml
version: '3'
services:
wordpress:
image: wordpress
links:
- mariadb:mysql
environment:
- WORDPRESS_DB_PASSWORD=${Database Name}
ports:
- "127.0.0.101:80:80"
hostname: ${Wordpress Host}
volumes:
- ./plugins:/var/www/html/wp-content/plugins:rw
- ./themes:/var/www/html/wp-content/themes:rw
restart: always
mariadb:
image: mariadb
environment:
- MYSQL_ROOT_PASSWORD=${Password}
- MYSQL_DATABASE=${Database Name}
volumes:
- ./database:/var/lib/mysql
restart: always
Versions
OS: Ubuntu 18.04 LTS
Docker: v18.05.0-ce, API v1.37
Editor (Visual Studio Code): v1.23.1
Please in VS terminal execute:
sudo chown -R $USER <directory_project>
I have a container A with Dockerfile as follows:
...
FROM ubuntu:latest
MAINTAINER arpitaggarwal "aggarwalarpit.89#gmail.com"
EXPOSE 8080
and another container B with Dockerfile as follows:
...
FROM ubuntu:latest
MAINTAINER arpitaggarwal "aggarwalarpit.89#gmail.com"
RUN apt-get install -q -y mysql-server
EXPOSE 3306
Then I started the container B using command:
docker run -P -it --name db B /bin/bash
And running the command: docker run --rm --name web --link db A env
gives me below output:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=a5dd66b72ca8
DB_PORT=tcp://172.17.0.2:3306
DB_PORT_3306_TCP=tcp://172.17.0.2:3306
DB_PORT_3306_TCP_ADDR=172.17.0.2
DB_PORT_3306_TCP_PORT=3306
DB_PORT_3306_TCP_PROTO=tcp
DB_NAME=/web/db
HOME=/root
Then I logged into container A and run the command: ping db
which gives me output:
PING db (172.17.0.2) 56(84) bytes of data.
64 bytes from db (172.17.0.2): icmp_seq=1 ttl=64 time=0.082 ms
64 bytes from db (172.17.0.2): icmp_seq=2 ttl=64 time=0.063 ms
64 bytes from db (172.17.0.2): icmp_seq=3 ttl=64 time=0.065 ms
64 bytes from db (172.17.0.2): icmp_seq=4 ttl=64 time=0.061 ms
64 bytes from db (172.17.0.2): icmp_seq=5 ttl=64 time=0.066 ms
And when I tried telnet command with port 3306, as follows:
telnet 172.17.0.2 3306
gives me output:
root#9b078c1fed82:/# telnet 172.17.0.2 3306
Trying 172.17.0.2...
telnet: Unable to connect to remote host: Connection refused
Any idea, how can I telnet linked container?
Any help will be appreciated!
MySQL usually only listens on 127.0.0.1 by default. You need to modify the my.cnf during build and set bind-address to 0.0.0.0. You could accomplish this with something like:
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
After receiving the answer by #Andy Shinn, I Updated the container B Dockerfile as below:
FROM ubuntu:latest
MAINTAINER arpitaggarwal "aggarwalarpit.89#gmail.com"
RUN apt-get install -q -y mysql-server
RUN apt-get install -q -y mysql-client
RUN sed -i -e"s/^bind-address\s*=\s*127.0.0.1/bind-address = 0.0.0.0/" /etc/mysql/my.cnf
EXPOSE 3306
And when I tried accessing mysql-server from container A as:
mysql -u root -ppassword -h 172.17.0.2
Gives me error:
ERROR 1130 (HY000): Host '172.17.0.3' is not allowed to connect to this MySQL server
Which I figured it out is because container A was trying to connect to db container through root user. But In mysql do not allow you to connect though root user remotely. So I work around creating another user in mysql-server with root privileges.