I have a small setup for a few services inside a docker-compose.yaml. For brevity, this is the service which is intended as the main API to use other services:
services:
fprint-api:
container_name: fprint-api-v2
image: "fprint-api:v0.0.1"
depends_on:
- fprint-svc
extra_hosts:
- "host.docker.internal:host-gateway"
ports:
- "8000:8000"
build:
context: ../.
dockerfile: docker/Dockerfile.fprint-api
# ...
fprint-api has a simple health-check endpoint like so:
#app.get("/health")
def health():
return "API OK"
If I just run docker-compose up on this, or use the Docker Compose run-configuration in PyCharm, everything works and I am able to make a GET request to http://localhost:8000.
However, if I use a remote python interpreter on said docker-compse.yaml and the fprint-api service, I can't reach this endpoint anymore. The system spins up, but the endpoint is not accessible and as such I am unable to debug my endpoints.
I am not sure what I'm missing here exactly.
Remote Interpreter Configuration
Run-Configuration for the fprint-api Service
Okay, that's an easy one.
uvicorn needs the --host flag set to 0.0.0.0 in order to be able to access the URL via the hostname localhost:
Related
I have 2 folders separated, one for backend and one for frontend services:
backend/docker-compose.yml
frontend/docker-compose.yml
The backend has a headless wordpress installation on nginx, with the scope to serve the frontend as an api service. The frontend runs on next.js. Here are the 2 different docker-compose.yml:
backend/docker-compose.yml
version: '3.9'
services:
nginx:
image: nginx:latest
container_name: my-app-nginx
ports:
- '80:80'
- '443:443'
- '8080:8080'
...
networks:
- internal-network
mysql:
...
networks:
- internal-network
wordpress:
...
networks:
- internal-network
networks:
internal-network:
external: true
frontend/docker-compose.yml
version: '3.9'
services:
nextjs:
build:
...
container_name: my-app-nextjs
restart: always
ports:
- 3000:3000
networks:
- internal-network
networks:
internal-network:
driver: bridge
name: internal-network
In the frontend I use the fetch api in nextjs as following:
fetch('http://my-app-nginx/wp-json/v1/enpoint', ...)
I tried also with ports 80 and 8080, without success.
The sequence of commands I run are:
docker network create internal-network
in backend/ folder, docker-compose up -d (all backend containers run fine, I can fetch data with Postman from WordPress api)
in frontend/ folder, docker-compose up -d fails with the error Error: getaddrinfo EAI_AGAIN my-app-nginx
I am not a very expert user of docker so I might miss something here, but I understand that there might be internal network issues over the containers. I read many answers regarding this topic but I couldn't figure it out.
Any recommendations?
Just to add a proper answer:
Generally you should NOT really want to be executing multiple docker-compose up -d commands
If you want to combine two separate docker-compose configs and run as one (slightly more preferable), you can use the extends keyword as described in the docs
However, I would suggest that you treat it as a single docker-compose project which can itself have multiple nested git repositories:
Example SO answer - Git repository setup for a Docker application consisting of multiple repositories
You can keep your code in a mono-repo or multiple repos, up to you
Real working example to backup using your applications that validates this approach:
headless-wordpress-nextjs-starter-kit and it's docker-compose.yml
I have found this thread here
Communication between multiple docker-compose projects
By looking at the most upvoted answers, I wonder if it is related to network prefix?
It seems like the internal-network would be prefixed with frontend_? On the other hand you can also try to locate the network by name in backend/docker-compose.yml:
networks:
internal-network:
external:
name: internal-network
The issue is external networks need the network name specified (because docker compose prefixes resources by default). Your backend docker compose network section should look like this:
networks:
internal-network:
name: internal-network
external: true
You are creating the network in your frontend docker compose so you should omit the docker network create ... command (just need to init frontend first). Or instead treat them both as external and keep the command. In which use the named external network as shown above in your frontend docker compose as well.
I have two docker containers (linux containers on Windows 10 host) that are built from the microsoft/aspnetcore base image. Both containers run fine when I start them individually. I am trying to use Docker Compose to start both containers up (one is an identity provider using IdentityServer4 and the other is an api resource protected by Identity Server). I have the following docker-compose.yml file:
version: '3'
services:
identityserver:
image: eventloom/identityserver
build:
context: ../Eventloom.Web.IdentityProvider/Eventloom.Web.IdentityProvider
dockerfile: DockerFile
ports:
- 8888:80
eventsite:
image: eventloom/eventsite
build:
context: ./Eventloom.Web.Eventsite
dockerfile: Dockerfile
ports:
- 8080:80
links:
- identityserver
depends_on:
- identityserver
environment:
IdentityServer: "http://identityserver"
the startup class for the "eventsite" container uses IdentityModel to ping the Discovery endpoint of "identityserver". For some reason, the startup is never able successfully get the discovery information, even though I can log into the eventsite container and get ping responses from identityserver. Is there something else I need to do to allow eventsite to communicate over port 80 with identityserver?
It turns out that the HTTP communication was working fine and using the internal DNS properly. The issue was with my IdentityModel.DiscoveryClient object and not configuring it to allow HTTP only. I had to use VS to debug as the app was starting inside the container to figure it out. Thanks.
I have the following situation:
My application consists of a single web service that calls an
external API (say, some SaaS service, ElasticSearch or so). For non-unit-testing purposes we want to control the external service and later also inject faults. The application and the "mocked" API are dockerized and
now I want to use docker-compose to spin all containers up.
Because the application has several addresses hardcoded (e.g. the hostname of external services) I cannot change them and need to work around.
The service container makes a call to http://external-service.com/getsomestuff.
My idea was to use some features that are provided by docker to reroute all outgoing traffic to the external http://external-service.com/getsomestuff to the mock container without changing the URL.
My docker-compose.yaml looks like:
version: '2'
services:
service:
build: ./service
container_name: my-service1
ports:
- "5000:5000"
command: /bin/sh -c "python3 app.py"
api:
build: ./api-mock
container_name: my-api-mock
ports:
- "5001:5000"
command: /bin/sh -c "python3 app.py"
Finally, I have a driver that just does the following:
curl -XGET localhost:5000/
curl -XPUT localhost:5001/configure?delay=10
curl -XGET localhost:5000/
where the second curl just sets the delay in the mock to 10 seconds.
There are several options I have considered:
Using iptables-fu (would require modifying Dockerfiles to install it)
Using docker networks (this is really unclear to me)
Is there any simple option to achieve what I want?
Edit:
For clarity, here is the relevant part of the service code:
import requests
#app.route('/')
def do_stuff():
r = requests.get('http://external-service.com/getsomestuff')
return process_api_response(r.text())
Docker runs an internal DNS server for user defined networks. Any unknown host lookups are forwarded to you normal DNS servers.
Version 2+ compose files will automatically create a network for compose to use so there's a number of ways to control the hostnames it resolves.
The simplest way is to name your container with the hostname:
version: "2"
services:
external-service.com:
image: busybox
command: sleep 100
ping:
image: busybox
command: ping external-service.com
depends_on:
- external-service.com
If you want to keep container names you can use links
version: "2"
services:
api:
image: busybox
command: sleep 100
ping:
image: busybox
links:
- api:external-service.com
command: ping external-service.com
depends_on:
- api
Or network aliases
version: "2"
services:
api:
image: busybox
command: sleep 100
networks:
pingnet:
aliases:
- external-service.com
ping:
image: busybox
command: ping external-service.com
depends_on:
- api
networks:
- pingnet
networks:
pingnet:
I'm not entirely clear what the problem is you're trying to solve, but if you're trying to make external-service.com inside the container direct traffic to your "mock" service, I think you should be able to do that using the extra_hosts directive in your docker-compose.yml file. For example, if I have this:
version: "2"
services:
example:
image: myimage
extra_hosts:
- google.com:172.23.254.1
That will result in /etc/hosts in the container containing:
172.23.254.1 google.com
And attempts to access http://google.com will hit my web server at 172.23.254.1.
I was able to solve this with -links, is there a way to do networks in docker-compose?
version: '3'
services:
MOCK:
image: api-mock:latest
container_name: api-mock-container
ports:
- "8081:80"
api:
image: my-service1:latest
links:
- MOCK:external-service.com
I'm trying to set up a dev environment for my project.
I have a container (ms1) which should be put in his own network ("services" in my case), and a container (apigateway) which should access that network while exposing an http port to the host's network.
Ideally my docker compose file would look like this:
version: '2'
services:
ms1:
expose:
- "13010"
networks:
services:
aliases:
- ms1
apigateway:
networks:
services:
aliases:
- api
network_mode: "host"
networks:
services:
docker-compose doesn't allow to use network_mode and networks at the same time.
Do I have other alternatives?
At the moment I'm using this:
apigateway:
networks:
services:
aliases:
- api
ports:
- "127.0.0.1:10000:13010"
and then apigateway container listens on 0.0.0.0:13010. It works but it is slow and it freezes if the host's internet connection goes down.
Also, I'm planning on using vagrant in the future upon docker, does it allow to solve in a clean way?
expose in docker-compose does not publish the port on the host. Since you probably don't need service linking anymore (instead you should rely on Docker networks as you do already), the option has limited value in general and seems to provide no value at all to you in your scenario.
I suspect you've come to using it by mistake and after realizing that it didn't seem to have any effect by itself, stumbled upon the fact that using the host network driver would "make it work". This had nothing to do with the expose property, mind you. It's just that the host network driver lets contained processes bind to the host network interface directly. Thanks to this, you could reach the API gateway process from the outside. You could remove the expose property and it would still work.
If this is the only reason why you picked the host network driver, then you've fallen victim of the X-Y problem:
(tl;dr)
You should never need to use the host network driver in normal situations, the default bridge network driver works just fine. What you're looking for is the ports property, not expose. This sets up the appropriate port forwarding behind the scenes.
In docker 1.13 you should be able to create a service to bridge between the two networks. I'm using something similar to fix another problem and I think this could also help here:
docker service create \
--name proxy \
--network proxy \
--publish mode=host,target=80,published=80 \
--publish mode=host,target=443,published=443 \
--constraint 'node.hostname == myproxynode' \
--replicas 1 \
letsnginx
I would try this :
1/ Find the host network
docker network ls
2/ Use this dockercompose file
services:
ms1:
ports:
- "13010"
networks:
- service
apigateway:
networks:
- front
- service
networks:
front:
service:
external:
name: "<ID of the network>"
I'm running gitlab-ce and gitlab-ci-multi-runner in separated docker containers, but on the same server.
Gitlab CE works fine, I can access it via browser and clone projects using both http and ssh.
However my runner cannot connect to Gitlab using domain/server ip. It can connect to it only via local docker network (for example using ip address 172.17.0.X or, if linked, by using service alias).
Ping to domain/server ip returns response.
I tried to link it as gitlab:example.domain.com but it didn't work, as somehow runner resolved server ip address instead of local network address
Checking for builds... failed: couldn't execute POST against http://example.domain.com/ci/api/v1/builds/register.json: Post http://example.domain.com/ci/api/v1/builds/register.json: dial tcp server.ip:80: i/o timeout
#Edit
docker-compose.yml
gitlab:
image: gitlab/gitlab-ce:8.2.2-ce.0
hostname: domain.name
privileged: true
volumes:
- ./gitlab-config:/etc/gitlab
- ./gitlab-data:/var/opt/gitlab
- ./gitlab-logs:/var/log/gitlab
restart: always
ports:
- server.ip:22:22
- server.ip:80:80
- server.ip:443:443
runner:
image: gitlab/gitlab-runner:alpine
restart: always
volumes:
- ./runner-config:/etc/gitlab-runner
- /var/run/docker.sock:/var/run/docker.sock
I have no clue what's the issue here.
I'd appreciate your help.
Thanks in advance! :)
Seems like it was a firewall problem. Unlocking docker0 interface allowed traffic from containers :)