Multi-container docker on AWS - Nginx use host machine /etc/hosts resolver - nginx

Question

Cross-Post from serverfault
Since I have seen similar issues posted here, I will X-post my question from ServerFault (https://serverfault.com/questions/855120/multi-container-docker-on-aws-nginx-use-host-machine-etc-hosts-resolver). I hope this
is permitted.
I have a multi-container docker environment on Amazon Elastic Beanstalk with the following Dockerrun.aws.json file:
{
"AWSEBDockerrunVersion": 2,
"containerDefinitions": [
{
"name": "web",
"memoryReservation": 256,
"image": "my/nginx/repo/image",
"portMappings": [
{
"hostPort": 80,
"containerPort": 80
}
],
"links": [
"api"
],
"essential": true
},
{
"name": "api",
"memoryReservation": 256,
"image": "my-api/repo",
"essential": true,
"portMappings": [
{
"hostPort": 3000,
"containerPort": 80
}
]
}
]
}
Ultimately I want the node app served by nginx to resolve requests to named addresses from linked containers, so in my web image (node app) I'd like to make a request to http://api/some/resource and let nginx resolve that to the api container.
Now, since docker adds a host entry for the api container due to the specified link, I want the nginx server to resolve addresses from the hosts etc/hosts file, however as I found out, nginx uses it's own resolver. After researching the issue a bit I found out that in non-Elastic Beanstalk multi-container solutions and with user-defined networks, the resolver would be provided by docker on 127.0.0.11, however since it is currently not possible to define user-defined networks in the Dockerrun.aws.json, I keep looking for a different solution. The links can be resolved inside the container, pinging api does work, however, nginx does it's own thing there.
I have read about dnsmasq as well, however, I wanted to get this running without installing this package, do I even have a choice here ?

Answer 1

This is a service discovery topic, and I don't think this solution is in a right direction.
It is valid that links can be resolved to the App's IP; however, it must follow the starting order - App first and then Nginx. If the App recreates, or scales after Nginx started, /etc/hosts in Nginx won't be changed. It is possible to recreate the Nginx for updating the /etc/hosts, but then all the connections to the Nginx will be disconnected.
Since you are using Amazon Elastic Beanstalk, I think a better solution is to use Consul (Service Discovery) + Registrator (Service Registration ) + Nginx (with SRV support, Plus or third party).
Personally, I have done a similar thing without AEB. I am using Docker Swarm (Service Discovery + Service Registration) + Nginx (with a modified HAproxy-SRV)
I hope this answer can help your decision.