ElasticBeanstalk MultiContainer docker with nginx - nginx

I have two applications that handle different, but related functionality. I would like to deploy them as a single entity on a single host:port.
My plan is to use elasticbeanstalk's multicontainer docker platform. Each application would be a container.
How can I tie them together? Is it possible to install and configure nginx on the eb host?

You need to define all containers that comprise your application (together with nginx container) in Dockerrun.aws.json.
{
"AWSEBDockerrunVersion": 2,
"volumes": [
{
"name": "nginx-proxy-conf",
"host": {
"sourcePath": "/var/app/current/conf.d"
}
}
],
"containerDefinitions": [
{
"name": "first-app",
"image": "FIRST_APP_IMAGE_NAME:FIRST_APP_TAG",
"environment": [],
"essential": true,
"memoryReservation": 200,
"mountPoints": [],
"portMappings": [
{
"hostPort": 8081,
"containerPort": 8080
}
]
},
{
"name": "secondapp",
"image": "SECOND_APP_IMAGE_NAME:SECOND_APP_TAG",
"environment": [],
"essential": true,
"memoryReservation": 200,
"mountPoints": [],
"portMappings": [
{
"hostPort": 8082,
"containerPort": 8080
}
]
}
{
"name": "nginx-proxy",
"image": "nginx",
"essential": true,
"memoryReservation": 128,
"portMappings": [
{
"hostPort": 80,
"containerPort": 80
}
],
"links": [
"firstapp", "secondapp"
],
"mountPoints": [
{
"sourceVolume": "nginx-proxy-conf",
"containerPath": "/etc/nginx/conf.d",
"readOnly": true
}
]
}
]
}
Now as we linked app containers to nginx container we can refer to them using their names as hostnames.
And then you need to deploy Dockerrun.aws.json zipped together with nginx config conf.d/default.conf file (put into conf.d folder) in which you need to specify
location /firstapp/ {
proxy_pass http://firstapp;
}
location /secondapp/ {
proxy_pass http://secondapp;
}
Please also refer to AWS example of nginx proxy before php application.
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_v2config.html

Related

Passing config file from one container to another in ecs ec2 launch type

I have been struggling with this issue for a while: I want to pass the config file located on Container A app/config-file/nginx/nginx.conf and pass it into etc/nginx/conf.d/default.conf so that the official nginx image that I'm running along side my default container can use my custom config file. I have tried to use bindMounts as shown in the aws docs to do this and even set a start-up dependency but for some reason, I am still not getting the correct config file to populate. It just resorts to the default config that comes with the nginx image. Below is my task definition template showing how I am trying to do this (I created named volumes outside of the task-definition)
{
"command": [],
"entrypoint": [
"uwsgi",
"--ini",
],
"cpu": 250,
"memoryReservation": 1024,
"image": "${image_url}",
"name": "${container_name}",
"mountPoints": [
{
"containerPath": "/app/config-file/nginx",
"sourceVolume": "nginx-config"
}
],
},
{
"name": "${container_name}-nginx",
"image": "nginx:1.19-alpine",
"essential": true,
"dependsOn": [
{
"containerName": "${container_name}",
"condition": "START"
}
],
"mountPoints": [
{
"containerPath": "/etc/nginx/conf.d/",
"sourceVolume": "nginx-config"
}
],
"cpu": 250,
"memoryReservation": 250,
"portMappings": [
{
"protocol": "tcp",
"containerPort": 4000,
"HostPort" : 0
}
]
}
]```
I do want to mention that I know there are ways to do this by using efs, s3, or by baking the custom config into the nginx image. I would prefer if I did not have to go down those routes though.

Ocelot Swagger MMLib.SwaggerForOcelot showing "No operations defined in spec!"

I am using Ocelot gateway and for swagger document using "MMLib.SwaggerForOcelot" library.
For some swagger Key, swagger UI is showing "No operations defined in spec!" and swagger JSON is coming without paths like
{
"openapi": "3.0.1",
"info": {
"title": "Admin API",
"version": "v1"
},
"paths": {},
"components": {
"schemas": {}
}
}
Ocelot Configuration Route is
{
"DownstreamPathTemplate": "/api/admin/v{version}/{everything} ",
"DownstreamScheme": "http",
"DownstreamHostAndPorts": [
{
"Host": "localhost",
"Port": 5000
}
],
"UpstreamPathTemplate": "/api/admin/v{version}/{everything}",
"UpstreamHttpMethod": [],
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 3,
"DurationOfBreak": 1000,
"TimeoutValue": 900000
},
"SwaggerKey": "AdminAPI"
}
and Swagger Configuration is
"SwaggerEndPoints": [
{
"Key": "AdminAPI",
"Config": [
{
"Name": "Admin API",
"Version": "v1",
"Url": "http://localhost:5000/swagger/v1/swagger.json"
}
]
}
]
after reviewing the MMLib.SwaggerForOcelot source code, it looks like something to do with the version in the downstream path, any clue on how this can be fixed?
The issue is that MMLib.SwaggerForOcelot is not considering {version} while doing Ocelot transformation.
RouteOptions has a property TransformByOcelotConfig which is set to true by default, so once swagger JSON is obtained from downstream, the transformation will be done.
here, it will try to find the route from the route configuration like below and if not found it will delete the route from swagger JSON
private static RouteOptions FindRoute(IEnumerable<RouteOptions> routes, string downstreamPath, string basePath)
{
string downstreamPathWithBasePath = PathHelper.BuildPath(basePath, downstreamPath);
return routes.FirstOrDefault(p
=> p.CanCatchAll
? downstreamPathWithBasePath.StartsWith(p.DownstreamPathWithSlash, StringComparison.CurrentCultureIgnoreCase)
: p.DownstreamPathWithSlash.Equals(downstreamPathWithBasePath, StringComparison.CurrentCultureIgnoreCase));
}
The problem is StartsWith will return false since swagger JSON path will be like
/api/admin/v{version}/Connections
and route config is like
/api/admin/v{version}/{everything}
and version will replace with the version given in swagger options so that it will become
/api/admin/v1/{everything}
Fix to this problem will be
Either set "TransformByOcelotConfig":false in swagger option
"SwaggerEndPoints": [
{
"Key": "AdminAPI",
"TransformByOcelotConfig":false,
"Config": [
{
"Name": "Admin API",
"Version": "v1",
"Url": "http://localhost:5000/swagger/v1/swagger.json"
}
]
}
]
Or Change the route, just to have {everything} keyword
{
"DownstreamPathTemplate": "/api/admin/{everything} ",
"DownstreamScheme": "http",
"DownstreamHostAndPorts": [
{
"Host": "localhost",
"Port": 5000
}
],
"UpstreamPathTemplate": "/api/admin/{everything}",
"UpstreamHttpMethod": [],
"QoSOptions": {
"ExceptionsAllowedBeforeBreaking": 3,
"DurationOfBreak": 1000,
"TimeoutValue": 900000
},
"SwaggerKey": "AdminAPI"
}

Testing NGINX configuration

I have a reverse proxy server with NGINX and I want to test its configuration automatically.
What I want to achieve in the end is to have a command that I can run, it starts the NGINX with the configuration, run several http requests, and then track and gather whether the right proxied server was called.
I've been thinking on setting up an environment with docker-compose and use curl/wget with the list of urls I want to test. The thing that I don't know is how to mock certain domains and track the forwarded requests.
Is there a tool to do that or should I write a server manually?
After experimenting a bit I managed to create this solution.
Use Docker Compose, Wiremock and Newman. The idea is to setup NGINX proxying requests to Wiremock (where you can control if the request matched the right structure), then with Newman, you can run a Postman collection that automatically checks that the stubbed responses are the right ones.
Example
Create all these files in a folder, get the testing environment by running
docker-compose up -d nginx wiremock
and then, to run the test suite
docker-compose run --rm newman
It should print the results of the collection.
Files
docker-compose.yml
version: "3"
services:
nginx:
image: nginx
ports:
- "80:80"
volumes:
- ./config:/etc/nginx
wiremock:
image: wiremock/wiremock:2.32.0
command: [ "--port", "80", "--verbose" ]
ports:
- "8080:80"
volumes:
- ./wiremock:/home/wiremock
networks:
default:
aliases:
- backend-service-1
- backend-service-2
newman:
image: postman/newman
volumes:
- ./newman:/etc/newman
command: [ "run", "example.postman_collection.json" ]
config/nginx.conf
events {
worker_connections 1024;
}
http {
resolver 127.0.0.11; # docker internal resolver
server {
listen 80 default_server;
location /some/path/ {
proxy_set_header X-Forwarded-Host $host;
proxy_pass http://backend-service-1/some/path;
}
location /other/path/ {
proxy_set_header X-Forwarded-Host $host;
proxy_pass http://backend-service-2/other/path;
}
}
}
wiremock/mappings/some-path.json
{
"request": {
"method": "GET",
"url": "/some/path",
"headers": {
"Host": {
"equalTo": "backend-service-1",
"caseInsensitive": true
}
}
},
"response": {
"status": 200,
"body": "{\"host\": \"from-1\"}",
"headers": {
"Content-Type": "application/json"
}
}
}
wiremock/mappings/other-path.json
{
"request": {
"method": "GET",
"url": "/other/path",
"headers": {
"Host": {
"equalTo": "backend-service-2",
"caseInsensitive": true
}
}
},
"response": {
"status": 200,
"body": "{\"host\": \"from-2\"}",
"headers": {
"Content-Type": "application/json"
}
}
}
newman/example.postman_collection.json
{
"info": {
"name": "example",
"schema": "https://schema.getpostman.com/json/collection/v2.1.0/collection.json"
},
"item": [
{
"name": "some path",
"event": [
{
"listen": "test",
"script": {
"exec": [
"pm.test(\"request backend service 1\", function () {",
" pm.response.to.have.status(200);",
"",
" var jsonData = pm.response.json();",
" pm.expect(jsonData.host).to.eql(\"from-1\");",
"});",
""
],
"type": "text/javascript"
}
}
],
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "http://nginx/some/path/",
"protocol": "http",
"host": [
"nginx"
],
"path": [
"some",
"path",
""
]
}
},
"response": []
},
{
"name": "other path",
"event": [
{
"listen": "test",
"script": {
"exec": [
"pm.test(\"request backend service 2\", function () {",
" pm.response.to.have.status(200);",
"",
" var jsonData = pm.response.json();",
" pm.expect(jsonData.host).to.eql(\"from-2\");",
"});",
""
],
"type": "text/javascript"
}
}
],
"request": {
"method": "GET",
"header": [],
"url": {
"raw": "http://nginx/other/path/",
"protocol": "http",
"host": [
"nginx"
],
"path": [
"other",
"path",
""
]
}
},
"response": []
}
]
}

Assign static outbound ip address to azure container instance

I need to make a setup where I can read and write to an external sql db from a python script residing in a azure container instance. I order to make this work I need to assign a static ip to the container.
As I cannot associate a container instance with a dedicated ip I have had to make a setup that use the following resources: a vnet, a gateway and a public IP.
I have partially borrowed the setup from https://godatadriven.com/blog/azure-container-instance-example/ where the setup is drawn as follows:
I have made a dev-ops build and release pipeline. I use an ARM template to create the release (the resources of the template are below):
"resources": [
{
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('vnetName')]",
"apiVersion": "2019-07-01",
"location": "[parameters('location')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[parameters('vnetAddressPrefix')]"
]
},
"subnets": [
{
"name": "[parameters('subnet2Name')]",
"properties": {
"addressPrefix": "[parameters('subnet2AddressPrefix')]",
"privateEndpointNetworkPolicies": "Enabled",
"privateLinkServiceNetworkPolicies": "Enabled"
}
},
{
"name": "[parameters('subnetName')]",
"properties": {
"addressPrefix": "[parameters('subnetAddressPrefix')]",
"delegations": [
{
"name": "DelegationService",
"properties": {
"serviceName": "Microsoft.ContainerInstance/containerGroups"
}
}
],
"privateEndpointNetworkPolicies": "Enabled",
"privateLinkServiceNetworkPolicies": "Enabled"
}
}
]
}
},
{
"apiVersion": "2018-07-01",
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('publicIPAddressName')]",
"location": "[parameters('location')]",
"sku": {
"name": "Standard",
"tier": "Regional"
},
"properties": {
"publicIPAddressVersion": "IPv4",
"publicIPAllocationMethod": "Static",
"idleTimeoutInMinutes": 4,
"dnsSettings": {
"domainNameLabel": "[parameters('dnsName')]"
}
}
},
{
"apiVersion": "2019-08-01",
"name": "[variables('applicationGatewayName')]",
"type": "Microsoft.Network/applicationGateways",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/virtualNetworks/', parameters('vnetName'))]",
"[resourceId('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]",
"[resourceId('Microsoft.ContainerInstance/containerGroups/', parameters('containerInstanceName'))]"
],
"properties": {
"sku": {
"name": "[parameters('skuName')]",
"tier": "Standard_v2",
"capacity": "[variables('capacity')]"
},
"gatewayIPConfigurations": [
{
"name": "appGatewayIpConfig",
"properties": {
"subnet": {
"id": "[variables('subnetRef')]"
}
}
}
],
"frontendIPConfigurations": [
{
"name": "appGatewayFrontendIP",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"PublicIPAddress": {
"id": "[variables('publicIPRef')]"
}
}
}
],
"frontendPorts": [
{
"name": "appGatewayFrontendPort",
"properties": {
"Port": 80
}
}
],
"backendAddressPools": [
{
"name": "appGatewayBackendPool",
"properties": {
"backendAddresses": [
{
"IpAddress": "[parameters('backendIP')]"
}
]
}
}
],
"backendHttpSettingsCollection": [
{
"name": "appGatewayBackendHttpSettings",
"properties": {
"Port": 80,
"Protocol": "Http",
"CookieBasedAffinity": "Disabled"
}
}
],
"httpListeners": [
{
"name": "appGatewayHttpListener",
"properties": {
"FrontendIPConfiguration": {
"Id": "[resourceId('Microsoft.Network/applicationGateways/frontendIPConfigurations', variables('applicationGatewayName'), 'appGatewayFrontendIP')]"
},
"FrontendPort": {
"Id": "[resourceId('Microsoft.Network/applicationGateways/frontendPorts', variables('applicationGatewayName'), 'appGatewayFrontendPort')]"
},
"Protocol": "Http",
"SslCertificate": null
}
}
],
"requestRoutingRules": [
{
"Name": "rule1",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[resourceId('Microsoft.Network/applicationGateways/httpListeners', variables('applicationGatewayName'), 'appGatewayHttpListener')]"
},
"backendAddressPool": {
"id": "[resourceId('Microsoft.Network/applicationGateways/backendAddressPools', variables('applicationGatewayName'), 'appGatewayBackendPool')]"
},
"backendHttpSettings": {
"id": "[resourceId('Microsoft.Network/applicationGateways/backendHttpSettingsCollection', variables('applicationGatewayName'), 'appGatewayBackendHttpSettings')]"
}
}
}
]
}
},
{
"name": "[parameters('networkProfileName')]",
"type": "Microsoft.Network/networkProfiles",
"apiVersion": "2018-07-01",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/virtualNetworks', parameters('vnetName'))]"
],
"properties": {
"containerNetworkInterfaceConfigurations": [
{
"name": "[variables('interfaceConfigName')]",
"properties": {
"ipConfigurations": [
{
"name": "[variables('interfaceIpConfig')]",
"properties": {
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('vnetName'), parameters('subnetName'))]"
}
}
}
]
}
}
]
}
},
{
"name": "[parameters('containerInstanceName')]",
"type": "Microsoft.ContainerInstance/containerGroups",
"apiVersion": "2018-10-01",
"location": "[parameters('location')]",
"dependsOn": [
"[resourceId('Microsoft.Network/networkProfiles', parameters('networkProfileName'))]"
],
"properties": {
"containers": [
{
"name": "[parameters('containerName')]",
"properties": {
"image": "[parameters('registryImageUri')]",
"ports": [{
"port": "[variables('port')]"
}],
"resources": {
"requests": {
"cpu": "[variables('cpuCores')]",
"memoryInGb": "[variables('memoryInGb')]"
}
}
}
}
],
"imageRegistryCredentials": [
{
"server": "[parameters('registryLoginServer')]",
"username": "[parameters('registryUserName')]",
"password": "[parameters('registryPassword')]"
}
],
"diagnostics": {
"logAnalytics": {
"workspaceId": "[parameters('LogAnalyticsID')]",
"workspaceKey": "[parameters('LogAnalyticsKEY')]"
}
},
"networkProfile": {
"Id": "[resourceId('Microsoft.Network/networkProfiles', parameters('networkProfileName'))]"
},
"osType": "Linux",
"ipAddress": {
"ports": [{
"protocol": "tcp",
"port": 80
}],
"type": "private",
"ip": "[parameters('backendIP')]"
},
"restartPolicy": "[parameters('restartPolicy')]"
}
}
]
The release works, but when I run I try to run the container instance, it use a different ip each time.
What am I doing wrong?
Since you are using an Azure provided SQL, I would recommend to leverage the private VNET offering that Azure provides.
you should look at configuring your ACI with a private subnet
https://learn.microsoft.com/en-us/azure/container-instances/container-instances-vnet
and also setup a vnet rule for your SQL server
https://learn.microsoft.com/en-us/azure/sql-database/sql-database-vnet-service-endpoint-rule-overview
Virtual network rules are one firewall security feature that controls whether the database server for your single databases and elastic pool in Azure SQL Database or for your databases in Azure Synapse Analytics accepts communications that are sent from particular subnets in virtual networks.
it's important that you enable the SQL service endpoint for SQL on the ACI subnet as well.
This will avoid you having to manage outbound IP whitelisting in your SQL firewall.
From the things you did, I think you misunderstanding the network of the Azure Container Instance. The Public or the Private type for the ACI is only available for the inbound traffic, not for the outbound. Even when you use the private type, the instance also can access the Internet without any other resource, but in this type, you cannot access it from the Internet.
Unfortunately, when you use the public type, the public IP address for the inbound and outbound may be even no the same. And for Azure Container Instance, we cannot control the IP address which we can use. So when you want to use a static public IP address to access the SQL DB, the Azure Container Instance is not suitable, I would recommend the VM, it's more controllable and appropriate.

Azure Resource Manager - Multiple VM NAT Rules

I am trying to create an ARM template that will provision multiple webservers with directly accessible ports. For instance I want a VM to have either port 9001 or 9002 open based on what the index of the VM is.
I am struggling to get the frontendPort parameter to accept a function. Here is the documentation that I have used.
Here is what the relevant portion of my template looks like:
"inboundNatRules": [
{
"copy": {
"name": "natCopy",
"count": "[parameters('numberOfVms')]"
},
"name": "[concat('directHttps-', copyIndex())]",
"properties": {
"frontendIPConfiguration": {
"id": "[concat(variables('lbID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"
},
"frontendPort": "[add(9001, copyIndex())]",
"backendPort": 9001,
"enableFloatingIP": false,
"idleTimeoutInMinutes": 4,
"protocol": "Tcp",
"backendIPConfiguration": {
"id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('vmNicName'), copyIndex()), 'ipconfig')]"
}
}
}
]
I was hoping that the this particular port would result in either "9001", or "9002".
"frontendPort": "[add(9001, copyIndex())]"
Instead, I see an error in Visual Studio's Intellisense, and when I try to deploy the solution.
Create template deployment 'deploymenttemplate-0107-1555'.
New-AzureRmResourceGroupDeployment : Resource Microsoft.Network/loadBalancers 'webserverLb'
failed with message 'Unable to process template language expressions for resource
'/subscriptions/some random guid/resourceGroups/webservers/providers/Microsoft.Network/loadBalancers/webserverLb'
at line '102' and column '10'. 'The template function 'copyIndex' is not expected at this location.
The function can only be used in a resource with copy specified.
Long story short, I'm simply trying to have the same number of NAT rules as I have VM's in the template, and dynamically assign the external port number.
Please let me know if I can provide any more information. Thank you.
Try:
[Concat(900,CopyIndex(1))]
which will offset the index (0 based) and give you the number you want.
This is the syntax that works for copying the NAT rules (I am adding an RDP rule on the standard back-end port):
"copy": [
{
"name": "inboundNatRules",
"count": "[parameters('numberOfWebInstances')]",
"input": {
"name": "[concat(parameters('lbNatRulePrefix'), copyindex('inboundNatRules'))]",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('lbFrontEndIpId')]"
},
"frontendPort": "[add(50001, copyIndex('inboundNatRules'))]",
"backendPort": 3389,
"enableFloatingIP": false,
"idleTimeoutInMinutes": 4,
"protocol": "tcp"
}
}
}
],
And then to apply the rules to the NIC, you actually need to add some code on the NIC itself. The following is for both LB rules and NAT rules:
"loadBalancerBackendAddressPools": [
{
"id": "[concat(variables('lbID'), '/backendAddressPools/', parameters('lbPoolName'))]"
}
],
"loadBalancerInboundNatRules": [
{
"id": "[concat(variables('lbID'),'/inboundNatRules/' , parameters('lbNatRulePrefix'), copyindex())]"
}
]
#Your script is wrong it should you are writing copyindex() but you need to pass the name of rule it should work.
"inboundNatRules": [
{
"copy": {
"name": "natCopy",
"count": "[parameters('numberOfVms')]"
},
"name": "[concat('directHttps-', copyIndex(natCopy,1))]",
"properties": {
"frontendIPConfiguration": {
"id": "[concat(variables('lbID'),'/frontendIPConfigurations/LoadBalancerFrontEnd')]"
},
"frontendPort": "[add(9001, copyIndex(natCopy,1))]",
"backendPort": 9001,
"enableFloatingIP": false,
"idleTimeoutInMinutes": 4,
"protocol": "Tcp",
"backendIPConfiguration": {
"id": "[resourceId('Microsoft.Network/networkInterfaces', concat(variables('vmNicName'), copyIndex(natCopy,1)), 'ipconfig')]"
}
}
}
$LoadBalancer = Get-AzureRmLoadBalancer -ResourceGroupName $ResourceGroupName -Name $LoadBalancerName
$publicIP1 = Get-AzureRmPublicIpAddress -name $pipName -resourcegroupname $ResourceGroupName
$frontendIP1 = Get-AzureRmLoadBalancerFrontendIpConfig -LoadBalancer $LoadBalancer -Name $FrontendIpConfigName
$LoadBalancer | Add-AzureRmLoadBalancerInboundNatRuleConfig -Name "nat_rule_tcp_IP1_49157" -FrontendIpConfiguration $frontendIP1 -IdleTimeoutInMinutes 4 -Protocol TCP -FrontendPort 49157 -BackendPort 49157 | Set-AzureRmLoadBalancer

Resources