ARM template for composite index in DocumentDB - azure-cosmosdb

There are multiple ways how to create composite index. But I see no ARM templates there and latest scheme for containers also has no it. When we can use ARM templates to define composite indexes?


Can you try something like the following:
{
"name": "string",
"type": "Microsoft.DocumentDB/databaseAccounts/sqlDatabases/containers",
"apiVersion": "2020-03-01",
"location": "string",
"tags": {},
"properties": {
"resource": {
"id": "string",
"indexingPolicy": {
"automatic": "boolean",
"indexingMode": "string",
"includedPaths": [
{
"path": "string",
"indexes": [
{
"dataType": "string",
"precision": "integer",
"kind": "string"
}
]
}
],
"excludedPaths": [
{
"path": "string"
}
],
"spatialIndexes": [
{
"path": "string",
"types": [
"string"
]
}
],
"compositeIndexes":[
[
{
"path":"/name",
"order":"ascending"
},
{
"path":"/age",
"order":"descending"
}
]
]
},
"partitionKey": {
"paths": [
"string"
],
"kind": "string",
"version": "integer"
},
"defaultTtl": "integer",
"uniqueKeyPolicy": {
"uniqueKeys": [
{
"paths": [
"string"
]
}
]
},
"conflictResolutionPolicy": {
"mode": "string",
"conflictResolutionPath": "string",
"conflictResolutionProcedure": "string"
}
},
"options": {
"additionalProperties": {},
"throughput": "string"
}
},
"resources": []
}
I simply took the template definition from here and added definition for composite index from here under indexing policy section.

Related

Updating CosmosDb indexing policy through ARM templates

I am trying to use ARM templates to update the indexing policy for cosmos container. I tried 2 methods, one to simply declare the indexing policy in while declaring the container in ARM.
{
"apiVersion": "[variables('cosmosDbApiVersion')]",
"type": "Microsoft.DocumentDB/databaseAccounts/apis/databases/containers",
"dependsOn": [ /* resourceId */ ],
"name": "/* containerName */",
"properties": {
"resource": {
"id": "/* id */",
"partitionKey": {
"paths": [
"/partitionKey"
],
"kind": "Hash"
},
"indexes": [
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Hash",
"dataType": "String",
"precision": 3
}
]
}
]
}
],
"defaultTtl": "[variables('defaultTtlValueToEnableTtl')]"
}
}
},
The second was to use to use ARM to deploy container setting as such:
{
"apiVersion": "[variables('cosmosDbApiVersion')]",
"type": "Microsoft.DocumentDB/databaseAccounts/apis/databases/containers/settings",
"name": "[/* name */",
"dependsOn": [ " /* container name */" ],
"properties": {
"resource": {
"throughput": "/* some throughput */",
"indexes": [
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Hash",
"dataType": "String",
"precision": 3
}
]
}
]
}
]
}
}
},
Both techniques do not fail deployment but the indexing policy does not change.
Would appreciate some help.

this is the example from the template reference (looks slightly different to what you are doing):
"resource": {
"id": "string",
"indexingPolicy": {
"automatic": "boolean",
"indexingMode": "string",
"includedPaths": [
{
"path": "string",
"indexes": [
{
"dataType": "string",
"precision": "integer",
"kind": "string"
}
]
}
],
"excludedPaths": [
{
"path": "string"
}
],
"spatialIndexes": [
{
"path": "string",
"types": [
"string"
]
}
]
},
xxx
}
https://learn.microsoft.com/en-us/azure/templates/microsoft.documentdb/2019-08-01/databaseaccounts/sqldatabases/containers

Range and hash index types are ignored by the Cosmos resource provider now for new containers or containers that were created within the past year or so. ARM does not validate the index policy which is why the template will deploy successfully.
Hash index was deprecated for these newer container because the performance of the range index in the new indexer is surpasses what hash index provided so was no longer necessary.
To create/modify index policy refer to this article below. There are multiple examples of index policies that implement everything from very simple to more complex policies that include composite indexes, spatial indexes and unique keys.
https://learn.microsoft.com/en-us/azure/cosmos-db/manage-sql-with-resource-manager#create-resource

How to get rid of "API must not have local definitions (i.e. only $refs are allowed)" Swaggerhub standardization error with Springfox

I have swagger api-docs.json definition generated by SpringFox.
Below minimal-reproducible-example:
{
"swagger": "2.0",
"info": {
"description": "Example REST API.",
"version": "15.11.02",
"title": "Example REST API",
"contact": {
"name": "ExampleTeam",
"url": "https://example.com/",
"email": "support#example.com"
},
"license": {
"name": "Apache License 2.0",
"url": "https://www.apache.org/licenses/LICENSE-2.0.txt"
}
},
"host": "d01088db.ngrok.io",
"basePath": "/cloud",
"tags": [
{
"name": "All Endpoints",
"description": " "
}
],
"paths": {
"/api/v2/users/{userId}/jobs/{jobId}": {
"get": {
"tags": [
"Builds",
"All Endpoints"
],
"summary": "Get job.",
"operationId": "getJobUsingGET",
"produces": [
"*/*"
],
"parameters": [
{
"name": "jobId",
"in": "path",
"description": "jobId",
"required": true,
"type": "integer",
"format": "int64"
},
{
"name": "userId",
"in": "path",
"description": "userId",
"required": true,
"type": "integer",
"format": "int64"
}
],
"responses": {
"200": {
"description": "OK",
"schema": {
"$ref": "#/definitions/APIPipelineJob"
}
},
"401": {
"description": "Unauthorized"
},
"403": {
"description": "Forbidden"
},
"404": {
"description": "Not Found"
}
},
"deprecated": false
}
}
},
"definitions": {
"APIPipelineJob": {
"type": "object",
"properties": {
"archiveTime": {
"type": "string",
"format": "date-time",
"example": "example"
},
"content": {
"type": "string",
"example": "example"
},
"createTime": {
"type": "string",
"format": "date-time",
"example": "example"
},
"id": {
"type": "integer",
"format": "int64",
"example": "example"
},
"name": {
"type": "string",
"example": "example"
},
"selfURI": {
"type": "string",
"example": "example"
},
"type": {
"type": "string",
"example": "example",
"enum": [
"BUILD",
"DEPLOY"
]
},
"userId": {
"type": "integer",
"format": "int64",
"example": "example"
}
},
"title": "APIPipelineJob",
"xml": {
"name": "APIPipelineJob",
"attribute": false,
"wrapped": false
}
}
}
}
When I import it to SwaggerHub I got standardization error:
'definitions.*' not allowed -> API must not have local definitions (i.e. only $refs are allowed)
I have found the recommended solution in SwaggerHub documentation
But here is my question how to achieve:
split into domains(then using a reference), or
inline schemas
with Springfox
Or maybe there is another way to get rid of the above standardization error?

If you go to your home page, then hover over your organization on the left hand side and go to settings > Standardization, you should see some options. Unselect "API must not have local definitions (i.e. only $refs are allowed)" at the bottom.
And don't forget to save at the top right!

How to use "jagged" objects in Azure Resource Manger templates to iterate over them?

I have object with not equal number of properties (and would like to keep it like this), i.e. second object is missing property "routeTable" and properties repeat i.e. "NSG-AllowAll"
"value": [
{
"name": "GatewaySubnet",
"addressPrefix": "10.2.0.0/24",
"networkSecurityGroup": "NSG-AllowAll",
"routeTable": "UDR-Default"
},
{
"name": "UnTrusted",
"addressPrefix": "10.2.1.0/24",
"networkSecurityGroup": "NSG-AllowAll"
},
{
"name": "routed",
"addressPrefix": "10.2.2.0/24",
"routeTable": "UDR-Default1"
}
]
The solution to create vnet works fine but only if exisiting properties don't repeat i.e. the second NSG above has different name from the first one. In my scenario there will be a lot of repeating property names

In that case you just need to remove nsg\udr from that template I've created. I think thats what I've told you there as well. check the ps in my previous answer.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"deploymentPrefix": {
"type": "string"
},
"subnets": {
"type": "array",
"defaultValue": [
{
"name": "GatewaySubnet",
"addressPrefix": "10.2.0.0/24",
"networkSecurityGroup": "NSG-AllowAll",
"routeTable": "UDR-Default"
},
{
"name": "UnTrusted",
"addressPrefix": "10.2.1.0/24",
"networkSecurityGroup": "NSG-AllowAll"
},
{
"name": "routed",
"addressPrefix": "10.2.2.0/24",
"routeTable": "UDR-Default"
}
]
}
},
"variables": {
"copy": [
{
"name": "subnetsBase",
"count": "[length(parameters('subnets'))]",
"input": {
"name": "[concat('subnet-', parameters('subnets')[copyIndex('subnetsBase')].name)]",
"properties": {
"addressPrefix": "[parameters('subnets')[copyIndex('subnetsBase')].addressPrefix]"
}
}
},
{
"name": "subnetsUDR",
"count": "[length(parameters('subnets'))]",
"input": {
"routeTable": {
"id": "[if(contains(parameters('subnets')[copyIndex('subnetsUDR')], 'routeTable'), resourceId('Microsoft.Network/routeTables', parameters('subnets')[copyIndex('subnetsUDR')].routeTable), 'skip')]"
}
}
},
{
"name": "subnetsNSG",
"count": "[length(parameters('subnets'))]",
"input": {
"networkSecurityGroup": {
"id": "[if(contains(parameters('subnets')[copyIndex('subnetsNSG')], 'networkSecurityGroup'), resourceId('Microsoft.Network/networkSecurityGroups', parameters('subnets')[copyIndex('subnetsNSG')].networkSecurityGroup), 'skip')]"
}
}
}
]
},
"resources": [
{
"apiVersion": "2017-06-01",
"type": "Microsoft.Network/virtualNetworks",
"name": "[concat(parameters('deploymentPrefix'), '-vNet')]",
"location": "[resourceGroup().location]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"10.2.0.0/16"
]
},
"copy": [
{
"name": "subnets",
"count": "[length(parameters('subnets'))]",
"input": {
"name": "[concat('subnet-', parameters('subnets')[copyIndex('subnets')].name)]",
"properties": "[union(variables('subnetsBase')[copyIndex('subnets')].properties, if(equals(variables('subnetsUDR')[copyIndex('subnets')].routetable.id, 'skip'), variables('subnetsBase')[copyIndex('subnets')].properties, variables('subnetsUDR')[copyIndex('subnets')]), if(equals(variables('subnetsNSG')[copyIndex('subnets')].networkSecurityGroup.id, 'skip'), variables('subnetsBase')[copyIndex('subnets')].properties, variables('subnetsNSG')[copyIndex('subnets')]))]"
}
}
]
}
}
]
}
I dont see any reason why this wont work if you have nsg\udr in place

How do I pass RegistrationKey to Azure DSC extenstion

I have template below which errors out during deployment with error below. Samples on documentation page seems to be erroneous and don't even compile.
"message": "VM has reported a failure when processing extension
'Microsoft.Powershell.DSC'. Error message: \"The DSC Extension failed
to install: Invalid type for parameter RegistrationKey of type
PSCredential.\nMore information about the failure can be found in the
logs located under
'C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\2.74.0.0'
on the VM.\nTo retry install, please remove the extension from the VM
first. \"."
Template
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[parameters('swarmmanager1Name')]",
"type": "Microsoft.Compute/virtualMachines",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"tags": {
"displayName": "swarmmanager1"
},
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('swarmmanager1VmSize')]"
},
"licenseType": "[parameters('LicenseType')]",
"osProfile": {
"computerName": "[parameters('swarmmanager1Name')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "[parameters('swarmmanager1ImagePublisher')]",
"offer": "[parameters('swarmmanager1ImageOffer')]",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"osDisk": {
"name": "swarmmanager1OSDisk",
"vhd": {
"uri": "[concat(reference(resourceId('Microsoft.Storage/storageAccounts', parameters('dockerswarmstorageaccountName')), '2016-01-01').primaryEndpoints.blob, parameters('swarmmanager1StorageAccountContainerName'), '/', parameters('swarmmanager1OSDiskName'), '.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', parameters('swarmmanager1NicName'))]"
}
]
}
},
"resources": [
{
"name": "Microsoft.Powershell.DSC",
"type": "extensions",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines', parameters('swarmmanager1Name'))]"
],
"tags": {
"displayName": "DSC"
},
"properties": {
"publisher": "Microsoft.Powershell",
"typeHandlerVersion": "2.26",
"type": "DSC",
"autoUpgradeMinorVersion": true,
"forceUpdateTag": "[parameters('DSCExtensionManagerTagVersion')]",
"settings": {
"wmfVersion": "latest",
"configurationArguments": {
//"RegistrationKey": {
// "UserName": "PLACEHOLDER_DONOTUSE",
// "Password": "PrivateSettingsRef:registrationKeyPrivate"
// },
"RegistrationKey": "[parameters('RegistrationKey')]",
"RegistrationUrl": "[parameters('registrationUrl')]",
"NodeConfigurationName": "SwarmManager.localhost",
"RebootNodeIfNeeded": true
}
},
"protectedSettings": {
"Items": {
"registrationKeyPrivate": "[parameters('RegistrationKey')]"
}
}
}
}
]
},
{
"name": "[parameters('dockerswarmstorageaccountName')]",
"type": "Microsoft.Storage/storageAccounts",
"location": "[resourceGroup().location]",
"apiVersion": "2016-01-01",
"sku": {
"name": "[parameters('dockerswarmstorageaccountType')]"
},
"dependsOn": [],
"tags": {
"displayName": "dockerswarmstorageaccount"
},
"kind": "Storage"
},
{
"name": "[parameters('swarmmanager1NicName')]",
"type": "Microsoft.Network/networkInterfaces",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-30",
"tags": {
"displayName": "swarmmanager1Nic"
},
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[parameters('swarmmanager1SubnetRef')]"
},
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', parameters('swarmmanagerpublicIPName'))]"
}
}
}
]
}
},
{
"apiVersion": "2016-03-30",
"dependsOn": [],
"location": "[resourceGroup().location]",
"name": "[parameters('swarmmanagerpublicIPName')]",
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "[parameters('swarmmanagerpublicIPDnsName')]"
}
},
"tags": {
"displayName": "swarmmanagerpublicIP"
},
"type": "Microsoft.Network/publicIPAddresses"
}
],
"parameters": {
"swarmmanager1Name": { "type": "string" },
"swarmmanager1VmSize": { "type": "string" },
"adminUsername": { "type": "string" },
"adminPassword": { "type": "securestring" },
"dockerswarmstorageaccountName": { "type": "string" },
"dockerswarmstorageaccountType": { "type": "string" },
"swarmmanager1NicName": { "type": "string" },
"swarmmanagerpublicIPName": { "type": "string" },
"swarmmanager1SubnetRef": { "type": "string" },
"swarmmanager1ImagePublisher": { "type": "string" },
"swarmmanager1ImageOffer": { "type": "string" },
"windowsOSVersion": { "type": "string" },
"swarmmanager1StorageAccountContainerName": { "type": "string" },
"swarmmanager1OSDiskName": { "type": "string" },
"swarmmanagerpublicIPDnsName": { "type": "string" },
"DSCConfigurationURL": { "type": "string" },
"DSCExtensionManagerTagVersion": { "type": "string" },
"RegistrationKey": { "type": "securestring" },
"RegistrationUrl": { "type": "string" },
"LicenseType": {"type": "string"}
},
"outputs": {
"returnedIPAddress": {
"type": "string",
"value": "[reference(parameters('swarmmanager1NicName')).ipConfigurations[0].properties.privateIPAddress]"
}
}
}

if you want to pass in ps credentials do this:
"protectedSettings": {
"configurationArguments": {
"RegistrationKey": {
"userName": "whatever",
"password": "[parameters('RegistrationKey')]"
}
}
}

How do I get private IP address of VM in nested ARM template?

I have a nested template which creates VM which works fine but I need to return private IP address of that machine to main template and as output. I create following output in my nested template but it fails with error message
Deployment template validation failed: 'The template reference 'swarmmanager1NetworkInterface' is not valid: could not find template resource or resource copy with this name. Please see https://aka.ms/arm-template-expressions/#reference for usage details.'.
Here is entire template
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01-preview/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
"vmSku": {
"type": "string",
"defaultValue": "Standard_A1",
"metadata": {
"description": "Size of VMs in the VM Scale Set."
}
},
"windowsOSVersion": {
"type": "string",
"defaultValue": "2016-Datacenter-with-Containers",
"allowedValues": [
"2008-R2-SP1",
"2012-Datacenter",
"2012-R2-Datacenter",
"2016-Datacenter-with-Containers"
],
"metadata": {
"description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version. Allowed values: 2008-R2-SP1, 2012-Datacenter, 2012-R2-Datacenter."
}
},
"registrationKey": {
"type": "securestring",
"metadata": {
"description": "Registration key to use to onboard to the Azure Automation DSC pull/reporting server"
}
},
"registrationUrl": {
"type": "string",
"metadata": {
"description": "Registration url of the Azure Automation DSC pull/reporting server"
}
},
"DSCExtensionTagVersion": {
"type": "string",
"metadata": {
"description": "Change this to redeploy DSC"
}
},
"nodeConfigurationName": {
"type": "string",
"defaultValue": "MyService.webServer",
"metadata": {
"description": "The name of the node configuration, on the Azure Automation DSC pull server, that this node will be configured as"
}
},
"configurationMode": {
"type": "string",
"defaultValue": "ApplyAndAutoCorrect",
"allowedValues": [
"ApplyOnly",
"ApplyAndMonitor",
"ApplyAndAutoCorrect"
],
"metadata": {
"description": "DSC agent (LCM) configuration mode setting. ApplyOnly, ApplyAndMonitor, or ApplyAndAutoCorrect"
}
},
"configurationModeFrequencyMins": {
"type": "int",
"defaultValue": 15,
"metadata": {
"description": "DSC agent (LCM) configuration mode frequency setting, in minutes"
}
},
"refreshFrequencyMins": {
"type": "int",
"defaultValue": 30,
"metadata": {
"description": "DSC agent (LCM) refresh frequency setting, in minutes"
}
},
"rebootNodeIfNeeded": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "DSC agent (LCM) rebootNodeIfNeeded setting"
}
},
"actionAfterReboot": {
"type": "string",
"defaultValue": "ContinueConfiguration",
"allowedValues": [
"ContinueConfiguration",
"StopConfiguration"
],
"metadata": {
"description": "DSC agent (LCM) actionAfterReboot setting. ContinueConfiguration or StopConfiguration"
}
},
"allowModuleOverwrite": {
"type": "bool",
"defaultValue": false,
"metadata": {
"description": "DSC agent (LCM) allowModuleOverwrite setting"
}
},
"automationAccountName": {
"type": "string",
"defaultValue": "myAutomationAccount",
"metadata": {
"description": "The name of the Automation account to use. Check the SKU and tags to make sure they match the existing account."
}
},
"automationRegionId": {
"type": "string",
"defaultValue": "East US 2",
"allowedValues": [
"Japan East",
"East US 2",
"West Europe",
"Southeast Asia",
"South Central US",
"Central India"
],
"metadata": {
"description": "The region the Automation account is located in."
}
},
"vmssName": {
"type": "string",
"metadata": {
"description": "String used as a base for naming resources. Must be 3-57 characters in length and globally unique across Azure. A hash is prepended to this string for some resources, and resource-specific information is appended."
},
"maxLength": 57
},
"instanceCount": {
"type": "int",
"metadata": {
"description": "Number of VM instances (100 or less)."
},
"maxValue": 100
},
"adminUsername": {
"type": "string",
"metadata": {
"description": "Admin username on all VMs."
}
},
"adminPassword": {
"type": "securestring",
"metadata": {
"description": "Admin password on all VMs."
}
},
"_artifactsLocation": {
"type": "string",
"metadata": {
"description": "Auto-generated container in staging storage account to receive post-build staging folder upload"
}
},
"_artifactsLocationSasToken": {
"type": "securestring",
"metadata": {
"description": "Auto-generated token to access _artifactsLocation"
}
},
"dockerswarmstorageaccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_ZRS",
"Standard_GRS",
"Standard_RAGRS",
"Premium_LRS"
]
},
"swarmmanager1Name": {
"type": "string",
"minLength": 1,
"defaultValue": "swarmmanager1"
},
"swarmmanagerpublicIPDnsName": {
"type": "string",
"defaultValue": "[uniqueString(subscription().subscriptionId)]",
"minLength": 1
}
},
"variables": {
"namingInfix": "[toLower(substring(concat(parameters('vmssName'), uniqueString(resourceGroup().id)), 0, 9))]",
"longNamingInfix": "[toLower(parameters('vmssName'))]",
"addressPrefix": "10.0.0.0/16",
"subnetPrefix": "10.0.8.0/21",
"virtualNetworkName": "[concat(variables('namingInfix'), 'vnet')]",
"natPoolName": "[concat(variables('namingInfix'), 'natpool')]",
"publicIPAddressName": "[concat(variables('namingInfix'), 'pip')]",
"subnetName": "[concat(variables('namingInfix'), 'subnet')]",
"nicName": "[concat(variables('namingInfix'), 'nic')]",
"ipConfigName": "[concat(variables('namingInfix'), 'ipconfig')]",
"imageReference": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"virtualNetworkID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
"appGwPublicIPAddressName": "[concat(variables('namingInfix'), 'appGwPip')]",
"bePoolName": "[concat(variables('namingInfix'), 'bepool')]",
"frontEndIPConfigID": "[concat(variables('lbID'),'/frontendIPConfigurations/loadBalancerFrontEnd')]",
"appGwName": "[concat(variables('namingInfix'), 'appGw')]",
"loadBalancerName": "[concat(variables('namingInfix'), 'lb')]",
"publicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"lbID": "[resourceId('Microsoft.Network/loadBalancers',variables('loadBalancerName'))]",
"appGwPublicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('appGwPublicIPAddressName'))]",
"appGwID": "[resourceId('Microsoft.Network/applicationGateways',variables('appGwName'))]",
"appGwSubnetName": "[concat(variables('namingInfix'),'appGwSubnet')]",
"appGwSubnetPrefix": "10.0.1.0/24",
"appGwSubnetID": "[concat(variables('virtualNetworkID'),'/subnets/',variables('appGwSubnetName'))]",
"appGwFrontendPort": 80,
"appGwBackendPort": 80,
"appGwBePoolName": "[concat(variables('namingInfix'), 'appGwBepool')]",
"computeApiVersion": "2016-04-30-preview",
"networkApiVersion": "2016-03-30",
"natStartPort": 50000,
"natEndPort": 50119,
"natBackendPort": 3389,
"DSCArchiveFolder": "DSC",
"DSCArchiveFileName": "DSC.zip",
"dockerswarmstorageaccountName": "[concat('sa', uniqueString(resourceGroup().id))]",
"swarmmanager1ImagePublisher": "MicrosoftWindowsServer",
"swarmmanager1ImageOffer": "WindowsServer",
"swarmmanager1OSDiskName": "swarmmanager1OSDisk",
"swarmmanager1VmSize": "Standard_D2_v2",
"swarmmanager1VnetID": "[resourceId('Microsoft.Network/virtualNetworks', variables('virtualNetworkName'))]",
"swarmmanager1SubnetRef": "[concat(variables('swarmmanager1VnetID'), '/subnets/', variables('subnetName'))]",
"swarmmanager1StorageAccountContainerName": "vhds",
"swarmmanager1NicName": "[concat(parameters('swarmmanager1Name'), 'NetworkInterface')]",
"swarmmanagerpublicIPName": "swarmmanagerpublicIP",
"swarmmanagerdeploymentTemplateFolder": "nestedtemplates",
"swarmmanagerdeploymentTemplateFileName": "swarmmanagerdeployment.json",
"swarmmanagerdeploymentTemplateParametersFileName": "swarmmanagerdeployment.parameters.json"
},
"resources": [
{
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('networkApiVersion')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
},
{
"name": "[variables('appGwSubnetName')]",
"properties": {
"addressPrefix": "[variables('appGwSubnetPrefix')]"
}
}
]
}
},
{
"type": "Microsoft.Network/loadBalancers",
"name": "[variables('loadBalancerName')]",
"location": "[resourceGroup().location]",
"apiVersion": "2017-04-01",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]"
],
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerFrontEnd",
"properties": {
"publicIPAddress": {
"id": "[variables('publicIPAddressID')]"
}
}
}
],
"backendAddressPools": [
{
"name": "[variables('bePoolName')]"
}
],
"inboundNatPools": [
{
"name": "[variables('natPoolName')]",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"protocol": "tcp",
"frontendPortRangeStart": "[variables('natStartPort')]",
"frontendPortRangeEnd": "[variables('natEndPort')]",
"backendPort": "[variables('natBackendPort')]"
}
}
]
}
},
{
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('appGwPublicIPAddressName')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('networkApiVersion')]",
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "appgwvip"
}
}
},
{
"apiVersion": "[variables('networkApiVersion')]",
"location": "[resourceGroup().location]",
"name": "[variables('publicIPAddressName')]",
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "rdpvip"
}
},
"type": "Microsoft.Network/publicIPAddresses"
},
{
"type": "Microsoft.Network/applicationGateways",
"name": "[variables('appGwName')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('networkApiVersion')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]",
"[concat('Microsoft.Network/publicIPAddresses/', variables('appGwPublicIPAddressName'))]"
],
"properties": {
"sku": {
"name": "Standard_Small",
"tier": "Standard",
"capacity": "1"
},
"gatewayIPConfigurations": [
{
"name": "appGwIpConfig",
"properties": {
"subnet": {
"id": "[variables('appGwSubnetID')]"
}
}
}
],
"frontendIPConfigurations": [
{
"name": "appGwFrontendIP",
"properties": {
"PublicIPAddress": {
"id": "[variables('appGwPublicIPAddressID')]"
}
}
}
],
"frontendPorts": [
{
"name": "appGwFrontendPort",
"properties": {
"Port": "[variables('appGwFrontendPort')]"
}
}
],
"backendAddressPools": [
{
"name": "[variables('appGwBePoolName')]"
}
],
"backendHttpSettingsCollection": [
{
"name": "appGwBackendHttpSettings",
"properties": {
"Port": "[variables('appGwBackendPort')]",
"Protocol": "Http",
"CookieBasedAffinity": "Disabled"
}
}
],
"httpListeners": [
{
"name": "appGwHttpListener",
"properties": {
"FrontendIPConfiguration": {
"Id": "[concat(variables('appGwID'), '/frontendIPConfigurations/appGwFrontendIP')]"
},
"FrontendPort": {
"Id": "[concat(variables('appGwID'), '/frontendPorts/appGwFrontendPort')]"
},
"Protocol": "Http",
"SslCertificate": null
}
}
],
"requestRoutingRules": [
{
"Name": "rule1",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[concat(variables('appGwID'), '/httpListeners/appGwHttpListener')]"
},
"backendAddressPool": {
"id": "[concat(variables('appGwID'), '/backendAddressPools/', variables('appGwBePoolName'))]"
},
"backendHttpSettings": {
"id": "[concat(variables('appGwID'), '/backendHttpSettingsCollection/appGwBackendHttpSettings')]"
}
}
}
]
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[variables('namingInfix')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('computeApiVersion')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]",
"[concat('Microsoft.Network/applicationGateways/', variables('appGwName'))]",
"[concat('Microsoft.Network/loadBalancers/', variables('loadBalancerName'))]"
],
"sku": {
"name": "[parameters('vmSku')]",
"tier": "Standard",
"capacity": "[parameters('instanceCount')]"
},
"properties": {
"overprovision": "false",
"singlePlacementGroup": "true",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {
"storageProfile": {
"osDisk": {
"caching": "ReadWrite",
"createOption": "FromImage"
},
"dataDisks": [],
"imageReference": "[variables('imageReference')]"
},
"osProfile": {
"computerNamePrefix": "[variables('namingInfix')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[variables('nicName')]",
"properties": {
"primary": "true",
"ipConfigurations": [
{
"name": "[variables('ipConfigName')]",
"properties": {
"subnet": {
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'), '/subnets/', variables('subnetName'))]"
},
"loadBalancerBackendAddressPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/loadBalancers/', variables('loadBalancerName'), '/backendAddressPools/', variables('bePoolName'))]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/loadBalancers/', variables('loadBalancerName'), '/inboundNatPools/', variables('natPoolName'))]"
}
],
"ApplicationGatewayBackendAddressPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/applicationGateways/', variables('appGwName'), '/backendAddressPools/', variables('appGwBePoolName'))]"
}
]
}
}
]
}
}
]
},
"extensionProfile": {
"extensions": [
{
"name": "Microsoft.Powershell.DSC",
"properties": {
"autoUpgradeMinorVersion": true,
"publisher": "Microsoft.Powershell",
"forceUpdateTag": "[parameters('DSCExtensionTagVersion')]",
"settings": {
"configuration": {
"url": "[concat(parameters('_artifactsLocation'), '/', variables('DSCArchiveFolder'), '/', variables('DSCArchiveFileName'), parameters('_artifactsLocationSasToken'))]",
"script": "DSC.ps1",
"function": "Main"
},
"configurationArguments": {
"RegistrationKey": "[parameters('registrationKey')]",
"RegistrationUrl": "[parameters('registrationUrl')]",
"NodeConfigurationName": "[parameters('nodeConfigurationName')]",
"ConfigurationMode": "[parameters('configurationMode')]",
"ConfigurationModeFrequencyMins": "[parameters('configurationModeFrequencyMins')]",
"RefreshFrequencyMins": "[parameters('refreshFrequencyMins')]",
"RebootNodeIfNeeded": "[parameters('rebootNodeIfNeeded')]",
"ActionAfterReboot": "[parameters('actionAfterReboot')]",
"AllowModuleOverwrite": "[parameters('allowModuleOverwrite')]"
}
},
"type": "DSC",
"typeHandlerVersion": "2.26"
}
}
]
}
}
}
},
{
"name": "swarmmanager",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2016-09-01",
"dependsOn": [],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[parameters('swarmmanager1Name')]",
"type": "Microsoft.Compute/virtualMachines",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('dockerswarmstorageaccountName'))]",
"[resourceId('Microsoft.Network/networkInterfaces', variables('swarmmanager1NicName'))]"
],
"tags": {
"displayName": "swarmmanager1"
},
"properties": {
"hardwareProfile": {
"vmSize": "[variables('swarmmanager1VmSize')]"
},
"osProfile": {
"computerName": "[parameters('swarmmanager1Name')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "[variables('swarmmanager1ImagePublisher')]",
"offer": "[variables('swarmmanager1ImageOffer')]",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"osDisk": {
"name": "swarmmanager1OSDisk",
"vhd": {
"uri": "[concat(reference(resourceId('Microsoft.Storage/storageAccounts', variables('dockerswarmstorageaccountName')), '2016-01-01').primaryEndpoints.blob, variables('swarmmanager1StorageAccountContainerName'), '/', variables('swarmmanager1OSDiskName'), '.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('swarmmanager1NicName'))]"
}
]
}
}
},
{
"name": "[variables('dockerswarmstorageaccountName')]",
"type": "Microsoft.Storage/storageAccounts",
"location": "[resourceGroup().location]",
"apiVersion": "2016-01-01",
"sku": {
"name": "[parameters('dockerswarmstorageaccountType')]"
},
"dependsOn": [],
"tags": {
"displayName": "dockerswarmstorageaccount"
},
"kind": "Storage"
},
{
"name": "[variables('swarmmanager1NicName')]",
"type": "Microsoft.Network/networkInterfaces",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-30",
"tags": {
"displayName": "swarmmanager1Nic"
},
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[variables('swarmmanager1SubnetRef')]"
},
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('swarmmanagerpublicIPName'))]"
}
}
}
]
}
},
{
"name": "[variables('swarmmanagerpublicIPName')]",
"type": "Microsoft.Network/publicIPAddresses",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-30",
"dependsOn": [],
"tags": {
"displayName": "swarmmanagerpublicIP"
},
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "[parameters('swarmmanagerpublicIPDnsName')]"
}
}
},
{
"outputs": {
"returnedIPAddress": {
"type": "string",
"value": "[reference(variables('swarmmanager1NicName')).ipConfigurations[0].properties.privateIPAddress]"
}
}
}
]
}
}
}
]
}

Ok, a couple things - you have your output in the nested template in the resources array - outputs should be a peer of the resources array. That's not getting flagged because of the inline reference() call fails before this.
First, reference() calls are often made very early in the deployment - sometimes earlier than you want them to be. When the resource being referenced is defined in the same template, it works as expected. When it's not defined in the same template, the GET happens immediately and the resource may not exist. That's a nuance of the reference() function that's a little tangential but important to know as you use it...
Combine that behavior with the inline template deployment. Inline deployments have a number of design quirks that make them pretty hard to use for anything "advanced". You won't be able to do what you're doing with an inline deployment. If you use a templateLink (i.e. another file), you can deploy the nic and output the ipconfig in the same template.
That help?

I have had a similar issue, we were deploying using Azure Pipelines, the approach i took was 1) Deploy Template and pass the RG name and VMSS name as template outputs 2) ARM Outputs - Use this snippet to publish the above results as vars to use downstream 3) Azure Powershell to get the IPs of all VMSS instances and publish them as pipeline variables 4) Write the pipeline vars file to a blob. Hope this helps
$iplist=#()
$i=1
$nicinfo=Get-AzureRmNetworkInterface -ResourceGroupName $(ResourceGroupName) -
VirtualMachineScaleSetName $(VmssName)
foreach($nic in $nicinfo){
$iplist += $nic.IpConfigurations.PrivateIpAddress
}
foreach($ip in $iplist){
Write-host Private ip $i is $ip
Write-Host "##vso[task.setvariable variable=Privateip$i;]$ip"
$i++
}

Resources