OperationTimedOut on Microsoft.DBforMySQL/servers arm deployment - azure-resource-manager

I've been trying to deploy the following arm template (removed the resources that are deploying successfully for brievety) a couple of times now.
I'm always getting the following error on the MySQL deployment part.
New-AzureRmResourceGroupDeployment : 9:21:31 AM - Resource Microsoft.DBforMySQL/servers 'webarm01' failed with
message '{
"status": "Failed",
"error": {
"code": "ResourceDeploymentFailure",
"message": "The resource operation completed with terminal provisioning state 'Failed'.",
"details": [
{
"code": "OperationTimedOut",
"message": "The operation timed out and automatically rolled back. Please retry the operation."
}
]
}
}'
Here is the arm template
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"administratorLogin": {
"type": "String"
},
"administratorLoginPassword": {
"type": "SecureString"
},
"servers_analytics_name": {
"defaultValue": "analyticsarmmodel",
"type": "String"
},
"databases_analytics_name": {
"defaultValue": "analytics",
"type": "String"
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
}
},
"variables": {
"databases_sys_name": "sys",
"databases_mysql_name": "mysql",
"databases_information_schema_name": "information_schema",
"databases_performance_schema_name": "performance_schema",
"firewallRules_AllowAllWindowsAzureIps_name": "AllowAllWindowsAzureIps"
},
"resources": [
{
"type": "Microsoft.DBforMySQL/servers",
"sku": {
"name": "B_Gen5_1",
"tier": "Basic",
"family": "Gen5",
"capacity": 1
},
"name": "[parameters('servers_analytics_name')]",
"apiVersion": "2017-12-01-preview",
"location": "[parameters('location')]",
"scale": null,
"properties": {
"administratorLogin": "[parameters('administratorLogin')]",
"administratorLoginPassword": "[parameters('administratorLoginPassword')]",
"storageProfile": {
"storageMB": 5120,
"backupRetentionDays": 7,
"geoRedundantBackup": "Disabled"
},
"version": "5.7",
"sslEnforcement": "Disabled"
},
"dependsOn": []
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('databases_information_schema_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "utf8",
"collation": "utf8_general_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', parameters('databases_analytics_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "utf8",
"collation": "utf8_general_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('databases_mysql_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "latin1",
"collation": "latin1_swedish_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('databases_performance_schema_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "utf8",
"collation": "utf8_general_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('databases_sys_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "utf8",
"collation": "utf8_general_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/firewallRules",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('firewallRules_AllowAllWindowsAzureIps_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"startIpAddress": "0.0.0.0",
"endIpAddress": "0.0.0.0"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
}
]
}
My question: how can I deploy a MySQL server, + databases using arm without running in a timeout?

After some internal discussions with Microsoft it appears that even though the administratorLoginPassword parameter is tagged typed as secure string you are supposed to pass in a clear text password.
The secure string will be too long and make the deployment hang.
Microsoft is working on better error messages and eventually supporting secure strings properly.
Also some parameters might be conflicting, here is an updated template that works for me.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"administratorLogin": {
"type": "String"
},
"administratorLoginPassword": {
"type": "SecureString"
},
"servers_analytics_name": {
"defaultValue": "analyticsarmmodel",
"type": "String"
},
"databases_analytics_name": {
"defaultValue": "analytics",
"type": "String"
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Location for all resources."
}
}
},
"variables": {
"databases_sys_name": "sys",
"databases_mysql_name": "mysql",
"databases_performance_schema_name": "performance_schema",
"firewallRules_AllowAllWindowsAzureIps_name": "AllowAllWindowsAzureIps"
},
"resources": [
{
"type": "Microsoft.DBforMySQL/servers",
"sku": {
"name": "B_Gen5_1",
"tier": "Basic",
"family": "Gen5",
"capacity": 1
},
"name": "[parameters('servers_analytics_name')]",
"apiVersion": "2017-12-01-preview",
"location": "[parameters('location')]",
"scale": null,
"properties": {
"administratorLogin": "[parameters('administratorLogin')]",
"administratorLoginPassword": "[parameters('administratorLoginPassword')]",
"storageProfile": {
"storageMB": 5120,
"backupRetentionDays": 7,
"geoRedundantBackup": "Disabled"
},
"version": "5.7",
"sslEnforcement": "Disabled"
},
"dependsOn": []
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('databases_information_schema_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "utf8",
"collation": "utf8_general_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', parameters('databases_analytics_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "utf8",
"collation": "utf8_general_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/databases",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('databases_mysql_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"charset": "latin1",
"collation": "latin1_swedish_ci"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
},
{
"type": "Microsoft.DBforMySQL/servers/firewallRules",
"name": "[concat(parameters('servers_analytics_name'), '/', variables('firewallRules_AllowAllWindowsAzureIps_name'))]",
"apiVersion": "2017-12-01-preview",
"scale": null,
"properties": {
"startIpAddress": "0.0.0.0",
"endIpAddress": "0.0.0.0"
},
"dependsOn": [
"[resourceId('Microsoft.DBforMySQL/servers', parameters('servers_analytics_name'))]"
]
}
]
}

Related

Not able to apply CMK encryption to Azure Storage Account through ARM Template

I am trying to attach CMK Encryption with Azure Storage Account through ARM Template but I am getting error as below. Need quick help with it. Able to apply it through portal after Storage Account is created but not able to do via ARM Template while creating Storage Account.
Error- [error]FeatureNotSupportedForAccount: Missing pre-requisites to enable EncryptionAtRest/Customer Managed Key for this storage account.
ARM:-
"resources": [
{​​​​​​​
"type": "Microsoft.Storage/storageAccounts",
"apiVersion": "2019-04-01",
"name": "[variables('storageaccountname')]",
"location": "[resourceGroup().location]",
"sku": {​​​​​​​
"name": "[parameters('storageaccountype')]"
}​​​​​​​,
"kind": "[parameters('storagekind')]",
"properties": {​​​​​​​
"supportsHttpsTrafficOnly": true,
"accesstier": "[parameters('accesstier')]",
"largeFileSharesState": "[parameters('largefilesharesstate')]",
"allowBlobPublicAccess": false,
"encryption": {​​​​​​​
"services": {​​​​​​​
"file": {​​​​​​​
"enabled": true
}​​​​​​​,
"blob": {​​​​​​​
"enabled": true
}​​​​​​​
}​​​​​​​,
"keySource": "Microsoft.Keyvault",
"keyvaultproperties": {​​​​​​​
"keyvaulturi": "[parameters('kvuri')]",
"keyname": "[parameters('keyname')]",
"keyversion": "[parameters('keyversion')]"
}​​​​​​​
}​​​​​​​
}​​​​​​​,
"tags": {​​​​​​​
"abcid": "[parameters('abcid')]"
}​​​​​​​
}​​​​​​​
According to the document, if you want to configure encryption with customer-managed keys stored in Azure key valt, we need to do the following steps
Create storage account and Enable Identity
Update Azure Key vault. Enable soft delete and purge protection.
Configure access policy for the storage account's Identity
Configure customer-managed keys for the storage account.
Regarding how to configure these with arm template, please refer to the following template
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"keyName": {
"type": "string",
"defaultValue": ""
},
"keyVersion": {
"type": "string",
"defaultValue": ""
},
"vaultName": {
"defaultValue": "",
"type": "String"
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]"
},
"accountNmae": {
"type": "string",
"defaultValue": "tetsdfgfgdffd"
},
},
"variables": {},
"resources": [{
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[ parameters('accountNmae')]",
"apiVersion": "2019-06-01",
"location": "[ parameters('location')]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"supportsHttpsTrafficOnly": true
},
"dependsOn": []
}, {
"type": "Microsoft.KeyVault/vaults",
"apiVersion": "2016-10-01",
"name": "[parameters('vaultName')]",
"location": "eastasia",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', parameters('accountNmae'))]"
],
"properties": {
"sku": {
"family": "A",
"name": "Standard"
},
"tenantId": "[subscription().tenantid]",
"accessPolicies": [],
"enabledForDeployment": true,
"enabledForDiskEncryption": true,
"enabledForTemplateDeployment": true,
"enableSoftDelete": true
}
}, {
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-07-01",
"name": "updateStorageAccount",
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults', parameters('vaultName'))]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "0.1.0.0",
"resources": [{
"type": "Microsoft.KeyVault/vaults/accessPolicies",
"name": "[concat(parameters('vaultName'), '/add')]",
"apiVersion": "2019-09-01",
"properties": {
"accessPolicies": [{
"tenantId": "[subscription().tenantid]",
"objectId": "[reference(resourceId('Microsoft.Storage/storageAccounts', parameters('accountNmae')),'2019-06-01', 'full').identity.principalId]",
"permissions": {
"keys": [
"wrapkey",
"unwrapkey",
"get"
],
"secrets": [],
"certificates": []
}
}
]
}
}, {
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[parameters('accountNmae')]",
"apiVersion": "2019-06-01",
"location": "[parameters('location')]",
"identity": {
"type": "SystemAssigned"
},
"properties": {
"encryption": {
"services": {
"file": {
"enabled": true
},
"blob": {
"enabled": true
}
},
"keySource": "Microsoft.Keyvault",
"keyvaultproperties": {
"keyvaulturi": "[reference(resourceId('Microsoft.KeyVault/vaults',parameters('vaultName')),'2016-10-01', 'full').properties.vaultUri]",
"keyname": "[parameters('keyName')]",
"keyversion": "[parameters('keyversion')]"
}
}
},
"dependsOn": [
"[resourceId('Microsoft.KeyVault/vaults/accessPolicies', parameters('vaultName'), 'add')]"
]
}
]
}
}
}
]
}
For more details, please refer to the blog

How to deploy ARM template with user managed identity and assign a subscription level role?

The ARM template below is supposed to create the following resources:
resource group
- user managed identity
- subscription level Contributor role assignment
Currently the deployment is failing with the error "error": { "code": "ResourceGroupNotFound", "message": "Resource group 'rg-myproject-deploy' could not be found." } apparently because the role assignment step seem to not be respecting the dependsOn statements that should enforce that it should only happen after the resource group is created. Is there a way to deploy all these resources in a single ARM template?
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"projectName": {
"type": "string",
"defaultValue": "myproject",
"maxLength": 11,
"metadata": {
"description": "The name of the project"
}
},
"location": {
"type": "string",
"defaultValue": "westus2",
"metadata": {
"description": "The region were to deploy assets"
}
}
},
"variables": {
"resourceGroupName": "[concat('rg-', parameters('projectName'), '-deploy')]",
"managedIdentityName": "[concat('msi-', parameters('projectName'), '-deploy')]",
"bootstrapRoleAssignmentId": "[guid(subscription().id, 'contributor')]",
"contributorRoleDefinitionId": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', 'b24988ac-6180-42a0-ab88-20f7382dd24c')]",
"managedIdentityId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', variables('resourceGroupName'), '/providers/Microsoft.ManagedIdentity/userAssignedIdentities/', variables('managedIdentityName'))]"
},
"resources": [
{
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2019-10-01",
"name": "[variables('resourceGroupName')]",
"location": "[parameters('location')]",
"properties": {}
},
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-10-01",
"name": "deployment-assets-except-role-assignment",
"resourceGroup": "[variables('resourceGroupName')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups/', variables('resourceGroupName'))]"
],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {},
"variables": {},
"resources": [
{
"type": "Microsoft.ManagedIdentity/userAssignedIdentities",
"name": "[variables('managedIdentityName')]",
"apiVersion": "2018-11-30",
"location": "[parameters('location')]"
}
],
"outputs": {}
}
}
}
,
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2017-09-01",
"name": "[variables('bootstrapRoleAssignmentId')]",
"dependsOn": [
"[subscriptionResourceId('Microsoft.Resources/resourceGroups', variables('resourceGroupName'))]",
"deployment-assets-except-role-assignment"
],
"properties": {
"roleDefinitionId": "[variables('contributorRoleDefinitionId')]",
"principalId": "[reference(variables('managedIdentityId'), '2018-11-30').principalId]",
"principalType": "ServicePrincipal",
"scope": "[subscription().id]"
}
}
],
"outputs": {}
}
I think you're running into this:
https://bmoore-msft.blog/2020/07/26/resource-not-found-dependson-is-not-working/
The fix was a little more involved than I thought, but to summarize:
the nested deployment that provisions the MI must be set to inner scope evaluation
output the principalId from that deployment and use that in your reference (i.e. don't directly reference)
Due to #1 I moved some stuff around (params/vars)
{
"$schema": "https://schema.management.azure.com/schemas/2018-05-01/subscriptionDeploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"projectName": {
"type": "string",
"defaultValue": "myproject",
"maxLength": 11,
"metadata": {
"description": "The name of the project"
}
},
"location": {
"type": "string",
"defaultValue": "westus2",
"metadata": {
"description": "The region were to deploy assets"
}
}
},
"variables": {
"identityDeploymentName": "deployment-assets-except-role-assignment",
"resourceGroupName": "[concat('rg-', parameters('projectName'), '-deploy')]",
"managedIdentityName": "[concat('msi-', parameters('projectName'), '-deploy')]",
"managedIdentityId": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', variables('resourceGroupName'), '/providers/Microsoft.ManagedIdentity/userAssignedIdentities/', variables('managedIdentityName'))]",
"bootstrapRoleAssignmentId": "[guid(subscription().id, variables('contributorRoleDefinitionId'),variables('managedIdentityId'))]",
"contributorRoleDefinitionId": "[subscriptionResourceId('Microsoft.Authorization/roleDefinitions', 'b24988ac-6180-42a0-ab88-20f7382dd24c')]",
},
"resources": [
{
"type": "Microsoft.Resources/resourceGroups",
"apiVersion": "2019-10-01",
"name": "[variables('resourceGroupName')]",
"location": "[parameters('location')]",
"properties": {}
},
{
"type": "Microsoft.Resources/deployments",
"apiVersion": "2019-10-01",
"name": "[variables('identityDeploymentName')]",
"resourceGroup": "[variables('resourceGroupName')]",
"dependsOn": [
"[resourceId('Microsoft.Resources/resourceGroups', variables('resourceGroupName'))]"
],
"properties": {
"mode": "Incremental",
"expressionEvaluationOptions":{
"scope": "inner"
},
"parameters": {
"location": {
"value": "[parameters('location')]"
},
"managedIdentityName": {
"value": "[variables('managedIdentityName')]"
}
},
"template": {
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"type": "string"
},
"managedIdentityName": {
"type": "string"
}
},
"variables": {},
"resources": [
{
"type": "Microsoft.ManagedIdentity/userAssignedIdentities",
"name": "[parameters('managedIdentityName')]",
"apiVersion": "2018-11-30",
"location": "[parameters('location')]"
}
],
"outputs": {
"principalId": {
"type": "string",
"value": "[reference(parameters('managedIdentityName')).principalId]"
}
}
}
}
}
,
{
"type": "Microsoft.Authorization/roleAssignments",
"apiVersion": "2020-04-01-preview",
"name": "[variables('bootstrapRoleAssignmentId')]",
"dependsOn": [
"[subscriptionResourceId('Microsoft.Resources/resourceGroups', variables('resourceGroupName'))]",
"[variables('identityDeploymentName')]"
],
"properties": {
"roleDefinitionId": "[variables('contributorRoleDefinitionId')]",
"principalId": "[reference(variables('identityDeploymentName')).outputs.principalId.value]",
"principalType": "ServicePrincipal",
"scope": "[subscription().id]"
}
}
]
}

How do I pass RegistrationKey to Azure DSC extenstion

I have template below which errors out during deployment with error below. Samples on documentation page seems to be erroneous and don't even compile.
"message": "VM has reported a failure when processing extension
'Microsoft.Powershell.DSC'. Error message: \"The DSC Extension failed
to install: Invalid type for parameter RegistrationKey of type
PSCredential.\nMore information about the failure can be found in the
logs located under
'C:\WindowsAzure\Logs\Plugins\Microsoft.Powershell.DSC\2.74.0.0'
on the VM.\nTo retry install, please remove the extension from the VM
first. \"."
Template
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[parameters('swarmmanager1Name')]",
"type": "Microsoft.Compute/virtualMachines",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"tags": {
"displayName": "swarmmanager1"
},
"properties": {
"hardwareProfile": {
"vmSize": "[parameters('swarmmanager1VmSize')]"
},
"licenseType": "[parameters('LicenseType')]",
"osProfile": {
"computerName": "[parameters('swarmmanager1Name')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "[parameters('swarmmanager1ImagePublisher')]",
"offer": "[parameters('swarmmanager1ImageOffer')]",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"osDisk": {
"name": "swarmmanager1OSDisk",
"vhd": {
"uri": "[concat(reference(resourceId('Microsoft.Storage/storageAccounts', parameters('dockerswarmstorageaccountName')), '2016-01-01').primaryEndpoints.blob, parameters('swarmmanager1StorageAccountContainerName'), '/', parameters('swarmmanager1OSDiskName'), '.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', parameters('swarmmanager1NicName'))]"
}
]
}
},
"resources": [
{
"name": "Microsoft.Powershell.DSC",
"type": "extensions",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"dependsOn": [
"[resourceId('Microsoft.Compute/virtualMachines', parameters('swarmmanager1Name'))]"
],
"tags": {
"displayName": "DSC"
},
"properties": {
"publisher": "Microsoft.Powershell",
"typeHandlerVersion": "2.26",
"type": "DSC",
"autoUpgradeMinorVersion": true,
"forceUpdateTag": "[parameters('DSCExtensionManagerTagVersion')]",
"settings": {
"wmfVersion": "latest",
"configurationArguments": {
//"RegistrationKey": {
// "UserName": "PLACEHOLDER_DONOTUSE",
// "Password": "PrivateSettingsRef:registrationKeyPrivate"
// },
"RegistrationKey": "[parameters('RegistrationKey')]",
"RegistrationUrl": "[parameters('registrationUrl')]",
"NodeConfigurationName": "SwarmManager.localhost",
"RebootNodeIfNeeded": true
}
},
"protectedSettings": {
"Items": {
"registrationKeyPrivate": "[parameters('RegistrationKey')]"
}
}
}
}
]
},
{
"name": "[parameters('dockerswarmstorageaccountName')]",
"type": "Microsoft.Storage/storageAccounts",
"location": "[resourceGroup().location]",
"apiVersion": "2016-01-01",
"sku": {
"name": "[parameters('dockerswarmstorageaccountType')]"
},
"dependsOn": [],
"tags": {
"displayName": "dockerswarmstorageaccount"
},
"kind": "Storage"
},
{
"name": "[parameters('swarmmanager1NicName')]",
"type": "Microsoft.Network/networkInterfaces",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-30",
"tags": {
"displayName": "swarmmanager1Nic"
},
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[parameters('swarmmanager1SubnetRef')]"
},
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', parameters('swarmmanagerpublicIPName'))]"
}
}
}
]
}
},
{
"apiVersion": "2016-03-30",
"dependsOn": [],
"location": "[resourceGroup().location]",
"name": "[parameters('swarmmanagerpublicIPName')]",
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "[parameters('swarmmanagerpublicIPDnsName')]"
}
},
"tags": {
"displayName": "swarmmanagerpublicIP"
},
"type": "Microsoft.Network/publicIPAddresses"
}
],
"parameters": {
"swarmmanager1Name": { "type": "string" },
"swarmmanager1VmSize": { "type": "string" },
"adminUsername": { "type": "string" },
"adminPassword": { "type": "securestring" },
"dockerswarmstorageaccountName": { "type": "string" },
"dockerswarmstorageaccountType": { "type": "string" },
"swarmmanager1NicName": { "type": "string" },
"swarmmanagerpublicIPName": { "type": "string" },
"swarmmanager1SubnetRef": { "type": "string" },
"swarmmanager1ImagePublisher": { "type": "string" },
"swarmmanager1ImageOffer": { "type": "string" },
"windowsOSVersion": { "type": "string" },
"swarmmanager1StorageAccountContainerName": { "type": "string" },
"swarmmanager1OSDiskName": { "type": "string" },
"swarmmanagerpublicIPDnsName": { "type": "string" },
"DSCConfigurationURL": { "type": "string" },
"DSCExtensionManagerTagVersion": { "type": "string" },
"RegistrationKey": { "type": "securestring" },
"RegistrationUrl": { "type": "string" },
"LicenseType": {"type": "string"}
},
"outputs": {
"returnedIPAddress": {
"type": "string",
"value": "[reference(parameters('swarmmanager1NicName')).ipConfigurations[0].properties.privateIPAddress]"
}
}
}
if you want to pass in ps credentials do this:
"protectedSettings": {
"configurationArguments": {
"RegistrationKey": {
"userName": "whatever",
"password": "[parameters('RegistrationKey')]"
}
}
}

ARM Template error error code invalidTemplate "Circular Dependancy Detected

I am new to deploying ARM templates. Before I created the ARM template I created a simple infrastructure consisting of 1 VM, 1 VNet, and 1 storage account. Once I finished provisioning the infrastructure, I decided to make a template for future use. I went under automation scripts, and I saved it to the library and then I also downloaded it. So, I wanted to change the SSD disk tier in the template from Standard D2S_V1 to E2S_V3. So I did that. I saved the changes and then I decided to test it out by deploying the template. After this, I got an error code that says the following:
Deployment template validation failed: 'Circular dependency detected on resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleRev2/providers/Microsoft.Network/networkInterfaces/simplevm18'. Please see https://aka.ms/arm-template/#resources for usage details.'. (Code: InvalidTemplate)
So, I tried to do my own research by checking out these links:
https://social.msdn.microsoft.com/Forums/en-US/c5782362-3a04-47fe-b5ec-87e5f488e844/deployment-failed-with-arm-template?forum=WAVirtualMachinesforWindows
https://learn.microsoft.com/en-us/azure/azure-resource-manager/resource-group-define-dependencies
I understand that the circular dependency error means that the dependOn property may be having issues. So I went in the template and looked at them, because it mainly has to do with the vnet resources specified in the template. What I'm not sure is, do I need to change the name of the vnet, pub ip, nsg, and nic? Or, is it something else. Here I'm going to attach the template in JSON:
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"virtualMachines_SimpleVM_name": {
"defaultValue": "SimpleVM",
"type": "String"
},
"virtualNetworks_SimpleVNet_name": {
"defaultValue": "SimpleVNet",
"type": "String"
},
"networkInterfaces_simplevm18_name": {
"defaultValue": "simplevm18",
"type": "String"
},
"publicIPAddresses_SimpleVM_ip_name": {
"defaultValue": "SimpleVM-ip",
"type": "String"
},
"storageAccounts_simplestorage1_name": {
"defaultValue": "simplestorage1",
"type": "String"
},
"networkSecurityGroups_SimpleVM_nsg_name": {
"defaultValue": "SimpleVM-nsg",
"type": "String"
},
"subnets_default_name": {
"defaultValue": "default",
"type": "String"
},
"securityRules_default_allow_rdp_name": {
"defaultValue": "default-allow-rdp",
"type": "String"
},
"virtualMachines_SimpleVM_id": {
"defaultValue": "/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Compute/disks/SimpleVM_OsDisk_1_e66aaeca090f4ff5a3021460b8255a30",
"type": "String"
}
},
"variables": {},
"resources": [
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Compute/virtualMachines/SimpleVM'.",
"type": "Microsoft.Compute/virtualMachines",
"name": "[parameters('virtualMachines_SimpleVM_name')]",
"apiVersion": "2017-03-30",
"location": "centralus",
"scale": null,
"properties": {
"hardwareProfile": {
"vmSize": "Standard_E2S_v3"
},
"storageProfile": {
"imageReference": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "2016-Datacenter",
"version": "latest"
},
"osDisk": {
"osType": "Windows",
"name": "[concat(parameters('virtualMachines_SimpleVM_name'),'_OsDisk_1_e66aaeca090f4ff5a3021460b8255a30')]",
"createOption": "FromImage",
"caching": "ReadWrite",
"managedDisk": {
"storageAccountType": "Premium_LRS",
"id": "[parameters('virtualMachines_SimpleVM_id')]"
},
"diskSizeGB": 128
},
"dataDisks": []
},
"osProfile": {
"computerName": "[parameters('virtualMachines_SimpleVM_name')]",
"adminUsername": "susan",
"windowsConfiguration": {
"provisionVMAgent": true,
"enableAutomaticUpdates": true
},
"secrets": []
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', parameters('networkInterfaces_simplevm18_name'))]"
}
]
},
"diagnosticsProfile": {
"bootDiagnostics": {
"enabled": true,
"storageUri": "https://loadbaldiag639.blob.core.windows.net/"
}
}
},
"dependsOn": [
"[resourceId('Microsoft.Network/networkInterfaces', parameters('networkInterfaces_simplevm18_name'))]"
]
},
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Network/networkInterfaces/simplevm18'.",
"type": "Microsoft.Network/networkInterfaces",
"name": "[parameters('networkInterfaces_simplevm18_name')]",
"apiVersion": "2017-10-01",
"location": "centralus",
"scale": null,
"properties": {
"provisioningState": "Succeeded",
"resourceGuid": "5cced9eb-2dee-4f5a-9768-7098c140bfba",
"ipConfigurations": [
{
"name": "ipconfig1",
"etag": "W/\"a2959053-241e-49fc-9ee6-ae6dd8f9a2ac\"",
"properties": {
"provisioningState": "Succeeded",
"privateIPAddress": "10.5.0.4",
"privateIPAllocationMethod": "Dynamic",
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', parameters('publicIPAddresses_SimpleVM_ip_name'))]"
},
"subnet": {
"id": "[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('virtualNetworks_SimpleVNet_name'), parameters('subnets_default_name'))]"
},
"primary": true,
"privateIPAddressVersion": "IPv4"
}
}
],
"dnsSettings": {
"dnsServers": [],
"appliedDnsServers": [],
"internalDomainNameSuffix": "4k34gmmtgvzetdldyabe4m2uwh.gx.internal.cloudapp.net"
},
"macAddress": "00-0D-3A-94-FA-26",
"enableAcceleratedNetworking": false,
"enableIPForwarding": false,
"networkSecurityGroup": {
"id": "[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSecurityGroups_SimpleVM_nsg_name'))]"
},
"primary": true,
"virtualMachine": {
"id": "[resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachines_SimpleVM_name'))]"
}
},
"dependsOn": [
"[resourceId('Microsoft.Network/publicIPAddresses', parameters('publicIPAddresses_SimpleVM_ip_name'))]",
"[resourceId('Microsoft.Network/virtualNetworks/subnets', parameters('virtualNetworks_SimpleVNet_name'), parameters('subnets_default_name'))]",
"[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSecurityGroups_SimpleVM_nsg_name'))]",
"[resourceId('Microsoft.Compute/virtualMachines', parameters('virtualMachines_SimpleVM_name'))]"
]
},
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Network/networkSecurityGroups/SimpleVM-nsg'.",
"type": "Microsoft.Network/networkSecurityGroups",
"name": "[parameters('networkSecurityGroups_SimpleVM_nsg_name')]",
"apiVersion": "2017-10-01",
"location": "centralus",
"scale": null,
"properties": {
"provisioningState": "Succeeded",
"resourceGuid": "0af138ab-8f42-4b94-b440-46525541955e",
"securityRules": [
{
"name": "default-allow-rdp",
"etag": "W/\"ece2fb9b-8e13-4434-ac2a-c5800b3b3c56\"",
"properties": {
"provisioningState": "Succeeded",
"protocol": "TCP",
"sourcePortRange": "*",
"destinationPortRange": "3389",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 1000,
"direction": "Inbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
}
}
],
"defaultSecurityRules": [
{
"name": "AllowVnetInBound",
"etag": "W/\"ece2fb9b-8e13-4434-ac2a-c5800b3b3c56\"",
"properties": {
"provisioningState": "Succeeded",
"description": "Allow inbound traffic from all VMs in VNET",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "*",
"sourceAddressPrefix": "VirtualNetwork",
"destinationAddressPrefix": "VirtualNetwork",
"access": "Allow",
"priority": 65000,
"direction": "Inbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
}
},
{
"name": "AllowAzureLoadBalancerInBound",
"etag": "W/\"ece2fb9b-8e13-4434-ac2a-c5800b3b3c56\"",
"properties": {
"provisioningState": "Succeeded",
"description": "Allow inbound traffic from azure load balancer",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "*",
"sourceAddressPrefix": "AzureLoadBalancer",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 65001,
"direction": "Inbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
}
},
{
"name": "DenyAllInBound",
"etag": "W/\"ece2fb9b-8e13-4434-ac2a-c5800b3b3c56\"",
"properties": {
"provisioningState": "Succeeded",
"description": "Deny all inbound traffic",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "*",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "*",
"access": "Deny",
"priority": 65500,
"direction": "Inbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
}
},
{
"name": "AllowVnetOutBound",
"etag": "W/\"ece2fb9b-8e13-4434-ac2a-c5800b3b3c56\"",
"properties": {
"provisioningState": "Succeeded",
"description": "Allow outbound traffic from all VMs to all VMs in VNET",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "*",
"sourceAddressPrefix": "VirtualNetwork",
"destinationAddressPrefix": "VirtualNetwork",
"access": "Allow",
"priority": 65000,
"direction": "Outbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
}
},
{
"name": "AllowInternetOutBound",
"etag": "W/\"ece2fb9b-8e13-4434-ac2a-c5800b3b3c56\"",
"properties": {
"provisioningState": "Succeeded",
"description": "Allow outbound traffic from all VMs to Internet",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "*",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "Internet",
"access": "Allow",
"priority": 65001,
"direction": "Outbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
}
},
{
"name": "DenyAllOutBound",
"etag": "W/\"ece2fb9b-8e13-4434-ac2a-c5800b3b3c56\"",
"properties": {
"provisioningState": "Succeeded",
"description": "Deny all outbound traffic",
"protocol": "*",
"sourcePortRange": "*",
"destinationPortRange": "*",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "*",
"access": "Deny",
"priority": 65500,
"direction": "Outbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
}
}
]
},
"dependsOn": []
},
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Network/publicIPAddresses/SimpleVM-ip'.",
"type": "Microsoft.Network/publicIPAddresses",
"sku": {
"name": "Basic"
},
"name": "[parameters('publicIPAddresses_SimpleVM_ip_name')]",
"apiVersion": "2017-10-01",
"location": "centralus",
"scale": null,
"properties": {
"provisioningState": "Succeeded",
"resourceGuid": "6ebcb25d-2340-4438-bfa5-8301ba372db9",
"ipAddress": "52.173.136.16",
"publicIPAddressVersion": "IPv4",
"publicIPAllocationMethod": "Dynamic",
"idleTimeoutInMinutes": 4
},
"dependsOn": []
},
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Network/virtualNetworks/SimpleVNet'.",
"type": "Microsoft.Network/virtualNetworks",
"name": "[parameters('virtualNetworks_SimpleVNet_name')]",
"apiVersion": "2017-10-01",
"location": "centralus",
"scale": null,
"properties": {
"provisioningState": "Succeeded",
"resourceGuid": "31e3bbf2-3593-4972-8d63-c0024f3394b7",
"addressSpace": {
"addressPrefixes": [
"10.5.0.0/16"
]
},
"subnets": [
{
"name": "default",
"etag": "W/\"add0de6e-d14d-4511-b107-739686a5dd70\"",
"properties": {
"provisioningState": "Succeeded",
"addressPrefix": "10.5.0.0/24"
}
}
],
"virtualNetworkPeerings": [],
"enableDdosProtection": false,
"enableVmProtection": false
},
"dependsOn": []
},
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Storage/storageAccounts/simplestorage1'.",
"type": "Microsoft.Storage/storageAccounts",
"sku": {
"name": "Standard_LRS",
"tier": "Standard"
},
"kind": "Storage",
"name": "[parameters('storageAccounts_simplestorage1_name')]",
"apiVersion": "2017-10-01",
"location": "centralus",
"identity": {
"principalId": "ba12dbb9-d280-4037-ae54-8ce938863a41",
"tenantId": "f3c4e7bf-60d1-40db-aa86-fcf229eb6d77",
"type": "SystemAssigned"
},
"tags": {},
"scale": null,
"properties": {
"networkAcls": {
"bypass": "AzureServices",
"virtualNetworkRules": [],
"ipRules": [],
"defaultAction": "Allow"
},
"supportsHttpsTrafficOnly": false,
"encryption": {
"services": {
"file": {
"enabled": true
},
"blob": {
"enabled": true
}
},
"keySource": "Microsoft.Storage"
}
},
"dependsOn": []
},
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Network/networkSecurityGroups/SimpleVM-nsg/securityRules/default-allow-rdp'.",
"type": "Microsoft.Network/networkSecurityGroups/securityRules",
"name": "[concat(parameters('networkSecurityGroups_SimpleVM_nsg_name'), '/', parameters('securityRules_default_allow_rdp_name'))]",
"apiVersion": "2017-10-01",
"scale": null,
"properties": {
"provisioningState": "Succeeded",
"protocol": "TCP",
"sourcePortRange": "*",
"destinationPortRange": "3389",
"sourceAddressPrefix": "*",
"destinationAddressPrefix": "*",
"access": "Allow",
"priority": 1000,
"direction": "Inbound",
"sourcePortRanges": [],
"destinationPortRanges": [],
"sourceAddressPrefixes": [],
"destinationAddressPrefixes": []
},
"dependsOn": [
"[resourceId('Microsoft.Network/networkSecurityGroups', parameters('networkSecurityGroups_SimpleVM_nsg_name'))]"
]
},
{
"comments": "Generalized from resource: '/subscriptions/7b790d29-944d-4f0f-861b-928e4774cde9/resourceGroups/SimpleVM/providers/Microsoft.Network/virtualNetworks/SimpleVNet/subnets/default'.",
"type": "Microsoft.Network/virtualNetworks/subnets",
"name": "[concat(parameters('virtualNetworks_SimpleVNet_name'), '/', parameters('subnets_default_name'))]",
"apiVersion": "2017-10-01",
"scale": null,
"properties": {
"provisioningState": "Succeeded",
"addressPrefix": "10.5.0.0/24"
},
"dependsOn": [
"[resourceId('Microsoft.Network/virtualNetworks', parameters('virtualNetworks_SimpleVNet_name'))]"
]
}
]
}
the template is pretty big, i wont be able to look at it. but the error means that you have a dependency loop in your template. you need to analyze dependsOn and reference() in the template to detect it.
because both of these act as a dependency "marker"

How do I get private IP address of VM in nested ARM template?

I have a nested template which creates VM which works fine but I need to return private IP address of that machine to main template and as output. I create following output in my nested template but it fails with error message
Deployment template validation failed: 'The template reference 'swarmmanager1NetworkInterface' is not valid: could not find template resource or resource copy with this name. Please see https://aka.ms/arm-template-expressions/#reference for usage details.'.
Here is entire template
{
"$schema": "http://schema.management.azure.com/schemas/2015-01-01-preview/deploymentTemplate.json",
"contentVersion": "1.0.0.0",
"parameters": {
"vmSku": {
"type": "string",
"defaultValue": "Standard_A1",
"metadata": {
"description": "Size of VMs in the VM Scale Set."
}
},
"windowsOSVersion": {
"type": "string",
"defaultValue": "2016-Datacenter-with-Containers",
"allowedValues": [
"2008-R2-SP1",
"2012-Datacenter",
"2012-R2-Datacenter",
"2016-Datacenter-with-Containers"
],
"metadata": {
"description": "The Windows version for the VM. This will pick a fully patched image of this given Windows version. Allowed values: 2008-R2-SP1, 2012-Datacenter, 2012-R2-Datacenter."
}
},
"registrationKey": {
"type": "securestring",
"metadata": {
"description": "Registration key to use to onboard to the Azure Automation DSC pull/reporting server"
}
},
"registrationUrl": {
"type": "string",
"metadata": {
"description": "Registration url of the Azure Automation DSC pull/reporting server"
}
},
"DSCExtensionTagVersion": {
"type": "string",
"metadata": {
"description": "Change this to redeploy DSC"
}
},
"nodeConfigurationName": {
"type": "string",
"defaultValue": "MyService.webServer",
"metadata": {
"description": "The name of the node configuration, on the Azure Automation DSC pull server, that this node will be configured as"
}
},
"configurationMode": {
"type": "string",
"defaultValue": "ApplyAndAutoCorrect",
"allowedValues": [
"ApplyOnly",
"ApplyAndMonitor",
"ApplyAndAutoCorrect"
],
"metadata": {
"description": "DSC agent (LCM) configuration mode setting. ApplyOnly, ApplyAndMonitor, or ApplyAndAutoCorrect"
}
},
"configurationModeFrequencyMins": {
"type": "int",
"defaultValue": 15,
"metadata": {
"description": "DSC agent (LCM) configuration mode frequency setting, in minutes"
}
},
"refreshFrequencyMins": {
"type": "int",
"defaultValue": 30,
"metadata": {
"description": "DSC agent (LCM) refresh frequency setting, in minutes"
}
},
"rebootNodeIfNeeded": {
"type": "bool",
"defaultValue": true,
"metadata": {
"description": "DSC agent (LCM) rebootNodeIfNeeded setting"
}
},
"actionAfterReboot": {
"type": "string",
"defaultValue": "ContinueConfiguration",
"allowedValues": [
"ContinueConfiguration",
"StopConfiguration"
],
"metadata": {
"description": "DSC agent (LCM) actionAfterReboot setting. ContinueConfiguration or StopConfiguration"
}
},
"allowModuleOverwrite": {
"type": "bool",
"defaultValue": false,
"metadata": {
"description": "DSC agent (LCM) allowModuleOverwrite setting"
}
},
"automationAccountName": {
"type": "string",
"defaultValue": "myAutomationAccount",
"metadata": {
"description": "The name of the Automation account to use. Check the SKU and tags to make sure they match the existing account."
}
},
"automationRegionId": {
"type": "string",
"defaultValue": "East US 2",
"allowedValues": [
"Japan East",
"East US 2",
"West Europe",
"Southeast Asia",
"South Central US",
"Central India"
],
"metadata": {
"description": "The region the Automation account is located in."
}
},
"vmssName": {
"type": "string",
"metadata": {
"description": "String used as a base for naming resources. Must be 3-57 characters in length and globally unique across Azure. A hash is prepended to this string for some resources, and resource-specific information is appended."
},
"maxLength": 57
},
"instanceCount": {
"type": "int",
"metadata": {
"description": "Number of VM instances (100 or less)."
},
"maxValue": 100
},
"adminUsername": {
"type": "string",
"metadata": {
"description": "Admin username on all VMs."
}
},
"adminPassword": {
"type": "securestring",
"metadata": {
"description": "Admin password on all VMs."
}
},
"_artifactsLocation": {
"type": "string",
"metadata": {
"description": "Auto-generated container in staging storage account to receive post-build staging folder upload"
}
},
"_artifactsLocationSasToken": {
"type": "securestring",
"metadata": {
"description": "Auto-generated token to access _artifactsLocation"
}
},
"dockerswarmstorageaccountType": {
"type": "string",
"defaultValue": "Standard_LRS",
"allowedValues": [
"Standard_LRS",
"Standard_ZRS",
"Standard_GRS",
"Standard_RAGRS",
"Premium_LRS"
]
},
"swarmmanager1Name": {
"type": "string",
"minLength": 1,
"defaultValue": "swarmmanager1"
},
"swarmmanagerpublicIPDnsName": {
"type": "string",
"defaultValue": "[uniqueString(subscription().subscriptionId)]",
"minLength": 1
}
},
"variables": {
"namingInfix": "[toLower(substring(concat(parameters('vmssName'), uniqueString(resourceGroup().id)), 0, 9))]",
"longNamingInfix": "[toLower(parameters('vmssName'))]",
"addressPrefix": "10.0.0.0/16",
"subnetPrefix": "10.0.8.0/21",
"virtualNetworkName": "[concat(variables('namingInfix'), 'vnet')]",
"natPoolName": "[concat(variables('namingInfix'), 'natpool')]",
"publicIPAddressName": "[concat(variables('namingInfix'), 'pip')]",
"subnetName": "[concat(variables('namingInfix'), 'subnet')]",
"nicName": "[concat(variables('namingInfix'), 'nic')]",
"ipConfigName": "[concat(variables('namingInfix'), 'ipconfig')]",
"imageReference": {
"publisher": "MicrosoftWindowsServer",
"offer": "WindowsServer",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"virtualNetworkID": "[resourceId('Microsoft.Network/virtualNetworks',variables('virtualNetworkName'))]",
"appGwPublicIPAddressName": "[concat(variables('namingInfix'), 'appGwPip')]",
"bePoolName": "[concat(variables('namingInfix'), 'bepool')]",
"frontEndIPConfigID": "[concat(variables('lbID'),'/frontendIPConfigurations/loadBalancerFrontEnd')]",
"appGwName": "[concat(variables('namingInfix'), 'appGw')]",
"loadBalancerName": "[concat(variables('namingInfix'), 'lb')]",
"publicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('publicIPAddressName'))]",
"lbID": "[resourceId('Microsoft.Network/loadBalancers',variables('loadBalancerName'))]",
"appGwPublicIPAddressID": "[resourceId('Microsoft.Network/publicIPAddresses',variables('appGwPublicIPAddressName'))]",
"appGwID": "[resourceId('Microsoft.Network/applicationGateways',variables('appGwName'))]",
"appGwSubnetName": "[concat(variables('namingInfix'),'appGwSubnet')]",
"appGwSubnetPrefix": "10.0.1.0/24",
"appGwSubnetID": "[concat(variables('virtualNetworkID'),'/subnets/',variables('appGwSubnetName'))]",
"appGwFrontendPort": 80,
"appGwBackendPort": 80,
"appGwBePoolName": "[concat(variables('namingInfix'), 'appGwBepool')]",
"computeApiVersion": "2016-04-30-preview",
"networkApiVersion": "2016-03-30",
"natStartPort": 50000,
"natEndPort": 50119,
"natBackendPort": 3389,
"DSCArchiveFolder": "DSC",
"DSCArchiveFileName": "DSC.zip",
"dockerswarmstorageaccountName": "[concat('sa', uniqueString(resourceGroup().id))]",
"swarmmanager1ImagePublisher": "MicrosoftWindowsServer",
"swarmmanager1ImageOffer": "WindowsServer",
"swarmmanager1OSDiskName": "swarmmanager1OSDisk",
"swarmmanager1VmSize": "Standard_D2_v2",
"swarmmanager1VnetID": "[resourceId('Microsoft.Network/virtualNetworks', variables('virtualNetworkName'))]",
"swarmmanager1SubnetRef": "[concat(variables('swarmmanager1VnetID'), '/subnets/', variables('subnetName'))]",
"swarmmanager1StorageAccountContainerName": "vhds",
"swarmmanager1NicName": "[concat(parameters('swarmmanager1Name'), 'NetworkInterface')]",
"swarmmanagerpublicIPName": "swarmmanagerpublicIP",
"swarmmanagerdeploymentTemplateFolder": "nestedtemplates",
"swarmmanagerdeploymentTemplateFileName": "swarmmanagerdeployment.json",
"swarmmanagerdeploymentTemplateParametersFileName": "swarmmanagerdeployment.parameters.json"
},
"resources": [
{
"type": "Microsoft.Network/virtualNetworks",
"name": "[variables('virtualNetworkName')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('networkApiVersion')]",
"properties": {
"addressSpace": {
"addressPrefixes": [
"[variables('addressPrefix')]"
]
},
"subnets": [
{
"name": "[variables('subnetName')]",
"properties": {
"addressPrefix": "[variables('subnetPrefix')]"
}
},
{
"name": "[variables('appGwSubnetName')]",
"properties": {
"addressPrefix": "[variables('appGwSubnetPrefix')]"
}
}
]
}
},
{
"type": "Microsoft.Network/loadBalancers",
"name": "[variables('loadBalancerName')]",
"location": "[resourceGroup().location]",
"apiVersion": "2017-04-01",
"dependsOn": [
"[concat('Microsoft.Network/publicIPAddresses/', variables('publicIPAddressName'))]"
],
"properties": {
"frontendIPConfigurations": [
{
"name": "LoadBalancerFrontEnd",
"properties": {
"publicIPAddress": {
"id": "[variables('publicIPAddressID')]"
}
}
}
],
"backendAddressPools": [
{
"name": "[variables('bePoolName')]"
}
],
"inboundNatPools": [
{
"name": "[variables('natPoolName')]",
"properties": {
"frontendIPConfiguration": {
"id": "[variables('frontEndIPConfigID')]"
},
"protocol": "tcp",
"frontendPortRangeStart": "[variables('natStartPort')]",
"frontendPortRangeEnd": "[variables('natEndPort')]",
"backendPort": "[variables('natBackendPort')]"
}
}
]
}
},
{
"type": "Microsoft.Network/publicIPAddresses",
"name": "[variables('appGwPublicIPAddressName')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('networkApiVersion')]",
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "appgwvip"
}
}
},
{
"apiVersion": "[variables('networkApiVersion')]",
"location": "[resourceGroup().location]",
"name": "[variables('publicIPAddressName')]",
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "rdpvip"
}
},
"type": "Microsoft.Network/publicIPAddresses"
},
{
"type": "Microsoft.Network/applicationGateways",
"name": "[variables('appGwName')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('networkApiVersion')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]",
"[concat('Microsoft.Network/publicIPAddresses/', variables('appGwPublicIPAddressName'))]"
],
"properties": {
"sku": {
"name": "Standard_Small",
"tier": "Standard",
"capacity": "1"
},
"gatewayIPConfigurations": [
{
"name": "appGwIpConfig",
"properties": {
"subnet": {
"id": "[variables('appGwSubnetID')]"
}
}
}
],
"frontendIPConfigurations": [
{
"name": "appGwFrontendIP",
"properties": {
"PublicIPAddress": {
"id": "[variables('appGwPublicIPAddressID')]"
}
}
}
],
"frontendPorts": [
{
"name": "appGwFrontendPort",
"properties": {
"Port": "[variables('appGwFrontendPort')]"
}
}
],
"backendAddressPools": [
{
"name": "[variables('appGwBePoolName')]"
}
],
"backendHttpSettingsCollection": [
{
"name": "appGwBackendHttpSettings",
"properties": {
"Port": "[variables('appGwBackendPort')]",
"Protocol": "Http",
"CookieBasedAffinity": "Disabled"
}
}
],
"httpListeners": [
{
"name": "appGwHttpListener",
"properties": {
"FrontendIPConfiguration": {
"Id": "[concat(variables('appGwID'), '/frontendIPConfigurations/appGwFrontendIP')]"
},
"FrontendPort": {
"Id": "[concat(variables('appGwID'), '/frontendPorts/appGwFrontendPort')]"
},
"Protocol": "Http",
"SslCertificate": null
}
}
],
"requestRoutingRules": [
{
"Name": "rule1",
"properties": {
"RuleType": "Basic",
"httpListener": {
"id": "[concat(variables('appGwID'), '/httpListeners/appGwHttpListener')]"
},
"backendAddressPool": {
"id": "[concat(variables('appGwID'), '/backendAddressPools/', variables('appGwBePoolName'))]"
},
"backendHttpSettings": {
"id": "[concat(variables('appGwID'), '/backendHttpSettingsCollection/appGwBackendHttpSettings')]"
}
}
}
]
}
},
{
"type": "Microsoft.Compute/virtualMachineScaleSets",
"name": "[variables('namingInfix')]",
"location": "[resourceGroup().location]",
"apiVersion": "[variables('computeApiVersion')]",
"dependsOn": [
"[concat('Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'))]",
"[concat('Microsoft.Network/applicationGateways/', variables('appGwName'))]",
"[concat('Microsoft.Network/loadBalancers/', variables('loadBalancerName'))]"
],
"sku": {
"name": "[parameters('vmSku')]",
"tier": "Standard",
"capacity": "[parameters('instanceCount')]"
},
"properties": {
"overprovision": "false",
"singlePlacementGroup": "true",
"upgradePolicy": {
"mode": "Automatic"
},
"virtualMachineProfile": {
"storageProfile": {
"osDisk": {
"caching": "ReadWrite",
"createOption": "FromImage"
},
"dataDisks": [],
"imageReference": "[variables('imageReference')]"
},
"osProfile": {
"computerNamePrefix": "[variables('namingInfix')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"networkProfile": {
"networkInterfaceConfigurations": [
{
"name": "[variables('nicName')]",
"properties": {
"primary": "true",
"ipConfigurations": [
{
"name": "[variables('ipConfigName')]",
"properties": {
"subnet": {
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/virtualNetworks/', variables('virtualNetworkName'), '/subnets/', variables('subnetName'))]"
},
"loadBalancerBackendAddressPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/loadBalancers/', variables('loadBalancerName'), '/backendAddressPools/', variables('bePoolName'))]"
}
],
"loadBalancerInboundNatPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/loadBalancers/', variables('loadBalancerName'), '/inboundNatPools/', variables('natPoolName'))]"
}
],
"ApplicationGatewayBackendAddressPools": [
{
"id": "[concat('/subscriptions/', subscription().subscriptionId,'/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Network/applicationGateways/', variables('appGwName'), '/backendAddressPools/', variables('appGwBePoolName'))]"
}
]
}
}
]
}
}
]
},
"extensionProfile": {
"extensions": [
{
"name": "Microsoft.Powershell.DSC",
"properties": {
"autoUpgradeMinorVersion": true,
"publisher": "Microsoft.Powershell",
"forceUpdateTag": "[parameters('DSCExtensionTagVersion')]",
"settings": {
"configuration": {
"url": "[concat(parameters('_artifactsLocation'), '/', variables('DSCArchiveFolder'), '/', variables('DSCArchiveFileName'), parameters('_artifactsLocationSasToken'))]",
"script": "DSC.ps1",
"function": "Main"
},
"configurationArguments": {
"RegistrationKey": "[parameters('registrationKey')]",
"RegistrationUrl": "[parameters('registrationUrl')]",
"NodeConfigurationName": "[parameters('nodeConfigurationName')]",
"ConfigurationMode": "[parameters('configurationMode')]",
"ConfigurationModeFrequencyMins": "[parameters('configurationModeFrequencyMins')]",
"RefreshFrequencyMins": "[parameters('refreshFrequencyMins')]",
"RebootNodeIfNeeded": "[parameters('rebootNodeIfNeeded')]",
"ActionAfterReboot": "[parameters('actionAfterReboot')]",
"AllowModuleOverwrite": "[parameters('allowModuleOverwrite')]"
}
},
"type": "DSC",
"typeHandlerVersion": "2.26"
}
}
]
}
}
}
},
{
"name": "swarmmanager",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2016-09-01",
"dependsOn": [],
"properties": {
"mode": "Incremental",
"template": {
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"resources": [
{
"name": "[parameters('swarmmanager1Name')]",
"type": "Microsoft.Compute/virtualMachines",
"location": "[resourceGroup().location]",
"apiVersion": "2015-06-15",
"dependsOn": [
"[resourceId('Microsoft.Storage/storageAccounts', variables('dockerswarmstorageaccountName'))]",
"[resourceId('Microsoft.Network/networkInterfaces', variables('swarmmanager1NicName'))]"
],
"tags": {
"displayName": "swarmmanager1"
},
"properties": {
"hardwareProfile": {
"vmSize": "[variables('swarmmanager1VmSize')]"
},
"osProfile": {
"computerName": "[parameters('swarmmanager1Name')]",
"adminUsername": "[parameters('adminUsername')]",
"adminPassword": "[parameters('adminPassword')]"
},
"storageProfile": {
"imageReference": {
"publisher": "[variables('swarmmanager1ImagePublisher')]",
"offer": "[variables('swarmmanager1ImageOffer')]",
"sku": "[parameters('windowsOSVersion')]",
"version": "latest"
},
"osDisk": {
"name": "swarmmanager1OSDisk",
"vhd": {
"uri": "[concat(reference(resourceId('Microsoft.Storage/storageAccounts', variables('dockerswarmstorageaccountName')), '2016-01-01').primaryEndpoints.blob, variables('swarmmanager1StorageAccountContainerName'), '/', variables('swarmmanager1OSDiskName'), '.vhd')]"
},
"caching": "ReadWrite",
"createOption": "FromImage"
}
},
"networkProfile": {
"networkInterfaces": [
{
"id": "[resourceId('Microsoft.Network/networkInterfaces', variables('swarmmanager1NicName'))]"
}
]
}
}
},
{
"name": "[variables('dockerswarmstorageaccountName')]",
"type": "Microsoft.Storage/storageAccounts",
"location": "[resourceGroup().location]",
"apiVersion": "2016-01-01",
"sku": {
"name": "[parameters('dockerswarmstorageaccountType')]"
},
"dependsOn": [],
"tags": {
"displayName": "dockerswarmstorageaccount"
},
"kind": "Storage"
},
{
"name": "[variables('swarmmanager1NicName')]",
"type": "Microsoft.Network/networkInterfaces",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-30",
"tags": {
"displayName": "swarmmanager1Nic"
},
"properties": {
"ipConfigurations": [
{
"name": "ipconfig1",
"properties": {
"privateIPAllocationMethod": "Dynamic",
"subnet": {
"id": "[variables('swarmmanager1SubnetRef')]"
},
"publicIPAddress": {
"id": "[resourceId('Microsoft.Network/publicIPAddresses', variables('swarmmanagerpublicIPName'))]"
}
}
}
]
}
},
{
"name": "[variables('swarmmanagerpublicIPName')]",
"type": "Microsoft.Network/publicIPAddresses",
"location": "[resourceGroup().location]",
"apiVersion": "2016-03-30",
"dependsOn": [],
"tags": {
"displayName": "swarmmanagerpublicIP"
},
"properties": {
"publicIPAllocationMethod": "Dynamic",
"dnsSettings": {
"domainNameLabel": "[parameters('swarmmanagerpublicIPDnsName')]"
}
}
},
{
"outputs": {
"returnedIPAddress": {
"type": "string",
"value": "[reference(variables('swarmmanager1NicName')).ipConfigurations[0].properties.privateIPAddress]"
}
}
}
]
}
}
}
]
}
Ok, a couple things - you have your output in the nested template in the resources array - outputs should be a peer of the resources array. That's not getting flagged because of the inline reference() call fails before this.
First, reference() calls are often made very early in the deployment - sometimes earlier than you want them to be. When the resource being referenced is defined in the same template, it works as expected. When it's not defined in the same template, the GET happens immediately and the resource may not exist. That's a nuance of the reference() function that's a little tangential but important to know as you use it...
Combine that behavior with the inline template deployment. Inline deployments have a number of design quirks that make them pretty hard to use for anything "advanced". You won't be able to do what you're doing with an inline deployment. If you use a templateLink (i.e. another file), you can deploy the nic and output the ipconfig in the same template.
That help?
I have had a similar issue, we were deploying using Azure Pipelines, the approach i took was 1) Deploy Template and pass the RG name and VMSS name as template outputs 2) ARM Outputs - Use this snippet to publish the above results as vars to use downstream 3) Azure Powershell to get the IPs of all VMSS instances and publish them as pipeline variables 4) Write the pipeline vars file to a blob. Hope this helps
$iplist=#()
$i=1
$nicinfo=Get-AzureRmNetworkInterface -ResourceGroupName $(ResourceGroupName) -
VirtualMachineScaleSetName $(VmssName)
foreach($nic in $nicinfo){
$iplist += $nic.IpConfigurations.PrivateIpAddress
}
foreach($ip in $iplist){
Write-host Private ip $i is $ip
Write-Host "##vso[task.setvariable variable=Privateip$i;]$ip"
$i++
}

Resources