I have an ARM template which sources the primaryMasterKey of a cosmosDb as follows:
{
"properties": {
"enabled": true,
"siteConfig": {
"appSettings": [
{
"name": "MongoDb:CnnDetails",
"value": "[listKeys(resourceId('Microsoft.DocumentDB/databaseAccounts', variables('cosmosdb_full')), '2015-04-08').primaryMasterKey]"
}
}
How to I modify it to get the actual connection string instead?
I've tried couple of things:
changed the word primaryMasterKey to primaryConnectionString. This gives an error saying:
'The language expression property 'primaryConnectionString' doesn't exist, available properties are 'primaryMasterKey, secondaryMasterKey, primaryReadonlyMasterKey, secondaryReadonlyMasterKey'
changed the work listKeys to listConnectionStrings. This is red underlined in my visual studio, but seems to work when put through azure devops
'The language expression property 'primaryConnectionString' doesn't exist, available properties are 'connectionStrings'
I went to https://learn.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/databaseaccounts/listconnectionstrings#code-try-0 to try it out. ListKeys returns a structure like this:
{
"primaryMasterKey": "[REDACTED]",
"secondaryMasterKey": "[REDACTED]",
"primaryReadonlyMasterKey": "[REDACTED]",
"secondaryReadonlyMasterKey": "[REDACTED]"
}
so I get why the .primaryMasterKey worked. But ListConnectionStrings returns:
{
"connectionStrings": [
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Primary MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Secondary MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Primary Read-Only MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Secondary Read-Only MongoDB Connection String"
}
]
}
Not sure how to "index into it"?
Any clues gratefully received.
For anyone else finding this question and wanting a fully complete ARM Template snippet, this is what I have used and is working:
"connectionStrings": [
{
"name": "CosmosConnection",
"connectionString": "[listConnectionStrings(resourceId('Microsoft.DocumentDB/databaseAccounts', parameters('cosmosDbAccountName')), '2019-12-12').connectionStrings[0].connectionString]",
"type": 3
}
]
like you normally would in almost any language:
ListConnectionStrings.connectionStrings[index].connectionString
index starts at 0.
you have a more "native" way of doing this:
first(ListConnectionStrings.connectionStrings).connectionString
but only available functions are first and last
The answer here by oatsoda is correct but it will only work if you are within the same resource group as the Cosmos DB you are getting the connection string for. If you have the scenario where you Cosmos DB is in a different resource group to the resource you are generating an ARM template for the following snippet is what I have used to generate the connection string for an App Service and is working.
"Cosmos": {
"value": "[listConnectionStrings(resourceId(parameters('cosmosResourceGroupName'),'Microsoft.DocumentDB/databaseAccounts', parameters('cosmosDbName')), '2019-12-12').connectionStrings[0].connectionString]",
"type": "Custom"
}
In the Cosmos linked ARM template named linkedTemplate_cosmos_db-gdp-event-ammi-dev-ne-001 I used the following code.
"outputs": {
"ConnectionString": {
"value": "[listConnectionStrings(resourceId('Microsoft.DocumentDB/databaseAccounts', parameters('accountName')), '2019-12-12').connectionStrings[0].connectionString]",
"name": "CosmosConnection",
"type": "string"
}
},
and then in the ARM template (linkedTemplate_Main) that uses the output parameter, the following, e.g a function app configuration setting
"COSMOS_CONNECTION_STRING": {
"value": "[reference('linkedTemplate_cosmos_db-gdp-event-ammi-dev-ne-001').outputs.ConnectionString.value]"
Related
I have two applications(one from Host and other from WebHost). Both .NET 6. I have multiple Serilog sinks as below in both apps configured in appsettings.json(its a valid json). I'm trying to use dictionary mode of configuring sinks in WriteTo section instead of array(see https://github.com/serilog/serilog-settings-configuration#writeto-enrich-auditto-destructure-sections).
In WebHost application appsettings are loaded without a problem.
But in Host application I get this error "The configuration value in Serilog:WriteTo:0:Name has no 'Name' element."
inside IHostBuilder.UseSerilog(...).
What am I getting wrong here?
I have checked with another WebHost application and it works. None of the Host applications are working.
"Serilog": {
"MinimumLevel": {
"Default": "Information"
},
"WriteTo": {
"FileSink": {
"Name": "File",
"Args": {
...
}
},
"ConsoleSink": {
"Name": "Console",
"Args": {
...
}
}
}
}
},
We've got ARM deployment template which is working fine, the only issue there is, as we're creating FunctionApp and adding custom key to it that will be referenced in API Management to connect API with FunctionApp backend. It looks like, even with dependsOn in backend resource in template referencing to FunctionKeys resource, listkeys() still fetches one from before deployment. Has anyone faced this scenario and is there anything in particular I'm missing? Or is dependsOn thinking that the key has been deployed already, and even tho future deployments update the key, backend isn't actually waiting for the completion of FunctionKeys resource deployment?
Template snippets:
backends
{
"type": "Microsoft.ApiManagement/service/backends",
"apiVersion": "2018-01-01",
"name": "[concat(parameters('apiManagementServiceName'), '/', variables('functionName'))]",
"dependsOn": [
"[resourceId('Microsoft.ApiManagement/service', parameters('apiManagementServiceName'))]",
"[resourceId('Microsoft.Web/sites', variables('functionName'))]",
"[resourceId('Microsoft.Web/sites/host/functionKeys', variables('functionName'),'default','apimanagement')]"
],
"properties": {
"url": "[concat('https://', variables('functionName'), '.azurewebsites.net/api')]",
"protocol": "http",
"resourceId": "[concat('https://management.azure.com/subscriptions/', subscription().subscriptionId, '/resourceGroups/', resourceGroup().name, '/providers/Microsoft.Web/sites/', variables('functionName'))]",
"credentials": {
"header": {
"x-functions-key": [
"[listkeys(concat(variables('functionAppId'), '/host/default/'),'2016-08-01').functionKeys.apimanagement]"
]
}
}
}
}
functionKeys
{
"type": "Microsoft.Web/sites/host/functionKeys",
"apiVersion": "2018-11-01",
"name": "[concat(variables('functionName'), '/default/apimanagement')]",
"dependsOn": [
"[resourceId('Microsoft.Web/sites', variables('functionName'))]"
],
"properties": {
"name": "apimanagement"
}
},
The listKeys call is scheduled too early... In a greenfield scenario the deployment would fail, in brownfield you "get the old key" instead of the new one. It's a "limitation" in the template engine that you can work around. Basically, you need to nest the deployment that uses the listKeys function (i.e. your backend resource). There's a little bit of detail on it here in the Use a Nested Deployment section
Question
How to handle complex structures, such as arrays, when using the ConfigurationBuilder's AddEnvironmentVariables approach for setting config.
i.e. What would the environment variables be called to represent the below structure?
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Detailed version with context
We're writing apps that will be hosted in containers. Initially we'd continued to use the AppSettings.json file to hold settings, populating this file's contents from environment variables at startup via an entrypoint.sh shell script.
We later realised that we could use the ConfigurationBuilder's AddEnvironmentVariables method to pull the values directly from the environment variables, and thus avoid the overhead of maintaining the entrypoint.sh script as being another place to update each time a new setting was added.
This works well for an app settings file such as:
{
"MyApp": {
"SmtpServer": "smtp.example.com",
"FromAddress": "JoeyBlogger#example.com"
},
"Logging": {
"LogLevel": "Default"
}
}
.. as this translates simply to:
export MyApp:SmtpServer=smtp.example.com
export MyApp:FromAddress=JoeyBlogger#example.com
export Logging:LogLevel=Default
However, I recently saw a developer's app settings file for some new monitoring, which looks like the below (note; this won't be the actual settings used in our hosted environments; it's just an example which triggered this question). In this case, using the above approach doesn't work so well, since there would be two different settings with the name Serilog:WriteTo:Name, and ambiguity over which entry anything starting Serilog:WriteTo:Args:* relates to.
{
"Serilog"
"WriteTo": [
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "File",
"Args": {
"path": ".\\log.txt",
"rollingInterval": "Day",
"retainedFileCountLimit": 7,
"buffered": true
}
}
]
}
},
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "Console"
}
]
}
}
]
}
}
My guess is we'd handle this via some sort of index; but I've not been able to find anything in the docs about this.
export Serilog:WriteTo[0]:Name=Async
export Serilog:WriteTo[0]:Args:configure[0]:Name=File
export Serilog:WriteTo[0]:Args:configure[0]:Args:path=.\\log.txt
export Serilog:WriteTo[0]:Args:configure[0]:Args:rollingInterval=Day
export Serilog:WriteTo[0]:Args:configure[0]:Args:retainedFileCountLimit=7
export Serilog:WriteTo[0]:Args:configure[0]:Args:buffered=true
export Serilog:WriteTo[1]:Name=Async
export Serilog:WriteTo[1]:Args:configure[0]:Name=Console
However, running a quick PoC shows that the above doesn't work. How can this be done?
I found the solution here: MS Extensions Configuration Deep Dive.
The answer is that indexes are used, but the index number is treated like a named element; i.e.
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Becomes:
export MyApp:SendAlertsTo:0:Name=Joe Blogger
export MyApp:SendAlertsTo:0:Email=JoeBlogger#example.com
export MyApp:SendAlertsTo:1:Name=Jane Doer
export MyApp:SendAlertsTo:1:Email=JaneDoer#example.com
Additionally, for Linux hosting there's another point I'd not been aware of / unrelated to the array question.. the : separator should be __, so the above becomes:
export MyApp__SendAlertsTo__0__Name=Joe Blogger
export MyApp__SendAlertsTo__0__Email=JoeBlogger#example.com
export MyApp__SendAlertsTo__1__Name=Jane Doer
export MyApp__SendAlertsTo__1__Email=JaneDoer#example.com
I am trying to update dozens of services to au.com.dius:pact-jvm-consumer-junit_2.12:3.5.12 from au.com.dius:pact-jvm-consumer-junit_2.11:3.2.13 but it seems that the new consumer version is generating pacts that the old provider version (au.com.dius:pact-jvm-provider-junit_2.11:3.2.13) cannot handle.
The old pact had a hashmap with matching rules being added at the root as show below
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodvider-amqp"
},
"messages": [
{
"description": "amqp contract",
"contents": {
"body": {
"guidProperty": "795ecfd5-a3a5-430f-a0cd-1569df61bff6"
}
},
"matchingRules": {
"$.body.body.guidProperty": {
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.2.13"
}
}
}
The new consumer adds and wrapper body around the matchers. Below is an example of the same pact generated with the new consumer version
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodiver-amqp"
},
"messages": [
{
"description": "contract",
"contents": {
"body": {
"guidProperty": "e2490de5-5bd3-43d5-b7c4-526e33f71304"
}
},
"matchingRules": {
"body": {
"$.guidProperty": {
"matchers": [
{
"match": "regex",
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
],
"combine": "AND"
}
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.5.12"
}
}
}
Because of that change the provider fails to parse the matching rules with the following error:
body
^
10:21:24.526 [main] DEBUG au.com.dius.pact.matchers.JsonBodyMatcher - compareValues: No matcher defined for path List($, body, body, guidProperty), using equality
10:21:24.527 [main] WARN au.com.dius.pact.matchers.Matchers$ - Path expression body is invalid, ignoring: [1.1] failure: `$' expected but `b' found
java.lang.AssertionError:
comparison
{$.body.body.guidProperty=Expected 'e2490de5-5bd3-43d5-b7c4-526e33f71304' but received 'aff876f5-5014-937c-6855-c099f9857437'
Looking at the v3 spec the new message seems to be valid, does the old provider library (v3.2.13) not support it? I looked through the code and found this commit which seems to me where the change was introduced.
From my testing the new provider library (3.5.12) can handle both old and new formats, but if both the new provider and old consumer libraries are present in the classpath http contract tests fail with run-time errors.
Questions:
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?
As J_A_X pointed out, it looks like the commit you referenced fixed a bug where pacts that were supposed to be version 3 were not fully version 3 compliant (they used the old matchers format)
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
Yes and yes. You should be able to fix this by setting the system property pact.provider.version to 2 - then both the old and the new versions will be able to read the generated pact.
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?
Yes, as long as you ask the old consumer version to generate version 2 pacts (but ideally, why not update both to the same version?)
I have a simple DSC Config file that contains a credential and string input parameter. I want this DSC configuration deployed with a VM deployed in an ARM template but am missing the concept of how to pass these two parameters securely. How do I accomplish this?
I was receiving the same error but, after some shenanigans, it is working for me. The important part is the settings/Properties/SqlAgentCred/password reference to protectedSettings/Items/AgentPassword. Below is the properties node under my Powershell.DSC extension resource in my template.
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.17",
"autoUpgradeMinorVersion": false,
"settings": {
"ModulesUrl": "https://blobstore.blob.core.windows.net/windows-powershell-dsc/DBServer.ps1.zip",
"ConfigurationFunction": "DBServer.ps1\\DBServer",
"Properties": {
"SqlAgentCred": {
"userName": "user#domain.com",
"password": "PrivateSettingsRef:AgentPassword"
}
},
"WmfVersion": "latest",
"Privacy": {
"DataCollection": "Disable"
}
},
"protectedSettings": {
"Items": {
"AgentPassword": "Pa$$word"
},
"DataBlobUri": ""
}
}
You will specify protected settings under protectedsettings section. Anything under ProtectedSettings are sent encrypted. Check https://blogs.msdn.microsoft.com/powershell/2016/02/26/arm-dsc-extension-settings/ for details.