Question
How to handle complex structures, such as arrays, when using the ConfigurationBuilder's AddEnvironmentVariables approach for setting config.
i.e. What would the environment variables be called to represent the below structure?
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Detailed version with context
We're writing apps that will be hosted in containers. Initially we'd continued to use the AppSettings.json file to hold settings, populating this file's contents from environment variables at startup via an entrypoint.sh shell script.
We later realised that we could use the ConfigurationBuilder's AddEnvironmentVariables method to pull the values directly from the environment variables, and thus avoid the overhead of maintaining the entrypoint.sh script as being another place to update each time a new setting was added.
This works well for an app settings file such as:
{
"MyApp": {
"SmtpServer": "smtp.example.com",
"FromAddress": "JoeyBlogger#example.com"
},
"Logging": {
"LogLevel": "Default"
}
}
.. as this translates simply to:
export MyApp:SmtpServer=smtp.example.com
export MyApp:FromAddress=JoeyBlogger#example.com
export Logging:LogLevel=Default
However, I recently saw a developer's app settings file for some new monitoring, which looks like the below (note; this won't be the actual settings used in our hosted environments; it's just an example which triggered this question). In this case, using the above approach doesn't work so well, since there would be two different settings with the name Serilog:WriteTo:Name, and ambiguity over which entry anything starting Serilog:WriteTo:Args:* relates to.
{
"Serilog"
"WriteTo": [
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "File",
"Args": {
"path": ".\\log.txt",
"rollingInterval": "Day",
"retainedFileCountLimit": 7,
"buffered": true
}
}
]
}
},
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "Console"
}
]
}
}
]
}
}
My guess is we'd handle this via some sort of index; but I've not been able to find anything in the docs about this.
export Serilog:WriteTo[0]:Name=Async
export Serilog:WriteTo[0]:Args:configure[0]:Name=File
export Serilog:WriteTo[0]:Args:configure[0]:Args:path=.\\log.txt
export Serilog:WriteTo[0]:Args:configure[0]:Args:rollingInterval=Day
export Serilog:WriteTo[0]:Args:configure[0]:Args:retainedFileCountLimit=7
export Serilog:WriteTo[0]:Args:configure[0]:Args:buffered=true
export Serilog:WriteTo[1]:Name=Async
export Serilog:WriteTo[1]:Args:configure[0]:Name=Console
However, running a quick PoC shows that the above doesn't work. How can this be done?
I found the solution here: MS Extensions Configuration Deep Dive.
The answer is that indexes are used, but the index number is treated like a named element; i.e.
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Becomes:
export MyApp:SendAlertsTo:0:Name=Joe Blogger
export MyApp:SendAlertsTo:0:Email=JoeBlogger#example.com
export MyApp:SendAlertsTo:1:Name=Jane Doer
export MyApp:SendAlertsTo:1:Email=JaneDoer#example.com
Additionally, for Linux hosting there's another point I'd not been aware of / unrelated to the array question.. the : separator should be __, so the above becomes:
export MyApp__SendAlertsTo__0__Name=Joe Blogger
export MyApp__SendAlertsTo__0__Email=JoeBlogger#example.com
export MyApp__SendAlertsTo__1__Name=Jane Doer
export MyApp__SendAlertsTo__1__Email=JaneDoer#example.com
Related
I'm struggling to figure this one out... So we have a web app with the following URL structure:
/city/posttype/slug
We want to implement dynamic linking, but we only want to open certain post types in the app, nothing else.
So for example these should link to the app:
/new-york/posttypeA/slug
/new-york/posttypeB/slug
/new-york/posttypeC/slug
Meanwhile, for example, these should not be opened in the app:
/new-york/posttypeD/slug
/new-york
/
Here is our AASA file:
{
"applinks": {
"details": [
{
"appID": "OUR_APP_ID",
"paths": [
{
"/": "/*/posttypeA/*",
"comment": "matches all posttypeA"
},
{
"/": "/*/posttypeB/*",
"comment": "matches all posttypeB"
},
{
"/": "/*/posttypeC/*",
"comment": "matches all posttypeC"
},
]
}
]
}
}
But our iOS app still wants to handle all links from our website. As I've read any time I do a change to our AASA file I need to reinstall the iOS app which I did but still no luck. Any ideas?
As a CosmosDB (SQL API) user I would like to index all non object or array properties inside of an object.
By default the index in cosmos /* will index every property, our data set is getting extremely large (expensive) and this strategy is no longer optimal. We store our metadata at the root and our customer data wrapped inside of an object property data.
Our platform restricts queries on the data path to be value type properties, this means that for us to index objects and arrays nested under the data path is just slowing down writes and costing RUs to store but never getting used.
I have tried several iterations of index policies but cannot find one that fits. Example:
{
"partitionKey": "f402a704-19bb-4f4d-93e6-801c50280cf6",
"id": "4a7a11e5-00b5-4def-8e80-132a8c083f24",
"data": {
"country": "Belgium",
"employee": 250,
"teammates": [
{ "name": "Jake", "id": 123 ...},
{ "name": "kyle", "id": 3252352 ...}
],
"user": {
"name": "Brian",
"addresses": [{ "city": "Moscow" ...}, { "city": "Moscow" ...}]
}
}
}
In this case I want to only index the root properties as well as /data/employee and /data/country.
Policies like /data/* will not work because it would then index /data/teammates/name ... and so on.
/data/? => assumes data is a value type which it never will be so this doesn't work.
/data/ and /data/*/? and /data/*? are not accepted by cosmos as valid policies.
Additionally I can't simply exclude /data/teammates/ and /data/user/ because what is inside of data is completely dynamic so while that might cover this use case there are several 100k others that it would not.
I have tried many iterations but it seems that options don't work for various reasons, is there a way to support what I am trying to do?
This indexing policy will index the properties you are asking for.
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/partitionKey/?"
},
{
"path": "/data/country/?"
},
{
"path": "/data/employee/?"
}
],
"excludedPaths": [
{
"path": "/*"
}
]
}
I have an ARM template which sources the primaryMasterKey of a cosmosDb as follows:
{
"properties": {
"enabled": true,
"siteConfig": {
"appSettings": [
{
"name": "MongoDb:CnnDetails",
"value": "[listKeys(resourceId('Microsoft.DocumentDB/databaseAccounts', variables('cosmosdb_full')), '2015-04-08').primaryMasterKey]"
}
}
How to I modify it to get the actual connection string instead?
I've tried couple of things:
changed the word primaryMasterKey to primaryConnectionString. This gives an error saying:
'The language expression property 'primaryConnectionString' doesn't exist, available properties are 'primaryMasterKey, secondaryMasterKey, primaryReadonlyMasterKey, secondaryReadonlyMasterKey'
changed the work listKeys to listConnectionStrings. This is red underlined in my visual studio, but seems to work when put through azure devops
'The language expression property 'primaryConnectionString' doesn't exist, available properties are 'connectionStrings'
I went to https://learn.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/databaseaccounts/listconnectionstrings#code-try-0 to try it out. ListKeys returns a structure like this:
{
"primaryMasterKey": "[REDACTED]",
"secondaryMasterKey": "[REDACTED]",
"primaryReadonlyMasterKey": "[REDACTED]",
"secondaryReadonlyMasterKey": "[REDACTED]"
}
so I get why the .primaryMasterKey worked. But ListConnectionStrings returns:
{
"connectionStrings": [
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Primary MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Secondary MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Primary Read-Only MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Secondary Read-Only MongoDB Connection String"
}
]
}
Not sure how to "index into it"?
Any clues gratefully received.
For anyone else finding this question and wanting a fully complete ARM Template snippet, this is what I have used and is working:
"connectionStrings": [
{
"name": "CosmosConnection",
"connectionString": "[listConnectionStrings(resourceId('Microsoft.DocumentDB/databaseAccounts', parameters('cosmosDbAccountName')), '2019-12-12').connectionStrings[0].connectionString]",
"type": 3
}
]
like you normally would in almost any language:
ListConnectionStrings.connectionStrings[index].connectionString
index starts at 0.
you have a more "native" way of doing this:
first(ListConnectionStrings.connectionStrings).connectionString
but only available functions are first and last
The answer here by oatsoda is correct but it will only work if you are within the same resource group as the Cosmos DB you are getting the connection string for. If you have the scenario where you Cosmos DB is in a different resource group to the resource you are generating an ARM template for the following snippet is what I have used to generate the connection string for an App Service and is working.
"Cosmos": {
"value": "[listConnectionStrings(resourceId(parameters('cosmosResourceGroupName'),'Microsoft.DocumentDB/databaseAccounts', parameters('cosmosDbName')), '2019-12-12').connectionStrings[0].connectionString]",
"type": "Custom"
}
In the Cosmos linked ARM template named linkedTemplate_cosmos_db-gdp-event-ammi-dev-ne-001 I used the following code.
"outputs": {
"ConnectionString": {
"value": "[listConnectionStrings(resourceId('Microsoft.DocumentDB/databaseAccounts', parameters('accountName')), '2019-12-12').connectionStrings[0].connectionString]",
"name": "CosmosConnection",
"type": "string"
}
},
and then in the ARM template (linkedTemplate_Main) that uses the output parameter, the following, e.g a function app configuration setting
"COSMOS_CONNECTION_STRING": {
"value": "[reference('linkedTemplate_cosmos_db-gdp-event-ammi-dev-ne-001').outputs.ConnectionString.value]"
I am trying to update dozens of services to au.com.dius:pact-jvm-consumer-junit_2.12:3.5.12 from au.com.dius:pact-jvm-consumer-junit_2.11:3.2.13 but it seems that the new consumer version is generating pacts that the old provider version (au.com.dius:pact-jvm-provider-junit_2.11:3.2.13) cannot handle.
The old pact had a hashmap with matching rules being added at the root as show below
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodvider-amqp"
},
"messages": [
{
"description": "amqp contract",
"contents": {
"body": {
"guidProperty": "795ecfd5-a3a5-430f-a0cd-1569df61bff6"
}
},
"matchingRules": {
"$.body.body.guidProperty": {
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.2.13"
}
}
}
The new consumer adds and wrapper body around the matchers. Below is an example of the same pact generated with the new consumer version
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodiver-amqp"
},
"messages": [
{
"description": "contract",
"contents": {
"body": {
"guidProperty": "e2490de5-5bd3-43d5-b7c4-526e33f71304"
}
},
"matchingRules": {
"body": {
"$.guidProperty": {
"matchers": [
{
"match": "regex",
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
],
"combine": "AND"
}
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.5.12"
}
}
}
Because of that change the provider fails to parse the matching rules with the following error:
body
^
10:21:24.526 [main] DEBUG au.com.dius.pact.matchers.JsonBodyMatcher - compareValues: No matcher defined for path List($, body, body, guidProperty), using equality
10:21:24.527 [main] WARN au.com.dius.pact.matchers.Matchers$ - Path expression body is invalid, ignoring: [1.1] failure: `$' expected but `b' found
java.lang.AssertionError:
comparison
{$.body.body.guidProperty=Expected 'e2490de5-5bd3-43d5-b7c4-526e33f71304' but received 'aff876f5-5014-937c-6855-c099f9857437'
Looking at the v3 spec the new message seems to be valid, does the old provider library (v3.2.13) not support it? I looked through the code and found this commit which seems to me where the change was introduced.
From my testing the new provider library (3.5.12) can handle both old and new formats, but if both the new provider and old consumer libraries are present in the classpath http contract tests fail with run-time errors.
Questions:
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?
As J_A_X pointed out, it looks like the commit you referenced fixed a bug where pacts that were supposed to be version 3 were not fully version 3 compliant (they used the old matchers format)
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
Yes and yes. You should be able to fix this by setting the system property pact.provider.version to 2 - then both the old and the new versions will be able to read the generated pact.
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?
Yes, as long as you ask the old consumer version to generate version 2 pacts (but ideally, why not update both to the same version?)
I have a simple DSC Config file that contains a credential and string input parameter. I want this DSC configuration deployed with a VM deployed in an ARM template but am missing the concept of how to pass these two parameters securely. How do I accomplish this?
I was receiving the same error but, after some shenanigans, it is working for me. The important part is the settings/Properties/SqlAgentCred/password reference to protectedSettings/Items/AgentPassword. Below is the properties node under my Powershell.DSC extension resource in my template.
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.17",
"autoUpgradeMinorVersion": false,
"settings": {
"ModulesUrl": "https://blobstore.blob.core.windows.net/windows-powershell-dsc/DBServer.ps1.zip",
"ConfigurationFunction": "DBServer.ps1\\DBServer",
"Properties": {
"SqlAgentCred": {
"userName": "user#domain.com",
"password": "PrivateSettingsRef:AgentPassword"
}
},
"WmfVersion": "latest",
"Privacy": {
"DataCollection": "Disable"
}
},
"protectedSettings": {
"Items": {
"AgentPassword": "Pa$$word"
},
"DataBlobUri": ""
}
}
You will specify protected settings under protectedsettings section. Anything under ProtectedSettings are sent encrypted. Check https://blogs.msdn.microsoft.com/powershell/2016/02/26/arm-dsc-extension-settings/ for details.