Pact JVM update backwards compatibility - pact

I am trying to update dozens of services to au.com.dius:pact-jvm-consumer-junit_2.12:3.5.12 from au.com.dius:pact-jvm-consumer-junit_2.11:3.2.13 but it seems that the new consumer version is generating pacts that the old provider version (au.com.dius:pact-jvm-provider-junit_2.11:3.2.13) cannot handle.
The old pact had a hashmap with matching rules being added at the root as show below
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodvider-amqp"
},
"messages": [
{
"description": "amqp contract",
"contents": {
"body": {
"guidProperty": "795ecfd5-a3a5-430f-a0cd-1569df61bff6"
}
},
"matchingRules": {
"$.body.body.guidProperty": {
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.2.13"
}
}
}
The new consumer adds and wrapper body around the matchers. Below is an example of the same pact generated with the new consumer version
{
"consumer": {
"name": "consumer-amqp"
},
"provider": {
"name": "prodiver-amqp"
},
"messages": [
{
"description": "contract",
"contents": {
"body": {
"guidProperty": "e2490de5-5bd3-43d5-b7c4-526e33f71304"
}
},
"matchingRules": {
"body": {
"$.guidProperty": {
"matchers": [
{
"match": "regex",
"regex": "[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}"
}
],
"combine": "AND"
}
}
}
}
],
"metadata": {
"pact-specification": {
"version": "3.0.0"
},
"pact-jvm": {
"version": "3.5.12"
}
}
}
Because of that change the provider fails to parse the matching rules with the following error:
body
^
10:21:24.526 [main] DEBUG au.com.dius.pact.matchers.JsonBodyMatcher - compareValues: No matcher defined for path List($, body, body, guidProperty), using equality
10:21:24.527 [main] WARN au.com.dius.pact.matchers.Matchers$ - Path expression body is invalid, ignoring: [1.1] failure: `$' expected but `b' found
java.lang.AssertionError:
comparison
{$.body.body.guidProperty=Expected 'e2490de5-5bd3-43d5-b7c4-526e33f71304' but received 'aff876f5-5014-937c-6855-c099f9857437'
Looking at the v3 spec the new message seems to be valid, does the old provider library (v3.2.13) not support it? I looked through the code and found this commit which seems to me where the change was introduced.
From my testing the new provider library (3.5.12) can handle both old and new formats, but if both the new provider and old consumer libraries are present in the classpath http contract tests fail with run-time errors.
Questions:
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?

As J_A_X pointed out, it looks like the commit you referenced fixed a bug where pacts that were supposed to be version 3 were not fully version 3 compliant (they used the old matchers format)
1) Is there a way to force the new consumer to create the pacts the old way, and is that form spec compliant?
Yes and yes. You should be able to fix this by setting the system property pact.provider.version to 2 - then both the old and the new versions will be able to read the generated pact.
2) Is there a way to update the provider to the new version and still have the old consumer library in the path and not get failures?
Yes, as long as you ask the old consumer version to generate version 2 pacts (but ideally, why not update both to the same version?)

Related

Using complex structures with AddEnvironmentVariables

Question
How to handle complex structures, such as arrays, when using the ConfigurationBuilder's AddEnvironmentVariables approach for setting config.
i.e. What would the environment variables be called to represent the below structure?
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Detailed version with context
We're writing apps that will be hosted in containers. Initially we'd continued to use the AppSettings.json file to hold settings, populating this file's contents from environment variables at startup via an entrypoint.sh shell script.
We later realised that we could use the ConfigurationBuilder's AddEnvironmentVariables method to pull the values directly from the environment variables, and thus avoid the overhead of maintaining the entrypoint.sh script as being another place to update each time a new setting was added.
This works well for an app settings file such as:
{
"MyApp": {
"SmtpServer": "smtp.example.com",
"FromAddress": "JoeyBlogger#example.com"
},
"Logging": {
"LogLevel": "Default"
}
}
.. as this translates simply to:
export MyApp:SmtpServer=smtp.example.com
export MyApp:FromAddress=JoeyBlogger#example.com
export Logging:LogLevel=Default
However, I recently saw a developer's app settings file for some new monitoring, which looks like the below (note; this won't be the actual settings used in our hosted environments; it's just an example which triggered this question). In this case, using the above approach doesn't work so well, since there would be two different settings with the name Serilog:WriteTo:Name, and ambiguity over which entry anything starting Serilog:WriteTo:Args:* relates to.
{
"Serilog"
"WriteTo": [
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "File",
"Args": {
"path": ".\\log.txt",
"rollingInterval": "Day",
"retainedFileCountLimit": 7,
"buffered": true
}
}
]
}
},
{
"Name": "Async",
"Args": {
"configure": [
{
"Name": "Console"
}
]
}
}
]
}
}
My guess is we'd handle this via some sort of index; but I've not been able to find anything in the docs about this.
export Serilog:WriteTo[0]:Name=Async
export Serilog:WriteTo[0]:Args:configure[0]:Name=File
export Serilog:WriteTo[0]:Args:configure[0]:Args:path=.\\log.txt
export Serilog:WriteTo[0]:Args:configure[0]:Args:rollingInterval=Day
export Serilog:WriteTo[0]:Args:configure[0]:Args:retainedFileCountLimit=7
export Serilog:WriteTo[0]:Args:configure[0]:Args:buffered=true
export Serilog:WriteTo[1]:Name=Async
export Serilog:WriteTo[1]:Args:configure[0]:Name=Console
However, running a quick PoC shows that the above doesn't work. How can this be done?
I found the solution here: MS Extensions Configuration Deep Dive.
The answer is that indexes are used, but the index number is treated like a named element; i.e.
{
"MyApp": {
"SendAlertsTo": [
{"Name": "Joe Blogger", "Email": "JoeBlogger#example.com"},
{"Name": "Jane Doer", "Email": "JaneDoer#example.com"}
]
}
}
Becomes:
export MyApp:SendAlertsTo:0:Name=Joe Blogger
export MyApp:SendAlertsTo:0:Email=JoeBlogger#example.com
export MyApp:SendAlertsTo:1:Name=Jane Doer
export MyApp:SendAlertsTo:1:Email=JaneDoer#example.com
Additionally, for Linux hosting there's another point I'd not been aware of / unrelated to the array question.. the : separator should be __, so the above becomes:
export MyApp__SendAlertsTo__0__Name=Joe Blogger
export MyApp__SendAlertsTo__0__Email=JoeBlogger#example.com
export MyApp__SendAlertsTo__1__Name=Jane Doer
export MyApp__SendAlertsTo__1__Email=JaneDoer#example.com

Get CosmosDb Primary Connection String in ARM template

I have an ARM template which sources the primaryMasterKey of a cosmosDb as follows:
{
"properties": {
"enabled": true,
"siteConfig": {
"appSettings": [
{
"name": "MongoDb:CnnDetails",
"value": "[listKeys(resourceId('Microsoft.DocumentDB/databaseAccounts', variables('cosmosdb_full')), '2015-04-08').primaryMasterKey]"
}
}
How to I modify it to get the actual connection string instead?
I've tried couple of things:
changed the word primaryMasterKey to primaryConnectionString. This gives an error saying:
'The language expression property 'primaryConnectionString' doesn't exist, available properties are 'primaryMasterKey, secondaryMasterKey, primaryReadonlyMasterKey, secondaryReadonlyMasterKey'
changed the work listKeys to listConnectionStrings. This is red underlined in my visual studio, but seems to work when put through azure devops
'The language expression property 'primaryConnectionString' doesn't exist, available properties are 'connectionStrings'
I went to https://learn.microsoft.com/en-us/rest/api/cosmos-db-resource-provider/databaseaccounts/listconnectionstrings#code-try-0 to try it out. ListKeys returns a structure like this:
{
"primaryMasterKey": "[REDACTED]",
"secondaryMasterKey": "[REDACTED]",
"primaryReadonlyMasterKey": "[REDACTED]",
"secondaryReadonlyMasterKey": "[REDACTED]"
}
so I get why the .primaryMasterKey worked. But ListConnectionStrings returns:
{
"connectionStrings": [
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Primary MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Secondary MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Primary Read-Only MongoDB Connection String"
},
{
"connectionString": "mongodb://[REDACTED]:10255/?ssl=true&replicaSet=globaldb",
"description": "Secondary Read-Only MongoDB Connection String"
}
]
}
Not sure how to "index into it"?
Any clues gratefully received.
For anyone else finding this question and wanting a fully complete ARM Template snippet, this is what I have used and is working:
"connectionStrings": [
{
"name": "CosmosConnection",
"connectionString": "[listConnectionStrings(resourceId('Microsoft.DocumentDB/databaseAccounts', parameters('cosmosDbAccountName')), '2019-12-12').connectionStrings[0].connectionString]",
"type": 3
}
]
like you normally would in almost any language:
ListConnectionStrings.connectionStrings[index].connectionString
index starts at 0.
you have a more "native" way of doing this:
first(ListConnectionStrings.connectionStrings).connectionString
but only available functions are first and last
The answer here by oatsoda is correct but it will only work if you are within the same resource group as the Cosmos DB you are getting the connection string for. If you have the scenario where you Cosmos DB is in a different resource group to the resource you are generating an ARM template for the following snippet is what I have used to generate the connection string for an App Service and is working.
"Cosmos": {
"value": "[listConnectionStrings(resourceId(parameters('cosmosResourceGroupName'),'Microsoft.DocumentDB/databaseAccounts', parameters('cosmosDbName')), '2019-12-12').connectionStrings[0].connectionString]",
"type": "Custom"
}
In the Cosmos linked ARM template named linkedTemplate_cosmos_db-gdp-event-ammi-dev-ne-001 I used the following code.
"outputs": {
"ConnectionString": {
"value": "[listConnectionStrings(resourceId('Microsoft.DocumentDB/databaseAccounts', parameters('accountName')), '2019-12-12').connectionStrings[0].connectionString]",
"name": "CosmosConnection",
"type": "string"
}
},
and then in the ARM template (linkedTemplate_Main) that uses the output parameter, the following, e.g a function app configuration setting
"COSMOS_CONNECTION_STRING": {
"value": "[reference('linkedTemplate_cosmos_db-gdp-event-ammi-dev-ne-001').outputs.ConnectionString.value]"

Is it possible to filter pact broker overview page/verification overview by tag?

The pact broker overview page (the index page) displays the verification status of each pact which is very useful. Is it possible to filter this view, by a query parameter or such, so that I can see the verifications for a certain tag? I would like to use this to have an overview of all the prod-tagged pacts and their verifications to make sure that all prod-pacts have been verified when making releases of components.
If not I'm contemplating having a separate pact broker instance for the prod pacts.
Thanks!
It's totally doable. It's just a matter me (or someone else) having the time to do it. I've raised an issue for it here:
https://github.com/pact-foundation/pact_broker/issues/146
In the meantime, you can make your own "dashboard" by making a CI build for each prod tag. I think I may have put this on the gitter channel already.
Call /pacts/provider/PROVIDER/consumer/CONSUMER/latest/prod. Follow the pb:consumer-version relation, then follow the pb:latest-verification-results-where-pacticipant-is-consumer relation.
You'll get this.
{
"success": true,
"providerSummary": {
"successful": [
"B"
],
"failed": [],
"unknown": []
},
"_embedded": {
"verificationResults": [
{
"providerName": "B",
"providerApplicationVersion": "2",
"success": true,
"verificationDate": "2017-10-09T12:34:17+08:00",
"_links": {
"self": {
"title": "Verification result",
"name": "Verification result 2 for Pact between A (v1) and B",
"href": "http://localhost:9292/pacts/provider/B/consumer/A/pact-version/7954fb780aa0db0ea451db4c3c1012fb7f0e1eb4/verification-results/2"
},
"pb:pact-version": {
"title": "Pact",
"name": "Pact between A (v1) and B",
"href": "http://localhost:9292/pacts/provider/B/consumer/A/version/1"
}
}
}
]
},
"_links": {
"self": {
"href": "http://localhost:9292/verification-results/consumer/A/version/1/latest",
"title": "Latest verification results for consumer A version 1"
}
}
}
I'll try and prioritise adding the tagged pacts to the UI.

Pact verify provider, what does Pact::UnexpectedIndex mean?

I am using Pact for Consumer Driven Contracts testing.
In my usecase, my consumer "some-market-service-consumer" is using provider "market-service". The contract "produced" at some-market-service-consumer, looks like this:
{
"provider": {
"name": "market-service"
},
"consumer": {
"name": "some-market-service-consumer"
},
"interactions": [
{
"description": "Get all markets",
"request": {
"method": "GET",
"path": "/markets"
},
"response": {
"status": 200,
"headers": {
"Content-Type": "application/json; charset=utf-8"
},
"body": {
"markets": [
{
"phone": "HBiQOAeQegaWtbcHfAro"
}
]
},
"matchingRules": {
"$.headers.Content-Type": {
"regex": "application/json; charset=utf-8"
},
"$.body.markets[*].phone": {
"match": "type"
},
"$.body.markets": {
"min": 1,
"match": "type"
}
}
}
}
],
"metadata": {
"pact-specification": {
"version": "2.0.0"
},
"pact-jvm": {
"version": "3.3.6"
}
}
}
On provider-site, I am using pact-provider-verifier-docker¹. Here is my test-result:
WARN: Ignoring unsupported matching rules {"min"=>1} for path $['body']['markets']
.....
## -1,7 +1,7 ##
{
"markets": [
... ,
- Pact::UnexpectedIndex,
+ Hash,
]
}
Key: - means "expected, but was not found".
+ means "actual, should not be found".
Values where the expected matches the actual are not shown.
It seems, as if the testing works fine - "phone" is tested valid.
But at this point, I have no clue, what (expected) "Pact::UnexpectedIndex" means, and why it fails. Where does this expectation come from, how to fix it?
¹ In special, my own version, which uses most recent external dependencies.
As you can see in this test case here the Pact::UnexpectedIndex is used to indicate than an array is longer than was expected. I think what it's trying to say is that there is an extra hash at the end of the array. (I agree that it is not clear at all! I wrote this code, so I apologise for its confusing nature. It turns out that the hardest part of writing pact code was writing helpful diff output.)
What is confusing about that error is that it should be allowing you to have extra elements as you've specified $.body.markets to have a minimum length of 1. Perhaps the docker verifier is using version 1 matching instead of version 2?

Passing Parameter Values to DSC Configuration from ARM Template

I have a simple DSC Config file that contains a credential and string input parameter. I want this DSC configuration deployed with a VM deployed in an ARM template but am missing the concept of how to pass these two parameters securely. How do I accomplish this?
I was receiving the same error but, after some shenanigans, it is working for me. The important part is the settings/Properties/SqlAgentCred/password reference to protectedSettings/Items/AgentPassword. Below is the properties node under my Powershell.DSC extension resource in my template.
"properties": {
"publisher": "Microsoft.Powershell",
"type": "DSC",
"typeHandlerVersion": "2.17",
"autoUpgradeMinorVersion": false,
"settings": {
"ModulesUrl": "https://blobstore.blob.core.windows.net/windows-powershell-dsc/DBServer.ps1.zip",
"ConfigurationFunction": "DBServer.ps1\\DBServer",
"Properties": {
"SqlAgentCred": {
"userName": "user#domain.com",
"password": "PrivateSettingsRef:AgentPassword"
}
},
"WmfVersion": "latest",
"Privacy": {
"DataCollection": "Disable"
}
},
"protectedSettings": {
"Items": {
"AgentPassword": "Pa$$word"
},
"DataBlobUri": ""
}
}
You will specify protected settings under protectedsettings section. Anything under ProtectedSettings are sent encrypted. Check https://blogs.msdn.microsoft.com/powershell/2016/02/26/arm-dsc-extension-settings/ for details.

Resources