Logic App > Cosmos PartitionKey Not Matching Error - azure-cosmosdb

I'm scared to put this out there because it should be so easy and I am facing the same issue as the post here, here and here and I have tried each of the answers to no avail. Below is the current Resulting Input (redacted) and Related CodeView of the inputs.
The Result
{
"method": "post",
"headers": {
"x-ms-documentdb-raw-partitionkey": "\"2020\""
},
"path": "/dbs/xxxx/colls/smtp/docs",
"host": {
"connection": {
"name": "/subscriptions/..."
}
},
"body": {
"category": [
[
"cat facts"
]
],
"email": "example#test.com",
"event": "processed",
"id": "yada",
"partitionKey": "\"2020\"",
"sg_event_id": "yada yada",
"sg_message_id": "yada",
"smtp-id": "yada",
"timestamp": 1604345542
}
}
The Code View
{
"inputs": {
"body": {
"category": [
"#items('For_each')['category']"
],
"email": "#items('For_each')['email']",
"event": "#items('For_each')['event']",
"id": "#items('For_each')['sg_message_id']",
"partitionKey": "\"#{formatDateTime(utcNow(),'yyyy')}\"",
"sg_event_id": "#items('For_each')['sg_event_id']",
"sg_message_id": "#items('For_each')['sg_message_id']",
"smtp-id": "#items('For_each')['smtp-id']",
"timestamp": "#items('For_each')['timestamp']"
},
"headers": {
"x-ms-documentdb-raw-partitionkey": "\"#{formatDateTime(utcNow(),'yyyy')}\""
}
}
The error I'm getting is the usual one - PartitionKey extracted from document doesn't match the one specified in the header
I just can't see what I'm missing here now.
Thanks all.

First, as Matias comments, check your partition key path.
Then, change this code "partitionKey": "\"#{formatDateTime(utcNow(),'yyyy')}\"", to "partitionKey": "#{formatDateTime(utcNow(),'yyyy')}", in your document.
It works fine on my side:

Related

The language expression property '0' can't be evaluated, property name must be a string - ARM Template error while adding Key Vault access policy

I've been working on an issue and seem to be stuck, so asking on so in case anyone can help.
To describe the issue, I've got an existing Azure Key Vault setup, and wish to add a number of access policies to this resource group. It needs to be conditional as if the function name is "false" then that function should not be added to key vault access policy.
variable section:
"variables": {
"functionAccess": {
"value": [
{
"name": "[parameters('Function_1')]"
},
{
"name": "[parameters('Function_2')]"
},
{
"name": "[parameters('Function_3')]"
}
]
}
}
My Template :
{
"apiVersion": "2016-10-01",
"condition": "[not(equals(variables('functionAccess')[CopyIndex()].name, 'false'))]",
"copy": {
"batchSize": 1,
"count": "[length(variables('functionAccess'))]",
"mode": "Serial",
"name": "accessPolicies"
},
"name": "[concat(parameters('KeyVault_Name'), '/add')]",
"properties": {
"accessPolicies": [
{
"tenantId": "[subscription().tenantId]",
"objectId": "[if(not(equals(variables('functionAccess')[CopyIndex()].name, 'false')), reference(concat('Microsoft.Web/sites/', variables('functionAccess')[CopyIndex()].name), '2016-08-01', 'Full').identity.principalId, json('null'))]",
"permissions": {
"keys": [
"get",
"list"
],
"secrets": [
"get",
"list"
],
"certificates": [
"get",
"list"
]
}
}
]
},
"type": "Microsoft.KeyVault/vaults/accessPolicies"
}
When I deploy my ARM template for the azure key vault I got this error message:
The language expression property '0' can't be evaluated, property name must be a string.
also tried below, but same error:
{
"apiVersion": "2018-02-14",
"name": "[concat(parameters('KeyVault_Name'), '/add')]",
"properties": {
"copy": [
{
"batchSize": 1,
"count": "[length(variables('functionAccess'))]",
"mode": "serial",
"name": "accessPolicies",
"input": {
"condition": "[not(equals(variables('functionAccess')[copyIndex('accessPolicies')].name, 'false'))]",
"tenantId": "[subscription().tenantId]",
"objectId": "[if(not(equals(variables('functionAccess')[copyIndex('accessPolicies')].name, 'false')), reference(concat('Microsoft.Web/sites/', variables('functionAccess')[copyIndex('accessPolicies')].name), '2016-08-01', 'Full').identity.principalId, json('null'))]",
"permissions": {
"keys": [
"get",
"list"
],
"secrets": [
"get",
"list"
],
"certificates": [
"get",
"list"
]
}
}
}
]
},
"type": "Microsoft.KeyVault/vaults/accessPolicies"
}
There are a few options for dealing with filtering an array for copy operation. I deploy my ARM templates from PowerShell scripts and use PowerShell to setup parameter values. When I need special logic handle different inputs for different environments, I let PowerShell handle it.
If you must handle the filtering in ARM and you have the option to input a CSV list of functions, then perhaps the following will work. You can then use the functionAccessArray to iterate over in the copy operation.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"functionAccessCsv": "Function-0,Function-1,false,Function-4,false,Function-6,Function-7",
"functionAccessFiltered": "[replace(replace(variables('functionAccessCsv'), 'false', ''), ',,', ',')]",
"functionAccessArray": "[split(variables('functionAccessFiltered'), ',')]"
},
"resources": [
],
"outputs": {
"functionAccessCsvFiltered": {
"type": "string",
"value": "[variables('functionAccessFiltered')]"
},
"functionAccessArray": {
"type": "array",
"value": "[variables('functionAccessArray')]"
}
}
}
The result:
I just had the same issue. By using an array parameter with a default value instead of a variable, I got it to work.
"parameters": {
"functionAccess": {
"type": "array",
"defaultValue": [
"value1",
"value2",
"value3"
]
}
}

Logic App workflow creating or updating documents in CosmosDB fails with conflict

I'm in the process of creating a Logic App which should copy all documents from specific containers in the database to a different database and its corresponding collections.
For this, I've created a workflow which retrieves all documents, loops through them and tries to do a Create or Update document on the target location.
As you can see on the image, this looks fairly straightforward.
I've specified the PartitionKey in the header and Upsert is set to true, so if a document already exists it'll be updated, otherwise created.
Originally, the body was filled with #items('[blurred]_-_For_each_document').
However, I got an error stating there was a conflict on the id.
I've also tried to remove all 'system' properties like so:
#removeProperty(removeProperty(removeProperty(removeProperty(removeProperty(items('[blurred]_-_For_each_document'), 'id'), '_rid'), '_self'), '_ts'), '_etag')
but it appears not having an id in place isn't valid, so now I've got this as I'm only interested in having the contents of the 'actual' document:
#removeProperty(removeProperty(removeProperty(removeProperty(items('[blurred]_-_For_each_document'), '_rid'), '_self'), '_ts'), '_etag')
This still fails.
The raw input looks a bit like this:
{
"method": "post",
"headers": {
"x-ms-documentdb-raw-partitionkey": "1050",
"x-ms-documentdb-is-upsert": "True"
},
"path": "/dbs/[myDatabase]/colls/[myCollection]/docs",
"host": {
"connection": {
"name": "/subscriptions/[subscriptionGuid]/resourceGroups/[resourcegroup]/providers/Microsoft.Web/connections/[connectionName]"
}
},
"body": {
"id": "2faee185-4a51-4797-bff9-3ce23a603690",
"MyPartitionKeyNumber": 1050,
"SomeValue": false,
}
}
With the following response:
{
"code": "Conflict",
"message": "Entity with the specified id already exists in the system., ... ResourceType: Document, OperationType: Upsert ..."
}
I know for a fact, the id doesn't exist, but changed the step to do a PUT anyway.
{
"method": "put",
"headers": {
"x-ms-documentdb-raw-partitionkey": "1050",
"x-ms-documentdb-is-upsert": "True"
},
"path": "/dbs/[myDatabase]/colls/[myCollection]/docs",
"host": {
"connection": {
"name": "/subscriptions/[subscriptionGuid]/resourceGroups/[resourcegroup]/providers/Microsoft.Web/connections/[connectionName]"
}
},
"body": {
"id": "2faee185-4a51-4797-bff9-3ce23a603690",
"MyPartitionKeyNumber": 1050,
"SomeValue": false,
}
}
With a response I'm expecting to see:
{
"statusCode": 404,
"message": "Resource not found"
}
I've also tried to do a Delete document before running the Create or Update document step, but got the same error(s).
There's some post here on Stack Overflow stating the x-ms-documentdb-raw-partitionkey should be x-ms-documentdb-partitionkey (without the raw-part), but doing so results in the following error message:
PartitionKey extracted from document doesn't match the one specified in the header
For completeness sake, these are the relevant steps of my workflow:
"[blurred]_-_Get_all_documents": {
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['documentdb_3']['connectionId']"
}
},
"method": "get",
"path": "/v2/dbs/#{encodeURIComponent('myDatabase')}/colls/#{encodeURIComponent(items('[blurred]_-_For_each_collection'))}/docs"
},
"runAfter": {}
},
"[blurred]_-_For_each_document": {
"type": "Foreach",
"foreach": "#body('[blurred]_-_Get_all_documents')?['value']",
"actions": {
"[blurred]_-_Create_or_Update_document_on_blurred2": {
"type": "ApiConnection",
"inputs": {
"host": {
"connection": {
"name": "#parameters('$connections')['documentdb_4']['connectionId']"
}
},
"method": "post",
"body": "#removeProperty(removeProperty(removeProperty(removeProperty(items('[blurred]_-_For_each_document'), '_rid'), '_self'), '_ts'), '_etag')",
"headers": {
"x-ms-documentdb-raw-partitionkey": "#{items('[blurred]_-_For_each_document')['MyPartitionKeyNumber']}",
"x-ms-documentdb-is-upsert": true
},
"path": "/dbs/#{encodeURIComponent('myDatabase')}/colls/#{encodeURIComponent(items('[blurred]_-_For_each_collection'))}/docs"
},
"runAfter": {}
}
},
"runAfter": {
"[blurred]_-_Get_all_documents": [
"Succeeded"
]
}
}

OneDrive API search based on a keyword gives only section information and doesnot give any information about page details

I want to know the page id or page url which has paternity in the content. When i use search functionality, i get the section details but i don't get any information about page.
Query:
https://graph.microsoft.com/v1.0/me/drive/root/search(q='paternity')
Result:
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#Collection(driveItem)",
"value": [
{
"#odata.type": "#microsoft.graph.driveItem",
"createdDateTime": "2018-07-11T11:06:28Z",
"id": "dfhsdfkhfklsdfsdkjf",
"lastModifiedDateTime": "2018-07-11T11:20:26Z",
"name": "Section1.one",
"webUrl": "https://microsoft-my.sharepoint.com/personal/abcd_contoso_com/_layouts/15/WopiFrame.aspx?sourcedoc=%7B7E1C4305-983D-4CE2-A15E-DBAF1B961423%7D&file=Section1.one&action=default&DefaultItemOpen=1",
"size": 390328,
"createdBy": {
"user": {
"email": "abcd#contoso.com",
"displayName": "abcd"
}
},
"lastModifiedBy": {
"user": {
"email": "abcd#contoso.com",
"displayName": "abcd"
}
},
"parentReference": {
"driveId": "b!QqRkFzhjsdgjkdhfkjdhXiDBfhDiNEmqz4NJGbg-Gcv-NrFDvVRJca8R9-3ylQ",
"driveType": "business",
"id": "01QsdjhdkjhdsdkjhHGT4FXINN2A"
},
"file": {
"mimeType": "application/msonenote"
},
"fileSystemInfo": {
"createdDateTime": "2018-07-11T11:06:28Z",
"lastModifiedDateTime": "2018-07-11T11:20:26Z"
},
"searchResult": {}
}
]
}
Please advise on how to get page level information
The OneNote API has search - you may try using that one:
https://blogs.msdn.microsoft.com/onenotedev/2014/11/17/introducing-the-onenote-search-api-beta-powered-by-bing/

Provide request header for Microsoft.Scheduler/jobCollections/jobs

I want to know how to provide from my Azure WebApp the user/password to provide header for my webjob
{
"name": "[concat('TraitementTableAzure-', parameters('HeliosEnvironnementName'), '-js')]",
"type": "jobs",
"apiVersion": "2016-03-01",
"location": "[resourceGroup().location]",
"properties": {
"action": {
"request": {
"method": "Post",
"uri": "[concat('https://', parameters('AzureWebAppWebJobs'), '.scm.azurewebsites.net/api/triggeredwebjobs/', parameters('HeliosEnvironnementName'), '_TraitementTableAzure/run')]",
"headers": {
"authorization": "[concat('Basic ', reference('???').???)]" }
},
"type": "Http",
"retryPolicy": {
"retryType": "Fixed"
}
},
"startTime": "[parameters('SchedulesStartTime').SchedulerTraitementTableAzureStartTime]",
"recurrence": {
"frequency": "Day",
"interval": 1
},
"state": "Enabled"
},
"dependsOn": [
"[resourceId('Microsoft.Scheduler/jobCollections', variables('AzureSchedulerName'))]"
],
"tags": {
"displayName": "Cedule_TraitementTableAzure"
}
},
I found information over Azure Portal but not in ARM Template under webjob Properties. How can reference information on the blue arrow over my ARM template ?
How can reference information on the blue arrow over my ARM template ?
If we want to get the publishingPassword, then we could use ListPublishingCredentials API in the ARM template via list function, list(concat('Microsoft.Web/sites/', parameters('websisteName') ,'/config/publishingcredentials'), '2016-08-01').properties.publishingPassword
According to your template, it seems the that you want to call WebJob REST API, If it is that case, the authorization header is base64(publishusername:publishpassword).
base64(concat(list(concat('Microsoft.Web/sites/', parameters('websisteName'),'/config/publishingcredentials'), '2016-08-01').properties.publishingUserName,':',list(concat('Microsoft.Web/sites/', parameters('websisteName') ,'/config/publishingcredentials'), '2016-08-01').properties.publishingPassword))
I write a demo to test it on my side, it works correctly .
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"websisteName": {
"type": "string"
}
},
"resources": [],
"outputs": {
"base64Output": {
"type": "string",
"value": "[base64(concat(list(concat('Microsoft.Web/sites/', parameters('websisteName'),'/config/publishingcredentials'), '2016-08-01').properties.publishingUserName,':',list(concat('Microsoft.Web/sites/', parameters('websisteName') ,'/config/publishingcredentials'), '2016-08-01').properties.publishingPassword))]"
}
}
}

copyIndex() inside a listKeys()

We're trying to deploy an ARM template which deploys a Stream Analytics job with n Event Hubs outputs depending on an input parameter.
Right now we're having success with all but the listKeys() function inside the outputs property copy loop function which gets each Event Hub's primary keys:
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[copyIndex('outputs')].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]"
We get the error:
17:44:31 - Error: Code=InvalidTemplate; Message=Deployment template
validation failed: 'The template resource
'tailor-router-axgf7t3gtspue' at line '129' and column '10' is not
valid: The template function 'copyIndex' is not expected at this
location. The function can only be used in a resource with copy
specified. Please see https://aka.ms/arm-copy for usage details..
Please see https://aka.ms/arm-template-expressions for usage
details.'.
However, if we change this to be a specific index:
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[0].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]"
it works.
Is copyIndex('propertyName') inside a listKeys() a supported function?
If not, is there a workaround that would achieve the same effect?
Kind regards,
Nick
Stream Analytics job resource definition:
{
"apiVersion": "2016-03-01",
"type": "Microsoft.StreamAnalytics/StreamingJobs",
"name": "[variables('routerStreamAnalyticsName')]",
"location": "[variables('location')]",
"dependsOn": [ "clientsEventHubCopy" ],
"tags": {
"boundedContext": "[variables('boundedContextName')]"
},
"properties": {
"sku": {
"name": "[parameters('routerStreamAnalyticsSkuTier')]"
},
"outputErrorPolicy": "drop",
"eventsOutOfOrderPolicy": "adjust",
"eventsOutOfOrderMaxDelayInSeconds": 0,
"eventsLateArrivalMaxDelayInSeconds": 5,
"dataLocale": "en-US",
"compatibilityLevel": "1.0",
"inputs": [
{
"name": "input0",
"properties": {
"type": "stream",
"serialization": {
"type": "Avro"
},
"datasource": {
"type": "Microsoft.ServiceBus/EventHub",
"properties": {
"serviceBusNamespace": "[parameters('input0EventHubNamespace')]",
"sharedAccessPolicyName": "[parameters('input0EventHubSharedAccessPolicyName')]",
"sharedAccessPolicyKey": "[parameters('input0EventHubSharedAccessPolicyKey')]",
"eventHubName": "[parameters('input0EventHubName')]"
}
}
}
}
],
"transformation": {
"name": "routing",
"properties": {
"streamingUnits": "[parameters('routerStreamAnalyticsSkuTier')]",
"query": "omitted"
}
},
"copy": [
{
"name": "outputs",
"count": "[length(parameters('clients'))]",
"input": {
"name": "[parameters('clients')[copyIndex('outputs')].id]",
"properties": {
"datasource": {
"type": "Microsoft.ServiceBus/EventHub",
"properties": {
"serviceBusNamespace": "[variables('clientEventHubNamespace')]",
"sharedAccessPolicyName": "[variables('clientEventHubClientSharedAccessName')]",
"sharedAccessPolicyKey": "[listKeys(resourceId('Microsoft.EventHub/namespaces/eventhubs/authorizationRules', variables('clientEventHubNamespace'), parameters('clients')[copyIndex('outputs')].id, variables('clientEventHubClientSharedAccessName')), '2015-08-01').primaryKey]",
"eventHubName": "[parameters('clients')[copyIndex('outputs')].id]"
}
},
"serialization": {
"type": "Avro"
}
}
}
}
]
}
},
Thanks for reporting this and sorry for the inconvenience.
I just talked to the ARM team, we had an issue when copyindex was inside the index tags eg 'array[copyindex()]'. It should be fixed now.
Let us know how it goes.
Thanks,
JS - Azure Stream Analytics

Resources