How to filter for a whole dictionary by a specific value inside it via JSONPath? - jsonpath

All I (just) leardned about JSONPath is that I can filter/select for parts of a JSON string by giving key names or array indices etc.
The result is that key/value I gave in the JSONPath expression.
However, I want to get, as a result, the whole dictionary where inside of it is a key/value pair I want to filter for – not only that key/value pair.
For example, imagine the following JSON string:
[
{
„date“: „2021-06-18T10:23:45.000+0000“,
„material“: {
„key1“: „value1“,
„key2“: „value2“
},
„preset“: {
„duration“: [ 120, 180 ],
„increments“: 3,
„id“: „16“,
„uniqueId“: „1357924680“
},
„userId“: 23456
},
{
„date“: „2021-06-18T08:15:45.000+0000“,
„material“: {
„key1“: „value1“,
„key2“: „value2“
},
„preset“: {
„duration“: [ 180 ],
„increments“: 1,
„id“: „10“,
„uniqueId“: „1797924680“
},
„userId“: 23456
},
{
„date“: „2021-06-16T17:12:23.000+0000“,
„material“: {
„key1“: „value1“,
„key2“: „value2“
},
„preset“: {
„duration“: [ 240 ],
„increments“: 1,
„id“: „10“,
„uniqueId“: „1357944680“
},
„userId“: 23456
},
{
„date“: „2021-06-09T08:45:45.000+0000“,
„material“: {
„key1“: „value1“,
„key2“: „value2“
},
„preset“: {
„duration“: [ 120, 180 ],
„increments“: 3,
„id“: „16“,
„uniqueId“: „2357924680“
},
„userId“: 23456
}
]
Now suppose I want to get those array elements (completely, with all their data) where inside the „preset“ named dictionary the value of „id“ is equal to „16“. What‘s the JSONPath query string for this?
Remember: I don‘t want to get just { „id“: „16“, „id“: „16“ } as a result but instead the whole dictionary with both results in this case:
{
„date“: „2021-06-18T10:23:45.000+0000“,
„material“: {
„key1“: „value1“,
„key2“: „value2“
},
„preset“: {
„duration“: [ 120, 180 ],
„increments“: 3,
„id“: „16“,
„uniqueId“: „1357924680“
},
„userId“: 23456
},
{
„date“: „2021-06-09T08:45:45.000+0000“,
„material“: {
„key1“: „value1“,
„key2“: „value2“
},
„preset“: {
„duration“: [ 120, 180 ],
„increments“: 3,
„id“: „16“,
„uniqueId“: „2357924680“
},
„userId“: 23456
}
And I don‘t want to do this inside some coding project/a program/source code in any programming language. Instead I have a REST API tool where I can filter JSON responses with a JSONPath query string (it‘s the app „HTTPBot“ for i[Pad]OS).
Here are a few JSONPath examples I tried but don‘t give the result I want:
$..preset – returns all the preset dicts (no filtering for those with a certain id value)
$..preset.id – returns all values of all id keys in all preset dicts (again, no filtering for a certain id)
$..preset[?(#.id==„16“)] – this is what I thought should do the trick but in HTTPBot this just returns the same as $..preset (maybe I should ask the author of HTTPBot if he implemented JSONPath completely?! 😳)

$..preset is going to perform a recursive search for any property (at any depth) called preset and return the collection of all the values in those properties.
To get the items, you need to have a selector that acts on them. Since you want to filter the items, you're looking at the filter selector [?(<expr>)].
Within JSON Path expressions, you can use the # to look at each item as the selector iterates the array, and it supports building a path from it. We want the #.preset.id to be 16, so we can use
$[?(#.preset.id=='16')]

Related

Remove Quotation from HTTP reponse(string datatype) to pass in post API

I have an API returning data(string format) like this
84, 101, 115, 116
But when I call this in power automate the response I get thru HTTP post is with quotation marks
"84, 101, 115, 116"
Is there any way I can get it without quotation mark " ", As I need the output to send in another API which fails if I send it
with "".
The reason being it needs to pass the response to another API that accepts byte array
like this [84, 101, 115, 116]
currently, inflow it is being sent as ["84, 101, 115, 116"] which fails the API
I tried replace function but it doesn't work for my case.
I think you need to split up your texts into an array.
Start with initialising three variables as shown below ...
... as you can see, I initialised a string variable with your values, it doesn't show the quotes but it's declared as a string so when it treats it, the quotes will be there.
In the next steps, I initialise an array variable that has the following expression ...
split(replace(variables('Response'), ' ', ''), ',')
... and then a blank array that will store the resulting number values.
After the above, loop through each value that was found and add it to the number array, this will convert the value as a string to a numeric value.
The expression above is ... int(item())
That then produces this result and it's that array that can then be passed through to your next API call.
This is the JOSN definition that you can load into your own tenant to see the answer in its entirety.
{
"definition": {
"$schema": "https://schema.management.azure.com/providers/Microsoft.Logic/schemas/2016-06-01/workflowdefinition.json#",
"actions": {
"For_each": {
"actions": {
"Append_to_array_variable": {
"inputs": {
"name": "Number Array",
"value": "#int(item())"
},
"runAfter": {},
"type": "AppendToArrayVariable"
}
},
"foreach": "#variables('String Array')",
"runAfter": {
"Initialize_Number_Array": [
"Succeeded"
]
},
"runtimeConfiguration": {
"concurrency": {
"repetitions": 1
}
},
"type": "Foreach"
},
"Initialize_Number_Array": {
"inputs": {
"variables": [
{
"name": "Number Array",
"type": "array"
}
]
},
"runAfter": {
"Initialize_String_Array": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Initialize_String_Array": {
"inputs": {
"variables": [
{
"name": "String Array",
"type": "array",
"value": "#split(replace(variables('Response'), ' ', ''), ',')"
}
]
},
"runAfter": {
"Initialize__Response": [
"Succeeded"
]
},
"type": "InitializeVariable"
},
"Initialize__Response": {
"inputs": {
"variables": [
{
"name": "Response",
"type": "string",
"value": "84, 101, 115, 116"
}
]
},
"runAfter": {},
"type": "InitializeVariable"
},
"Initialize_variable": {
"inputs": {
"variables": [
{
"name": "Result",
"type": "array",
"value": "#variables('Number Array')"
}
]
},
"runAfter": {
"For_each": [
"Succeeded"
]
},
"type": "InitializeVariable"
}
},
"contentVersion": "1.0.0.0",
"outputs": {},
"parameters": {},
"triggers": {
"Recurrence": {
"evaluatedRecurrence": {
"frequency": "Month",
"interval": 12
},
"recurrence": {
"frequency": "Month",
"interval": 12
},
"type": "Recurrence"
}
}
},
"parameters": {}
}

Map to an array within an object within an array

I am still having trouble understanding how to use the map function. In this case my payload is a JSON object that contains an array of "orders" with each "order" being an object... How do I create a map that would let me get to the array of "ContactEmailAddresses"?
{
"orders": [
{
"OrderGroupNumber": 1,
"Requester": {
"Name": "Mickey Mouse"
},
"ContactEmailAddresses": [
"user1#abc.com",
"user2#abc.com"
],
"CreatedByEmailAddress": "user1#abc.com"
},
{
"OrderGroupNumber": 2,
"Requester": {
"Name": "Donald Duck"
},
"ContactEmailAddresses": [
"user3#abc.com",
"user4#abc.com"
],
"CreatedByEmailAddress": "user3#abc.com"
},
{
"OrderGroupNumber": 3,
"Requester": {
"Name": "Goofy"
},
"ContactEmailAddresses": [
"user5#abc.com",
"user6#abc.com"
]
}
]
}
My current attempt that doesn't work is:
payload.*orders map (order, index) ->
{
order.contactEmailAddresses
}
%dw 2.0
output application/json
---
payload.orders flatMap $.ContactEmailAddresses
Output:
[
"user1#abc.com",
"user2#abc.com",
"user3#abc.com",
"user4#abc.com",
"user5#abc.com",
"user6#abc.com"
]

CosmosDB, help flatten and filter by nested array

I'm trying to flatten and filter my json data that is in a CosmosDB.
The data looks like below and I would like to flatten everything in the array Variables and then filter by specific _id and Timestamp inside of the array:
{
"_id": 21032,
"FirstConnected": {
"$date": 1522835868346
},
"LastUpdated": {
"$date": 1523360279908
},
"Variables": [
{
"_id": 99999,
"Values": [
{
"Timestamp": {
"$date": 1522835868347
},
"Value": 1
}
]
},
{
"_id": 99998,
"Values": [
{
"Timestamp": {
"$date": 1523270312001
},
"Value": 8888
}
]
}
]
}
If you want to flatten data from the Variables array with properties from the root object you can query your collection like this:
SELECT root._id, root.FirstConnected, root.LastUpdated, var.Values
FROM root
JOIN var IN root.Variables
WHERE var._id = 99998
This will result into:
[
{
"_id": 21032,
"FirstConnected": {
"$date": 1522835868346
},
"LastUpdated": {
"$date": 1523360279908
},
"Values": [
{
"Timestamp": {
"$date": 1523270312001
},
"Value": 8888
}
]
}
]
If you want to even flatten the Values array you will need to write something like this:
SELECT root._id, root.FirstConnected, root.LastUpdated,
var.Values[0].Timestamp, var.Values[0]["Value"]
FROM root
JOIN var IN root.Variables
WHERE var._id = 99998
Note that CosmosDB considers "Value" as a reserved keyword and you need to use an escpape syntax. The result for this query is:
[
{
"_id": 21032,
"FirstConnected": {
"$date": 1522835868346
},
"LastUpdated": {
"$date": 1523360279908
},
"Timestamp": "1970-01-01T00:00:00Z",
"Value": 8888
}
]
Check for more details https://learn.microsoft.com/en-us/azure/cosmos-db/sql-api-sql-query#Advanced
If you're only looking for filtering by the nested '_id' property then you could use ARRAY_CONTAINS w/ the partial_match argument set to true. The query would look something like this:
SELECT VALUE c
FROM c
WHERE ARRAY_CONTAINS(c.Variables, {_id: 99998}, true)
If you also want to flatten the array, then you could use JOIN
SELECT VALUE v
FROM v IN c.Variables
WHERE v._id = 99998

API Platform filter entity data

I just start to use Api platform and immediately stuck with problem how to filter data.
I have entity User and i want to filter data that are present in response ( JSON API format)
{
"links": {
"self": "/api/users"
},
"meta": {
"totalItems": 2,
"itemsPerPage": 30,
"currentPage": 1
},
"data": [
{
"id": "/api/users/1",
"type": "User",
"attributes": {
"_id": 1,
"username": "jonhdoe",
"isActive": true,
"address": null
}
},
{
"id": "/api/users/3",
"type": "User",
"attributes": {
"_id": 3,
"username": "test",
"isActive": true,
"address": null
}
}
]
}
so I want to remove e.g. User with id 3, but not use filters sent via request. I just want to set filter that will be always run when someone go to /api/users.
I look to api-platform extensions but this will be applied on each request e.g. /api/trucks. So at end I just want to get something like
{
"links": {
"self": "/api/users"
},
"meta": {
"totalItems": 1,
"itemsPerPage": 30,
"currentPage": 1
},
"data": [
{
"id": "/api/users/1",
"type": "User",
"attributes": {
"_id": 1,
"username": "jonhdoe",
"isActive": true,
"address": null
}
}
]
}
As you pointed out, extensions are the way to go.
The applyToCollection method gets a $resourceClass parameter containing the current resource class.
So you can apply the WHERE clause only for a specific class in this method like:
public function applyToCollection(QueryBuilder $queryBuilder, QueryNameGeneratorInterface $queryNameGenerator, string $resourceClass, string $operationName = null)
{
if (User::class === $resourceClass) {
$queryBuilder->doSomething();
}
// Do nothing for other classes
}

Exclude Path in Azure Cosmos DB

What is the correct JSON to exclude certain keys from an input json to be not indexed by Azure CosmosDB. We are using the CosmosDB in mongodb mode. Was planning to change the index configuration on the Azure Portal after creating the collection.
Sample Input Json being
{
"name": "test",
"age": 1,
"location": "l1",
"height":5.7
}
If I were to include name and age in the index and remove location and height from the index, what does the includedPaths and excludedPaths look like.
Finally got it to work with the below spec:-
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [{
"path": "/*",
"indexes": [{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Hash",
"dataType": "String",
"precision": 3
}
]
}],
"excludedPaths": [{
"path": "/\"location\"/?"
},
{
"path": "/\"height\"/?"
}
]
}
It looks like underlying implementation has changed and at the time of writing the documentation does not cover changing indexPolicy in MongoDB flavoured CosmosBD. Because the documents are really stored in a wired way where all the keys start from root $v and all scalar fields are stored as documents, containing value and type information. So your document will be stored something like:
{
'_etag': '"2f00T0da-0000-0d00-0000-4cd987940000"',
'id': 'SDSDFASDFASFAFASDFASDF',
'_self': 'dbs/sMsxAA==/colls/dVsxAI8MBXI=/docs/sMsxAI8MBXIKAAAAAAAAAA==/',
'_rid': 'sMsxAI8MBXIKAAAAAAAAAA==',
'$t': 3,
'_attachments': 'attachments/',
'$v': {
'_id': {
'$t': 7,
'$v': "\\Ù\x87\x14\x01\x15['\x18m\x01ú"
},
'name': {
'$t': 2,
'$v': 'test'
},
'age': {
'$t': 1,
'$v': 1
},
...
},
'_ts': 1557759892
}
Therefore the indexingPolicy paths need to include the root $v and use /* (objects) instead of /? (scalars).
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number"
},
{
"kind": "Hash",
"dataType": "String"
}
]
}
],
"excludedPaths": [
{"path": "/\"$v\"/\"location\"/*"},
{"path": "/\"$v\"/\"height\"/*"}
]
}
PS:
Also mongo api can be used to drop all the default indexes and create specific indexes as required
https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb-indexing

Resources