Simplify array of objects with children using JQ - jq

From Wikidata, I get the following json:
# Sparql query
query=$(cat ./myquery.sparql)
response=$(curl -G --data-urlencode query="${query}" https://wikidata.org/sparql?format=json)
echo "${response}" | jq '.results.bindings'
[
{
"language": {
"type": "uri",
"value": "https://lingualibre.org/entity/Q100"
},
"wikidata": {
"type": "literal",
"value": "Q36157"
},
"code": {
"type": "literal",
"value": "lub"
}
},
{
"language": {
"type": "uri",
"value": "https://lingualibre.org/entity/Q101"
},
"wikidata": {
"type": "literal",
"value": "Q36284"
},
"code": {
"type": "literal",
"value": "srr"
}
}
]
I would like to have the keys directly paired with their values, such as :
[
{
"language": "https://lingualibre.org/entity/Q100",
"wikidata": "Q36157",
"iso": "lub"
},
{
"language": "https://lingualibre.org/entity/Q101",
"wikidata": "Q36284",
"iso": "srr"
}
]
I currently have a non-resilient code, which will break whenever the key names change :
jq 'map({"language":.language.value,"wikidata":.wikidata.value,"iso":.code.value})'
How to pair the keys with their values in a resilient way (not naming the keys) ?
I want to "prune" the child objects so to only keep the value.

You could use map_values which works like the outer map but for objects, i.e. it retains the object structure, including the field names:
jq 'map(map_values(.value))'
[
{
"language": "https://lingualibre.org/entity/Q100",
"wikidata": "Q36157",
"code": "lub"
},
{
"language": "https://lingualibre.org/entity/Q101",
"wikidata": "Q36284",
"code": "srr"
}
]
Note that this solution lacks the name conversion from code to iso.

Related

ECS Task Definition container_definitions is invalid: Error decoding JSON: invalid character 'f' after object key:value pair

I´m getting: [Error] invalid character 'f' after object key:value pair How can I tell what is the (f) that is causing the error? is not quite descriptive the output like in some other languages that tells you the line number at least.
{
"name": "${name}",
"ulimits": [
{
"name": "nofile",
"hardLimit": 204800,
"softLimit": 102400
}
],
"image": "${ecr_url}/${image_name}:${image_tag}",
"cpu": "${container_cpu}",
"memory": null,
"memoryReservation": "${container_memory}",
"command": [
"/bin/sh",
"-c",
"export DD_AGENT_HOST=$(curl --silent http://169.254.169.254/latest/meta-data/local-ipv4); /usr/bin/$APPLICATION_ARTIFACTS"
],
"essential": true,
"mountPoints": [],
"portMappings": [
{
"containerPort": "${container_port}",
"hostPort": 0,
"protocol": "tcp"
}
],
"volumesFrom": [],
"dockerLabels": {
"application": "${application}",
"environment": "${environment}",
"version": "${image_tag}"
},
"logConfiguration": {
"logDriver": "${log_driver}",
"options": "${log_configuration}"
},
"environment": [
{
"name": "AWS_REGION",
"value": "${task_region}"
},
{
"name": "APPLICATION_ARTIFACTS",
"value": "${application_artifacts}"
},
{
"name": "ENVIRONMENT",
"value": "${environment}"
},
{
"name": "VERSION",
"value": "${image_tag}"
},
{
"name": "SERVER_PORT",
"value": "${container_port}"
},
{
"name": "DD_TRACE_SAMPLE_RATE",
"value": "1.0"
},
{
"name": "DD_LOGS_INJECTION",
"value": "false"
},
{
"name": "DD_ENV",
"value": "${DD_ENV}"
},
{
"name": "DD_SERVICE",
"value": "${DD_SERVICE}"
},
{
"name": "TABLE_NAME",
"value": "${table_name}"
}
]
}
I´m getting: [Error] invalid character 'f' after object key:value pair
How can I tell what is the (f) that is causing the error? is not quite descriptive the output like in some other languages that tells you the line number at least.
Error: ECS Task Definition container_definitions is invalid: Error decoding JSON: invalid character 'f' after object key:value pair
on deployment/terraform/service-blue.tf line 89, in resource "aws_ecs_task_definition" "default_blue":
89: container_definitions = local.require_blue_service ? data.aws_s3_object.container_definition_blue[0].body : data.template_file.container_definitions.rendered

How to use ObjectSizeGreaterThan and ObjectSizeLessThan xml tags for AWS S3 command 'PutBucketLifeCycleConfiguration"?

Trying with a json input like this:
{
"Rules": [
{
"Expiration": {
"Date": "2023-01-01T00:00:00.000Z"
},
"ID": "id1",
"Filter": {
"And": {
"Tags": [
{
"Key": "k1",
"Value": "k1val1"
},
{
"Key": "k1u",
"Value": "k1vadl1"
}
],
"ObjectSizeGreaterThan": 100,
"ObjectSizeLessThan": 1000
}
},
"Status": "Enabled"
}
]
}
However, I am getting an error like this from the aws client:
Parameter validation failed:
Unknown parameter in LifecycleConfiguration.Rules[0].Filter.And: "ObjectSizeGreaterThan", must be one of: Prefix, Tags
Any idea why I get this error and how to correct it?
This seems to be working but only with the latest cli version aws-cli/2.7.30
I am getting the same error on aws-cli/1.19.39
{
"Rules": [
{
"Expiration": {
"Date": "2024-01-01T00:00:00.000Z"
},
"ID": "123",
"Filter": {
"And": {
"Prefix": "sricli",
"Tags": [
{
"Key": "ke",
"Value": "k1"
},
{
"Key": "je",
"Value": "k2"
}
],
"ObjectSizeGreaterThan": 102400,
"ObjectSizeLessThan": 204800
}
},
"Status": "Enabled"
}
]
}
Sri

JSON path help required for a request

I am new at JSON path request and I have a quite complex request to build.
I work with a JSOn strusture having 2 arrays like this example :
{
"WideXml": {
"Guid": "9cf379c5-dc12-4a63-922a-d242efe9a777",
"ApplicationGuid": "24df8af4-58c2-40dd-8ce8-70becb2df96f",
"Action": "Approval",
"Values": {
"Date": {
"TimeStamp": "2021-11-23T04:00:00Z",
"Value": [{
"Guid": "9c64fb06-60f5-4541-a006-3a92ac576e13",
"Value": "6.7169265747070313",
"Unit": "t",
"UserFields": {
"Field": [{
"Value": "131",
"Key": "BWART"
},
{
"Value": "14702-00-BULK",
"Key": "MATNR"
}
]
}
},
{
"Guid": "6c048d70-1521-4fa1-a462-669730d6b1ed",
"Value": "84.824371337890625",
"Unit": "t",
"UserFields": {
"Field": [{
"Value": "261",
"Key": "BWART"
}, {
"Value": "14366-00-WA0R",
"Key": "MATNR"
}]
}
}
]
}
}
}
}
I need to find the "MATNR" code by searching with Key = 'BWART' and Value = '131'.
I can find the Field document by the request
$.WideXml.Values.Date.Value[*].UserFields.Field[?(#.Key=='BWART' && #.Value=='131')]
But I don't manage to build the query to get the MATNR after having this result...
Can someone help?
Regards.
if the key BWART is always at the same index position i.e 0 in this example, you can try the expression
$.WideXml.Values.Date.Value[?(#.UserFields.Field[0].Key== "BWART" && #.UserFields.Field[0].Value == "131")].UserFields.Field[*]

The language expression property '0' can't be evaluated, property name must be a string - ARM Template error while adding Key Vault access policy

I've been working on an issue and seem to be stuck, so asking on so in case anyone can help.
To describe the issue, I've got an existing Azure Key Vault setup, and wish to add a number of access policies to this resource group. It needs to be conditional as if the function name is "false" then that function should not be added to key vault access policy.
variable section:
"variables": {
"functionAccess": {
"value": [
{
"name": "[parameters('Function_1')]"
},
{
"name": "[parameters('Function_2')]"
},
{
"name": "[parameters('Function_3')]"
}
]
}
}
My Template :
{
"apiVersion": "2016-10-01",
"condition": "[not(equals(variables('functionAccess')[CopyIndex()].name, 'false'))]",
"copy": {
"batchSize": 1,
"count": "[length(variables('functionAccess'))]",
"mode": "Serial",
"name": "accessPolicies"
},
"name": "[concat(parameters('KeyVault_Name'), '/add')]",
"properties": {
"accessPolicies": [
{
"tenantId": "[subscription().tenantId]",
"objectId": "[if(not(equals(variables('functionAccess')[CopyIndex()].name, 'false')), reference(concat('Microsoft.Web/sites/', variables('functionAccess')[CopyIndex()].name), '2016-08-01', 'Full').identity.principalId, json('null'))]",
"permissions": {
"keys": [
"get",
"list"
],
"secrets": [
"get",
"list"
],
"certificates": [
"get",
"list"
]
}
}
]
},
"type": "Microsoft.KeyVault/vaults/accessPolicies"
}
When I deploy my ARM template for the azure key vault I got this error message:
The language expression property '0' can't be evaluated, property name must be a string.
also tried below, but same error:
{
"apiVersion": "2018-02-14",
"name": "[concat(parameters('KeyVault_Name'), '/add')]",
"properties": {
"copy": [
{
"batchSize": 1,
"count": "[length(variables('functionAccess'))]",
"mode": "serial",
"name": "accessPolicies",
"input": {
"condition": "[not(equals(variables('functionAccess')[copyIndex('accessPolicies')].name, 'false'))]",
"tenantId": "[subscription().tenantId]",
"objectId": "[if(not(equals(variables('functionAccess')[copyIndex('accessPolicies')].name, 'false')), reference(concat('Microsoft.Web/sites/', variables('functionAccess')[copyIndex('accessPolicies')].name), '2016-08-01', 'Full').identity.principalId, json('null'))]",
"permissions": {
"keys": [
"get",
"list"
],
"secrets": [
"get",
"list"
],
"certificates": [
"get",
"list"
]
}
}
}
]
},
"type": "Microsoft.KeyVault/vaults/accessPolicies"
}
There are a few options for dealing with filtering an array for copy operation. I deploy my ARM templates from PowerShell scripts and use PowerShell to setup parameter values. When I need special logic handle different inputs for different environments, I let PowerShell handle it.
If you must handle the filtering in ARM and you have the option to input a CSV list of functions, then perhaps the following will work. You can then use the functionAccessArray to iterate over in the copy operation.
{
"$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
},
"variables": {
"functionAccessCsv": "Function-0,Function-1,false,Function-4,false,Function-6,Function-7",
"functionAccessFiltered": "[replace(replace(variables('functionAccessCsv'), 'false', ''), ',,', ',')]",
"functionAccessArray": "[split(variables('functionAccessFiltered'), ',')]"
},
"resources": [
],
"outputs": {
"functionAccessCsvFiltered": {
"type": "string",
"value": "[variables('functionAccessFiltered')]"
},
"functionAccessArray": {
"type": "array",
"value": "[variables('functionAccessArray')]"
}
}
}
The result:
I just had the same issue. By using an array parameter with a default value instead of a variable, I got it to work.
"parameters": {
"functionAccess": {
"type": "array",
"defaultValue": [
"value1",
"value2",
"value3"
]
}
}

Conditional select using jq

I have this below json format, I want to take the list of "id" which satisfies the condition
in this below I want to take the id which has matchers.value as dev-stack and status.state as active
{
"status": "success",
"data": [
{
"id": "b5e7f85d",
"matchers": [
{
"name": "stack",
"value": "dev-stack",
"isRegex": true
}
],
"startsAt": "2020-07-13T07:17:36Z",
"endsAt": "2020-07-15T07:15:44Z",
"updatedAt": "2020-07-13T07:15:59.643692023Z",
"createdBy": "api",
"comment": "Silence",
"status": {
"state": "active"
}
},
{
"id": "1fdaa4b5",
"matchers": [
{
"name": "stack",
"value": "qa-stack",
"isRegex": true
}
],
"startsAt": "2020-07-10T13:19:12Z",
"endsAt": "2020-07-10T13:20:55.510739499Z",
"updatedAt": "2020-07-10T13:20:55.510739499Z",
"createdBy": "api",
"comment": "Silence",
"status": {
"state": "expired"
}
}
]
}
Here is a solution which uses update assignment |=, map and select to update .data.
Note it avoids an undesirable cartesian product if multiple .matchers meet the criteria by using any.
.data |= map(select(
(.matchers | any(.value=="dev-stack")) and (.status.state=="active")
))
Try it online!

Resources