transform json input using jq command - jq

I have the following json input from a internet service:
{
"sunarme": "foo",
"id": "foo-id",
"name": "Foo bar",
"profile": [
{
"id": "test1",
"products": [
"product1",
"product2"
],
"description": "test1 description"
},
{
"id": "test2",
"products": [
"product3",
"product4",
"product5"
],
"description": "test2 description"
},
{
"id": "test3",
"products": [
"product6",
"product7",
"product8"
],
"description": "test2 description"
}
]
}
So I need to transform profile key from array to json object. This is the desired output:
{
"sunarme": "foo",
"id": "foo-id",
"name": "Foo bar",
"profile": {
"test1": [
"product1",
"product2"
],
"test2": [
"product3",
"product4",
"product5"
],
"test3": [
"product6",
"product7",
"product8"
]
}
}
I don't have any idea how to do it in jq command, please, could you help me?
Thanks in advance.

Use with_entries which lets you convert the array into an object if you adjust the .keys accordingly.
jq '.profile |= with_entries(.key = .value.id | .value |= .products)'
Demo
Or use reduce to build the object by iterating through the array.
jq '.profile |= reduce .[] as $p ({}; .[$p.id] = $p.products)'
Demo
Or use map to convert each array item into an object, then merge them using add.
jq '.profile |= (map({(.id): .products}) | add)'
Demo
Output is:
{
"sunarme": "foo",
"id": "foo-id",
"name": "Foo bar",
"profile": {
"test1": [
"product1",
"product2"
],
"test2": [
"product3",
"product4",
"product5"
],
"test3": [
"product6",
"product7",
"product8"
]
}
}

Related

`jq` hangs when I provide `--arg` and a template json

I am trying to use jq to create a json from a template json file using --args and the template file. When I execute the below command, jq just hangs in there forever.
I am a rookie with jq, would really appreciate if someone can point out what am I doing wrong.
template.jq
{
"channel": "channel",
"attachments": [
{
"color": "#a7dbb5",
"blocks": [
{
"type": "header",
"text": {
"type": "plain_text",
"text": $SUMMARY,
"emoji": true
}
},
{
"type": "divider"
},
{
"type": "section",
"text": {
"type": "mrkdwn",
"text": "*Build ID: * <\($BUILD_URL)|\($BUILD_ID)>\n*Duration:* \($DURATION)\n*User: *<\($USER_EMAIL)|\($USER_NAME)>\n*Test Cases:* \($TEST_CASES)"
},
"accessory": {
"type": "image",
"image_url": "https://raw.githubusercontent.com/sudas-px/dev-repo/main/check.png",
"alt_text": "status thumbnail"
}
},
{
"type": "divider"
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Portworx*\nv\($PX_VERSION)"
},
{
"type": "mrkdwn",
"text": "*PX Backup*\nv\($PX_BACKUP_VERSION)"
}
]
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Stork Image:*\n\($STORK_IMAGE)"
},
{
"type": "mrkdwn",
"text": "*Kubernetes:*\nv\($K8S_VERSION)"
}
]
},
{
"type": "divider"
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Github Repository*"
},
{
"type": "mrkdwn",
"text": $GITHUB_REPO
}
]
},
{
"type": "section",
"fields": [
{
"type": "mrkdwn",
"text": "*Branch*"
},
{
"type": "mrkdwn",
"text": $GITHUB_BRANCH
}
]
},
{
"type": "divider"
},
{
"type": "actions",
"block_id": "actionblock789",
"elements": [
{
"type": "button",
"text": {
"type": "plain_text",
"text": "View Pipeline"
},
"style": "primary",
"url": $BUILD_URL
},
{
"type": "button",
"text": {
"type": "plain_text",
"text": "View Logs"
},
"url": $KIBANA_URL
}
]
}
]
}
]
}
This is the command I ran
jq --arg SUMMARY "Summary" --arg BUILD_ID "BUILD_ID" --arg BUILD_URL "BUILD_URL" --arg DURATION "DURATION" --arg USER_EMAIL "EMAIL" --arg USER_NAME "USER" --arg TEST_CASES 3 --arg PX_VERSION "VERSION" --arg PX_BACKUP_VERSION "PX_VERSION" --arg STORK_IMAGE "IMAGE_STORK" --arg K8S_VERSION "1.23.0" --arg GITHUB_BRANCH "branch" --arg GITHUB_REPO "repo" --arg KIBANA_URL "url" -f template.jq
Your "template" is just a filter that requires no input, but you forgot to tell jq that the filter won't need any input. As a result, jq is waiting to read from standard input. Use the -n option to tell jq it doesn't need to read from standard input.
jq -n <lots of --args> -f template.jq

mapping array of keys to array of values for substitution

Given this data, which is two distinct objects, one having an array of keys and the other a dictionary of key value pairs:
{
"servers": [
{
"location": "server4",
"services": [
"srv07",
"srv06",
"srv01",
"srv04"
]
},
{
"location": "server2",
"services": [
"srv07",
"srv02",
"srv05",
"srv03"
]
}
],
"release": {
"id": "release1",
"services": [
{
"service": "srv01",
"URL": "/srv01_service/v1.20.0"
},
{
"service": "srv02",
"URL": "/srv02_service/v1.14.0"
},
{
"service": "srv03",
"URL": "/srv03_service/v1.15.0"
},
{
"service": "srv04",
"URL": "/srv04_service/v1.18.0"
},
{
"service": "srv05",
"URL": "/srv05_service/v1.14.0"
},
{
"service": "srv06",
"URL": "/srv06_serv/v1.13.0"
},
{
"service": "srv07",
"URL": "/srv07_service/v1.19.0"
}
]
}
}
I am trying to produce this, the first object with the keys replaced with the values from the dictionary. NOTE: I would be fine with renaming services[] to URLs[] if it makes things easier.
{
"servers": [
{
"location": "server4",
"services": [
"/srv07_service/v1.19.0",
"/srv06_serv/v1.13.0",
"/srv01_service/v1.20.0",
"/srv04_service/v1.18.0"
]
},
{
"location": "server2",
"services": [
"/srv07_service/v1.19.0",
"/srv02_service/v1.14.0",
"/srv05_service/v1.14.0",
"/srv03_service/v1.15.0"
]
}
]
}
My latest attempt is close but returns something akin to a Cartesian.
. | .servers[].services[] = (.servers[] as $s | .release.services[] | select(.service as $v | $s.services[] | index($v)).URL) | {servers}
Create an INDEX to lookup in, then map each element you want to change according to the index.
jq '
INDEX(.release.services[]; .service) as $index
| {servers} | .servers[].services[] |= $index[.].URL
'
{
"servers": [
{
"location": "server4",
"services": [
"/srv07_service/v1.19.0",
"/srv06_serv/v1.13.0",
"/srv01_service/v1.20.0",
"/srv04_service/v1.18.0"
]
},
{
"location": "server2",
"services": [
"/srv07_service/v1.19.0",
"/srv02_service/v1.14.0",
"/srv05_service/v1.14.0",
"/srv03_service/v1.15.0"
]
}
]
}
Demo

jq Contactenate arrays from two different files and save the output in the first file

Here's what I'm looking to do.
file1.json
{
"info": {
"id": "",
"name": "Text Fields",
"schema": "url"
},
"item": [
{
"name": "CompanyName Field",
"item": [
{
"name": "CompanyName is CompanyName1"
}
]
}
]
}
file2.json
[
{
"name": "Phone Field",
"item": [
{
"name": "Phone is 1234"
}
]
},
{
"name": "Job Field",
"item": [
{
"name": "Job is Job1"
}
]
}
]
Expected output after running jq
file1.json
{
"info": {
"id": "",
"name": "Text Fields",
"schema": "url"
},
"item": [
{
"name": "CompanyName Field",
"item": [
{
"name": "CompanyName is CompanyName1"
}
]
},
{
"name": "Phone Field",
"item": [
{
"name": "Phone is 1234"
}
]
},
{
"name": "Job Field",
"item": [
{
"name": "Job is Job1"
}
]
}
]
}
As a first step I tried to at least concatenate the arrays of the two files and get that as an output before trying to get them in the first file itself but that itself is not happening.
Here's what I tried
jq '.item .' file1.json file2.json
but I get the following error:
jq: error: syntax error, unexpected $end, expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.item .
jq: 1 compile error
I tried searching a lot, trust me. There are a lot of queries with similar titles but they all seem to be very specific problems when you look into each one. Please help.
Use --argfile to read in the second file into a variable, then += to add it to the existing array in .item
jq --argfile f file2.json '.item += $f' file1.json
{
"info": {
"id": "",
"name": "Text Fields",
"schema": "url"
},
"item": [
{
"name": "CompanyName Field",
"item": [
{
"name": "CompanyName is CompanyName1"
}
]
},
{
"name": "Phone Field",
"item": [
{
"name": "Phone is 1234"
}
]
},
{
"name": "Job Field",
"item": [
{
"name": "Job is Job1"
}
]
}
]
}

jq - extract multiple fields from a list, with a nested list of key/value pairs

I have the following structure:
{
"Subnets": [
{
"SubnetId": "foo1",
"Id": "bar1",
"Tags": [
{
"Key": "Name",
"Value": "foo"
},
{
"Key": "Status",
"Value": "dev"
}
]
},
{
"SubnetId": "foo2",
"Id": "bar2",
"Tags": [
{
"Key": "Name",
"Value": "foo"
},
{
"Key": "Status",
"Value": "dev"
}
]
}
]
}
I can extract multiple keys at the "top level" like so:
cat subnets.json| jq '.Subnets[] | "\(.Id) \(.SubnetId)"'
Anyone know how I can also display one of the tags by key name, let's say I also want the Status tag displayed on the same line as the Id and SubnetId.
Thx for any help,
Is this what you are looking for?
jq '.Subnets[] | "\(.Id) \(.SubnetId) \(.Tags | from_entries | .Status)"' subnets.json

Use jq extract two values and build new output

Looking to extract values from api_http array. I am looking for output that looks like the following. Each element should have the name and the url value attached a key called api.
{ "name": "lookproduct1", "api": "http://testapi.api.com"}
{ "name": "lookproduct2", "api": "http://testapi2.api.com"}
{ "name": "lookproduct3", "api": "http://testapi3.api.com"}
{ "name": "lookproduct4", "api": "http://testapi4.api.com"}
the JSON data:
{
"meta": {
"details": {
"value": "Details"
},
"network": {
"label": "Network:",
"value": "test"
},
"title": {
"value": "Test Report"
},
"update": {
"label": "Validation last update:",
"value": "2020-07-15 17:40 UTC"
}
},
"report": {
"api_http": [
[
{
"html_name": "Product 1",
"name": "lookproduct1",
"rank": 3
},
"http://testapi.api.com",
"GB",
"TEST"
],
[
{
"html_name": "Product 2",
"name": "lookproduct2",
"rank": 3
},
"http://testapi2.api.com",
"GB",
"TEST"
],
[
{
"html_name": "Product 3",
"name": "lookproduct3",
"rank": 3
},
"http://testapi3.api.com",
"GB",
"TEST"
],
[
{
"html_name": "Product 4",
"name": "lookproduct4",
"rank": 3
},
"http://testapi.api.com",
"GB",
"TEST"
]
]
}
}
I got the following, but unsure to extract those final two values and create the new output.
.report[] | .[]
Try:
.report.api_http[]|{name:values[0]["name"],api:values[1]}
My output is:
{
"name": "lookproduct1",
"api": "http://testapi.api.com"
}
{
"name": "lookproduct2",
"api": "http://testapi2.api.com"
}
{
"name": "lookproduct3",
"api": "http://testapi3.api.com"
}
{
"name": "lookproduct4",
"api": "http://testapi.api.com"
}
You could use the -c command-line option in conjunction with the following jq filter:
.report.api_http[]
| {name: .[0].name, api: .[1]}

Resources