JQ Group By by Key, Merge Value, then Flatten Object - jq

With the following inputs:
# file1.json
{
"uid": "1",
"name": "jack"
}
{
"uid": "2",
"name": "jill"
}
# file2.json
{
"fid": "a",
"file": "sample1.txt",
"uid": "1"
}
{
"fid": "b",
"file": "sample2.txt",
"uid": "1"
}
{
"fid": "c",
"file": "sample3.txt",
"uid": "2"
}
How do I go about inserting the name key-value pair to the object in the file2.json. The output I'm trying to get is as follows:
{
"fid": "a",
"file": "sample1.txt",
"uid": "1",
"name": "jack"
}
{
"fid": "b",
"file": "sample2.txt",
"uid": "1",
"name": "jack"
}
{
"fid": "c",
"file": "sample3.txt",
"uid": "2",
"name": "jill"
}
Solutions posted on merge json objects with jq and join two json files based on common key with jq utility or alternative way from command line both seems to only return the last matching pair. See below.
{"uid":"1","name":"jack","fid":"b","file":"sample2.txt"}
{"uid":"2","name":"jill","fid":"c","file":"sample3.txt"}

You will need to "slurp" file1.json, e.g. by invoking jq as follows:
jq -n -f merge.jq --slurpfile file1 file1.json file2.json
where merge.jq contains:
INDEX($file1[]; .uid) as $dict
| inputs
| . + $dict[.uid]
def INDEX
If your jq does not have INDEX/2, then simply add its def:
def INDEX(stream; idx_expr):
reduce stream as $row ({}; .[$row|idx_expr|tostring] = $row);

Related

jq - if a name is in a disctionary list exists, return a row of values

Given the following input
{
"environment": [
{
"name": "AAA",
"value": "1111"
},
{
"name": "BBB",
"value": "2222"
},
{
"name": "CCC",
"value": "3333"
},
{
"name": "DDD",
"value": "4444"
}
]
}
If CCC exists, then write out on a single line of certain values
1111, 2222, 4444
Else
Write Nothing
jq -r '.environment[] | select (.name == "CCC") | [.name, .value] | #csv'
Don't know how to write the other values.
Would like to see the values
1111, 2222, 3333
The sample is not quite valid JSON, but after fixing it, the following will produce the output you first mention (without the "3333"), in accordance with my best guess as to your requirements:
jq -r '.environment
| (map(.name) | index("CCC")) as $ix
| select($ix)
| del(.[$ix])
| map(.value)
| join(", ")
'
For very large arrays, you would probably want to tweak this a bit to avoid mapping the entire array.
jq -r --arg wanted "CCC" '
.environment
| if any(.name == $wanted)
then map(.value) | #csv
else empty
end
' << END_INPUT
{
"environment": [
{
"name": "AAA",
"value": "1111"
},
{
"name": "BBB",
"value": "2222"
},
{
"name": "CCC",
"value": "3333"
},
{
"name": "DDD",
"value": "4444"
}
]
}
END_INPUT
"1111","2222","3333","4444"

How to return an array from `capture` with `global` filter in `jq`

Given the following input:
{
"text": "a1\nb2"
}
How do I get the following output:
[
{
"letter": "a",
"number": 1
},
{
"letter": "b",
"number": 2
}
]
I've tried using capture with the "g" flag, but this yields two documents instead of a single document with an array of captured inputs:
$ echo '{
"text": "a1\\nb2"
}' | jq '.text | capture("(?<letter>[a-z])(?<number>[0-9])";"g")'
{
"letter": "a",
"number": "1"
}
{
"letter": "b",
"number": "2"
}
Here is a link to the jqplay example.
Why not just wrap the capture in a new array:
.text | [ capture("(?<letter>[a-z])(?<number>[0-9])";"g") ]
JqPlay

Pass array index into children entries with jq

If you think the title of this post is off, so do I. I'm happy to reformulate but it's hard to explain.
Given this:
[
{
"A": "a",
"B": "b",
"C": "c"
},
{
"D": "d",
"E": "e"
}
]
I want this:
[
{ index: 0, key: "A", value: "a" },
{ index: 0, key: "B", value: "b" },
{ index: 0, key: "C", value: "c" },
{ index: 1, key: "D", value: "d" },
{ index: 1, key: "E", value: "e" }
]
Now whether it's a single array in the end or something else, it doesn't really matter.
What I'm really looking for is to transform those elements in the input array into key-value pairs, and add the index of that element from the array.
So the first object in the array is "index: 0", second is "index: 1", etc.
I can get the index from the array just fine:
jq 'to_entries | .[]' sample.json
which gives me:
{
"key": 0,
"value": {
"A": "a",
"B": "b",
"C": "c"
}
}
{
"key": 1,
"value": {
"D": "d",
"E": "e"
}
}
And I can get the key-value pairs just fine too:
jq 'to_entries | .[] | .value | to_entries' sample.json
which gives me:
[
{
"key": "A",
"value": "a"
},
{
"key": "B",
"value": "b"
},
{
"key": "C",
"value": "c"
}
]
[
{
"key": "D",
"value": "d"
},
{
"key": "E",
"value": "e"
}
]
but I can't figure out how to combine the two.
This gives expected result :
jq 'to_entries|map({index:.key}+(.value|to_entries[]))' sample.json

JQ applying function to dict but print surrounding datas

I'd like
"1": {
"id": "1",
"type": "select",
"label": "country :",
"choices": {
"1": {
"label": "Canada CAN",
"value": "",
},
"2": {
"label": "United States USA",
"value": ""
}
}
}
to produce
"1": {
"id": "1",
"type": "select",
"label": "country :",
"choices": {
"1": {
"label": "Canada",
"value": "CAN",
},
"2": {
"label": "United States",
"value": "USA"
}
}
}
By now I have a two-step solution consisting of a sed-like function
def do_extract:
if .value | test("^$") then
(.value = (.label | capture(".* (?<code>...)")).code) | .label = (.label | capture("(?<name>.*) ...$").name)
else
.
end;
and a direct access to the sub-tree []."1"."choices"|keys_unsorted as $key|map_values(do_extract) but I have to manually copy-paste the output in place of the original "choices" dict.
Is there a way to do the function but still print the surrounding datas ?
Thanks
Well to get your desired result, you could do this:
.[].choices[] |= (.label | capture("^(?<label>.*?) (?<value>[^ ]+)$"))
To do that within your function, I'd change it to this:
def do_extract:
if .value == "" then
(.label | capture("^(?<label>.*?) (?<value>[^ ]+)$"))
else
.
end;
Then use it:
.[].choices[] |= do_extract

JQ Cross reference or how to replace one value with another part of the input

I want to parse terraform.tfstate (where openstack provider is used), to return instance name and it's internal + floating IP (if assigned).
First select what we are interested in:
jq -r '.modules?[]|.resources[]?|select(.type == "openstack_compute_floatingip_v2", .type == "openstack_compute_instance_v2")' < terraform.tfstate
For simplicity, pre-parsed example with the above part (one FIP and one instance):
{
"type": "openstack_compute_floatingip_v2",
"depends_on": [
"openstack_networking_router_interface_v2.management"
],
"primary": {
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"attributes": {
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"instance_id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"pool": "public",
"region": "RegionOne"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
{
"type": "openstack_compute_instance_v2",
"depends_on": [
"openstack_compute_floatingip_v2.management",
"openstack_compute_secgroup_v2.ssh_only",
"openstack_networking_network_v2.management"
],
"primary": {
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"attributes": {
"access_ip_v4": "10.10.10.5",
"access_ip_v6": "",
"all_metadata.%": "1",
"all_metadata.habitat": "sup",
"availability_zone": "nova",
"flavor_id": "eb36e84e-17c1-42ab-b359-4380f6f524ae",
"flavor_name": "m1.large",
"force_delete": "false",
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"image_id": "c574aeed-e47c-4fb7-9da0-75550b76ee56",
"image_name": "ubuntu-16.04",
"key_pair": "vault-etcd_test_tf",
"metadata.%": "1",
"metadata.habitat": "sup",
"name": "ctl01",
"network.#": "1",
"network.0.access_network": "false",
"network.0.fixed_ip_v4": "10.10.10.5",
"network.0.fixed_ip_v6": "",
"network.0.floating_ip": "",
"network.0.mac": "02:c6:61:f9:ee:7e",
"network.0.name": "management",
"network.0.port": "",
"network.0.uuid": "f2468669-e321-4eb4-9ede-003e362a8988",
"region": "RegionOne",
"security_groups.#": "1",
"security_groups.1845949017": "vault-etcd_test_ssh_only",
"stop_before_destroy": "false"
},
"meta": {
"e2bfb730-ecaa-11e6-8f88-34363bc7c4c0": {
"create": 1800000000000,
"delete": 1800000000000,
"update": 1800000000000
}
},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
Required is to take from "type": "openstack_compute_floatingip_v2" replace .primary.attributes.address and .fixed_ip and from corresponding .instance_id the .name.
So, sth like:
{"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"}
Well, I came with an idea while using walk, but miss how to actually assign the proper value from corresponding instance id:
jq -r "$(cat floating.jq)" terraform.tfstate
floating.jq:
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
.modules?[]|.resources[]?|select(.type ==
"openstack_compute_floatingip_v2", .type ==
"openstack_compute_instance_v2")|
.primary|walk( if type == "object" and .attributes.address then
.attributes.instance_id |= "REFERRED VALUE HERE") else . end)
Let's assume the two related objects are in a file named two.json. Then one way to merge the information from both objects is using the -s command-line option, e.g.
jq -s '
(.[0].primary.attributes | {address, fixed_ip})
+ {name: .[1].primary.attributes.name}' two.json
Output
With your example input, the output would be:
{
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"
}

Resources