def gather($ary):
INDEX(.[]; .column) as $dict
| $ary
| map( $dict[.] | .value );
.[] | gather(["h1", "h2", "h3"])
Given the above jq file, I got the following output. I'd like to make "column" and "value" additional argument to gather() so that users can easily change them. Could you show me how to make such scalar as function arguments? Thanks.
$ jq -f ./main.jq <<EOF
[
[
{ "column": "h1", "value": "v1" },
{ "column": "h2", "value": "v2" },
{ "column": "h3", "value": "v3" }
],
[
{ "column": "h1", "value": "v4" },
{ "column": "h2", "value": "v5" },
{ "column": "h3", "value": "v6" }
]
]
EOF
[
"v1",
"v2",
"v3"
]
[
"v4",
"v5",
"v6"
]
Function arguments are separated by ; rather than ,; note also that, as the jq manual says:
Arguments are passed as filters (functions with no arguments), not as values...
Thus you could define:
def gather($ary; column; value):
....
and invoke it as:
def gather(["h1", "h2", "h3"]; .column; .value)
See also https://github.com/stedolan/jq/wiki/How-to:-Avoid-Pitfalls#multi-arity-functions-and-commasemi-colon-confusability
Related
Given the following input:
{
"text": "a1\nb2"
}
How do I get the following output:
[
{
"letter": "a",
"number": 1
},
{
"letter": "b",
"number": 2
}
]
I've tried using capture with the "g" flag, but this yields two documents instead of a single document with an array of captured inputs:
$ echo '{
"text": "a1\\nb2"
}' | jq '.text | capture("(?<letter>[a-z])(?<number>[0-9])";"g")'
{
"letter": "a",
"number": "1"
}
{
"letter": "b",
"number": "2"
}
Here is a link to the jqplay example.
Why not just wrap the capture in a new array:
.text | [ capture("(?<letter>[a-z])(?<number>[0-9])";"g") ]
JqPlay
I have an array of array as following
[
{
"devices": [
{
"files": [
{
"id": "2",
"type": "file"
}
],
"path": "/tmp/file1"
},
{
"files": [
{
"id": "3",
"type": "file"
}
],
"path": "/tmp/file2"
}
],
"name": "a"
},
{
"devices": [
{
"files": [
{
"id": "4",
"type": "file"
}
],
"path": "/tmp/tfile"
},
{
"files": [
{
"id": "5",
"type": "file"
}
],
"path": "/var/mfile"
}
],
"name": "b"
}
]
I'm looking for output like this
a /tmp/file1 2
a /tmp/file2 3
b /tmp/tfile 4
b /var/mfile 5
What I have tried
cat myfile.json | jq '.[] | "\(.name), \(.devices[].path) \(.devices[].files[].id)"'
and result is
"a, /tmp/file1 2"
"a, /tmp/file2 2"
"a, /tmp/file1 3"
"a, /tmp/file2 3"
"b, /tmp/tfile 4"
"b, /var/mfile 4"
"b, /tmp/tfile 5"
"b, /var/mfile 5"
but I'm unable to select a key for the value is in parent map.
I want to for each of the elements matching name == a, for each of a[path], get a[path][id].
Using .devices[] twice will iterate over the same .devices twice, giving you the cartesian product. Extract from within one iteration.
You can nest string iterpolation; otherwise collect into an array (here you'd need , as separator) and use join(" ") to concatenate
Use the -r (or --raw-output) flag to output raw text
jq -r '.[] | "\(.name) \(.devices[] | "\(.path) \(.files[].id)")"'
a /tmp/file1 2
a /tmp/file2 3
b /tmp/tfile 4
b /var/mfile 5
Demo
With the following inputs:
# file1.json
{
"uid": "1",
"name": "jack"
}
{
"uid": "2",
"name": "jill"
}
# file2.json
{
"fid": "a",
"file": "sample1.txt",
"uid": "1"
}
{
"fid": "b",
"file": "sample2.txt",
"uid": "1"
}
{
"fid": "c",
"file": "sample3.txt",
"uid": "2"
}
How do I go about inserting the name key-value pair to the object in the file2.json. The output I'm trying to get is as follows:
{
"fid": "a",
"file": "sample1.txt",
"uid": "1",
"name": "jack"
}
{
"fid": "b",
"file": "sample2.txt",
"uid": "1",
"name": "jack"
}
{
"fid": "c",
"file": "sample3.txt",
"uid": "2",
"name": "jill"
}
Solutions posted on merge json objects with jq and join two json files based on common key with jq utility or alternative way from command line both seems to only return the last matching pair. See below.
{"uid":"1","name":"jack","fid":"b","file":"sample2.txt"}
{"uid":"2","name":"jill","fid":"c","file":"sample3.txt"}
You will need to "slurp" file1.json, e.g. by invoking jq as follows:
jq -n -f merge.jq --slurpfile file1 file1.json file2.json
where merge.jq contains:
INDEX($file1[]; .uid) as $dict
| inputs
| . + $dict[.uid]
def INDEX
If your jq does not have INDEX/2, then simply add its def:
def INDEX(stream; idx_expr):
reduce stream as $row ({}; .[$row|idx_expr|tostring] = $row);
I want to parse terraform.tfstate (where openstack provider is used), to return instance name and it's internal + floating IP (if assigned).
First select what we are interested in:
jq -r '.modules?[]|.resources[]?|select(.type == "openstack_compute_floatingip_v2", .type == "openstack_compute_instance_v2")' < terraform.tfstate
For simplicity, pre-parsed example with the above part (one FIP and one instance):
{
"type": "openstack_compute_floatingip_v2",
"depends_on": [
"openstack_networking_router_interface_v2.management"
],
"primary": {
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"attributes": {
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"instance_id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"pool": "public",
"region": "RegionOne"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
{
"type": "openstack_compute_instance_v2",
"depends_on": [
"openstack_compute_floatingip_v2.management",
"openstack_compute_secgroup_v2.ssh_only",
"openstack_networking_network_v2.management"
],
"primary": {
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"attributes": {
"access_ip_v4": "10.10.10.5",
"access_ip_v6": "",
"all_metadata.%": "1",
"all_metadata.habitat": "sup",
"availability_zone": "nova",
"flavor_id": "eb36e84e-17c1-42ab-b359-4380f6f524ae",
"flavor_name": "m1.large",
"force_delete": "false",
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"image_id": "c574aeed-e47c-4fb7-9da0-75550b76ee56",
"image_name": "ubuntu-16.04",
"key_pair": "vault-etcd_test_tf",
"metadata.%": "1",
"metadata.habitat": "sup",
"name": "ctl01",
"network.#": "1",
"network.0.access_network": "false",
"network.0.fixed_ip_v4": "10.10.10.5",
"network.0.fixed_ip_v6": "",
"network.0.floating_ip": "",
"network.0.mac": "02:c6:61:f9:ee:7e",
"network.0.name": "management",
"network.0.port": "",
"network.0.uuid": "f2468669-e321-4eb4-9ede-003e362a8988",
"region": "RegionOne",
"security_groups.#": "1",
"security_groups.1845949017": "vault-etcd_test_ssh_only",
"stop_before_destroy": "false"
},
"meta": {
"e2bfb730-ecaa-11e6-8f88-34363bc7c4c0": {
"create": 1800000000000,
"delete": 1800000000000,
"update": 1800000000000
}
},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
Required is to take from "type": "openstack_compute_floatingip_v2" replace .primary.attributes.address and .fixed_ip and from corresponding .instance_id the .name.
So, sth like:
{"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"}
Well, I came with an idea while using walk, but miss how to actually assign the proper value from corresponding instance id:
jq -r "$(cat floating.jq)" terraform.tfstate
floating.jq:
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
.modules?[]|.resources[]?|select(.type ==
"openstack_compute_floatingip_v2", .type ==
"openstack_compute_instance_v2")|
.primary|walk( if type == "object" and .attributes.address then
.attributes.instance_id |= "REFERRED VALUE HERE") else . end)
Let's assume the two related objects are in a file named two.json. Then one way to merge the information from both objects is using the -s command-line option, e.g.
jq -s '
(.[0].primary.attributes | {address, fixed_ip})
+ {name: .[1].primary.attributes.name}' two.json
Output
With your example input, the output would be:
{
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"
}
I have some json data that looks like:
{
"p": {
"d": {
"a" : {
"r": "foo",
"g": 1
},
"b": {
"r": "bar",
"g": 2
}
},
"c": {
"e": {
"r": "baz",
"g": 1
}
},
...
}
}
I want something like:
{
"d": [
"a",
"b"
],
"c": [
"e"
]
}
I can get the list of keys on the first level under "p" with jq '.p|keys', and the structure and keys on the second level with jq '.p|map(.|keys)', but I can't figure out how to combine it.
Use map_values instead of map to map the values of a JSON object while preserving the keys:
jq '.p | map_values(keys)'
On jq versions lower than 1.5, map_values is not defined: instead, you can use []|=:
jq '.p | . []|= keys'
In general
Top level keys:
curl -s https://crates.io/api/v1/crates/atty | jq '. |= keys'
[
"categories",
"crate",
"keywords",
"versions"
]
Two levels of keys:
curl -s https://crates.io/api/v1/crates/atty | jq '.| map_values(keys)'
{
"crate": [
"badges",
"categories",
"created_at",
"description",
"documentation",
"downloads",
"exact_match",
"homepage",
"id",
"keywords",
"links",
"max_version",
"name",
"newest_version",
"recent_downloads",
"repository",
"updated_at",
"versions"
],
"versions": [
0,
1,
2,
3,
4,
5,
6,
7,
8,
9,
10,
11,
12,
13,
14,
15,
16
],
"keywords": [
0,
1,
2
],
"categories": []
}
Method versions
topLevelJsonKeys() {
curl -s $1 | jq '. |= keys'
# EXAMPLE:
# topLevelJsonKeys https://crates.io/api/v1/crates/atty
}
topLevelJsonKeys2() {
curl -s $1 | jq '.| map_values(keys)'
# EXAMPLE:
# topLevelJsonKeys2 https://crates.io/api/v1/crates/atty
}
Here is a solution which uses reduce and setpath
.p
| reduce keys[] as $k (
.
; setpath([$k]; .[$k] | keys)
)