What I need is simply to extract id, sales from sorted_employees AND full name for that employee from employees.. like this:
{ "Full name": "John Doe", "ID": "employee1", "Sales": "26" }
{ "Full name": "Sam Jones", "ID": "employee2", "Sales": "119" }
What would be the easiest way to combine these 2 arrays ( employees & sorted_employees ) ? I have problems merging the result, I have tried to cast results into an array and I tried purely with jq filters.. but it doesn't want to give me what I need..
employees_id=$( echo "${json_data}" | jq -r '.employees[] | .id' )
sorted_employees_id=$( echo "${json_data}" | jq -r '.sorted_employees[] | .id' )
sorted_employees_sales=$( echo "${json_data}" | jq '.sorted_employees[] | .sales' )
{
"employees": [ {
"started_at": "2018-05-01 12.00",
"id": "employee1",
"facebook": "https://fb/john_doe",
"full name": "John Doe"
}, {
"started_at": "2017-05-01 12.00",
"id": "employee2",
"facebook": "https://fb/sam_jones",
"full name": "Sam Jones"
}, {
"started_at": "2016-05-01 12.00",
"id": "employee3",
"facebook": "https://fb/jane_roe",
"full name": "Jane Roe"
}],
"sorted_employees": [{
"id": "employee1",
"sales": 26
}, {
"id": "employee2",
"sales": 119
}, {
"id": "employee3",
"sales": 84
}]
}
You could combine the two arrays and just group by the common ID field and form the desired output object
jq '.employees + .sorted_employees | group_by(.id) |
map({"Full name": .[0]."full name", ID: .[0].id, "Sales": .[1].sales})'
Related
Suppose I have a JSON:
[
{
"title": "Title1",
"reference": [
"123"
]
},
{
"title": "Title2",
"reference": [
"234",
"345"
]
}
]
Id like to modify each element of the reference array so that the reference appears twice. I'd like to achieve:
[
{
"title": "Title1",
"reference": [
"123 is 123"
]
},
{
"title": "Title2",
"reference": [
"234 is 234",
"345 is 345"
]
}
]
I've tried:
jq '.[] | .reference = [("\(.reference[]) is \(.reference[])")]'
but this fails where the array has more than one item:
{
"title": "Title1",
"reference": [
"123 is 123"
]
}
{
"title": "Title2",
"reference": [
"234 is 234",
"345 is 234",
"234 is 345",
"345 is 345"
]
}
How can I modify the above jq to achieve the desired result?
Thanks in advance!
map(.reference |= map(. + " is " + .))
Will change each .reference to be .reference is .reference
[
{
"title": "Title1",
"reference": [
"123 is 123"
]
},
{
"title": "Title2",
"reference": [
"234 is 234",
"345 is 345"
]
}
]
Demo
This should work just fine:
jq '.[].reference[] |= "\(.) is \(.)"'
It replaces every item of the reference arrays with a string which contains itself two times and the word "is"
I have an array of array as following
[
{
"devices": [
{
"files": [
{
"id": "2",
"type": "file"
}
],
"path": "/tmp/file1"
},
{
"files": [
{
"id": "3",
"type": "file"
}
],
"path": "/tmp/file2"
}
],
"name": "a"
},
{
"devices": [
{
"files": [
{
"id": "4",
"type": "file"
}
],
"path": "/tmp/tfile"
},
{
"files": [
{
"id": "5",
"type": "file"
}
],
"path": "/var/mfile"
}
],
"name": "b"
}
]
I'm looking for output like this
a /tmp/file1 2
a /tmp/file2 3
b /tmp/tfile 4
b /var/mfile 5
What I have tried
cat myfile.json | jq '.[] | "\(.name), \(.devices[].path) \(.devices[].files[].id)"'
and result is
"a, /tmp/file1 2"
"a, /tmp/file2 2"
"a, /tmp/file1 3"
"a, /tmp/file2 3"
"b, /tmp/tfile 4"
"b, /var/mfile 4"
"b, /tmp/tfile 5"
"b, /var/mfile 5"
but I'm unable to select a key for the value is in parent map.
I want to for each of the elements matching name == a, for each of a[path], get a[path][id].
Using .devices[] twice will iterate over the same .devices twice, giving you the cartesian product. Extract from within one iteration.
You can nest string iterpolation; otherwise collect into an array (here you'd need , as separator) and use join(" ") to concatenate
Use the -r (or --raw-output) flag to output raw text
jq -r '.[] | "\(.name) \(.devices[] | "\(.path) \(.files[].id)")"'
a /tmp/file1 2
a /tmp/file2 3
b /tmp/tfile 4
b /var/mfile 5
Demo
I'm using jq to try and merge 2 json files into one unique file.
The result is close to what I was looking for, but not just right.
File 1:
{
"series": "Harry Potter Movie Series",
"writer": "J.K. Rowling",
"movies": [
{
"title": "Harry Potter and the Philosopher's Stone",
"actors": [
{
"names": [
"Emma Watson",
"Other actor"
],
"other": "Some value"
}
]
},
{
"title": "Harry Potter and the Chamber of Secrets",
"actors": [
{
"names": [
"Emma Watson"
],
"other": "Some value"
}
]
}
]
}
File 2:
{
"series": "Harry Potter Movie Series",
"producer": "David Heyman",
"movies": [
{
"title": "Harry Potter and the Philosopher's Stone",
"year": "2001"
},
{
"title": "Harry Potter and the Chamber of Secrets",
"year": "2002"
}
]
}
Expected result:
{
"series": "Harry Potter Movie Series",
"writer": "J.K. Rowling",
"movies": [
{
"title": "Harry Potter and the Philosopher's Stone",
"year": "2001",
"actors": [
{
"names": [
"Emma Watson",
"Other actor"
],
"other": "Some value"
}
]
},
{
"title": "Harry Potter and the Chamber of Secrets",
"year": "2001",
"actors": [
{
"names": [
"Emma Watson"
],
"other": "Some value"
}
]
}
],
"producer": "David Heyman"
}
Best result I've got so far (only arrays with actors are missing):
{
"series": "Harry Potter Movie Series",
"writer": "J.K. Rowling",
"movies": [
{
"title": "Harry Potter and the Philosopher's Stone",
"year": "2001"
},
{
"title": "Harry Potter and the Chamber of Secrets",
"year": "2002"
}
],
"producer": "David Heyman"
}
Using one of the commands below:
jq -s '.[0] * .[1]' file1 file2
jq --slurp 'add' file1 file2
jq '. * input' file1 file2
If I switch order of files I either end up losing 'actors' from file1 or 'year' from file2.
How it should work:
the elements in file 2 will be leading and should replace the matching elements in file 1.
the elements in file 1 that doesn't exist in file 2 (like writer and movies[].actors elements) shouldn't be deleted
the elements in file 2 that doesn't exist yet in file 1 will be added (like producer and movies[].year).
a title is unique and should by default not occur more then once, but if it does remove the duplicates.
I would assume there is a solution to get these movies arrays perfectly merged with jq.
You are looking for a solution that "merges" objects and arrays. For the former you have already found + (or add) for a top-level merge, and * for a recursive merge, but merging arrays (namely the two .movies fields) needs more specification from your end as there is no canonical solution for that.
In a comment you state
.movies[0] always correspond to the same movie in both files
This enables you to use transpose to align the items from both arrays, and then apply object-merging on each pair of corresponding items. If you want to merge deeper arrays as well (e.g. .movies[].actors or .movies[].actors[].names) you need to extend this approach accordingly. Here's a solution using plain add for the merging of the array items as well as of the other top-level fields:
jq -s 'add + {movies: map(.movies) | transpose | map(add)}' file1 file2
{
"series": "Harry Potter Movie Series",
"writer": "J.K. Rowling",
"movies": [
{
"title": "Harry Potter and the Philosopher's Stone",
"actors": [
{
"names": [
"Emma Watson",
"Other actor"
],
"other": "Some value"
}
],
"year": "2001"
},
{
"title": "Harry Potter and the Chamber of Secrets",
"actors": [
{
"names": [
"Emma Watson"
],
"other": "Some value"
}
],
"year": "2002"
}
],
"producer": "David Heyman"
}
Demo
I want to parse terraform.tfstate (where openstack provider is used), to return instance name and it's internal + floating IP (if assigned).
First select what we are interested in:
jq -r '.modules?[]|.resources[]?|select(.type == "openstack_compute_floatingip_v2", .type == "openstack_compute_instance_v2")' < terraform.tfstate
For simplicity, pre-parsed example with the above part (one FIP and one instance):
{
"type": "openstack_compute_floatingip_v2",
"depends_on": [
"openstack_networking_router_interface_v2.management"
],
"primary": {
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"attributes": {
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"instance_id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"pool": "public",
"region": "RegionOne"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
{
"type": "openstack_compute_instance_v2",
"depends_on": [
"openstack_compute_floatingip_v2.management",
"openstack_compute_secgroup_v2.ssh_only",
"openstack_networking_network_v2.management"
],
"primary": {
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"attributes": {
"access_ip_v4": "10.10.10.5",
"access_ip_v6": "",
"all_metadata.%": "1",
"all_metadata.habitat": "sup",
"availability_zone": "nova",
"flavor_id": "eb36e84e-17c1-42ab-b359-4380f6f524ae",
"flavor_name": "m1.large",
"force_delete": "false",
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"image_id": "c574aeed-e47c-4fb7-9da0-75550b76ee56",
"image_name": "ubuntu-16.04",
"key_pair": "vault-etcd_test_tf",
"metadata.%": "1",
"metadata.habitat": "sup",
"name": "ctl01",
"network.#": "1",
"network.0.access_network": "false",
"network.0.fixed_ip_v4": "10.10.10.5",
"network.0.fixed_ip_v6": "",
"network.0.floating_ip": "",
"network.0.mac": "02:c6:61:f9:ee:7e",
"network.0.name": "management",
"network.0.port": "",
"network.0.uuid": "f2468669-e321-4eb4-9ede-003e362a8988",
"region": "RegionOne",
"security_groups.#": "1",
"security_groups.1845949017": "vault-etcd_test_ssh_only",
"stop_before_destroy": "false"
},
"meta": {
"e2bfb730-ecaa-11e6-8f88-34363bc7c4c0": {
"create": 1800000000000,
"delete": 1800000000000,
"update": 1800000000000
}
},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
Required is to take from "type": "openstack_compute_floatingip_v2" replace .primary.attributes.address and .fixed_ip and from corresponding .instance_id the .name.
So, sth like:
{"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"}
Well, I came with an idea while using walk, but miss how to actually assign the proper value from corresponding instance id:
jq -r "$(cat floating.jq)" terraform.tfstate
floating.jq:
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
.modules?[]|.resources[]?|select(.type ==
"openstack_compute_floatingip_v2", .type ==
"openstack_compute_instance_v2")|
.primary|walk( if type == "object" and .attributes.address then
.attributes.instance_id |= "REFERRED VALUE HERE") else . end)
Let's assume the two related objects are in a file named two.json. Then one way to merge the information from both objects is using the -s command-line option, e.g.
jq -s '
(.[0].primary.attributes | {address, fixed_ip})
+ {name: .[1].primary.attributes.name}' two.json
Output
With your example input, the output would be:
{
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"
}
For an input file that looks like this:
{
"employees": [
{
"number": "101",
"tags": [
{
"value": "yes",
"key": "management"
},
{
"value": "joe",
"key": "login"
},
{
"value": "joe blogs",
"key": "name"
}
]
},
{
"number": "102",
"tags": [
{
"value": "no",
"key": "management"
},
{
"value": "jane",
"key": "login"
},
{
"value": "jane doe",
"key": "name"
}
]
},
{
"number": "103",
"tags": [
{
"value": "no",
"key": "management"
},
{
"value": "john",
"key": "login"
},
{
"value": "john doe",
"key": "name"
}
]
}
]
}
... I'd like to get details for all non-management employees so that the desired output looks like this:
{
"number": "102",
"name": "jane doe",
"login": "jane"
}
{
"number": "103",
"name": "john doe",
"login": "john"
}
I can't figure out how to limit results based on a key without selecting that key (in this case "management")
The following is a slightly more succinct solution:
.employees[]
| .tags |= from_entries
| select(.tags.management == "no")
| {number, "name": .tags.name, "login": .tags.login}
Using from_entries, this worked for me:
$ jq '.employees[] | {number: .number, tags: .tags | from_entries} | select(.tags.management=="no") | {number: .number, name: .tags.name, login: .tags.login}' input
... and the output is:
{
"number": "102",
"name": "jane blogs",
"login": "jane"
}
{
"number": "103",
"name": "john doe",
"login": "john"
}
There may be a better way to achieve what I wanted, so I'll leave the question open for a while if someone wants to offer a better solution.
Here is another solution which uses from_entries
.employees[]
| {number} + (.tags | from_entries)
| if .management == "no" then {number, name, login} else empty end