Can I output the ip and source id only when source id is duplicate it should out put all ip in one array if no duplicate ip can be output with corresponding source id
{"ip":"192.134.5.31","access_key":"223434354656767","source_id":"2e74a68a-2fef-443544-815d-87"}
{"ip":"172.23.54.4","saccess_key":"223434354656767","source_id":"2e74a68a-2fef-443544-815d-87"}
{"ip":"182,555,44.44","access_key":"223434354656767","source_id":"2e74a68a-2fef-443544-815d-222"}
I dont care about access key here also if this access key can be done with Jq would be great
unique_by(.ip) |{ip.source_id[]}
.ip| select(.source_id[])
You should group_by to group all the matching source_id.
Then you can create the desired output, for example:
group_by(.source_id)[] | { ip: map(.ip), source_id: (first.source_id) }
Will output:
{
"ip": [
"182,555,44.44"
],
"source_id": "2e74a68a-2fef-443544-815d-222"
}
{
"ip": [
"192.134.5.31",
"172.23.54.4"
],
"source_id": "2e74a68a-2fef-443544-815d-87"
}
Since
We group on source_id
We create an object for each group, containing
An map() from all the lower ip's and the source_id taken from the first object
Use the --slurp option to combine those objects in to an array:
jq --slurp 'group_by(.source_id)[] | { ip: map(.ip), source_id: (first.source_id) }'
JqPlay Demo
I'm not sure what your expected output is, but if you are trying to group IPs by source id like such:
{
"2e74a68a-2fef-443544-815d-222": [
"182,555,44.44"
],
"2e74a68a-2fef-443544-815d-87": [
"192.134.5.31",
"172.23.54.4"
]
}
Then you can group by your source_id and then transform the output:
group_by(.source_id) | map({(.[0].source_id): map(.ip)}) | add
Or by using a custom function:
def group(k): group_by(k) | map({key:first|k, value:.}) | from_entries;
group(.source_id) | map_values(map(.ip))
or
def group(k;v): group_by(k) | map({key:first|k, value:map(v)}) | from_entries;
group(.source_id;.ip)
Using code below to pull data from a local json file.
The file is very large and is nested with objects and arrays. There are multiple objects in the .ratings[] that I would like to extract.
How can I use the pipe operator in the .ratings[] array so that I don't have to retype .ratings[] for each piece of data that I would like to pull?
jq -r '.players[] | [.firstName,.lastName,.tid,.pid,.ratings[].spd,.ratings[].jmp] | join(", ")'
You can enclose it in () to use the pipe sign:
.players[] | [.firstName, .lastName, .tid, .pid, (.ratings[] | .spd, .jmp)] | join(", ")
Try it online
You didn't specify the expected output, so it is not clear if your proposed solution gives you the output you want.
Given the following input:
{
"players": [
{
"firstName": "fname1",
"lastName": "lname1",
"tid": "tid1",
"pid": "pid1",
"ratings": [
{
"spd": "spd1-1",
"jmp": "jmp1-1"
}
]
},
{
"firstName": "fname2",
"lastName": "lname2",
"tid": "tid2",
"pid": "pid2",
"ratings": [
{
"spd": "spd2-1",
"jmp": "jmp2-1"
},
{
"spd": "spd2-2",
"jmp": "jmp2-2"
}
]
},
{
"firstName": "fname3",
"lastName": "lname3",
"tid": "tid3",
"pid": "pid3",
"ratings": [
{
"spd": "spd3-1",
"jmp": "jmp3-2"
},
{
"spd": "spd3-2",
"jmp": "jmp3-2"
},
{
"spd": "spd3-3",
"jmp": "jmp3-3"
}
]
}
]
}
Your solution and the answer from 0ston0 will give you 1 line per player, but a different number of columns per line:
.players[] | [.firstName,.lastName,.tid,.pid,(.ratings[]|.spd,.jmp)] | join(", ")
generates:
fname1, lname1, tid1, pid1, spd1-1, jmp1-1
fname2, lname2, tid2, pid2, spd2-1, jmp2-1, spd2-2, jmp2-2
fname3, lname3, tid3, pid3, spd3-1, jmp3-2, spd3-2, jmp3-2, spd3-3, jmp3-3
This might or might not be what want your result to look like.
A different solution will print one line per rating, but duplicate the players' names. Running:
.players[] | [.firstName,.lastName,.tid,.pid] + (.ratings[]|[.spd,.jmp]) | join(", ")
will result in:
fname1, lname1, tid1, pid1, spd1-1, jmp1-1
fname2, lname2, tid2, pid2, spd2-1, jmp2-1
fname2, lname2, tid2, pid2, spd2-2, jmp2-2
fname3, lname3, tid3, pid3, spd3-1, jmp3-2
fname3, lname3, tid3, pid3, spd3-2, jmp3-2
fname3, lname3, tid3, pid3, spd3-3, jmp3-3
Both solutions are valid for different use cases and depending on how you are going to subsequently process the data.
I have a JSON object where the relevant parts are of the form
{
"_meta": {
"hostvars": {
"name_1": {
"ansible_host": "10.0.0.1"
},
"name_2": {
"ansible_host": "10.0.0.2"
},
"name_3": {
"ansible_host": "10.0.0.3"
}
}
},
...
"nodes": {
"hosts": [
"name_1",
"name_2"
]
}
}
(the output of ansible-inventory --list, for reference).
I would like to use jq to produce a list of IPs of the nodes hosts by looking up the names in ._meta.hostvars. In the example, the output should be:
10.0.0.1
10.0.0.2
Note that 10.0.0.3 should not be included because name_3 is not in the .nodes.hosts list. So just doing jq -r '._meta.hostvars[].ansible_host' doesn't work.
I've tried jq '.nodes.hosts[] | ._meta.hostvars[.].ansible_host' but that fails because ._meta doesn't scan from the root after the pipe.
You can store the root in a variable before changing the context:
jq -r '. as $root | .nodes.hosts[] | $root._meta.hostvars[.].ansible_host'
But a better solution is to just inline the "hosts" query:
jq -r '._meta.hostvars[.nodes.hosts[]].ansible_host'
Given the json
{ "games": [
{
"id":1,
"files": [ "foo.mp4" ]
},
{
"id":2,
"files": [ "foo.ogv", "bar.ogv" ]
},
{
"id":3,
"files": [ "bar.ogv" ]
}
]}
and the command
jq -r '.games[] | select(.files[] | contains("ogv"))' foo.json, json outputs an element once for every time it matches ogv in the subelement array. How do I get jq to output each matching element only once?
Using any would be more efficient than relying on unique. E.g.
jq -r '.games[] | select(any(.files[]; test("ogv")))'
jq -r '[.games[] | select(.files[] | contains("ogv"))] | unique | .[]' foo.json
or, since what I really want is just the id,
jq -r '[.games[] | select(.files[] | contains("ogv")) | .id] | unique | .[]' foo.json
I have an array of objects of various types which reference one another with UUIDs (a terraform.tfstate file). I'd like to select one value from one such object based on the appearance of a different value in another object, where the two objects are related by one of those UUIDs.
By way of example, I can do this:
$ jq '.modules[].resources[]
| select(.type == "openstack_compute_instance_v2" and
.primary.attributes.name == "jumpbox").primary.id' terraform.tfstate
"5edfe2bf-94df-49d5-8118-3e91fb52946b"
$ jq '.modules[].resources[]
| select(.type =="openstack_compute_floatingip_associate_v2" and
.primary.attributes.instance_id == "5edfe2bf-94df-49d5-8118-3e91fb52946b").primary.attributes.floating_ip' terraform.tfstate
"10.120.241.21"
Giving me the external floating IP of the 'jumpbox' VM based on its name.
I'd like to make that all one jq call. Is that possible?
This would be easier to answer if you provided more sample data but
working backwards from your commands (with some reformatting)
$ jq '
.modules[].resources[]
| select(.type == "openstack_compute_instance_v2" and .primary.attributes.name == "jumpbox")
| .primary.id
' terraform.tfstate
"5edfe2bf-94df-49d5-8118-3e91fb52946b"
$ jq '
.modules[].resources[]
| select(.type =="openstack_compute_floatingip_associate_v2" and .primary.attributes.instance_id == "5edfe2bf-94df-49d5-8118-3e91fb52946b")
| .primary.attributes.floating_ip
' terraform.tfstate
"10.120.241.21"
we can infer you have data which looks like
{
"modules": [
{
"resources": [
{
"type": "openstack_compute_instance_v2",
"primary": {
"id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"attributes": {
"name": "jumpbox"
}
}
},
{
"type": "openstack_compute_floatingip_associate_v2",
"primary": {
"attributes": {
"instance_id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"floating_ip": "10.120.241.21"
}
}
}
]
}
]
}
The following filter demonstrates a solution using functions, variables and parenthesis ():
def get_primary_id($name):
select(.type == "openstack_compute_instance_v2"
and .primary.attributes.name == $name)
| .primary.id
;
def get_floating_ip($id):
select(.type =="openstack_compute_floatingip_associate_v2"
and .primary.attributes.instance_id == $id)
| .primary.attributes.floating_ip
;
.modules[]
| ( .resources[] | get_primary_id("jumpbox") ) as $id
| ( .resources[] | get_floating_ip($id) ) as $fip
| ($id, $fip)
if this filter is in filter.jq and data.json contains the sample data above
then
$ jq -M -f filter.jq data.json
produces the output:
"5edfe2bf-94df-49d5-8118-3e91fb52946b"
"10.120.241.21"