Here my jq script:
def pick_nationality:
select(.NACIONALITAT) |
{nation: {country: .NACIONALITAT, code: "some code"} };
def pick_surname:
select(.SURNAME) |
{name: {surname: .SURNAME, code: "some code"} };
def pick_extension:
{ use: "official", extension: [pick_nationality, pick_surname] };
map(pick_extension)
Input json is like:
{
"SURNAME": "surname1"
}
{
"NACIONALITAT": "nacionalitat1"
}
However, sometimes any input objects don't contain any look up field:
{
"field1": "value1"
}
{
"field2": "value2"
}
Above script returns:
[
{
"use": "official",
"extension": []
},
{
"use": "official",
"extension": []
}
]
I'd like extension doesn't appear:
[
{
"use": "official"
},
{
"use": "official"
}
]
Any ideas?
You can simply add
| del(..|select(. == []))
as a trailing to your script in order to remove all such empty arrays
Demo
extend your function pick_extension for the desired output:
def pick_extension:
[pick_nationality, pick_surname] as $extension
| { use: "official" }
| if $extension | length > 0 then . + {extension: $extension} else . end;
If no extension could be picked, the empty array will no longer be added to the json object this way.
Related
So, given two files with JSON data from the same source type. The JSON objects look something like:
file1:
[
{
"data": {
"id": "2",
"nodes": [
{
"stuff": "foo"
}
]
}
},
{
"data": {
"id": "6",
"nodes": [
{
"stuff": "bar"
}
]
}
},
{
"data": {
"id": "61",
"nodes": [
{
"stuff": "baz"
}
]
}
},
{
"data": {
"id": "63",
"nodes": [
{
"stuff": "qux"
}
]
}
}
]
file2:
[
{
"data": {
"id": "61",
"nodes": [
{
"stuff": "baz"
}
]
}
},
{
"data": {
"id": "63",
"nodes": [
{
"stuff": "qux"
}
]
}
}
]
I'm trying to remove objects in the array in the first file with the matching IDs in the second file so that the resultant output would be:
[
{
"data": {
"id": "2",
"nodes": [
{
"stuff": "foo"
}
]
}
},
{
"data": {
"id": "6",
"nodes": [
{
"stuff": "bar"
}
]
}
}
]
I've tried a bunch of ways to accomplish this, but I haven't found a proper solution yet.
A couple of attempts have been various permutations of the following with accompanying errors:
jq -n --argfile src /var/tmp/w-src.json --argfile dst /var/tmp/w-dst.json '
$dst
| [.data[].id] as $ids
| $src
| .data | map(select(.id | in($ids[])))
jq: error: select/0 is not defined at <top-level>, line 5:
| .data | map($ids | map(select .id == .))
jq: 1 compile error
jq -n --argfile src /var/tmp/w-src.json --argfile dst /var/tmp/w-dst.json '
$dst
| [.data[].id] as $ids
| $src
| .data[] | select(.id | in($ids[]))
'
jq: error (at <unknown>): Cannot check whether string has a string key
Ideally it would be super cool to do some kind of operation like:
$src.data[] - $dst.data[]
(kinda Ruby-ish like would be cool) and I admit, I haven't tried this but I will for kicks and giggles.
I'm trying not to have to use a function and I want to accomplish this using jq. I'm probably not too far off, but I'm at a loss. Any thoughts?
You could compile a list of IDs from the second file using input, check against it using IN, and either use del to delete the matching, or map to keep those that do not match:
jq '
(input | map(.data.id)) as $del | del(.[] | select(IN(.data.id; $del[])))
' file1.json file2.json
or
jq '
(input | map(.data.id)) as $del | map(select(IN(.data.id; $del[]) | not))
' file1.json file2.json
If you can assert that objects with identical IDs also are identical in their other parts, and you don't have many items (because it's costly), you can even just subtract the second file from the first:
jq '. - input' file1.json file2.json
I am trying to delete a key whose value is duplicated elsewhere. That is I would like to delete all occurences(duplicates) after the first occurence. Here is a sample json file I am working with
{
"clouds":{
"finfolk-vmaas":{
"auth-types":[
"oauth1"
],
"endpoint":"http://10.125.0.10:5240/MAAS/",
"type":"maas"
},
"vsphere":{
"auth-types":[
"userpass"
],
"endpoint":"10.247.0.3",
"regions":{
"QA":{
"endpoint":"10.247.0.3"
}
},
"type":"vsphere"
}
}
}
I would like to get this after the deletion:
{
"clouds":{
"finfolk-vmaas":{
"auth-types":[
"oauth1"
],
"endpoint":"http://10.125.0.10:5240/MAAS/",
"type":"maas"
},
"vsphere":{
"auth-types":[
"userpass"
],
"endpoint":"10.247.0.3",
"regions":{
"QA":{}
},
"type":"vsphere"
}
}
}
Essentially I want to remove this duplicate key:pair "endpoint":"10.247.0.3" and leave the enclosing parentheses {}
Here is a simple jq query that I am trying to play with:
jq -cs 'unique_by(.endpoint)' clouds.json
For each object in .clouds[], this saves the object reduced to its enpoint as $endpoint, then recursively traverses to all child objects, from which, if it contains the previously stored endpoint, (only) the endpoint field will be deleted.
.clouds[] |= ({endpoint} as $endpoint | .[] |= walk(
(objects | select(contains($endpoint))) |= del(.endpoint)
))
{
"clouds": {
"finfolk-vmaas": {
"auth-types": [
"oauth1"
],
"endpoint": "http://10.125.0.10:5240/MAAS/",
"type": "maas"
},
"vsphere": {
"auth-types": [
"userpass"
],
"endpoint": "10.247.0.3",
"regions": {
"QA": {}
},
"type": "vsphere"
}
}
}
Demo
I am new to jq and stuck with this problem for a while. Any help is appreciable.
I have two json files,
In file1.json:
{
"version": 4,
"group1": [
{
"name":"olditem1",
"content": "old content"
}
],
"group2": [
{
"name":"olditem2"
}
]
}
And in file2.json:
{
"group1": [
{
"name" : "newitem1"
},
{
"name":"olditem1",
"content": "new content"
}
],
"group2": [
{
"name" : "newitem2"
}
]
}
Expected result is:
{
"version": 4,
"group1": [
{
"name":"olditem1",
"content": "old content"
},
{
"name" : "newitem1"
}
],
"group2": [
{
"name":"olditem2"
},
{
"name" : "newitem2"
}
]
}
Criterial for merge:
Has to merge only group1 and group2
Match only by name
I have tried
jq -S '.group1+=.group1|.group1|unique_by(.name)' file1.json file2.json
but this is filtering group1 and all other info are lost.
This approach uses INDEX to create a dictionary of unique elements based on their .name field, reduce to iterate over the group fields to be considered, and an initial state created by combining the slurped (-s) input files using add after removing the group fileds to be processed separately using del.
jq -s '
[ "group1", "group2" ] as $gs | . as $in | reduce $gs[] as $g (
map(del(.[$gs[]])) | add; .[$g] = [INDEX($in[][$g][]; .name)[]]
)
' file1.json file2.json
{
"version": 4,
"group1": [
{
"name": "olditem1",
"content": "new content"
},
{
"name": "newitem1"
}
],
"group2": [
{
"name": "olditem2"
},
{
"name": "newitem2"
}
]
}
Demo
Here my document:
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": [
{
"system": {
"value": "urn:oid:1.3.6.1.4.1.19126.3"
},
"value": {
"value": "Y3454867M"
}
},
{
"system": {
"value": "urn:oid:2.16.724.4.9.10.2"
},
"value": {
"value": "108505134"
}
}
]
}
]
I need to pick only .identifier[where .system.value == "urn:oid:1.3.6.1.4.1.19126.3"] and project .identifier.value.value.
Desired output:
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": "Y3454867M"
}
]
I've been playing with map and select but I don't quite figure out what's the right way to get it.
Any ideas?
This approach uses first to get the first result, in case there is more than one array item matching the criteria.
jq --arg v "urn:oid:1.3.6.1.4.1.19126.3" '
map(.identifier |= first(.[] | select(.system.value == $v).value.value))
'
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": "Y3454867M"
}
]
Demo
Right on the money with the good ol' select tool, since you need data from an arbitrary index. I fumbled a bit before I unwrapped the inner array that gets piped to my select.
jq -r '.[] | [{id: .id, identifier: .identifier | .[] | select(.system.value | contains("urn:oid:1.3.6.1.4.1.19126.3")) | .value.value }]'
Still new to jq myself, so any feedback is welcome.
This is my jq code:
def pick_nationality:
{nation: {country: .NACIONALITAT, code: "some code"} };
def pick_surname:
{name: {surname: .SURNAME, code: "some code"} };
map([pick_nationality, pick_surname])
Problem here appears when some of .NACIONALITAT or .SURNAME is not present on input objects:
{
"SURNAME": "surname1"
}
{
"NACIONALITAT": "nacionalitat1"
}
Result:
[
[
{
"nation": {
"country": null,
"code": "some code"
}
},
{
"name": {
"surname": "surname1",
"code": "some code"
}
}
],
[
{
"nation": {
"country": "nacionalitat1",
"code": "some code"
}
},
{
"name": {
"surname": null,
"code": "some code"
}
}
]
]
Problem is I need to avoid pick_natinality when .NATIONALITAT field is not present...
Desired result would be:
[
[
{
"name": {
"surname": "surname1",
"code": "some code"
}
}
],
[
{
"nation": {
"country": "nacionalitat1",
"code": "some code"
}
}
]
]
Any ideas?
Just have the functions return nothing when the relevant field is missing.
def pick_nationality:
select(.NACIONALITAT) |
{nation: {country: .NACIONALITAT, code: "some code"} };
def pick_surname:
select(.SURNAME) |
{name: {surname: .SURNAME, code: "some code"} };
map([pick_nationality, pick_surname])
Demo on jqplay
You can use del for the respective null values such as
jq -r 'del(.[][] | select(.nation.country == null and .name.surname== null))'
Demo
You can add a test before capture the value :
def pick_nationality:
if has("NACIONALITAT")
then {nation: {country: .NACIONALITAT, code: "some code"} }
else empty end;
def pick_surname:
if has("SURNAME")
then {name: {surname: .SURNAME, code: "some code"} }
else empty end;
map([pick_nationality, pick_surname])