I am trying to retrieve the values containg contoso. So my output in this case should just readdata_contoso and writedata_contoso but I am having issues. I am trying jq -e -r ".roles[] | select(.name | contains("cont"))"
{
"roles": [
{
"name": "readdata_contoso",
"permissions": {
"": [
"ReadData"
]
}
},
{
"name": "writedata_contoso",
"permissions": {
"": [
"WriteData"
]
}
}
]
}
You are almost there. First, extract only the name of each role, then select those names which contain "contoso":
.roles[].name | select(contains("contoso"))
roles[].name generates a list of all names
select(contains("contoso")) only selects those names which contain "contoso"
Related
So, given two files with JSON data from the same source type. The JSON objects look something like:
file1:
[
{
"data": {
"id": "2",
"nodes": [
{
"stuff": "foo"
}
]
}
},
{
"data": {
"id": "6",
"nodes": [
{
"stuff": "bar"
}
]
}
},
{
"data": {
"id": "61",
"nodes": [
{
"stuff": "baz"
}
]
}
},
{
"data": {
"id": "63",
"nodes": [
{
"stuff": "qux"
}
]
}
}
]
file2:
[
{
"data": {
"id": "61",
"nodes": [
{
"stuff": "baz"
}
]
}
},
{
"data": {
"id": "63",
"nodes": [
{
"stuff": "qux"
}
]
}
}
]
I'm trying to remove objects in the array in the first file with the matching IDs in the second file so that the resultant output would be:
[
{
"data": {
"id": "2",
"nodes": [
{
"stuff": "foo"
}
]
}
},
{
"data": {
"id": "6",
"nodes": [
{
"stuff": "bar"
}
]
}
}
]
I've tried a bunch of ways to accomplish this, but I haven't found a proper solution yet.
A couple of attempts have been various permutations of the following with accompanying errors:
jq -n --argfile src /var/tmp/w-src.json --argfile dst /var/tmp/w-dst.json '
$dst
| [.data[].id] as $ids
| $src
| .data | map(select(.id | in($ids[])))
jq: error: select/0 is not defined at <top-level>, line 5:
| .data | map($ids | map(select .id == .))
jq: 1 compile error
jq -n --argfile src /var/tmp/w-src.json --argfile dst /var/tmp/w-dst.json '
$dst
| [.data[].id] as $ids
| $src
| .data[] | select(.id | in($ids[]))
'
jq: error (at <unknown>): Cannot check whether string has a string key
Ideally it would be super cool to do some kind of operation like:
$src.data[] - $dst.data[]
(kinda Ruby-ish like would be cool) and I admit, I haven't tried this but I will for kicks and giggles.
I'm trying not to have to use a function and I want to accomplish this using jq. I'm probably not too far off, but I'm at a loss. Any thoughts?
You could compile a list of IDs from the second file using input, check against it using IN, and either use del to delete the matching, or map to keep those that do not match:
jq '
(input | map(.data.id)) as $del | del(.[] | select(IN(.data.id; $del[])))
' file1.json file2.json
or
jq '
(input | map(.data.id)) as $del | map(select(IN(.data.id; $del[]) | not))
' file1.json file2.json
If you can assert that objects with identical IDs also are identical in their other parts, and you don't have many items (because it's costly), you can even just subtract the second file from the first:
jq '. - input' file1.json file2.json
I am trying to delete a key whose value is duplicated elsewhere. That is I would like to delete all occurences(duplicates) after the first occurence. Here is a sample json file I am working with
{
"clouds":{
"finfolk-vmaas":{
"auth-types":[
"oauth1"
],
"endpoint":"http://10.125.0.10:5240/MAAS/",
"type":"maas"
},
"vsphere":{
"auth-types":[
"userpass"
],
"endpoint":"10.247.0.3",
"regions":{
"QA":{
"endpoint":"10.247.0.3"
}
},
"type":"vsphere"
}
}
}
I would like to get this after the deletion:
{
"clouds":{
"finfolk-vmaas":{
"auth-types":[
"oauth1"
],
"endpoint":"http://10.125.0.10:5240/MAAS/",
"type":"maas"
},
"vsphere":{
"auth-types":[
"userpass"
],
"endpoint":"10.247.0.3",
"regions":{
"QA":{}
},
"type":"vsphere"
}
}
}
Essentially I want to remove this duplicate key:pair "endpoint":"10.247.0.3" and leave the enclosing parentheses {}
Here is a simple jq query that I am trying to play with:
jq -cs 'unique_by(.endpoint)' clouds.json
For each object in .clouds[], this saves the object reduced to its enpoint as $endpoint, then recursively traverses to all child objects, from which, if it contains the previously stored endpoint, (only) the endpoint field will be deleted.
.clouds[] |= ({endpoint} as $endpoint | .[] |= walk(
(objects | select(contains($endpoint))) |= del(.endpoint)
))
{
"clouds": {
"finfolk-vmaas": {
"auth-types": [
"oauth1"
],
"endpoint": "http://10.125.0.10:5240/MAAS/",
"type": "maas"
},
"vsphere": {
"auth-types": [
"userpass"
],
"endpoint": "10.247.0.3",
"regions": {
"QA": {}
},
"type": "vsphere"
}
}
}
Demo
I am new to jq and stuck with this problem for a while. Any help is appreciable.
I have two json files,
In file1.json:
{
"version": 4,
"group1": [
{
"name":"olditem1",
"content": "old content"
}
],
"group2": [
{
"name":"olditem2"
}
]
}
And in file2.json:
{
"group1": [
{
"name" : "newitem1"
},
{
"name":"olditem1",
"content": "new content"
}
],
"group2": [
{
"name" : "newitem2"
}
]
}
Expected result is:
{
"version": 4,
"group1": [
{
"name":"olditem1",
"content": "old content"
},
{
"name" : "newitem1"
}
],
"group2": [
{
"name":"olditem2"
},
{
"name" : "newitem2"
}
]
}
Criterial for merge:
Has to merge only group1 and group2
Match only by name
I have tried
jq -S '.group1+=.group1|.group1|unique_by(.name)' file1.json file2.json
but this is filtering group1 and all other info are lost.
This approach uses INDEX to create a dictionary of unique elements based on their .name field, reduce to iterate over the group fields to be considered, and an initial state created by combining the slurped (-s) input files using add after removing the group fileds to be processed separately using del.
jq -s '
[ "group1", "group2" ] as $gs | . as $in | reduce $gs[] as $g (
map(del(.[$gs[]])) | add; .[$g] = [INDEX($in[][$g][]; .name)[]]
)
' file1.json file2.json
{
"version": 4,
"group1": [
{
"name": "olditem1",
"content": "new content"
},
{
"name": "newitem1"
}
],
"group2": [
{
"name": "olditem2"
},
{
"name": "newitem2"
}
]
}
Demo
Here my document:
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": [
{
"system": {
"value": "urn:oid:1.3.6.1.4.1.19126.3"
},
"value": {
"value": "Y3454867M"
}
},
{
"system": {
"value": "urn:oid:2.16.724.4.9.10.2"
},
"value": {
"value": "108505134"
}
}
]
}
]
I need to pick only .identifier[where .system.value == "urn:oid:1.3.6.1.4.1.19126.3"] and project .identifier.value.value.
Desired output:
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": "Y3454867M"
}
]
I've been playing with map and select but I don't quite figure out what's the right way to get it.
Any ideas?
This approach uses first to get the first result, in case there is more than one array item matching the criteria.
jq --arg v "urn:oid:1.3.6.1.4.1.19126.3" '
map(.identifier |= first(.[] | select(.system.value == $v).value.value))
'
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": "Y3454867M"
}
]
Demo
Right on the money with the good ol' select tool, since you need data from an arbitrary index. I fumbled a bit before I unwrapped the inner array that gets piped to my select.
jq -r '.[] | [{id: .id, identifier: .identifier | .[] | select(.system.value | contains("urn:oid:1.3.6.1.4.1.19126.3")) | .value.value }]'
Still new to jq myself, so any feedback is welcome.
I am trying to extract kubeconfig data with jq.
kubectl config view --raw -o json | jq ...
There is a json produced of this kind:
{
"kind": "Config",
"apiVersion": "v1",
"preferences": {},
"clusters": [
{
"name": "some-name",
"cluster": {
"server": "https://some-url",
"certificate-authority-data": "some-cert"
}
},
{
"name": "another-name",
"cluster": {
"server": "https://another-url",
"certificate-authority-data": "another-cert"
}
}
],
"users": [
{
"name": "some-name",
"user": {
"username": "some-user",
"password": "some-password"
}
},
{
"name": "another-name",
"user": {
"username": "another-user",
"password": "another-password"
}
}
],
"contexts": [],
"current-context": "some-context"
}
Question #1:
For a given name ,"some-name", I'd like to extract json:
{
url: "https://some-url",
cert: "some-cert",
username: "some-user",
password: "some-password"
}
Question #2:
"users" sub-section can have other format
"users": [
{
"name": "...",
"user": {
"exec": {
...
}
Where .user.username or .user.password or both can be missing
In this case overall query should return "{}", even though, "clusters" query/branch has result
Question 3, as a follow up to Jeff Mercado answer:
I want to get all clusters, joined (grouped by) name:
Looking at the manual, https://stedolan.github.io/jq/manual/#Builtinoperatorsandfunctions ,
section "Multiplication, division, modulo: *, /, and %", example:
jq '{"k": {"a": 1, "b": 2}} * {"k": {"a": 0,"c": 3}}' => {"k": {"a": 0, "b": 2, "c": 3}}'
gives presumably right result, assuming "k" is value of "name". So, grouping by "k", merging (*) the results.
I produced following query:
echo "${json}" | jq -r '(.clusters[] | {(.name): {url: .cluster.server, cert: .cluster["certificate-authority-data"]}}) * (.users[] | {(.name): {user: .user.username, password: .user.password}})'
First part returns {"name": {url: cert}}, second part is {"name": {username, password}}
However, result is not merge as in jq Manual, but something else ... product ?
{
"some-name": {
"url": "https://some-url",
"cert": "some-cert",
"user": "some-user",
"password": "some-password"
}
}
{
"another-name": {
"url": "https://another-url",
"cert": "another-cert"
},
"some-name": {
"user": "some-user",
"password": "some-password"
}
}
{
"some-name": {
"url": "https://some-url",
"cert": "some-cert"
},
"another-name": {
"user": "another-user",
"password": "another-password"
}
}
{
"another-name": {
"url": "https://another-url",
"cert": "another-cert",
"user": "another-user",
"password": "another-password"
}
}
Why/what is it ? Kind of following idea of the product ('*') but not of the jq tutorial as I (most likely, incorrectly) understand it
Experimentation:
I have 2 queries now producing partial result.
Let's grab original json (above) in and parse:
read -d '' json << EOF
...
EOF
queries:
echo "${json}" | jq -r '.clusters[] | select(.name=="some-name") | .cluster | {url: .server, cert: .["certificate-authority-data"]}' &&\
echo "${json}" | jq -r '.users[] | select(.name=="some-name") | .user | {user: .username, password: .password}'
Will produce the split output:
{
"url": "https://some-url",
"cert": "some-cert"
}
{
"user": "some-user",
"password": "some-password"
}
Or, with key added for further merge:
echo "${json}" | jq -r '.clusters[] | select(.name=="some-name") | {name: .name, url: .cluster.server, cert: .cluster["certificate-authority-data"]}' &&\
echo "${json}" | jq -r '.users[] | select(.name=="some-name") | {name: .name, user: .user.username, password: .user.password}'
Will produce:
{
"name": "some-name",
"url": "https://some-url",
"cert": "some-cert"
}
{
"name": "some-name",
"user": "some-user",
"password": "some-password"
}
"name" is not needed but can be used as a join operation
So you already know how to get the cluster and user by name separately, first step is to select them both within a single filter:
(.clusters[] | select(.name == $name).cluster), (.users[] | select(.name == $name).user)
This will yield two separate objects, the cluster, then the user. But we want to merge them. There's plenty of ways to do this. You could add them (+) directly or merge them (*) but no real difference there. You'll just want to remap the properties to the names you wanted where needed.
(.clusters[] | select(.name == $name).cluster | {url: .server, cert: ."certificate-authority-data"})
+
(.users[] | select(.name == $name).user | {username, password})
Pass the name in as a parameter to your filter;
$ kubectl config view --raw -o json | jq --arg name some-name '
(.clusters[] | select(.name == $name).cluster | {url: .server, cert: ."certificate-authority-data"})
+
(.users[] | select(.name == $name).user | {username, password})
'
For the second part of your question, if it turns out the mapped user is missing key properties and you want to omit them, just add another select filter to the end to test for those properties and replace with an empty object if nothing is found:
... | select(has("username") and has("password")) // {}
jqplay