example.json
{
"foo": {
"bar": {
"name": "bob",
"role": "gopherthis"
},
"baz": {
"name": "tom",
"role": "gopherthat"
}
}
}
example query:
jq -r '.foo[] | " \(.name),\(.role) " ' example.json
bob,gopherthis
tom,gopherthat
My question is how do I get bar and baz associated with their name and role? Where the output would be:
bar,bob,gopherthis
baz,tom,gopherthat
jq solution:
jq -r '.foo | to_entries[] | "\(.key),\(.value.name),\(.value.role)"' example.json
The output:
bar,bob,gopherthis
baz,tom,gopherthat
Related
So, given two files with JSON data from the same source type. The JSON objects look something like:
file1:
[
{
"data": {
"id": "2",
"nodes": [
{
"stuff": "foo"
}
]
}
},
{
"data": {
"id": "6",
"nodes": [
{
"stuff": "bar"
}
]
}
},
{
"data": {
"id": "61",
"nodes": [
{
"stuff": "baz"
}
]
}
},
{
"data": {
"id": "63",
"nodes": [
{
"stuff": "qux"
}
]
}
}
]
file2:
[
{
"data": {
"id": "61",
"nodes": [
{
"stuff": "baz"
}
]
}
},
{
"data": {
"id": "63",
"nodes": [
{
"stuff": "qux"
}
]
}
}
]
I'm trying to remove objects in the array in the first file with the matching IDs in the second file so that the resultant output would be:
[
{
"data": {
"id": "2",
"nodes": [
{
"stuff": "foo"
}
]
}
},
{
"data": {
"id": "6",
"nodes": [
{
"stuff": "bar"
}
]
}
}
]
I've tried a bunch of ways to accomplish this, but I haven't found a proper solution yet.
A couple of attempts have been various permutations of the following with accompanying errors:
jq -n --argfile src /var/tmp/w-src.json --argfile dst /var/tmp/w-dst.json '
$dst
| [.data[].id] as $ids
| $src
| .data | map(select(.id | in($ids[])))
jq: error: select/0 is not defined at <top-level>, line 5:
| .data | map($ids | map(select .id == .))
jq: 1 compile error
jq -n --argfile src /var/tmp/w-src.json --argfile dst /var/tmp/w-dst.json '
$dst
| [.data[].id] as $ids
| $src
| .data[] | select(.id | in($ids[]))
'
jq: error (at <unknown>): Cannot check whether string has a string key
Ideally it would be super cool to do some kind of operation like:
$src.data[] - $dst.data[]
(kinda Ruby-ish like would be cool) and I admit, I haven't tried this but I will for kicks and giggles.
I'm trying not to have to use a function and I want to accomplish this using jq. I'm probably not too far off, but I'm at a loss. Any thoughts?
You could compile a list of IDs from the second file using input, check against it using IN, and either use del to delete the matching, or map to keep those that do not match:
jq '
(input | map(.data.id)) as $del | del(.[] | select(IN(.data.id; $del[])))
' file1.json file2.json
or
jq '
(input | map(.data.id)) as $del | map(select(IN(.data.id; $del[]) | not))
' file1.json file2.json
If you can assert that objects with identical IDs also are identical in their other parts, and you don't have many items (because it's costly), you can even just subtract the second file from the first:
jq '. - input' file1.json file2.json
Here my document:
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": [
{
"system": {
"value": "urn:oid:1.3.6.1.4.1.19126.3"
},
"value": {
"value": "Y3454867M"
}
},
{
"system": {
"value": "urn:oid:2.16.724.4.9.10.2"
},
"value": {
"value": "108505134"
}
}
]
}
]
I need to pick only .identifier[where .system.value == "urn:oid:1.3.6.1.4.1.19126.3"] and project .identifier.value.value.
Desired output:
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": "Y3454867M"
}
]
I've been playing with map and select but I don't quite figure out what's the right way to get it.
Any ideas?
This approach uses first to get the first result, in case there is more than one array item matching the criteria.
jq --arg v "urn:oid:1.3.6.1.4.1.19126.3" '
map(.identifier |= first(.[] | select(.system.value == $v).value.value))
'
[
{
"id": "9f0e27fe-3b8f-4857-8e1d-e57e7a3f4c31",
"identifier": "Y3454867M"
}
]
Demo
Right on the money with the good ol' select tool, since you need data from an arbitrary index. I fumbled a bit before I unwrapped the inner array that gets piped to my select.
jq -r '.[] | [{id: .id, identifier: .identifier | .[] | select(.system.value | contains("urn:oid:1.3.6.1.4.1.19126.3")) | .value.value }]'
Still new to jq myself, so any feedback is welcome.
I am trying to extract kubeconfig data with jq.
kubectl config view --raw -o json | jq ...
There is a json produced of this kind:
{
"kind": "Config",
"apiVersion": "v1",
"preferences": {},
"clusters": [
{
"name": "some-name",
"cluster": {
"server": "https://some-url",
"certificate-authority-data": "some-cert"
}
},
{
"name": "another-name",
"cluster": {
"server": "https://another-url",
"certificate-authority-data": "another-cert"
}
}
],
"users": [
{
"name": "some-name",
"user": {
"username": "some-user",
"password": "some-password"
}
},
{
"name": "another-name",
"user": {
"username": "another-user",
"password": "another-password"
}
}
],
"contexts": [],
"current-context": "some-context"
}
Question #1:
For a given name ,"some-name", I'd like to extract json:
{
url: "https://some-url",
cert: "some-cert",
username: "some-user",
password: "some-password"
}
Question #2:
"users" sub-section can have other format
"users": [
{
"name": "...",
"user": {
"exec": {
...
}
Where .user.username or .user.password or both can be missing
In this case overall query should return "{}", even though, "clusters" query/branch has result
Question 3, as a follow up to Jeff Mercado answer:
I want to get all clusters, joined (grouped by) name:
Looking at the manual, https://stedolan.github.io/jq/manual/#Builtinoperatorsandfunctions ,
section "Multiplication, division, modulo: *, /, and %", example:
jq '{"k": {"a": 1, "b": 2}} * {"k": {"a": 0,"c": 3}}' => {"k": {"a": 0, "b": 2, "c": 3}}'
gives presumably right result, assuming "k" is value of "name". So, grouping by "k", merging (*) the results.
I produced following query:
echo "${json}" | jq -r '(.clusters[] | {(.name): {url: .cluster.server, cert: .cluster["certificate-authority-data"]}}) * (.users[] | {(.name): {user: .user.username, password: .user.password}})'
First part returns {"name": {url: cert}}, second part is {"name": {username, password}}
However, result is not merge as in jq Manual, but something else ... product ?
{
"some-name": {
"url": "https://some-url",
"cert": "some-cert",
"user": "some-user",
"password": "some-password"
}
}
{
"another-name": {
"url": "https://another-url",
"cert": "another-cert"
},
"some-name": {
"user": "some-user",
"password": "some-password"
}
}
{
"some-name": {
"url": "https://some-url",
"cert": "some-cert"
},
"another-name": {
"user": "another-user",
"password": "another-password"
}
}
{
"another-name": {
"url": "https://another-url",
"cert": "another-cert",
"user": "another-user",
"password": "another-password"
}
}
Why/what is it ? Kind of following idea of the product ('*') but not of the jq tutorial as I (most likely, incorrectly) understand it
Experimentation:
I have 2 queries now producing partial result.
Let's grab original json (above) in and parse:
read -d '' json << EOF
...
EOF
queries:
echo "${json}" | jq -r '.clusters[] | select(.name=="some-name") | .cluster | {url: .server, cert: .["certificate-authority-data"]}' &&\
echo "${json}" | jq -r '.users[] | select(.name=="some-name") | .user | {user: .username, password: .password}'
Will produce the split output:
{
"url": "https://some-url",
"cert": "some-cert"
}
{
"user": "some-user",
"password": "some-password"
}
Or, with key added for further merge:
echo "${json}" | jq -r '.clusters[] | select(.name=="some-name") | {name: .name, url: .cluster.server, cert: .cluster["certificate-authority-data"]}' &&\
echo "${json}" | jq -r '.users[] | select(.name=="some-name") | {name: .name, user: .user.username, password: .user.password}'
Will produce:
{
"name": "some-name",
"url": "https://some-url",
"cert": "some-cert"
}
{
"name": "some-name",
"user": "some-user",
"password": "some-password"
}
"name" is not needed but can be used as a join operation
So you already know how to get the cluster and user by name separately, first step is to select them both within a single filter:
(.clusters[] | select(.name == $name).cluster), (.users[] | select(.name == $name).user)
This will yield two separate objects, the cluster, then the user. But we want to merge them. There's plenty of ways to do this. You could add them (+) directly or merge them (*) but no real difference there. You'll just want to remap the properties to the names you wanted where needed.
(.clusters[] | select(.name == $name).cluster | {url: .server, cert: ."certificate-authority-data"})
+
(.users[] | select(.name == $name).user | {username, password})
Pass the name in as a parameter to your filter;
$ kubectl config view --raw -o json | jq --arg name some-name '
(.clusters[] | select(.name == $name).cluster | {url: .server, cert: ."certificate-authority-data"})
+
(.users[] | select(.name == $name).user | {username, password})
'
For the second part of your question, if it turns out the mapped user is missing key properties and you want to omit them, just add another select filter to the end to test for those properties and replace with an empty object if nothing is found:
... | select(has("username") and has("password")) // {}
jqplay
{
"status": "DOWN",
"components": {
"ping": {
"status": "UP",
"details": {
"version": "1.0.0",
"description": "dobre application",
"name": "SO-3113"
}
},
"bridge.lock": {
"status": "UP",
"details": {
"description": "test1"
}
},
"Configuration converted": {
"status": "DOWN",
"details": {
"description": "test2"
}
},
"app started": {
"status": "DOWN",
"details": {
"description": "test3"
}
}
}
}
I need to get the name of first component with DOWN status ("Configuration converted" in above json). So far i managed to get only .details.description of it:
jq -c '.components| .[] | select( .status | contains("DOWN")) .details.description' | head -1
How can I get the name (key) of component? ("Configuration converted")
You can use to_entries to get the key/values of your components , then select the first one with down status:
first(.components | to_entries | .[] | select( .value.status == "DOWN") | .key)
Run it on jqplay
You could use to_entries function to convert the objects within .components into key/value pairs and provide an expression that selects the first object matching the condition and retrieve its key
.components |
to_entries |
map(select(.value.status == "DOWN"))[0].key
For an input below:
[{
"commit": {
"author": {
"name": "Stephen Dolan",
"email": "mu#netsoc.tcd.ie",
"date": "2013-06-22T16:30:59Z"
},
"committer": {
"name": "Stephen Dolan",
"email": "mu#netsoc.tcd.ie",
"date": "2013-06-22T16:30:59Z"
},
"message": "Merge pull request #162 from stedolan/utf8-fixes\n\nUtf8 fixes. Closes #161"
"url":"https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f"
},
{
...
}
}]
How can JQ generate a delimited string from different objects as shown below?
"Stephen Dolan", "https://api.github.com/repos/stedolan/jq/commits/d25341478381063d1c76e81b3a52e0592a7c997f", "2013-06-22T16:30:59Z"
Collect the fields you want in an array and use #csv to convert to a CSV row. Make sure you get the raw output.
jq -r '.[] | [ .commit.author.name, .commit.url, .commit.author.date ] | #csv' input.json