I have json:
"spec": {
"background": true,
"failurePolicy": "Fail",
"rules": [
{
"exclude": {
"resources": {}
},
"generate": {
"clone": {}
},
"match": {
"resources": {
"kinds": [
"networking.k8s.io/v1/NetworkPolicy"
]
}
},
"mutate": {},
"name": "validate-nodeport",
"validate": {
"message": "Services of type NodePort are not allowed.",
"pattern": {
"spec": {
"type": "!NodePort"
}
}
}
}
],
"validationFailureAction": "audit"
},
"status": {
"ready": true
}
}
I have jq command:
(.spec.rules[0].match.resources.kinds[] / "/")
| [select(.[1])[0] // null, select(.[2])[1] // null, last]
as [$version,$group,$kind]
| {$version,$group,$kind}
But sometimes field "version" or "group" maybe empty. I need to write with the following condition - if the string is null, then it is not necessary to write it.
The filter you have is already a pretty efficient way to deal with formation of objects, when all the individual fields (version, group and kind) are present and not required to re-write the whole thing to deal with one specific case.
To skip the null fields, just pipe your previous filter to below
with_entries(select(.value!=null))
i.e. the whole filter being below. See jqplay demo
(.spec.rules[0].match.resources.kinds[] / "/")
| [select(.[1])[0] // null, select(.[2])[1] // null, last]
as [$version,$group,$kind]
| {$version,$group,$kind}
| with_entries(select(.value!=null))
Use select filter:
jq '(.spec.rules[0].match.resources.kinds[] / "/")
| [select(.[1])[0] // null, select(.[2])[1] // null, last]
as [$version,$group,$kind]
| {$version,$group,$kind}
| select(.version != "")
| select(.group != "")
' \
spec.json
Or if you want to only reduce output:
jq '(.spec.rules[0].match.resources.kinds[] / "/")
| [select(.[1])[0] // null, select(.[2])[1] // null, last]
as [$version,$group,$kind]
| {$version,$group,$kind}
| if .version != "" then
if .group != "" then
{$version,$group,$kind}
else
{$version,$kind}
end
else
if .group != "" then
{$group,$kind}
else
{$kind}
end
end
' \
spec.json
The {$version,$group,$kind} part from my other answer was included to show how to make use of the variables you requested to be created. Thus, if you don't want it to be written, don't generate it in the first place only to modify it later with with_entries(select(.value)) or the like. Instead, generate it differently, according to your needs.
For instance, instead of unconditionally using all three parts as in {$version,$group,$kind} you may pick just the ones that are not null using [{$version},{$group},{$kind} | select(.[])] | add: (Demo)
(.spec.rules[0].match.resources.kinds / "/")
| [select(.[1])[0] // null, select(.[2])[1] // null, last]
as [$version,$group,$kind]
| [{$version},{$group},{$kind} | select(.[])] | add
Better yet, modify the [...] as {...} part directly to generate what you need, for instance: (Demo)
(.spec.rules[0].match.resources.kinds[] / "/")
| ({version:select(.[1])[0]} // {})
+ ({group:select(.[2])[1]} // {})
+ {kind: last}
Related
Is there a generic mechanism within jq to select arbitrary elements of a JSON file but return the full structural context of those elements? For example, if I have the following:
{
"foo": {
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
},
"bar": [
1,
4,
5
],
"baz": true
}
using the filter .foo,.baz would normally result in:
{
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
}
true
but what I'd like is to get:
{
"foo": {
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
},
"baz": true
}
I can solve that specifically for the given filter using select, but I'd like something generic, to be able to run the same code with a different filter and get the same type of result, e.g. running with the filter .foo.three,.bar[1] would result in:
{
"foo": {
"three": {
"hello": "world"
}
},
"bar": [
4
]
}
Thanks!
This would give exactly the result you wanted for .foo.three,.bar[1] :
jq 'def extract(f):
. as $input |
reduce path(f) as $path (
null;
if ($path | last | type) == "string"
then setpath($path; $input | getpath($path))
else setpath(($path|.[:-1]);
getpath($path|.[:-1]) +
[$input | getpath($path)]
)
end
);
extract(.foo.three, .bar[1])' data.json
At best you could do an object construction directly by naming the key names under {..} and apply a further transformation to get only the desired paths
{foo, bar} | .foo |= {three} | .bar |= [.[1]]
jqplay demo
You could convert your queries into paths, and the input into a stream, select the pieces matching the query path, and rebuild it to a single output:
def extract(f):
reduce (
path(f) as $path | tostream
| select(length > 1 and (.[0] | index($path) == 0))
) as $set (
null;
setpath($set[0]; $set[1])
);
First example using .foo and .baz:
jq 'def extract(f): …; extract(.foo, .baz)'
{
"foo": {
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
},
"baz": true
}
Demo
As with sparse arrays though, it'll fill up the missing items with null, as otherwise the index wouldn't match anymore. Second example using .foo.three and .bar[1]:
jq 'def extract(f): …; extract(.foo.three, .bar[1])'
{
"foo": {
"three": {
"hello": "world"
}
},
"bar": [
null,
4
]
}
Demo
Goal
I'd like to add a proxy-url field to the currently active clusters entry of my kubeconfig file. The "active" cluster is identified by the "active" context, which is itself identified by a top-level key current-context. Simplified, the JSON object looks something like:
{
"clusters":[
{
"name":"cluster1",
"field":"field1"
},
{
"name":"cluster2",
"field":"field2"
}
],
"contexts":[
{
"name":"context1",
"context": {
"cluster":"cluster1"
}
},
{
"name":"context2",
"context": {
"cluster":"cluster2"
}
}
],
"current-context": "context1"
}
And I'd like to update the clusters entry for cluster1 from:
{
"name":"cluster1",
"field":"field1"
}
to
{
"name":"cluster1",
"field":"field1",
"proxy-url":"my-url"
}
First attempt
jq '. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name) as $context
| $o.clusters[] | select(.name == $context.context.cluster)
| .proxy_id |= "my-url"'
gives me
{
"name": "cluster1",
"field": "field1",
"proxy_id": "my-url"
}
-- great! But I need the rest of the object too.
Parentheses almost work
With parentheses, I can get the whole object back & add a "proxy-url" field to the active context, but I can't take it one step further to update the active cluster. This filter:
jq '(. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name)
| ."proxy-url")
|= "my-url"'
works mint:
{
"clusters": [...], // omitted for brevity, unchanged
"contexts": [
{
"name": "context1",
"context": {
"cluster": "cluster1"
},
"proxy-url": "my-url" // tada!
},
{...} // omitted for brevity, unchanged
],
"current-context": "context1"
}
Trying to take it one step further (to update the cluster identified by that context, instead):
jq '(. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name) as $context
| $o.clusters[] | select(.name == $context.context.cluster)
| ."proxy-url")
|= "my-url"'
gives me the following error:
jq: error (at <stdin>:26): Invalid path expression near attempt to access element "clusters" of {"clusters":[{"name":"clus...
exit status 5
How can I use the $context.context.cluster result to update the relevant clusters entry? I don't understand why this approach works for adding something to contexts but not to clusters.
Dirty solution
I can kludge together a new clusters entry & merge that with the top-level object:
jq '. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name) as $context
| $o + {"clusters": [($o.clusters[] | select(.name == $context.context.cluster)."proxy-url" |= "my-url")]}
but this feels a bit fragile.
This solution retrieves the active cluster using an INDEX construction, then just sets the new field directly without modifying the context:
jq '
INDEX(.contexts[]; .name)[."current-context"].context.cluster as $cluster
| (.clusters[] | select(.name == $cluster))."proxy-url" = "my-url"
'
{
"clusters": [
{
"name": "cluster1",
"field": "field1",
"proxy-url": "my-url"
},
{
"name": "cluster2",
"field": "field2"
}
],
"contexts": [
{
"name": "context1",
"context": {
"cluster": "cluster1"
}
},
{
"name": "context2",
"context": {
"cluster": "cluster2"
}
}
],
"current-context": "context1"
}
Demo
I need to swap a value on a dynamoDB entry. The difficulty is the coworker who put the table together used nested json on the field I need to edit. I'm having trouble figuring out to how to edit a value when the layout is asymmetrical.
Let's say a row in dynamoDB shows up like this with get-item:
{
"Item": {
"aws:rep:deleting": {
"BOOL": false
},
"service": {
"S": "g"
},
"settings": {
"L": [
{
"M": {
"fe_enabled": {
"BOOL": false
},
"stack_type": {
"S": "all"
},
"label": {
"S": "helm_chart_name"
},
"value": {
"S": "gate"
}
}
},
{
"M": {
"fe_enabled": {
"BOOL": true
},
"stack_type": {
"S": "all"
},
"label": {
"S": "helm_chart_version"
},
"value": {
"S": "0.1.1"
}
}
}
]
},
"label": {
"S": "gate"
},
"service_children": {
"L": []
},
"independent": {
"BOOL": true
}
}
}
I need to change helm_chart_version's value to - say - 0.1.2.
I've started by trying to isolate the value. I've tried variations of the following with no luck:
jq -r '.Item[].L.M | select(.label == "helm_4G_chart_version") | .value.S'
# errors because there are multiple L's so it's iterating over null
jq '..|.label? | select(type != "null")'
# discovered this neat syntax, but I can't figure out how to get value from that
jq --arg argName "argValue" '()' file.json | sponge file.json
# this is likely the "replace" solution, but I don't know what to put in the () to get the right value
Also open to hearing that I'm tackling this incorrectly - I'm a little out of my depth here. I'm planning to save the edited json to a file and then doing a put-item on it. Is this the right way to edit a dynamoDB entry?
If you want to traverse individual objects and find the object to update and set its value, you can do
jq '(.Item.settings.L[] | select(.M.label.S == "helm_chart_version")) |= (.M.value.S = "0.1.2")'
the part before the |= does the identification of the right object matching your string. Once the object is identified, the update operator |= just modifies the field with the desired value. See it working on jq-play
If convenience is at a premium, then using walk has much to recommend it:
walk(if type == "object" and .label.S == "helm_chart_version"
then .value.S = "0.1.2" else . end)
This of course assumes that "S" is known ahead of time. If that's not the case, then one possibility to consider would be:
walk(if type == "object" and has("label") and any(.label[]; . == "helm_chart_version")
then .value |= map_values("0.1.2") else . end)
Variations
If you don't want to walk the entire object, you could simply walk over .settings:
.Item.settings
|= walk(if type == "object"
and has("label")
and any(.label[]; . == "helm_chart_version")
then .value |= map_values("0.1.2")
else .
end)
Or if "label" is also an unknown:
.Item.settings
|= (. as $in
| [paths
| . as $p
| select(($in|getpath($p)) == "helm_chart_version")]
| reduce .[] as $p ($in; setpath($p | (.[-2] = "value"); "0.1.2")))
I can't seem to find the secret sauce to make jq do what I want.
Given the following contrived input:
{
"node1": {
"1": {
"Aliases": ["one", "uno"]
},
"2": {
"Aliases": ["two", "dos"]
}
},
"node2": {
"a": {
"Aliases": ["alpha"]
},
"b": {
"Aliases": ["bravo"]
}
}
}
I want to return the keys of the ancestors of Aliases when Aliases contains a particular value.
For example, given the search key dos, I want to return node1 and 2.
You can play with this data in jqplay. Any help would be appreciated.
(paths | select(.[-2] == "Aliases")) as $p
| select( "dos" == getpath($p))
| $p[:-2][]
Notice that there's no need for an additional variable (. as $in).
I have a query:
.modules[].resources | select (.[]!=null)
and after it I have got:
{ somestuff } { somestuff } { somestuff }
when I add legth after all:
.modules[].resources | select (.[]!=null) | length
I have got:
1 1 1
but I need to count elements, so I need 3 in an output. How can I implement it ?
In fact it would be very useful to create an array from the first query output to operate with it furthure
[ { somestuff } , { somestuff } , { somestuff } ]
You can put the results of the query into a list and get the length of this list:
[ .modules[].resources | select (.[]!=null) ] | length
Since you indicated it would be very useful to create an array from the first query output, what you probably want to use here is map.
Let's assume your data is something like
the data from this question: jq: search by value from another array element
{
"modules": [
{
"resources": [
{
"type": "openstack_compute_instance_v2",
"primary": {
"id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"attributes": {
"name": "jumpbox"
}
}
},
{
"type": "openstack_compute_floatingip_associate_v2",
"primary": {
"attributes": {
"instance_id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"floating_ip": "10.120.241.21"
}
}
}
]
}
]
}
with this data your filter
.modules[].resources
| select (.[]!=null) #< do you really want this `[]` ?
would produce two copies of the entire .resources array. What you may want instead is
.modules[].resources
| map(select(.!=null))
which will give you an array
[
{
"type": "openstack_compute_instance_v2",
"primary": {
"id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"attributes": {
"name": "jumpbox"
}
}
},
{
"type": "openstack_compute_floatingip_associate_v2",
"primary": {
"attributes": {
"instance_id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"floating_ip": "10.120.241.21"
}
}
}
]
to get the length, just add length:
.modules[].resources
| map(select(.!=null))
| length
in this example giving
2
Because map(f) is defined as [ .[] | f ] the above filter is really
.modules[].resources
| [
.[]
| select(.!=null)
]
| length
In this form you can clearly see the construction of the intermediate array.
Note also that jq provides a built-in values filter which is defined as select(.!=null) so this example could be further simplified to just
.modules[].resources
| map(values)
| length
As mentioned at jq: count nest object values which satisfy condition
the best solution (e.g. because it does not involve construction of an intermediate array) would be to use count, defined as follows:
def count(s): reduce s as $i (0; .+1);
With this definition, you should be able simply to wrap count(...) around your query since it produces a stream:
count(.modules[].resources | select (.[]!=null))
Or maybe you want something closer to:
count(.modules[].resources | objects)