I need to swap a value on a dynamoDB entry. The difficulty is the coworker who put the table together used nested json on the field I need to edit. I'm having trouble figuring out to how to edit a value when the layout is asymmetrical.
Let's say a row in dynamoDB shows up like this with get-item:
{
"Item": {
"aws:rep:deleting": {
"BOOL": false
},
"service": {
"S": "g"
},
"settings": {
"L": [
{
"M": {
"fe_enabled": {
"BOOL": false
},
"stack_type": {
"S": "all"
},
"label": {
"S": "helm_chart_name"
},
"value": {
"S": "gate"
}
}
},
{
"M": {
"fe_enabled": {
"BOOL": true
},
"stack_type": {
"S": "all"
},
"label": {
"S": "helm_chart_version"
},
"value": {
"S": "0.1.1"
}
}
}
]
},
"label": {
"S": "gate"
},
"service_children": {
"L": []
},
"independent": {
"BOOL": true
}
}
}
I need to change helm_chart_version's value to - say - 0.1.2.
I've started by trying to isolate the value. I've tried variations of the following with no luck:
jq -r '.Item[].L.M | select(.label == "helm_4G_chart_version") | .value.S'
# errors because there are multiple L's so it's iterating over null
jq '..|.label? | select(type != "null")'
# discovered this neat syntax, but I can't figure out how to get value from that
jq --arg argName "argValue" '()' file.json | sponge file.json
# this is likely the "replace" solution, but I don't know what to put in the () to get the right value
Also open to hearing that I'm tackling this incorrectly - I'm a little out of my depth here. I'm planning to save the edited json to a file and then doing a put-item on it. Is this the right way to edit a dynamoDB entry?
If you want to traverse individual objects and find the object to update and set its value, you can do
jq '(.Item.settings.L[] | select(.M.label.S == "helm_chart_version")) |= (.M.value.S = "0.1.2")'
the part before the |= does the identification of the right object matching your string. Once the object is identified, the update operator |= just modifies the field with the desired value. See it working on jq-play
If convenience is at a premium, then using walk has much to recommend it:
walk(if type == "object" and .label.S == "helm_chart_version"
then .value.S = "0.1.2" else . end)
This of course assumes that "S" is known ahead of time. If that's not the case, then one possibility to consider would be:
walk(if type == "object" and has("label") and any(.label[]; . == "helm_chart_version")
then .value |= map_values("0.1.2") else . end)
Variations
If you don't want to walk the entire object, you could simply walk over .settings:
.Item.settings
|= walk(if type == "object"
and has("label")
and any(.label[]; . == "helm_chart_version")
then .value |= map_values("0.1.2")
else .
end)
Or if "label" is also an unknown:
.Item.settings
|= (. as $in
| [paths
| . as $p
| select(($in|getpath($p)) == "helm_chart_version")]
| reduce .[] as $p ($in; setpath($p | (.[-2] = "value"); "0.1.2")))
Related
Is there a generic mechanism within jq to select arbitrary elements of a JSON file but return the full structural context of those elements? For example, if I have the following:
{
"foo": {
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
},
"bar": [
1,
4,
5
],
"baz": true
}
using the filter .foo,.baz would normally result in:
{
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
}
true
but what I'd like is to get:
{
"foo": {
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
},
"baz": true
}
I can solve that specifically for the given filter using select, but I'd like something generic, to be able to run the same code with a different filter and get the same type of result, e.g. running with the filter .foo.three,.bar[1] would result in:
{
"foo": {
"three": {
"hello": "world"
}
},
"bar": [
4
]
}
Thanks!
This would give exactly the result you wanted for .foo.three,.bar[1] :
jq 'def extract(f):
. as $input |
reduce path(f) as $path (
null;
if ($path | last | type) == "string"
then setpath($path; $input | getpath($path))
else setpath(($path|.[:-1]);
getpath($path|.[:-1]) +
[$input | getpath($path)]
)
end
);
extract(.foo.three, .bar[1])' data.json
At best you could do an object construction directly by naming the key names under {..} and apply a further transformation to get only the desired paths
{foo, bar} | .foo |= {three} | .bar |= [.[1]]
jqplay demo
You could convert your queries into paths, and the input into a stream, select the pieces matching the query path, and rebuild it to a single output:
def extract(f):
reduce (
path(f) as $path | tostream
| select(length > 1 and (.[0] | index($path) == 0))
) as $set (
null;
setpath($set[0]; $set[1])
);
First example using .foo and .baz:
jq 'def extract(f): …; extract(.foo, .baz)'
{
"foo": {
"one": true,
"two": false,
"three": {
"hello": "world"
},
"four": true
},
"baz": true
}
Demo
As with sparse arrays though, it'll fill up the missing items with null, as otherwise the index wouldn't match anymore. Second example using .foo.three and .bar[1]:
jq 'def extract(f): …; extract(.foo.three, .bar[1])'
{
"foo": {
"three": {
"hello": "world"
}
},
"bar": [
null,
4
]
}
Demo
Goal
I'd like to add a proxy-url field to the currently active clusters entry of my kubeconfig file. The "active" cluster is identified by the "active" context, which is itself identified by a top-level key current-context. Simplified, the JSON object looks something like:
{
"clusters":[
{
"name":"cluster1",
"field":"field1"
},
{
"name":"cluster2",
"field":"field2"
}
],
"contexts":[
{
"name":"context1",
"context": {
"cluster":"cluster1"
}
},
{
"name":"context2",
"context": {
"cluster":"cluster2"
}
}
],
"current-context": "context1"
}
And I'd like to update the clusters entry for cluster1 from:
{
"name":"cluster1",
"field":"field1"
}
to
{
"name":"cluster1",
"field":"field1",
"proxy-url":"my-url"
}
First attempt
jq '. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name) as $context
| $o.clusters[] | select(.name == $context.context.cluster)
| .proxy_id |= "my-url"'
gives me
{
"name": "cluster1",
"field": "field1",
"proxy_id": "my-url"
}
-- great! But I need the rest of the object too.
Parentheses almost work
With parentheses, I can get the whole object back & add a "proxy-url" field to the active context, but I can't take it one step further to update the active cluster. This filter:
jq '(. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name)
| ."proxy-url")
|= "my-url"'
works mint:
{
"clusters": [...], // omitted for brevity, unchanged
"contexts": [
{
"name": "context1",
"context": {
"cluster": "cluster1"
},
"proxy-url": "my-url" // tada!
},
{...} // omitted for brevity, unchanged
],
"current-context": "context1"
}
Trying to take it one step further (to update the cluster identified by that context, instead):
jq '(. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name) as $context
| $o.clusters[] | select(.name == $context.context.cluster)
| ."proxy-url")
|= "my-url"'
gives me the following error:
jq: error (at <stdin>:26): Invalid path expression near attempt to access element "clusters" of {"clusters":[{"name":"clus...
exit status 5
How can I use the $context.context.cluster result to update the relevant clusters entry? I don't understand why this approach works for adding something to contexts but not to clusters.
Dirty solution
I can kludge together a new clusters entry & merge that with the top-level object:
jq '. as $o
| $o."current-context" as $current_context_name
| $o.contexts[] | select(.name == $current_context_name) as $context
| $o + {"clusters": [($o.clusters[] | select(.name == $context.context.cluster)."proxy-url" |= "my-url")]}
but this feels a bit fragile.
This solution retrieves the active cluster using an INDEX construction, then just sets the new field directly without modifying the context:
jq '
INDEX(.contexts[]; .name)[."current-context"].context.cluster as $cluster
| (.clusters[] | select(.name == $cluster))."proxy-url" = "my-url"
'
{
"clusters": [
{
"name": "cluster1",
"field": "field1",
"proxy-url": "my-url"
},
{
"name": "cluster2",
"field": "field2"
}
],
"contexts": [
{
"name": "context1",
"context": {
"cluster": "cluster1"
}
},
{
"name": "context2",
"context": {
"cluster": "cluster2"
}
}
],
"current-context": "context1"
}
Demo
Here my current jq script:
def reduce_generateId:
.[] | (. + {generalPractitionerCode: (.UAB_UP + "-" + .UAB_COD_UAB)});
def reduce_generalPractitioner($practitionerRole):
(reduce $practitionerRole[] as $g (
{};
.[$g.oid1 + "-" + $g.oid2].generalPractitioner += ($g | [.id])
)) as $dict
| $dict;
reduce_generateId | reduce_generalPractitioner($generalPractitioner)
My jq command is:
jq -s -f merge-patient.jq --argfile generalPractitioner reduced_ids.json reduced_pacient.json
Where reduced_pacient.json:
{ "UAB_UP": "00003", "UAB_COD_UAB": "3212", "INVENTAT": "02"}
{ "UAB_UP": "00006", "UAB_COD_UAB": "5881", "INVENTAT": "102"}
{ "UAB_UP": "00006", "UAB_COD_UAB": "5751", "INVENTAT": "102"}
and reduced_ids.json:
[
{
"id": "3e67b455-8cdb-4bc0-a5e1-f90253870fc9",
"oid1": "04374",
"oid2": "INFP3"
},
{
"id": "0f22e5ff-70bc-457f-bdaf-7afe86d478de",
"oid1": "04376",
"oid2": "INF07"
}
]
As you can see, redux_generalPractitioner returns $dict straightforwardly. I'm getting this:
{
"04374-INFP3": {
"generalPractitioner": [
"3e67b455-8cdb-4bc0-a5e1-f90253870fc9"
]
},
"04376-INF07": {
"generalPractitioner": [
"0f22e5ff-70bc-457f-bdaf-7afe86d478de"
]
}
}
{
"04374-INFP3": {
"generalPractitioner": [
"3e67b455-8cdb-4bc0-a5e1-f90253870fc9"
]
},
"04376-INF07": {
"generalPractitioner": [
"0f22e5ff-70bc-457f-bdaf-7afe86d478de"
]
}
}
{
"04374-INFP3": {
"generalPractitioner": [
"3e67b455-8cdb-4bc0-a5e1-f90253870fc9"
]
},
"04376-INF07": {
"generalPractitioner": [
"0f22e5ff-70bc-457f-bdaf-7afe86d478de"
]
}
}
As you can see, I'm getting $dict three times.
I don't quite figure out why it's generating three $dict instead of one.
If I only perform reduce_generalPractitioner($generalPractitioner), I'm getting:
{
"04374-INFP3": {
"generalPractitioner": [
"3e67b455-8cdb-4bc0-a5e1-f90253870fc9"
]
},
"04376-INF07": {
"generalPractitioner": [
"0f22e5ff-70bc-457f-bdaf-7afe86d478de"
]
}
}
Any ideas?
Your program input, reduced_pacient.json is a stream of three objects. As you have set the --slurp flag, the initial context of your script is an array of three elements.
First, you're calling reduce_generateId which by iteration .[] decomposes the array into its three items. Thus reduce_generateId has three outputs.
Next, these are fed by pipe into your reduce_generalPractitioner function which consequently is run three times. Each run produces your $dict once, yielding three times in total.
Remember: Although your function reduce_generalPractitioner has an input parameter, it also has (just like every filter in jq) a general input context (addressed through piping), which determines the value of . and subsequently the number of runs.
I can't seem to find the secret sauce to make jq do what I want.
Given the following contrived input:
{
"node1": {
"1": {
"Aliases": ["one", "uno"]
},
"2": {
"Aliases": ["two", "dos"]
}
},
"node2": {
"a": {
"Aliases": ["alpha"]
},
"b": {
"Aliases": ["bravo"]
}
}
}
I want to return the keys of the ancestors of Aliases when Aliases contains a particular value.
For example, given the search key dos, I want to return node1 and 2.
You can play with this data in jqplay. Any help would be appreciated.
(paths | select(.[-2] == "Aliases")) as $p
| select( "dos" == getpath($p))
| $p[:-2][]
Notice that there's no need for an additional variable (. as $in).
I have a query:
.modules[].resources | select (.[]!=null)
and after it I have got:
{ somestuff } { somestuff } { somestuff }
when I add legth after all:
.modules[].resources | select (.[]!=null) | length
I have got:
1 1 1
but I need to count elements, so I need 3 in an output. How can I implement it ?
In fact it would be very useful to create an array from the first query output to operate with it furthure
[ { somestuff } , { somestuff } , { somestuff } ]
You can put the results of the query into a list and get the length of this list:
[ .modules[].resources | select (.[]!=null) ] | length
Since you indicated it would be very useful to create an array from the first query output, what you probably want to use here is map.
Let's assume your data is something like
the data from this question: jq: search by value from another array element
{
"modules": [
{
"resources": [
{
"type": "openstack_compute_instance_v2",
"primary": {
"id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"attributes": {
"name": "jumpbox"
}
}
},
{
"type": "openstack_compute_floatingip_associate_v2",
"primary": {
"attributes": {
"instance_id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"floating_ip": "10.120.241.21"
}
}
}
]
}
]
}
with this data your filter
.modules[].resources
| select (.[]!=null) #< do you really want this `[]` ?
would produce two copies of the entire .resources array. What you may want instead is
.modules[].resources
| map(select(.!=null))
which will give you an array
[
{
"type": "openstack_compute_instance_v2",
"primary": {
"id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"attributes": {
"name": "jumpbox"
}
}
},
{
"type": "openstack_compute_floatingip_associate_v2",
"primary": {
"attributes": {
"instance_id": "5edfe2bf-94df-49d5-8118-3e91fb52946b",
"floating_ip": "10.120.241.21"
}
}
}
]
to get the length, just add length:
.modules[].resources
| map(select(.!=null))
| length
in this example giving
2
Because map(f) is defined as [ .[] | f ] the above filter is really
.modules[].resources
| [
.[]
| select(.!=null)
]
| length
In this form you can clearly see the construction of the intermediate array.
Note also that jq provides a built-in values filter which is defined as select(.!=null) so this example could be further simplified to just
.modules[].resources
| map(values)
| length
As mentioned at jq: count nest object values which satisfy condition
the best solution (e.g. because it does not involve construction of an intermediate array) would be to use count, defined as follows:
def count(s): reduce s as $i (0; .+1);
With this definition, you should be able simply to wrap count(...) around your query since it produces a stream:
count(.modules[].resources | select (.[]!=null))
Or maybe you want something closer to:
count(.modules[].resources | objects)