I realize there are several other JmesPath join questions here, but I'm having trouble with a separate problem that I haven't found any examples for, where I need to concatenate (ie, join) a set of JSON values that have dynamically-named keys into a single element.
If I start with the following JSON data structure:
{
"data": [
{
"secmeetingdays":
{
"dayset_01":
{
"day_01": "M",
"day_02": "W",
"day_03": "F"
},
"dayset_02":
{
"day_01": "T",
"day_02": "TH"
}
},
}]
}
I would like to end up with something like this:
[
[
"M,W,F"
],
[
"T,TH"
]
]
I've started the query to flatten the data down, but am completely stuck with the join syntax. Nothing I try seems to be working.
Attempt 1: data[].secmeetingdays | [0].*.*
[
[
"M",
"W",
"F"
],
[
"T",
"TH"
]
]
Almost, but not quite there.
Attempt 2: data[].secmeetingdays | [0].*.* | {join(',',#)}
fails
Attempt 3: data[].secmeetingdays | [0].*.*.join(',',#)
fails
Attempt 4: data[].secmeetingdays | {join(',',#[0].*.*)}
fails
I tried avoiding 2 flattens to have some reference to grab onto inside the join.
Attempt 4 data[].secmeetingdays | [0].* | join(',',#[]).
fails
Attempt 6 data[].secmeetingdays | [0].*.* | #.join(',',[]) Gives a result, but it's not what I want:
"M,W,F,T,TH"
Update:
Attempt 7 data[].secmeetingdays[].*.* | [].join(',',#) gets me a lot closer but is also not exactly what I need:
[
"M,W,F",
"T,TH"
]
I might be able to work with this solution, but will leave this open in case someone has the accurate answer to the question.
The example here https://jmespath.org/ has a join, but it is only on a single list of items. How can I join the sub-arrays without affecting the structure of the parents?
data[*].secmeetingdays.values(#)[].values(#).join(',', #).to_array(#)
Gives you the example desired output but I see no benefit to wrapping each single string in an extra array.
data[].secmeetingdays.values(#) | [*][*].values(#).join(',', #)
Produces more logical output (to me) because it gives an array of daysets for each item in the data array:
[
[
"M,W,F",
"T,TH"
]
]
Note that the proper way to deal with such data is to write a script that iterates the objects, parses the keys and guarantees ordered output after sorting the items. JSON parsers have no obligation to keep object properties ordered the same as they were stored/read, so blindly converting to an array as above is not certain to be the order you desire. Using key names to store order is superfluous. Chronologically ordered data should be stored as arrays like so:
{
"data": [
{
"secmeetingdays": [
[
"M",
"W",
"F"
],
[
"T",
"TH"
]
]
}
]
}
[[0].title,[1].title].join(',', #).to_array(#)
RESULT: ["some1,some2"]
[[0].title,[1].title].join(',', #)
RESULT: "some1,some2"
[[0].title,[1].title]
RESULT: ["some1,some2"]
Related
I have multiple files containing both A array elements at the top level, and files containing A array elements nested in B elements. Is it possible to extract the data in a single jq line?
File 1:
{ "A" : [
{ "x" : " y" }
]
}
File 2:
{ "B" : [
{ "A" : [
{ "x" : "y" }
] }
] }
I have tried the following command
jq -r 'select(.A[] != null or .B[].A[] != null) | .A[] | .x'
without expected results.
It should be straightforward with using getpath/1. Dynamically identify all paths where the leaf value is A
getpath(paths | select(.[-1] == "A")) | .[].x
jqplay - Demo 1, Demo 2
or a hacky-way to rely on the fact that A is an array and x will always be one of the leaf paths
getpath(paths | select(.[-3] == "A" and .[-1] == "x"))
To visualize how the solution works, run the command paths on your input JSON, which breaks down the original JSON to all possible root-leaf paths. getpath/1 works by getting the value at a given path, where the last element in the array i.e. -1 is the leaf path x and 2 places before last is A.
Another option would be to use the .[]? (see the manual) which works
like .[], but no errors will be output if . is not an array or object.
jq -r '., .B[]? | .A[]?.x'
Demo
I have a large file of records that contain fields that look something like this:
{
"id": "1000001",
"updatedDate": "2018-12-21T01:52:00Z",
"createdDate": "1993-11-30T02:59:25Z",
"varFields": [
{
"fieldTag": "b",
"content": "1000679727"
},
{
"fieldTag": "v",
"content": "v.1"
}
}
I need to extract the .content element along with other things, but only when the fieldTag associated with it is "v". Only some records contain a fieldTag "v".
When I try to parse using
(.varFields[] |select(.fieldTag=="v") | "\(.content)") // ""
it works fine so long as v is present. However, when it is not present, I get
jq: error (at <stdin>:353953): Cannot iterate over null (null)
I tried to get rid of the error with multiple variations, including things to the effect of
(select((.varFields[] |select(.fieldTag=="v") | .content) != null) | .varFields[] |select(.fieldTag=="v") | "\(.content)") // ""
but I'm still getting the same error. What am I missing?
Take a look at the error suppression operator ? that works a bit like the new ?. nullable chaining operator in Javascript.
The ? operator, used as EXP?, is shorthand for try EXP.
Example:
jq '[.[]|(.a)?]'
Input [{}, true, {"a":1}]
Output [null, 1]
They have a slightly simpler demonstrable example of this at https://jqplay.org/jq?q=%5B.%5B%5D%7C(.a)%3F%5D&j=%5B%7B%7D%2C%20true%2C%20%7B%22a%22%3A1%7D%5D and the try-catch operator is similar if all you need is custom error handling (or just error ignoring...).
I have a list in a DynamoDB table and would like to move items to different positions in the same list, is there a way to do this in a single update?
At the moment, I'm looking at having to read the list, modify it, then write it back again, but would prefer doing it all in a single update, is there a way to do this?
Edit to add example
So here's some noddy data that shows what I'd like to do:
If the data started like this:
Item: { COLUMN: [ "Element_0", "Element_1", "Element_2", "Element_3" ] }
Then I'd give it from and to indices and it would move the element. So for example if I gave it a from index of 0 and to index of 2 the data should end up like this:
Item: { COLUMN: [ "Element_1", "Element_2", "Element_0", "Element_3" ] }
You can do this with an Update Expression, but it's a little tricky, since you don't have the data.
Basically, you have to create a dynamic update statement that sets every value you want to move. Something like this works:
aws dynamodb update-item --table-name test --key '{"pk":{"S":"1"}}' --update-expression "SET #list[1] = #list[2], #list[2] = #list[1]" --region us-west-2 --profile jw-test --expression-attribute-names '{"#list": "list"}'
I created a table with a key of pk, with a value of 1. The list before the update was like this:
[
'one',
'two',
'three',
'four'
]
After the update it looks like this:
[
'one',
'three',
'two',
'four'
]
Default answer if this isn't possible in a single update.
Read out the list, modify it, then write it back. It's not elegant, but it works and isn't that ugly either.
It's not atomic though so any answer that can do it in a single update will get the check mark.
I have written a fairly complicated code for my ABM (634 agents having interactions, each having different variables some of which are lists with multiple values that are updated each tick). As I need to save the updated values for all agents, I have defined a global variable using table:make. This table has 634 keys (each key for one agent), and each key has a list of those values (from that agents-own list variable) for the correspondent agent. But when I use the name of this table to be reported as one of my outputs in Behavior Space, the result in csv file is a table with no keys and it has only a number in it: {{table: 1296}}. So, I was wondering how I could change this variable to be able to have all values.
If you're happy to do some post-processing with R or something after the fact, then table:to-list might be all you need. For example, with a simple setup example like:
extensions [ table ]
globals [ example-table ]
turtles-own [ turtle-list ]
to setup
ca
crt 3 [
set turtle-list ( list random 10 one-of [ "A" "B" "C" ] random 100 )
]
set example-table table:make
foreach sort turtles [
t ->
table:put example-table ( word "turtle_" [who] of t ) [turtle-list] of t
]
reset-ticks
end
And a to-report to clean each table item such that the first item is the key and all other items are the items in the list:
to-report easier-read-table [ table_ ]
let out []
foreach table:to-list table_ [ i ->
set out lput ( reduce sentence i ) out
]
report out
end
You can set up your BehaviorSpace experiment such that one of your reporters is that reporter, something like:
To get a .csv file like:
Where the reporter column outputs a list of lists that you can process how you like.
However, I probably wouldn't use the basic BehaviorSpace output for this, but instead have a call in the experiment to call a manual table output procedure. For example, using the csv extension to make this output-table procedure:
to output-table [ filename_ table_ ]
let out [["key" "col1" "col2" "col3"]]
foreach table:to-list table_ [ i ->
set out lput ( reduce sentence i ) out
]
csv:to-file filename_ out
end
This outputs a much more analysis-ready table if you're less comfortable cleaning the output of a list-of-lists that as far as I know is what you would get from the BehaviorSpace output. So, you can either call it at the end of your experiment, like:
To get a table like:
Which is a little nicer to deal with. You can obviously modify this to report more often if needed, for example:
which would output a table at each tick of the experiment (you can also do this in your code to make it a little easier).
I have a record
record = [ [ name1:'value1', name2:'value2', name3:'value3' ],
[ name1:'value6', name2:'value7', name3:'value8' ] ]
I would like to add two more key/value pairs to with values as boolean(true/false) as below
record = [ [ name1:'value1', name2:'value2', name3:'value3', name4:false, name5:true ],
[ name1:'value6', name2:'value7', name3:'value8', name4:false, name5:true ] ]
When I tried to use add or put functions, doesnt seem to work(either replacing the existing values or not doing anything)
Just do:
record = record.collect { it + [ name4:false, name5:true ] }
Or you can also do:
record = record*.plus( name4:false, name5:true )
To add to Patricks answer above (+1), a Map contains a set not a list so all keys must be unique. So you cannot assign multiple values to a single key directly.
Among many solutions, you can alternatively, save an object:
Map<String, myObject>
that holds many different values and this will still maintain the uniqueness of the set since there will only be one key.