How can I sum values from json object with jq
Example input JSON object
{
"orderNumber": 2346999,
"workStep": 110,
"good": 8,
"bad": 0,
"type": "1",
"date": "2022-11-08T07:17:09",
"time": 0,
"result": 1
}
{
"orderNumber": 2346999,
"workStep": 110,
"good": 8,
"bad": 0,
"type": "1",
"date": "2022-11-08T07:26:57",
"time": 0,
"result": 1
}
jq condition
. | select(.orderNumber==2346999 and .workStep==110) | .good
result
8
8
and I liketo have
16
A simple approach using add to get the sum of the numbers.
Use map() with --slurp to create an array with just the .good and apply add:
map(select(.orderNumber==2346999 and .workStep==110).good) | add
Gives: 16
Online demo
An efficient approach that avoids the need to slurp is to use the general utility:
def count(s): reduce s as $_ (0; .+1);
With this, you can use your filter using jq's -n option instead of the -s option:
count(inputs | select(.orderNumber==2346999 and .workStep==110) | .good)
Notice that the leading . in your filter is unnecessary.
Related
jq . has the side-effect of pretty printing the input.
$ echo '{"foo":"bar", "baz":[1,2,3]}' | jq .
{
"foo": "bar",
"baz": [
1,
2,
3
]
}
But if I want to use jq to incorporate the input with some surrounding text, the input renders compactly
$ echo '{"foo":"bar", "baz":[1,2,3]}' | jq -r '"My value is:\n\(.)\nSome other stuff"'
My value is:
{"foo":"bar","baz":[1,2,3]}
Some other stuff
Is there any way to force pretty printing here? I'd like the output to be
My value is:
{
"foo": "bar",
"baz": [
1,
2,
3
]
}
Some other stuff
I did find a solution while I was writing the question: don't put the input inside string interpolation, output a stream of things:
echo '{"foo":"bar", "baz":[1,2,3]}' | jq -r '"My value is:", . , "Some other stuff"'
# .........................................................^^^^^
outputs
My value is:
{
"foo": "bar",
"baz": [
1,
2,
3
]
}
Some other stuff
This might not fit your actual use case, but instead of creating a single JSON string in jq, just use a shell command group.
echo '{"foo":"bar", "baz":[1,2,3]}' | { echo "My value is:"; jq .; echo "Some other stuff"; }
So I have big json, where I need to take some subtree and copy it to other place, but with some properties updated (a lot of them). So for example:
{
"items": [
{ "id": 1, "other": "abc"},
{ "id": 2, "other": "def"},
{ "id": 3, "other": "ghi"}
]
}
and say, that i'd like to duplicate record having id == 2, and replace char e in other field with char x using regex. That could go (I'm sure there is a better way, but I'm beginner) something like:
jq '.items |= . + [.[]|select (.id == 2) as $orig | .id=4 | .other=($orig.other | sub("e";"x"))]'<sample.json
producing
{
"items": [
{
"id": 1,
"other": "abc"
},
{
"id": 2,
"other": "def"
},
{
"id": 3,
"other": "ghi"
},
{
"id": 4,
"other": "dxf"
}
]
}
Now that's great. But suppose, that there ins't just one other field. There are multitude of them, and over deep tree. Well I can issue multiple sub operations, but assuming, that replacement pattern is sufficiently selective, maybe we can turn the whole JSON subtree to string (trivial, tostring method) and replace all occurences using singe sub call. But how to turn that substituted string back to — is it call object? — to be able to add it back to items array?
Here's a program that might be a solution to the general problem you are describing, but if not at least illustrates how problems of this type can be solved. Note in particular that there is no explicit reference to a field named "other", and that (thanks to walk) the update function is applied to all candidate JSON objects in the input.
def update($n):
if .items | length > 0
then ((.items[0]|keys_unsorted) - ["id"]) as $keys
| if ($keys | length) == 1
then $keys[0] as $key
| (.items|map(.id) | max + 1) as $newid
| .items |= . + [.[] | select(.id == $n) as $orig | .id=$newid | .[$key]=($orig[$key] | sub("e";"x"))]
else .
end
else .
end;
walk(if type == "object" and has("items") then update(2) else . end)
I've the following Json :
{
"hits": {
"hits": [
{
"_source": {
"offers_data": [
{
"base_price": 198.89,
"shop_id": 2002,
"shop_name": "TheOtherShop"
},
{
"base_price": 223,
"shop_id": 2247,
"shop_name": "MainShop"
},
{
"base_price": 225,
"shop_id": 2247,
"shop_name": "MainShop"
}
],
"search_result_data": {
"identifiers": {
"id": 32116
},
"shop": {
"id": 2247,
"name": "MainShop"
}
}
}
}
]
}
}
I'm writing the following command :
jq -c --raw-output '.hits.hits[]|{products_ids: ._source.search_result_data.identifiers.id,
best_shop_id: ._source.search_result_data.shop.id,
best_shop_name: (if ._source.search_result_data.shop.id>0 then ._source.search_result_data.shop.id as $shop_id|._source.offers_data[]|select(.shop_id==$shop_id).shop_name else "" end),
best_offer_base_price: (if ._source.search_result_data.shop.id>0 then ._source.search_result_data.shop.id as $shop_id|._source.offers_data[]|select(.shop_id==$shop_id).base_price else "" end)}'
and I get this result :
{"products_ids":32116,"best_shop_id":2247,"best_shop_name":"MainShop","best_offer_base_price":223}
{"products_ids":32116,"best_shop_id":2247,"best_shop_name":"MainShop","best_offer_base_price":225}
{"products_ids":32116,"best_shop_id":2247,"best_shop_name":"MainShop","best_offer_base_price":223}
{"products_ids":32116,"best_shop_id":2247,"best_shop_name":"MainShop","best_offer_base_price":225}
As you can see I get 2 duplicates : Of course I've two offers from MainShop, so it's normal that I get 2 records, but if I'm also fetching the base prices, the it duplicates the result again. In my real world case I get 32 records instead of 2 legitimate ones, because I'm fetching other fields. So I'd like to avoid this extra duplication each time I fetch a field.
The icing on the cake would be to be able to only get one record, the one where amongst Mainshop offers the base_price is the minimum.
Thanks
Icing
... the one where the base_price is the minimum.
The following two interpretations of the problem both assume we can take, as the "minimum" item, any of the admissible items that has the minimal value.
First Interpretation of the original question
.hits.hits[]._source
| (.offers_data | min_by(.base_price)) as $min_offers_data
| .search_result_data
| {products_ids: .identifiers.id}
+ ($min_offers_data
| {best_shop_id: .shop_id,
best_shop_name: .shop_name,
best_offer_base_price: .base_price})
Output:
{
"products_ids": 32116,
"best_shop_id": 2002,
"best_shop_name": "TheOtherShop",
"best_offer_base_price": 198.89
}
Second Interpretation
Restrict consideration to .search_result_data.shop.id:
.hits.hits[]._source
| (.search_result_data.shop.id) as $shop
| (.offers_data | map(select(.shop_id == $shop)) | min_by(.base_price)) as $min_offers_data
| .search_result_data
| {products_ids: .identifiers.id}
+ ($min_offers_data
| {best_shop_id: .shop_id,
best_shop_name: .shop_name,
best_offer_base_price: .base_price})
Output
{
"products_ids": 32116,
"best_shop_id": 2247,
"best_shop_name": "MainShop",
"best_offer_base_price": 223
}
My input file looks something like this:
{
"login": "dmaxfield",
"id": 7449977,
...
}
{
"login": "dmaxfield",
"id": 7449977,
...
}
I can get all the login names with this : cat members | jq '.[].login'
but I have not been able to crack the syntax to get both the login and id?
You can use jq '.[] | .login, .id' to obtain each login followed by its id.
This works for me:
> echo '{"a":1,"b":2,"c":3}{"a":1,"b":2,"c":3}' | jq '{a,b}'
{
"a": 1,
"b": 2
}
{
"a": 1,
"b": 2
}
Just provide one more example here (jq-1.6):
Walk through an array and select a field of an object element and a field of object in that object
echo '[{"id":1, "private_info": {"name": "Ivy", "age": 18}}, {"id":2, "private_info": {"name": "Tommy", "aga": 18}}]' | jq ".[] | {id: .id, name: .private_info.name}" -
{
"id": 1,
"name": "Ivy"
}
{
"id": 2,
"name": "Tommy"
}
Without the example data:
jq ".[] | {id, name: .private_info.name}" -
.[]: walk through an array
{id, name: .private_info.name}: take .id and .private_info.name and wrap it into an object with field name "id" and "name" respectively
In order to select values which are indented to different levels (i.e. both first and second level), you might use the following:
echo '[{"a":{"aa":1,"ab":2},"b":3,"c":4},{"a":{"aa":5,"ab":6},"b":7,"c":8}]' \
| jq '.[]|[.a.aa,.a.ab,.b]'
[
1,
2,
3
]
[
5,
6,
7
]
I'm using js tool to parse some JSONs/strings. My minimal example is the following command:
echo '"foo foo"' | jq 'match("(foo)"; "g")'
Which results in the following output:
{
"offset": 0,
"length": 3,
"string": "foo",
"captures": [
{
"offset": 0,
"length": 3,
"string": "foo",
"name": null
}
]
}
{
"offset": 4,
"length": 3,
"string": "foo",
"captures": [
{
"offset": 4,
"length": 3,
"string": "foo",
"name": null
}
]
}
I want my final output for this example to be:
"foo,foo"
But in this case I get two separate objects instead of an array or similar that I could call implode on. I guess either the API isn't made for my UC or my understanding of it is very wrong. Please, advise.
The following script takes the string value from each of the separate objects with .string, wraps them in an array [...] and then joins the members of the array with commas using join.
I modified the regex because you didn't actually need a capture group for the given use case, but if you wanted to access the capture groups you could do .captures[].string instead of .string.
echo '"foo foo"' | jq '[match("foo"; "g").string] | join(",")'