jq not getting expected output - jq

My JSON looks like this.
"[{
changes": [
{
"change": "{users=[7], submitted=true}",
"date": "2016-11-13T14:34:27.353Z",
"user": "abcd"
}
]
}]
Expected Output:
{
id: null,
date: "2016-11-13T14:34:27.353Z",
type: "submission",
user: abcd,
_processDate: todaysDate
}
JQ I tried
[.[][] as $source |
$source.changes[] as $log |
$log.change |
{
submitted: .| (scan("submitted=(?<submitted>[^,}]+)") // [""] ) | .[0],
rejected: .| (scan("rejected=(?<rejected>[^,}]+)") // [""] ) | .[0]
} as $change |
[
(
select($change.submitted == "true") |
{
id: $source.id,
date: $log.date,
type: "submission",
user: $log.user,
_processDate: now | todate
}),
(select($change.rejected == "true") |
{
id: $source.id,
date: $log.date,
type: "rejection",
user: $log.user,
_processDate: now | todate
}
)
] |
.[]]
There could 'rejections' in the json and the output should display the rejections .
My JQ is not yielding expected output.
Any pointers on how to fix this query.
Thank you for your help.
Appreciate it.

I'd take a completely different approach and do this instead.
(now | todate) as $_processDate | [
.[].changes[]
| (.change
| scan("\\b(submitted|rejected)=true\\b")[]
| {submitted:"submission",rejected:"rejection"}[.]) as $type
| {id, date, $type, user, $_processDate}
]
First take note of the current date at the beginning. Then determine what type of change it is. Assuming that "submitted" and "rejected" (or other "types") are mutually exclusive, easier to match on the key name. Then build up the result. This will keep the results in an array.
https://jqplay.org/s/S65ySzbF30

Related

Conditionally output a field?

In this example I only want isGreaterThanOne field to be shown if it's true. Here's what I started with (always shown)
echo '[{"a":5},{"a":1}]' | jq '[.[] | {value:.a, isGreaterThanOne:(.a>1)}]'
I inserted an if statement
echo '[{"a":5},{"a":1}]' | jq '[.[] | {value:.a, X:(if .a>1 then "Y" else "N" end) }]'
Then got stuck trying to move the field into the conditional. Also it seems like I must have an else with an if
echo '[{"a":5},{"a":1}]' | jq '[.[] | {value:.a, (if .a>1 then (K:"Y)" else (L:"N") end) }]'
I want the below as the result (doesn't need to be pretty printed)
[
{
"value": 5,
"X": "Y"
},
{
"value": 1,
}
]
Using if, make one branch provide an empty object {} which wouldn't contain the extra field:
map({value: .a} + if .a > 1 then {X: "Y"} else {} end)
Demo
Alternatively, equip only selected items with the extra field:
map({value: .a} | select(.value > 1).X = "Y")
Demo
Output:
[
{
"value": 5,
"X": "Y"
},
{
"value": 1
}
]

Double condition in array with JQ

My JSON is an array of one object like this:
[{
"id": 125650,
"status": "success",
"name": "build_job",
"artifacts": [
{
"file_type": "archive",
"size": 72720116,
"filename": "artifacts.zip",
"file_format": "zip"
},
{
"file_type": "metadata",
"size": 1406,
"filename": "metadata.gz",
"file_format": "gzip"
}
]
}]
I want to select only the object ID if the following conditions matches:
status == success
name == build_job
artifacts.size > 0 where file_type == archive
I'm stuck on the last condition, I can select artifacts with size > 0, OR artifacts where file_type = archive, but not both at the same time.
Here's my current query :
| jq '.[0] | select(.name == "build_job" and .status == "success" and .artifacts[].file_type == "archive") | .id'
Can you help me with that ?
For the last condition, you presumably mean something like:
all(.artifacts[];
if .file_type == "archive" then .size > 0 else true end)
which can also be written as:
all(.artifacts[] | select(.file_type == "archive");
.size > 0)
I’d recommend using either all or any, depending on your requirements.
Try this:
.[0] | select(
.name == "build_job" and .status == "success" and (
.artifacts[] | select(.file_type == "archive") | length > 0
)
) | .id
This selects successful build_jobs containing one or more archive artifacts. Unfortunately, multiple ids are returned if there's more than one such artifacts. Here's how to wrap the expression to fix that:
[
.[] | select(
.name == "build_job" and .status == "success" and (
.artifacts[] | select(.file_type == "archive") | length > 0
)
)
] | unique | .[].id
For the last condition, take the array .artifacts, reduce it to those elements matching your criteria map(select(.file_type == "archive")) and test the resulting array's length length > 0.
All together:
.[0] | select(
.name == "build_job" and
.status == "success" and (
(.artifacts | map(select(.file_type == "archive"))) | length > 0
)
)
| .id

Get the output in the indicated format without selected data

Using jq, how to get the output in the indicated format without selected groups, e.g. group_3. All data is to be selected from each selected group: name_x.
Input:
{
"groups": {
"group_1": {
"name_1": {
"field_111": "value_111",
"field_112": "value_112",
"field_11n": "value_11n"
},
"name_2": {
"field_121": "value_121",
"field_122": "value_122",
"field_12n": "value_12n"
}
},
"group_2": {
"name_3": {
"field_231": "value_231",
"field_232": "value_232",
"field_23n": "value_23n"
}
},
"group_3": {
"name_4": {
"field_341": "value_341",
"field_342": "value_342",
"field_34n": "value_34n"
}
}
}
}
Ouptut:
name: name_1, group: group_1, field_A_value: value_111, field_B_value: value_112
name: name_2, group: group_1, field_A_value: value_121, field_B_value: value_122
name: name_3, group: group_2, field_A_value: value_231, field_B_value: value_232
Try something like this:
jq --raw-output '
.groups | to_entries[] | select(.key | IN("group_3") | not)
| [.key] + (.value | to_entries[] | [.key, .value[]])
| "name: \(.[1]), group: \(.[0]), field_A_value: \(.[2]), field_B_value: \(.[3])"
' input.json
Demo
If you want to exclude more than just one group, change IN("group_3") to, say, IN("group_3", "group_5", "group_7")

Extract nested properties from an array of objects

I have the following JSON file :
{
"filter": [
{
"id": "id_1",
"criteria": {
"from": "mail#domain1.com",
"subject": "subject_1"
},
"action": {
"addLabelIds": [
"Label_id_1"
],
"removeLabelIds": [
"INBOX",
"SPAM"
]
}
},
{
"id": "id_2",
"criteria": {
"from": "mail#domain2.com",
"subject": "subject_1"
},
"action": {
"addLabelIds": [
"Label_id_2"
],
"removeLabelIds": [
"INBOX",
"SPAM"
]
}
}
]
}
And I would like to extract emails values : mail#domain1.com and mail#domain2.com
I have tried this command:
jq --raw-output '.filter[] | select(.criteria.from | test("mail"; "i")) | .id'
But does not work, I get this error :
jq: error (at <stdin>:1206): null (null) cannot be matched, as it is
not a string exit status 5
Another point : how to display the value of "id" key, where "from" key value = mail#domain1.com ?
So in my file id = id_1
Do you have an idea ?
If you only need to extract the emails from .criteria.from then this filter is enough as far as I can tell:
jq --raw-output '.filter[].criteria.from' file.json
If some objects don't have a criteria object then you can filter out nulls with:
jq --raw-output '.filter[].criteria.from | select(. != null)' file.json
If you want to keep the emails equal to "mail#domain1.com":
jq --raw-output '.filter[].criteria.from | select(. == "mail#domain1.com")' file.json
If you want to keep the emails that start with "mail#":
jq --raw-output '.filter[].criteria.from | select(. != null) | select(startswith("mail#"))' file.json
I would like to extract emails values
There is a wide spectrum of possible answers, with these
amongst the least specific with respect to where in the JSON the email addresses occur:
.. | objects | .from | select(type=="string")
.. | strings | select(test("#([a-z0-9]+[.])+[a-z]+$"))

Transforming a list of objects into a table in Kusto

I am trying to get the json data (in form of a list of key-value pairs) in one of my data table cells and convert that into a dynamic table of sorts.
T
| where id == "xyz"
| project telem_obj
The data in the telem_obj cell is of the format
[
{
"Value": "SomeKey01",
"Key": "0"
},
{
"Value": "SomeKey02",
"Key": "1"
}
]
My end objective is to get a table of the form;
|Key | Value |
|SomeValue01 | 0 |
|SomeValue02 | 1 |
I have managed to do this by taking out the static data and creating atable out of it.
print EnumVals = dynamic(
[
{
"Value": "SomeKey01",
"Key": "0"
},
{
"Value": "SomeKey02",
"Key": "1"
}
]
)
| mvexpand EnumVals
| evaluate bag_unpack(EnumVals)
I am not sure how can I go about taking result of my query, extracting this list of json objects from it and convert it into a new dynamic table. I cannot find any example which works on a list of objects.
After a good night's sleep, i found how to do it
T
| take 1
| mvexpand telem_obj
| evaluate bag_unpack(telem_obj)
| project Value, Key
my mistake was I was trying to force the actual query inside a dynamic function.
print EnumVals = dynamic(
T
| where id == "xyz"
| project telem_obj
)
| mvexpand EnumVals
| evaluate bag_unpack(EnumVals)

Resources