Using jq I want to modify multiple values - jq

I have a json like this:
{
"bla": { "body": {
"mode": "raw",
"raw": "{\n \"accountId\": \"1111\",\n \"monetaryAmount\": {\n \"amount\": 111,\n \"exponent\": 2,\n \"currency\": \"aaa\"\n },\n \"remarks\": \"consequat quis\"\n}",
"options": {
"raw": {
"language": "json"
}
}
}},
"auth": {
"type": "bearer",
"bearer": [
{
"key": "token",
"value": "{{token}}",
"type": "string"
}
]
},
"segg": {
"key": "txn_id",
"value": "{{token}}",
"type": "string"
},
"slugg": {
"key": "companyId",
"value": "{{token}}",
"type": "string"
},
"blu": [ {
"key": "teamMemberId",
"value": "{{token}}",
"type": "string"
} ]
}
Conditions: There can be anywhere any deep ..body.raw strings, on these I want to do a simple search and replace (Or better: Parse the string as JSon and do some jq on that?).
And any deep objects with .key and .value (strings), in this I just want to replace the .value="{{"+.key+"}}"
Thanks
This is what I could do:
del(.. | objects | .auth)|
(.. | objects | select(.key == "teamMemberId")).value="{{teamMemberId}}"|
(.. | objects | select(.key == "accountId")).value="{{accountId}}"|
(.. | objects | select(.key == "companyId")).value="{{companyId}}"|
(.. | objects | select(.key == "transactionId")).value="{{transactionId}}"|
(.. | objects | select(.key == "txn_id")).value="{{transactionId}}"|
(.. | objects | select(.key == "limitType")).value="MONTHLY"|
(.. | objects | select(.raw | type == "string")).raw=null
In case of ..raw I can not find a way to replace, and in case of .key -> .value I would prefer something like find by regex, and replace as mentioned. So that the whole jq expression is more compact.

One way to process the .body.raw values would be to start with:
(.. | try .body.raw // empty) |= fromjson
You can elaborate fromjson as per your requirements.
One way to update the .value values would be to include the following in your jq pipeline:
walk(if type == "object" and has("key") and has("value")
then .value = "{{\(.key)}}" else . end)

Related

Flattening a nested json while printing outer json using jq

I have a file like below (excerpt)
cat input.json
{
"Metrics": [
{
"Namespace": "CWAgent",
"MetricName": "disk_used_percent",
"Dimensions": [
{
"Name": "path",
"Value": "/aem"
},
{
"Name": "host",
"Value": "uat2-author01.uat.cloud.abc.com.au"
},
{
"Name": "device",
"Value": "mapper/aem-aem"
},
{
"Name": "fstype",
"Value": "ext4"
}
]
},
{
"Namespace": "CWAgent",
"MetricName": "mem_used_percent",
"Dimensions": [
{
"Name": "host",
"Value": "uat2-dispatch02.uat.cloud.abc.com.au"
}
]
}
]
}
As you can see the "Dimensions" is a list of json objects of varying length. I am required to print each objects under "Metrics" in a single line like below -
CWAgent|disk_used_percent|path:/aem|host:uat2-author01.uat.cloud.abc.com.au|device:mapper/aem-aem|fstype:ext4
CWAgent|mem_used_percent|host:uat2-dispatch02.uat.cloud.abc.com.au
I used this -
cat input.json | jq -r ".Metrics[] | .Namespace + \"|\" + .MetricName + \"|\" + (.Dimensions[] | .Name + \":\" + .Value + \"|\")"
And the output is like below:
CWAgent|disk_used_percent|path:/aem|
CWAgent|disk_used_percent|host:uat2-author01.uat.cloud.abc.com.au|
CWAgent|disk_used_percent|device:mapper/aem-aem|
CWAgent|disk_used_percent|fstype:ext4|
CWAgent|mem_used_percent|host:uat2-dispatch02.uat.cloud.abc.com.au|
As you can see, we are getting 4 lines for first object each having different object under Dimensions. Can anyone please suggest what will be the jq command to get desired output. I am novice in jq and wasted too much time on this experimenting.
Thanks,
Bijitesh
You're looking for this:
jq -r '.Metrics[] | [.Namespace, .MetricName] + (.Dimensions | map("\(.Name):\(.Value)")) | join("|")' input.json

Parsing on jq with_entries

I have the following data
Command output:
| jq '.rrsets[]'
{
"comments": [],
"name": "name1.",
"records": [
{
"content": "10.10.10.10",
"disabled": false
}
],
"ttl": 60,
"type": "A"
}
{
"comments": [],
"name": "name2.",
"records": [
{
"content": "20.20.20.20",
"disabled": false
}
],
"ttl": 60,
"type": "CNAME"
}
I want to get names where type is A.
Help, tell me how to do this?
| jq '.rrsets[] | with_entries(select(.key== "name", .value == "A"))'
{
"name": "name1."
}
{
"name": "name2.",
"type": "A"
}
Displays all the lines, but I only need where type = A
I'm not sure if this is what you are looking for, but wouldn't simply ... | select(.type == "A") do the trick?
... | jq '.rrsets[] | select(.type == "A")'
{
"comments": [],
"name": "name1.",
"records": [
{
"content": "10.10.10.10",
"disabled": false
}
],
"ttl": 60,
"type": "A"
}
Demo
And then just get the .name if you want only that (using -r to get rid of the JSON formatting):
... | jq -r '.rrsets[] | select(.type == "A").name'
name1.
Demo

JQ: How do I extract item which has part of subitem value of "FOO" [duplicate]

I have a JSON file that looks like this:
{
"InstanceId": "i-9KwoRGF6jbhYdZi823aE4qN",
"Tags": [
{
"Key": "blah",
"Value": "server-blah"
},
{
"Key": "environment",
"Value": "ops"
},
{
"Key": "server_role",
"Value": "appserver"
},
{
"Key": "Name",
"Value": "some_name"
},
{
"Key": "product",
"Value": "some_server"
}
]
}
{
...more objects like the above...
}
I need to display the InstanceId where "Key" == "environment" and "Value" == "ops".
I have jq-1.6.
If I say:
cat source.json | jq '
{ InstanceId, Tags } |
(.Tags[] | select( .Key == "environment" ))
'
I get some of what I want, but I cannot figure out how to include InstanceId in the output nor how to incorporate the "and" part of the select.
Here is a simple but efficient approach using any:
select( any(.Tags[]; .Key=="environment" and .Value == "ops") )
| .InstanceId
An alternative approach that avoids .Tags[]:
{"Key": "environment", "Value": "ops"} as $object
| select( .Tags | index($object) )
| .InstanceId
I'm not sure if this is the exact output you're looking for (comment if it isn't), but this will output the InstanceIds of JSON objects that contain a Tag with Key environment and Value ops.
jq 'select( .Tags[] | (.Key == "environment" and .Value == "ops")) | .InstanceId' < source.json

`jq` selection based on condition of different levels

I'd like to select the following node in a long json file.
The conditions are "locator": "mmc1" and "#name": "section-title". This nested structure makes the jq query very complex. Notice that I have to specify things like "locator" and "section-title" multiple times.
.. | .["$$"]?
| select((.[]? | ."#name" == "section-title"?) and (..[]? | .locator? | test("mmc[0-9]+")?))
| [
(..[]? | select(.locator? | test("mmc[0-9]+")?) | .locator)
, (.[] | select(."#name" == "section-title") | ._)
] | #tsv
Could anybody let me know how to make the query simpler in jq yet make sure its function is exact the same as the original query? Just making the following simplified test input producing the same output should not be considered as an equivalent query. Thanks.
...
"$$": [
{
"#name": "label",
"_": "Appendix A"
},
{
"#name": "section-title",
"$": {
"id": "sectitle0145"
},
"_": "Supplementary data"
},
{
"#name": "para",
"$": {
"id": "p0210",
"view": "all"
},
"$$": [
{
"#name": "__text__",
"_": "The following is the supplementary data related to this article:"
},
{
"#name": "display",
"$$": [
{
"#name": "e-component",
"$": {
"id": "ec1"
},
"$$": [
{
"#name": "link",
"$": {
"locator": "mmc1",
"type": "simple",
"role": "http://data.elsevier.com/vocabulary/ElsevierContentTypes/46.1",
"href": "pii:S2212877817302818/mmc1",
"id": "aep-link-id8"
}
}
]
}
]
}
]
}
]
...
See the output below.
$ jq '.. | .["$$"]? | select((.[]? | ."#name" == "section-title"?) and (..[]? | .locator? | test("mmc[0-9]+")?)) | [ (..[]? | select(.locator? | test("mmc[0-9]+")?) | .locator) , (.[] | select(."#name" == "section-title") | ._) ] | #tsv' < 1.json
"mmc1\tSupplementary data"
$ cat 1.json
{
"$$": [
{
"#name": "label",
"_": "Appendix A"
},
{
"#name": "section-title",
"$": {
"id": "sectitle0145"
},
"_": "Supplementary data"
},
{
"#name": "para",
"$": {
"id": "p0210",
"view": "all"
},
"$$": [
{
"#name": "__text__",
"_": "The following is the supplementary data related to this article:"
},
{
"#name": "display",
"$$": [
{
"#name": "e-component",
"$": {
"id": "ec1"
},
"$$": [
{
"#name": "link",
"$": {
"locator": "mmc1",
"type": "simple",
"role": "http://data.elsevier.com/vocabulary/ElsevierContentTypes/46.1",
"href": "pii:S2212877817302818/mmc1",
"id": "aep-link-id8"
}
}
]
}
]
}
]
}
]
}
If you want to get the "locator" value off of all objects unconditionally, you could use this:
..|objects.locator|strings
To find all objects with #name == "section-title" and select the _ value:
..|select(objects."#name" == "section-title")._
Putting it all together:
[(..|objects.locator|strings), (..|select(objects."#name" == "section-title")._)] | #tsv
https://jqplay.org/s/xHWg8aGSSS
The following is simpler in at least some respects, produces the desired result, and seems to reflect the requirements:
..
| .["$$"]?
| (.. | objects | .locator | strings | select(test("mmc[0-9]+")) ) as $locator
| (.. | objects | select(.["#name"] == "section-title") | ._) as $st
| [$locator, $st]
| #tsv

Interaction between {} and select

Here's my test data:
[
{
"id": "id-1",
"tags": {
"key": "name",
"value": "name-1"
}
},
{
"id": "id-2"
}
]
I'm trying to simplify the output, to show the 'name' field if present, and always show the id. For example, this script almost works:
~ $ cat testdata | jq '.[] | {id, name: .tags.value}'
{
"id": "id-1",
"name": "name-1"
}
{
"id": "id-2",
"name": null
}
When I try to add in a guard against .keys not existing and filter for the section of 'keys' I care about, here's what happens:
~ $ cat testdata | jq '.[] | {id, name: (select(.tags.key == "name") | .tags.value)}'
{
"id": "id-1",
"name": "name-1"
}
I assume {} is somehow ending up with a zero-length array instead of 'null'. What should I be using instead of |? What am I misunderstanding?
I ended up solving the problem using: [POSSIBLY_MATCHED_EXPRESSION][0], in this case:
cat testdata | jq '.[] | {id, name: ([select(.tags.key == "name") | .tags.value][0])}'
If I'm understanding correctly, if you wanted to include a name only if it existed, I'd do this:
map({id} + with_entries(select(.key == "tags") | .value))
Otherwise if you don't mind null names:
map({id, name: with_entries(select(.key == "tags") | .value) | .name})
Here's a more general solution if you have other "tags" so it's not hardcoded to only accept name values.
This assumes that any object value is actually a key/value pair.
map(with_entries(if .value | type == "object" then .value else . end))
Or if tags is the only dynamic property:
map(with_entries(if .key == "tags" then .value else . end))
If the goal is to produce:
{"id":"id-1","name":"name-1"}
{"id":"id-2"}
then the following three expressions are essentially equivalent solutions:
.[] | if .tags.key == "name" then {id, name: .tags.value} else {id} end
.[] | {id} + (if .tags.key == "name" then {name: .tags.value} else {} end)
.[] | (select(.tags.key == "name") | {id, name: .tags.value}) // {id}
You could just add
| if .name == null then del(.name) else . end
to the end of your filter to get rid of the .name key when its value is null.
With your test data, the following
.[]
| {id, name:.tags.value }
| if .name == null then del(.name) else . end
produces
{
"id": "id-1",
"name": "name-1"
}
{
"id": "id-2"
}

Resources