I am trying to pull out of this list the ip_prefix based on where region contains us-west- and list them in csv. When I try jq '.prefixes[] | .ip_prefix,.region | select(contains("us")?)' file.json I only get a list of regions.
{
"syncToken": "15985471",
"createDate": "2021-10-08-20-13-16",
"prefixes": [
{
"ip_prefix": "3.5.140.0/22",
"region": "ap-northeast-2",
"service": "AMAZON",
"network_border_group": "ap-northeast-2"
},
{
"ip_prefix": "13.34.37.64/27",
"region": "ap-southeast-4",
"service": "AMAZON",
"network_border_group": "ap-southeast-4"
},
{
"ip_prefix": "35.180.0.0/16",
"region": "eu-west-3",
"service": "AMAZON",
"network_border_group": "eu-west-3"
},
{
"ip_prefix": "52.93.153.170/32",
"region": "eu-west-2",
"service": "AMAZON",
"network_border_group": "eu-west-2"
},
{
"ip_prefix": "52.93.178.234/32",
"region": "us-west-1",
"service": "AMAZON",
"network_border_group": "us-west-1"
}
]
}
You have the filters in the wrong order. Try:
jq '.prefixes[] | select(.region|startswith("us-west")) | .ip_prefix' <your json file>
What this is doing is getting the prefixes array, filtering for elements where the region starts with "us-west" and then selecting the ip_prefix.
Related
From Wikidata, I get the following json:
# Sparql query
query=$(cat ./myquery.sparql)
response=$(curl -G --data-urlencode query="${query}" https://wikidata.org/sparql?format=json)
echo "${response}" | jq '.results.bindings'
[
{
"language": {
"type": "uri",
"value": "https://lingualibre.org/entity/Q100"
},
"wikidata": {
"type": "literal",
"value": "Q36157"
},
"code": {
"type": "literal",
"value": "lub"
}
},
{
"language": {
"type": "uri",
"value": "https://lingualibre.org/entity/Q101"
},
"wikidata": {
"type": "literal",
"value": "Q36284"
},
"code": {
"type": "literal",
"value": "srr"
}
}
]
I would like to have the keys directly paired with their values, such as :
[
{
"language": "https://lingualibre.org/entity/Q100",
"wikidata": "Q36157",
"iso": "lub"
},
{
"language": "https://lingualibre.org/entity/Q101",
"wikidata": "Q36284",
"iso": "srr"
}
]
I currently have a non-resilient code, which will break whenever the key names change :
jq 'map({"language":.language.value,"wikidata":.wikidata.value,"iso":.code.value})'
How to pair the keys with their values in a resilient way (not naming the keys) ?
I want to "prune" the child objects so to only keep the value.
You could use map_values which works like the outer map but for objects, i.e. it retains the object structure, including the field names:
jq 'map(map_values(.value))'
[
{
"language": "https://lingualibre.org/entity/Q100",
"wikidata": "Q36157",
"code": "lub"
},
{
"language": "https://lingualibre.org/entity/Q101",
"wikidata": "Q36284",
"code": "srr"
}
]
Note that this solution lacks the name conversion from code to iso.
have a very large JSON data like below
{
"10.10.10.1": {
"asset_id": 1,
"referencekey": "ASSET-00001",
"hostname": "testDev01",
"fqdn": "ip-10-10.10.1.ap-northeast-2.compute.internal",
"network_zone": [
"DEV",
"Dev"
],
"service": {
"name": "TEST_SVC",
"account": "AWS_TEST",
"billing": "Testpay"
},
"aws": {
"tags": {
"Name": "testDev01",
"Service": "TEST_SVC",
"Usecase": "Dev",
"billing": "Testpay",
"OsVersion": "20.04"
},
"instance_type": "t3.micro",
"ami_imageid": "ami-e000001",
"state": "running"
}
},
"10.10.10.2": {
"asset_id": 3,
"referencekey": "ASSET-47728",
"hostname": "Infra_Live01",
"fqdn": "ip-10-10-10-2.ap-northeast-2.compute.internal",
"network_zone": [
"PROD",
"Live"
],
"service": {
"name": "Infra",
"account": "AWS_TEST",
"billing": "infra"
},
"aws": {
"tags": {
"Name": "Infra_Live01",
"Service": "Infra",
"Usecase": "Live",
"billing": "infra",
"OsVersion": "16.04"
},
"instance_type": "r5.large",
"ami_imageid": "ami-e592398b",
"state": "running"
}
}
}
Can I use JQ to make the conversion like below?
Or is there an easier way to solve it?
Thank you
Expected result
_key,asset_id,referencekey,hostname,fqdn,network_zone/0,network_zone/1,service/name,service/account,service/billing,aws/tags/Name,aws/tags/Service,aws/tags/Usecase,aws/tags/billing,aws/tags/OsVersion,aws/instance_type,aws/ami_imageid,aws/state
10.10.10.1,1,ASSET-00001,testDev01,ip-10-10.10.1.ap-northeast-2.compute.internal,DEV,Dev,TEST_SVC,AWS_TEST,Testpay,testDev01,TEST_SVC,Dev,Testpay,20.04,t3.micro,ami-e000001,running
10.10.10.2,3,ASSET-47728,Infra_Live01,ip-10-10-10-2.ap-northeast-2.compute.internal,PROD,Live,Infra,AWS_TEST,infra,Infra_Live01,Infra,Live,infra,16.04,r5.large,ami-e592398b,running
jq let's you do the conversion to CSV easily. The following code produces the desired output:
jq -r 'to_entries
| map([.key,
.value.asset_id, .value.referencekey, .value.hostname, .value.fqdn,
.value.network_zone[0], .value.network_zone[1],
.value.service.name, .value.service.account, .value.service.billing,
.value.aws.tags.Name, .value.aws.tags.Service, .value.aws.tags.Usecase, .value.aws.tags.billing, .value.aws.tags.OsVersion,
.value.aws.instance_type, .value.aws.ami_imageid, .value.aws.state])
| ["_key","asset_id","referencekey","hostname","fqdn","network_zone/0","network_zone/1","service/name","service/account","service/billing","aws/tags/Name","aws/tags/Service","aws/tags/Usecase","aws/tags/billing","aws/tags/OsVersion","aws/instance_type","aws/ami_imageid","aws/state"]
, .[]
| #csv' "$INPUT"
Remarks
If some nodes in the input JSON are missing, the code does not break but fills in empty values in the CSV file.
If more than two network zones are given, only the first two are covered in the CSV file
Given this data, which is two distinct objects, one having an array of keys and the other a dictionary of key value pairs:
{
"servers": [
{
"location": "server4",
"services": [
"srv07",
"srv06",
"srv01",
"srv04"
]
},
{
"location": "server2",
"services": [
"srv07",
"srv02",
"srv05",
"srv03"
]
}
],
"release": {
"id": "release1",
"services": [
{
"service": "srv01",
"URL": "/srv01_service/v1.20.0"
},
{
"service": "srv02",
"URL": "/srv02_service/v1.14.0"
},
{
"service": "srv03",
"URL": "/srv03_service/v1.15.0"
},
{
"service": "srv04",
"URL": "/srv04_service/v1.18.0"
},
{
"service": "srv05",
"URL": "/srv05_service/v1.14.0"
},
{
"service": "srv06",
"URL": "/srv06_serv/v1.13.0"
},
{
"service": "srv07",
"URL": "/srv07_service/v1.19.0"
}
]
}
}
I am trying to produce this, the first object with the keys replaced with the values from the dictionary. NOTE: I would be fine with renaming services[] to URLs[] if it makes things easier.
{
"servers": [
{
"location": "server4",
"services": [
"/srv07_service/v1.19.0",
"/srv06_serv/v1.13.0",
"/srv01_service/v1.20.0",
"/srv04_service/v1.18.0"
]
},
{
"location": "server2",
"services": [
"/srv07_service/v1.19.0",
"/srv02_service/v1.14.0",
"/srv05_service/v1.14.0",
"/srv03_service/v1.15.0"
]
}
]
}
My latest attempt is close but returns something akin to a Cartesian.
. | .servers[].services[] = (.servers[] as $s | .release.services[] | select(.service as $v | $s.services[] | index($v)).URL) | {servers}
Create an INDEX to lookup in, then map each element you want to change according to the index.
jq '
INDEX(.release.services[]; .service) as $index
| {servers} | .servers[].services[] |= $index[.].URL
'
{
"servers": [
{
"location": "server4",
"services": [
"/srv07_service/v1.19.0",
"/srv06_serv/v1.13.0",
"/srv01_service/v1.20.0",
"/srv04_service/v1.18.0"
]
},
{
"location": "server2",
"services": [
"/srv07_service/v1.19.0",
"/srv02_service/v1.14.0",
"/srv05_service/v1.14.0",
"/srv03_service/v1.15.0"
]
}
]
}
Demo
I have the following structure:
{
"Subnets": [
{
"SubnetId": "foo1",
"Id": "bar1",
"Tags": [
{
"Key": "Name",
"Value": "foo"
},
{
"Key": "Status",
"Value": "dev"
}
]
},
{
"SubnetId": "foo2",
"Id": "bar2",
"Tags": [
{
"Key": "Name",
"Value": "foo"
},
{
"Key": "Status",
"Value": "dev"
}
]
}
]
}
I can extract multiple keys at the "top level" like so:
cat subnets.json| jq '.Subnets[] | "\(.Id) \(.SubnetId)"'
Anyone know how I can also display one of the tags by key name, let's say I also want the Status tag displayed on the same line as the Id and SubnetId.
Thx for any help,
Is this what you are looking for?
jq '.Subnets[] | "\(.Id) \(.SubnetId) \(.Tags | from_entries | .Status)"' subnets.json
I try to reduce the size of a jq call. The current command is:
jq '.result | .[].newValueJson.action |= (. | tostring // .) | .[].oldValueJson.action |= (. | tostring // .) | .[].metadata.value |= (. | tostring // .)'
As you can see, the function "tostring" is being applied to "newValueJson.action", "oldValueJson.action" and "metadata.value" in just the same way. I am wondering if there is a more compact syntax so I only need to apply tostring once?
I extracted sample data show what is being done (this is not the full json tree).
Source:
{
"result": [{
"id": 1,
"action": {
"result": true,
"type": "filter_create"
},
"newValueJson": {
"action": "simulate"
},
"oldValueJson": {
"action": "enforce"
},
"metadata": {
"value": false
}
},
{
"id": 2,
"action": {
"result": true,
"type": "filter_create"
},
"newValueJson": {
"action": {
"mode": "simulate",
"timeout": 3600
}
},
"oldValueJson": {
"action": {
"mode": "enforce",
"timeout": 3600
}
},
"metadata": {
"value": "off"
}
}
]
}
Result:
[{
"id": 1,
"action": {
"result": true,
"type": "filter_create"
},
"newValueJson": {
"action": "simulate"
},
"oldValueJson": {
"action": "enforce"
},
"metadata": {
"value": "false"
}
},
{
"id": 2,
"action": {
"result": true,
"type": "filter_create"
},
"newValueJson": {
"action": "{\"mode\":\"simulate\",\"timeout\":3600}"
},
"oldValueJson": {
"action": "{\"mode\":\"enforce\",\"timeout\":3600}"
},
"metadata": {
"value": "off"
}
}
]
Thanks and Best.
You can group the actions together using (..) as below. Also instead of using the array notation .[], you can use map(..) to handle apply the filter expression inside the result array
.result | map((.newValueJson.action, .oldValueJson.action, .metadata.value ) |= (. | tostring // .))