Using jq, I want to get a list of application names, all versions and server name. The list should not contain information about the latest version.
Input:
{
"software": {
"app_1": {
"0.0.1": {
"properties_1": {
"lang": "en"
},
"server": "vm123-4.domain.com"
},
"0.0.2": {
"properties_2": {
"arch": "x86"
},
"server": "vm123-5.comain.com"
},
"latest_version": "0.0.2"
},
"app_2": {
"1.0.1": {
"properties_2": {
"arch": "x86"
},
"server": "vm333-1.domain.com"
},
"latest_version": "1.0.1"
},
"app_33": {
"0.44.1": {
"properties_1": {
"lang": "en"
},
"properties_2": {
"arch": "x86"
},
"properties_3": {
"boot": "true"
},
"server": "vm888-9.domain.com"
},
"1.2.2": {
"properties_3": {
"boot": "yes"
},
"server": "vm123-4.domain.com"
},
"latest_version": "1.2.2"
}
}
}
Desired output:
"app_1, 0.0.1, vm123-4.domain.com"
"app_1, 0.0.2, vm123-5.comain.com"
"app_2, 1.0.1, vm333-1.comain.com"
"app_33, 0.44.1, vm888-9.comain.com"
"app_33, 1.2.2, vm123-4.comain.com"
My request will only list one version of the app, but not all. I do not know how to do this.
.software | to_entries[] | [(.key), (.value | to_entries[] | select(.key | IN("latest_version") | not))] | "\(.[0]) \(.[1].key) \(.[1].value.server)"
My output:
"app_1 0.0.1 vm123-4.domain.com"
"app_2 1.0.1 vm333-1.domain.com"
"app_33 0.44.1 vm888-9.domain.com"
.software | to_entries[] | "\(.key) \(.value | to_entries[] | select(.key != "latest_version") | "\(.key) \(.value.server)")"
Will produce
"app_1 0.0.1 vm123-4.domain.com"
"app_1 0.0.2 vm123-5.comain.com"
"app_2 1.0.1 vm333-1.domain.com"
"app_33 0.44.1 vm888-9.domain.com"
"app_33 1.2.2 vm123-4.domain.com"
As you can test in this online demo.
Using --raw-output with tabs \t, we can create a column like output:
app_1 0.0.1 vm123-4.domain.com
app_1 0.0.2 vm123-5.comain.com
app_2 1.0.1 vm333-1.domain.com
app_33 0.44.1 vm888-9.domain.com
app_33 1.2.2 vm123-4.domain.com
Demo
jq -r '.software | to_entries[] | .key as $app | .value | to_entries[] | select((.value | objects)) | [$app, .key, .value.server] | #csv'
Related
I have following json input file. I need to get the 2 and 3rd keys like below.
Need out put as:
test, 2021.1.2
dev, 2021.2.2
test, 2021.1.2
dev, 2021.2.1
test, 2021.3.1
Input json:
[
{
"builds": {
"test": {
"2021.1.2": {
"stages": [
"functional",
"integration"
]
}
},
"dev": {
"2021.2.2": {
"stages": [
"junit",
"nls"
]
}
}
},
"dockertag": "0.1.1"
},
{
"builds": {
"test": {
"2021.1.2": {
"stages": [
"functional"
]
}
},
"dev": {
"2021.2.1": {
"stages": [
"junit",
"nls"
]
}
}
},
"dockertag": "0.1.2"
},
{
"builds": {
"test": {
"2021.3.1": {
"stages": [
"functional",
"integration"
]
}
}
},
"dockertag": "0.1.3"
}
]
I tried following code but it is incomplete. I can get the 2nd key but not sure how to further extract 3rd key:
jq -r '.[].builds | keys[] as $k | "\($k), \(.[$k] | .)"' test.json
Thanks for the help.
.[].builds | to_entries[] | "\(.key), \(.value | keys[])"
Will generate
"test, 2021.1.2"
"dev, 2021.2.2"
"test, 2021.1.2"
"dev, 2021.2.1"
"test, 2021.3.1"
Use jq --raw-output to remove the "'s around each line
Online demo
Two approaches using the path:
jq -r '.[].builds | paths | select(length == 2) | join(", ")'
test, 2021.1.2
dev, 2021.2.2
test, 2021.1.2
dev, 2021.2.1
test, 2021.3.1
Demo
jq -r 'path(.[].builds[][])[2:] | join(", ")'
test, 2021.1.2
dev, 2021.2.2
test, 2021.1.2
dev, 2021.2.1
test, 2021.3.1
Demo
Here is a ruby if your jq is broken that day:
ruby -r json -e 'JSON.parse($<.read).
each{|h| h.select{|k,v| k=="builds"}.
each{|k,v| ["test","dev"].
each{|t| puts "#{t}, #{v[t].keys[0]}" if v.key?(t)}}} ' file
test, 2021.1.2
dev, 2021.2.2
test, 2021.1.2
dev, 2021.2.1
test, 2021.3.1
I am trying to extract kubeconfig data with jq.
kubectl config view --raw -o json | jq ...
There is a json produced of this kind:
{
"kind": "Config",
"apiVersion": "v1",
"preferences": {},
"clusters": [
{
"name": "some-name",
"cluster": {
"server": "https://some-url",
"certificate-authority-data": "some-cert"
}
},
{
"name": "another-name",
"cluster": {
"server": "https://another-url",
"certificate-authority-data": "another-cert"
}
}
],
"users": [
{
"name": "some-name",
"user": {
"username": "some-user",
"password": "some-password"
}
},
{
"name": "another-name",
"user": {
"username": "another-user",
"password": "another-password"
}
}
],
"contexts": [],
"current-context": "some-context"
}
Question #1:
For a given name ,"some-name", I'd like to extract json:
{
url: "https://some-url",
cert: "some-cert",
username: "some-user",
password: "some-password"
}
Question #2:
"users" sub-section can have other format
"users": [
{
"name": "...",
"user": {
"exec": {
...
}
Where .user.username or .user.password or both can be missing
In this case overall query should return "{}", even though, "clusters" query/branch has result
Question 3, as a follow up to Jeff Mercado answer:
I want to get all clusters, joined (grouped by) name:
Looking at the manual, https://stedolan.github.io/jq/manual/#Builtinoperatorsandfunctions ,
section "Multiplication, division, modulo: *, /, and %", example:
jq '{"k": {"a": 1, "b": 2}} * {"k": {"a": 0,"c": 3}}' => {"k": {"a": 0, "b": 2, "c": 3}}'
gives presumably right result, assuming "k" is value of "name". So, grouping by "k", merging (*) the results.
I produced following query:
echo "${json}" | jq -r '(.clusters[] | {(.name): {url: .cluster.server, cert: .cluster["certificate-authority-data"]}}) * (.users[] | {(.name): {user: .user.username, password: .user.password}})'
First part returns {"name": {url: cert}}, second part is {"name": {username, password}}
However, result is not merge as in jq Manual, but something else ... product ?
{
"some-name": {
"url": "https://some-url",
"cert": "some-cert",
"user": "some-user",
"password": "some-password"
}
}
{
"another-name": {
"url": "https://another-url",
"cert": "another-cert"
},
"some-name": {
"user": "some-user",
"password": "some-password"
}
}
{
"some-name": {
"url": "https://some-url",
"cert": "some-cert"
},
"another-name": {
"user": "another-user",
"password": "another-password"
}
}
{
"another-name": {
"url": "https://another-url",
"cert": "another-cert",
"user": "another-user",
"password": "another-password"
}
}
Why/what is it ? Kind of following idea of the product ('*') but not of the jq tutorial as I (most likely, incorrectly) understand it
Experimentation:
I have 2 queries now producing partial result.
Let's grab original json (above) in and parse:
read -d '' json << EOF
...
EOF
queries:
echo "${json}" | jq -r '.clusters[] | select(.name=="some-name") | .cluster | {url: .server, cert: .["certificate-authority-data"]}' &&\
echo "${json}" | jq -r '.users[] | select(.name=="some-name") | .user | {user: .username, password: .password}'
Will produce the split output:
{
"url": "https://some-url",
"cert": "some-cert"
}
{
"user": "some-user",
"password": "some-password"
}
Or, with key added for further merge:
echo "${json}" | jq -r '.clusters[] | select(.name=="some-name") | {name: .name, url: .cluster.server, cert: .cluster["certificate-authority-data"]}' &&\
echo "${json}" | jq -r '.users[] | select(.name=="some-name") | {name: .name, user: .user.username, password: .user.password}'
Will produce:
{
"name": "some-name",
"url": "https://some-url",
"cert": "some-cert"
}
{
"name": "some-name",
"user": "some-user",
"password": "some-password"
}
"name" is not needed but can be used as a join operation
So you already know how to get the cluster and user by name separately, first step is to select them both within a single filter:
(.clusters[] | select(.name == $name).cluster), (.users[] | select(.name == $name).user)
This will yield two separate objects, the cluster, then the user. But we want to merge them. There's plenty of ways to do this. You could add them (+) directly or merge them (*) but no real difference there. You'll just want to remap the properties to the names you wanted where needed.
(.clusters[] | select(.name == $name).cluster | {url: .server, cert: ."certificate-authority-data"})
+
(.users[] | select(.name == $name).user | {username, password})
Pass the name in as a parameter to your filter;
$ kubectl config view --raw -o json | jq --arg name some-name '
(.clusters[] | select(.name == $name).cluster | {url: .server, cert: ."certificate-authority-data"})
+
(.users[] | select(.name == $name).user | {username, password})
'
For the second part of your question, if it turns out the mapped user is missing key properties and you want to omit them, just add another select filter to the end to test for those properties and replace with an empty object if nothing is found:
... | select(has("username") and has("password")) // {}
jqplay
{
"status": "DOWN",
"components": {
"ping": {
"status": "UP",
"details": {
"version": "1.0.0",
"description": "dobre application",
"name": "SO-3113"
}
},
"bridge.lock": {
"status": "UP",
"details": {
"description": "test1"
}
},
"Configuration converted": {
"status": "DOWN",
"details": {
"description": "test2"
}
},
"app started": {
"status": "DOWN",
"details": {
"description": "test3"
}
}
}
}
I need to get the name of first component with DOWN status ("Configuration converted" in above json). So far i managed to get only .details.description of it:
jq -c '.components| .[] | select( .status | contains("DOWN")) .details.description' | head -1
How can I get the name (key) of component? ("Configuration converted")
You can use to_entries to get the key/values of your components , then select the first one with down status:
first(.components | to_entries | .[] | select( .value.status == "DOWN") | .key)
Run it on jqplay
You could use to_entries function to convert the objects within .components into key/value pairs and provide an expression that selects the first object matching the condition and retrieve its key
.components |
to_entries |
map(select(.value.status == "DOWN"))[0].key
I am trying to read the CIDR blocks from the VPCs in AWS on the AWS CLI. I will use this in a script when I'm done. I am using jq to parse the info:
aws ec2 describe-vpcs --region=us-east-1 | jq -r '.Vpcs[].CidrBlock'
10.200.3.0/24
However, jq only returns one of the two CIDR blocks in the VPC. This is the original json:
{
"Vpcs": [
{
"CidrBlock": "10.200.3.0/24",
"DhcpOptionsId": "dopt-d0aa95ab",
"State": "available",
"VpcId": "vpc-00de11103235ec567",
"OwnerId": "046480487130",
"InstanceTenancy": "default",
"Ipv6CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-assoc-09f19d81c2e4566b9",
"Ipv6CidrBlock": "2600:1f18:1f7:300::/56",
"Ipv6CidrBlockState": {
"State": "associated"
},
"NetworkBorderGroup": "us-east-1"
}
],
"CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-assoc-0511a5d459f937899",
"CidrBlock": "10.238.3.0/24",
"CidrBlockState": {
"State": "associated"
}
},
{
"AssociationId": "vpc-cidr-assoc-05ad73e8c515a470f",
"CidrBlock": "100.140.0.0/27",
"CidrBlockState": {
"State": "associated"
}
}
],
"IsDefault": false,
"Tags": [
{
"Key": "environment",
"Value": "int01"
},
{
"Key": "Name",
"Value": "company-int01-vpc"
},
{
"Key": "project",
"Value": "company"
}
]
}
]
}
Why does jq only return part of the info I'm after? I need to get all VPC CIDR blocks in the output.
You have two keys CidrBlock and CidrBlockAssociationSet under the Vpcs array.
aws ec2 describe-vpcs --region=us-east-1 |
jq -r '.Vpcs[] | .CidrBlock, .CidrBlockAssociationSet[].CidrBlock'
10.200.3.0/24
10.238.3.0/24
100.140.0.0/27
and this is an invariant solution:
aws ... | jq -r '.. | if type == "object" and has("CidrBlock") then .CidrBlock else empty end'
and, inspired by jq170727's answer, a less expressive form:
aws ... | jq -r '.. | objects | .CidrBlock // empty'
Here is a filter inspired by Dmitry's answer which is slightly shorter: .. | .CidrBlock? | values
Try it online!
example.json
{
"foo": {
"bar": {
"name": "bob",
"role": "gopherthis"
},
"baz": {
"name": "tom",
"role": "gopherthat"
}
}
}
example query:
jq -r '.foo[] | " \(.name),\(.role) " ' example.json
bob,gopherthis
tom,gopherthat
My question is how do I get bar and baz associated with their name and role? Where the output would be:
bar,bob,gopherthis
baz,tom,gopherthat
jq solution:
jq -r '.foo | to_entries[] | "\(.key),\(.value.name),\(.value.role)"' example.json
The output:
bar,bob,gopherthis
baz,tom,gopherthat