First off thanks to everyone who takes time to answer questions for us who are learning or struggling. I have this code that works as I need, I had help from knittl earlier.
jq -r '["Username", "Full name", "Superuser", "Active"],
> (.User[]
> | select(.is_superuser)
> | [ .username, .full_name, .is_superuser, .is_active ])
> | #csv' lbc.noprod1.xxx.json > lbc.noprod1.xxx_Superuser.csv
I have tried to adjust it so that I can use the same concept but to get other information and have this
jq -r '["Interface Name", "Vlan ID", "IP address", "Subnet Mask"],
(.ServiceEngine[].data_vnics[].vlan_interfaces[]
| [ .if_name, .vlan_id])
(.ServiceEngine[].data_vnics[].vlan_interfaces[].vnic_networks[].ip[].ip_addr[]
| [.addr])
(.ServiceEngine[].data_vnics[].vlan_interfaces[].vnic_networks[].ip[]
| [.mask])
| #csv' 20221004_lbc.prod1.xxx.json > 20221004_lbc.prod1.xxx_VNIC.csv
I receive the error below
jq: error: syntax error, unexpected '(', expecting $end (Unix shell quoting issues?) at , line 4:
(.ServiceEngine[].data_vnics[].vlan_interfaces[].vnic_networks[].ip[].ip_addr[]
jq: 1 compile error
If it helps this is an extract from the JSON with fields that aren't relevant removed but all formatting kept
"ServiceEngine": [
{
"data_vnics": [
{
"vlan_interfaces": [
{
"if_name": "bond1.109",
"vlan_id": 109,
"vnic_networks": [
{
"ctlr_alloc": false,
"ip": {
"ip_addr": {
"addr": "123.123.123.123",
"type": "V4"
},
"mask": 24
},
"mode": "STATIC"
}
],
"vrf_ref": "/api/vrfcontext/?tenant=admin&name=branch-pci-client-vlan109&cloud=Default-Cloud"
},
{
"dhcp_enabled": true,
"enabled": true,
"if_name": "bond1.1622",
"ip6_autocfg_enabled": true,
"is_mgmt": false,
"vlan_id": 1622,
"vnic_networks": [
{
"ctlr_alloc": false,
"ip": {
"ip_addr": {
"addr": "456.456.456.456",
"type": "V4"
},
"mask": 22
},
"mode": "STATIC"
}
],
"vrf_ref": "/api/vrfcontext/?tenant=admin&name=onprem-pci-prod-vlan1622&cloud=Default-Cloud"
},
As always if you have any tips and suggestions I would be greatful.
Kind regards
I think you are looking for this:
["Interface Name", "Vlan ID", "IP address", "Subnet Mask"],
(
.ServiceEngine[].data_vnics[].vlan_interfaces[]
| . as { $if_name, $vlan_id }
| .vnic_networks[].ip
| [ $if_name, $vlan_id, .ip_addr.addr, .mask]
)
| #csv
or
["Interface Name", "Vlan ID", "IP address", "Subnet Mask"],
(
.ServiceEngine[].data_vnics[].vlan_interfaces[]
| [ .if_name, .vlan_id ] + (.vnic_networks[].ip | [.ip_addr.addr, .mask])
)
| #csv
or
["Interface Name", "Vlan ID", "IP address", "Subnet Mask"],
(
.ServiceEngine[].data_vnics[].vlan_interfaces[]
| { if_name, vlan_id, ips: .vnic_networks[].ip }
| [ .if_name, .vlan_id, (.ips|.ip_addr.addr, .mask) ]
)
| #csv
All three generate the same CSV with one line per IP. Interface name and VLAN id will be printed multiple times if they contain more than one IP.
Related
I have a file like below (excerpt)
cat input.json
{
"Metrics": [
{
"Namespace": "CWAgent",
"MetricName": "disk_used_percent",
"Dimensions": [
{
"Name": "path",
"Value": "/aem"
},
{
"Name": "host",
"Value": "uat2-author01.uat.cloud.abc.com.au"
},
{
"Name": "device",
"Value": "mapper/aem-aem"
},
{
"Name": "fstype",
"Value": "ext4"
}
]
},
{
"Namespace": "CWAgent",
"MetricName": "mem_used_percent",
"Dimensions": [
{
"Name": "host",
"Value": "uat2-dispatch02.uat.cloud.abc.com.au"
}
]
}
]
}
As you can see the "Dimensions" is a list of json objects of varying length. I am required to print each objects under "Metrics" in a single line like below -
CWAgent|disk_used_percent|path:/aem|host:uat2-author01.uat.cloud.abc.com.au|device:mapper/aem-aem|fstype:ext4
CWAgent|mem_used_percent|host:uat2-dispatch02.uat.cloud.abc.com.au
I used this -
cat input.json | jq -r ".Metrics[] | .Namespace + \"|\" + .MetricName + \"|\" + (.Dimensions[] | .Name + \":\" + .Value + \"|\")"
And the output is like below:
CWAgent|disk_used_percent|path:/aem|
CWAgent|disk_used_percent|host:uat2-author01.uat.cloud.abc.com.au|
CWAgent|disk_used_percent|device:mapper/aem-aem|
CWAgent|disk_used_percent|fstype:ext4|
CWAgent|mem_used_percent|host:uat2-dispatch02.uat.cloud.abc.com.au|
As you can see, we are getting 4 lines for first object each having different object under Dimensions. Can anyone please suggest what will be the jq command to get desired output. I am novice in jq and wasted too much time on this experimenting.
Thanks,
Bijitesh
You're looking for this:
jq -r '.Metrics[] | [.Namespace, .MetricName] + (.Dimensions | map("\(.Name):\(.Value)")) | join("|")' input.json
Given this input small sample:
{
"_format_version": "1.1",
"_workspace": "test",
"services": [
{
"connect_timeout": 60000,
"host": "host-name-test.com",
"name": "name-of-service",
"path": "/test/oauthpass",
"port": 777,
"protocol": "http",
"read_timeout": 1000,
"retries": 1,
"write_timeout": 1000,
"routes": [
{
"hosts": [
"Google.com"
],
"name": "com.testing.active.oauth",
"methods": [
"POST"
],
"paths": [
"/vendors/otest/pass/?$"
],
"path_handling": "v8",
"preserve_host": false,
"protocols": [
"https"
],
"regex_priority": 0,
"strip_path": true,
"https_redirect_status_code": 426,
"request_buffering": true,
"response_buffering": true
}
]
}
}
trying to get a listing from the data pulling certain values like the listing below:
host-name-test.com, Google.com, POST, HTTPS
the command that I have working so far is
cat /tmp/petecar.json | jq -r ' .services[] | .routes[] | ( .hosts[] + "/" + .paths[]) ' | more
but I can't access the values under services, please provide some sample on how to get the values
routes has an array value and as such cannot be concatenated with a string. You can use join to turn that array into a single string:
jq -r '.services[] | .host + " " + (.routes[].hosts | join(","))'
Output:
host-name-test.com Google.com
Alternatively, using string interpolation) which will automatically serialize any values into their string representation:
jq -r '.services[] | "\(.host) \(.routes[].hosts)"'
Output:
host-name-test.com ["Google.com"]
join and string interpolation can be combined, giving you the identical output of the first command:
jq -r '.services[] | "\(.host) \(.routes[].hosts|join(","))"'
I have the following data
Command output:
| jq '.rrsets[]'
{
"comments": [],
"name": "name1.",
"records": [
{
"content": "10.10.10.10",
"disabled": false
}
],
"ttl": 60,
"type": "A"
}
{
"comments": [],
"name": "name2.",
"records": [
{
"content": "20.20.20.20",
"disabled": false
}
],
"ttl": 60,
"type": "CNAME"
}
I want to get names where type is A.
Help, tell me how to do this?
| jq '.rrsets[] | with_entries(select(.key== "name", .value == "A"))'
{
"name": "name1."
}
{
"name": "name2.",
"type": "A"
}
Displays all the lines, but I only need where type = A
I'm not sure if this is what you are looking for, but wouldn't simply ... | select(.type == "A") do the trick?
... | jq '.rrsets[] | select(.type == "A")'
{
"comments": [],
"name": "name1.",
"records": [
{
"content": "10.10.10.10",
"disabled": false
}
],
"ttl": 60,
"type": "A"
}
Demo
And then just get the .name if you want only that (using -r to get rid of the JSON formatting):
... | jq -r '.rrsets[] | select(.type == "A").name'
name1.
Demo
I'd like to select the following node in a long json file.
The conditions are "locator": "mmc1" and "#name": "section-title". This nested structure makes the jq query very complex. Notice that I have to specify things like "locator" and "section-title" multiple times.
.. | .["$$"]?
| select((.[]? | ."#name" == "section-title"?) and (..[]? | .locator? | test("mmc[0-9]+")?))
| [
(..[]? | select(.locator? | test("mmc[0-9]+")?) | .locator)
, (.[] | select(."#name" == "section-title") | ._)
] | #tsv
Could anybody let me know how to make the query simpler in jq yet make sure its function is exact the same as the original query? Just making the following simplified test input producing the same output should not be considered as an equivalent query. Thanks.
...
"$$": [
{
"#name": "label",
"_": "Appendix A"
},
{
"#name": "section-title",
"$": {
"id": "sectitle0145"
},
"_": "Supplementary data"
},
{
"#name": "para",
"$": {
"id": "p0210",
"view": "all"
},
"$$": [
{
"#name": "__text__",
"_": "The following is the supplementary data related to this article:"
},
{
"#name": "display",
"$$": [
{
"#name": "e-component",
"$": {
"id": "ec1"
},
"$$": [
{
"#name": "link",
"$": {
"locator": "mmc1",
"type": "simple",
"role": "http://data.elsevier.com/vocabulary/ElsevierContentTypes/46.1",
"href": "pii:S2212877817302818/mmc1",
"id": "aep-link-id8"
}
}
]
}
]
}
]
}
]
...
See the output below.
$ jq '.. | .["$$"]? | select((.[]? | ."#name" == "section-title"?) and (..[]? | .locator? | test("mmc[0-9]+")?)) | [ (..[]? | select(.locator? | test("mmc[0-9]+")?) | .locator) , (.[] | select(."#name" == "section-title") | ._) ] | #tsv' < 1.json
"mmc1\tSupplementary data"
$ cat 1.json
{
"$$": [
{
"#name": "label",
"_": "Appendix A"
},
{
"#name": "section-title",
"$": {
"id": "sectitle0145"
},
"_": "Supplementary data"
},
{
"#name": "para",
"$": {
"id": "p0210",
"view": "all"
},
"$$": [
{
"#name": "__text__",
"_": "The following is the supplementary data related to this article:"
},
{
"#name": "display",
"$$": [
{
"#name": "e-component",
"$": {
"id": "ec1"
},
"$$": [
{
"#name": "link",
"$": {
"locator": "mmc1",
"type": "simple",
"role": "http://data.elsevier.com/vocabulary/ElsevierContentTypes/46.1",
"href": "pii:S2212877817302818/mmc1",
"id": "aep-link-id8"
}
}
]
}
]
}
]
}
]
}
If you want to get the "locator" value off of all objects unconditionally, you could use this:
..|objects.locator|strings
To find all objects with #name == "section-title" and select the _ value:
..|select(objects."#name" == "section-title")._
Putting it all together:
[(..|objects.locator|strings), (..|select(objects."#name" == "section-title")._)] | #tsv
https://jqplay.org/s/xHWg8aGSSS
The following is simpler in at least some respects, produces the desired result, and seems to reflect the requirements:
..
| .["$$"]?
| (.. | objects | .locator | strings | select(test("mmc[0-9]+")) ) as $locator
| (.. | objects | select(.["#name"] == "section-title") | ._) as $st
| [$locator, $st]
| #tsv
I want to parse terraform.tfstate (where openstack provider is used), to return instance name and it's internal + floating IP (if assigned).
First select what we are interested in:
jq -r '.modules?[]|.resources[]?|select(.type == "openstack_compute_floatingip_v2", .type == "openstack_compute_instance_v2")' < terraform.tfstate
For simplicity, pre-parsed example with the above part (one FIP and one instance):
{
"type": "openstack_compute_floatingip_v2",
"depends_on": [
"openstack_networking_router_interface_v2.management"
],
"primary": {
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"attributes": {
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"id": "48b039fc-a9fa-4672-934a-32d6d267f280",
"instance_id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"pool": "public",
"region": "RegionOne"
},
"meta": {},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
{
"type": "openstack_compute_instance_v2",
"depends_on": [
"openstack_compute_floatingip_v2.management",
"openstack_compute_secgroup_v2.ssh_only",
"openstack_networking_network_v2.management"
],
"primary": {
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"attributes": {
"access_ip_v4": "10.10.10.5",
"access_ip_v6": "",
"all_metadata.%": "1",
"all_metadata.habitat": "sup",
"availability_zone": "nova",
"flavor_id": "eb36e84e-17c1-42ab-b359-4380f6f524ae",
"flavor_name": "m1.large",
"force_delete": "false",
"id": "597e75e8-834d-4f05-8408-e2e6e733577e",
"image_id": "c574aeed-e47c-4fb7-9da0-75550b76ee56",
"image_name": "ubuntu-16.04",
"key_pair": "vault-etcd_test_tf",
"metadata.%": "1",
"metadata.habitat": "sup",
"name": "ctl01",
"network.#": "1",
"network.0.access_network": "false",
"network.0.fixed_ip_v4": "10.10.10.5",
"network.0.fixed_ip_v6": "",
"network.0.floating_ip": "",
"network.0.mac": "02:c6:61:f9:ee:7e",
"network.0.name": "management",
"network.0.port": "",
"network.0.uuid": "f2468669-e321-4eb4-9ede-003e362a8988",
"region": "RegionOne",
"security_groups.#": "1",
"security_groups.1845949017": "vault-etcd_test_ssh_only",
"stop_before_destroy": "false"
},
"meta": {
"e2bfb730-ecaa-11e6-8f88-34363bc7c4c0": {
"create": 1800000000000,
"delete": 1800000000000,
"update": 1800000000000
}
},
"tainted": false
},
"deposed": [],
"provider": "provider.openstack"
}
Required is to take from "type": "openstack_compute_floatingip_v2" replace .primary.attributes.address and .fixed_ip and from corresponding .instance_id the .name.
So, sth like:
{"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"}
Well, I came with an idea while using walk, but miss how to actually assign the proper value from corresponding instance id:
jq -r "$(cat floating.jq)" terraform.tfstate
floating.jq:
def walk(f):
. as $in
| if type == "object" then
reduce keys[] as $key
( {}; . + { ($key): ($in[$key] | walk(f)) } ) | f
elif type == "array" then map( walk(f) ) | f
else f
end;
.modules?[]|.resources[]?|select(.type ==
"openstack_compute_floatingip_v2", .type ==
"openstack_compute_instance_v2")|
.primary|walk( if type == "object" and .attributes.address then
.attributes.instance_id |= "REFERRED VALUE HERE") else . end)
Let's assume the two related objects are in a file named two.json. Then one way to merge the information from both objects is using the -s command-line option, e.g.
jq -s '
(.[0].primary.attributes | {address, fixed_ip})
+ {name: .[1].primary.attributes.name}' two.json
Output
With your example input, the output would be:
{
"address": "209.66.89.143",
"fixed_ip": "10.10.10.5",
"name": "ctl01"
}