I'm new to JSONPath and am having some difficulty getting the right syntax to the element that I need. I'm working with responses from the Census Bureau, and the typical response I get is:
{
results: [
{
layerId: 0,
layerName: "Census Tracts",
displayFieldName: "BASENAME",
value: "2105",
attributes: {
MTFCC: "G5020",
OID: "20790282931526",
GEOID: "42091210500",
STATE: "42",
COUNTY: "091",
TRACT: "210500",
BASENAME: "2105",
NAME: "Census Tract 2105",
LSADC: "CT",
FUNCSTAT: "S",
AREALAND: "3379105",
AREAWATER: "0",
STGEOMETRY: "Polygon",
CENTLAT: "+40.0836945",
CENTLON: "-075.1929315",
INTPTLAT: "+40.0836945",
INTPTLON: "-075.1929315",
OBJECTID: "48987"
}
}
]
}
I need the "TRACT" element, but I cannot get the format right. I'm grabbing the 'value' element with
$.results[0].value
But I need the TRACT element as well. Is there a tool like Xmplify that would give me the JSONPaths for a particular element?
Thanks all.
You can get the "TRACT" values by using path $.results[*].attributes.TRACT. To test your Jsonpath you can use the utility app at http://jsonpath.herokuapp.com/.
For more information, you can refer the documentation at https://github.com/json-path/JsonPath
Related
I am new to jq and can't seem to quite get the syntax right for what I want to do. I am executing a command and piping its JSON output into jq. The structure looks like this:
{
"timestamp": 1658186185,
"nodes": {
"x3006c0s13b1n0": {
"Mom": "x3006c0s13b1n0.hsn.cm",
"Port": 15002,
"state": "free",
"pcpus": 64,
"resources_available": {
"arch": "linux",
"gputype": "A100",
"host": "x3006c0s13b1n0",
"mem": "527672488kb",
"ncpus": 64,
"ngpus": 4,
"system": "polaris",
"tier0": "x3006-g1",
"tier1": "g1",
"vnode": "x3006c0s13b1n0"
},
"resources_assigned": {},
"comment": "CHC- Offlined due to node health check failure",
"resv_enable": "True",
"sharing": "default_shared",
"license": "l",
"last_state_change_time": 1658175652,
"last_used_time": 1658175652
},
And so on with a record for each node. In psuedocode, what I want to do is this:
if state is not free then display nodename : {comment = "Why is the node down"}
The nodename is the key, but could be extracted from a field inside the record. However, for future reference, I would like to understand how to get the key. I figured out (I think) that you can't use == on strings, but instead have to use the regex functions.
This gives me the if state is not free part:
<stdin> | jq '.nodes[] | .state | test("free") | not'
This gives me an object with the Mom (which includes the key) and the comment:
jq '.nodes[] | {Mom: .Mom, comment: .comment}'
The question is how do I put all that together? And as for the keys, this gives me a list of the keys: jq '.nodes | keys' but that uses the non-array version of nodes.
One way without touching the keys would be to only select those array items that match the condition, and map the remaining items' value to the comment itself using map_values:
jq '.nodes | map_values(select(.state != "free").comment)'
{
"x3006c0s13b1n0": "CHC- Offlined due to node health check failure"
}
Keeping the whole comments object, which is closer to your desired output, would be similar:
jq '.nodes | map_values(select(.state != "free") | {comment})'
{
"x3006c0s13b1n0": {
"comment": "CHC- Offlined due to node health check failure"
}
}
Accessing the keys directly is still possible though. You may want to have a look at keys, keys_unsorted or to_entries.
I have an output that i am getting in this format :-
[
{
"_class": "hudson.model.FreeStyleProject",
"name": "my-name",
"id": "123"
},
{
"_class": "hudson.model.FreeStyleProject",
"name": "my-name2",
"id": "456"
},
{
"_class": "hudson.model.FreeStyleProject",
"name": "my-name3",
"id": "789"
}
]
How can i parse the name and id using jq?
I tried to use [].name
but i get curl: (23) Failed writing body (320 != 1338)
Any help will be appreciated. Thank you.
You failed to mention the relevant error:
jq: error (at <stdin>:17): Cannot index array with string "name"
The program should be
.[].name
Because you provided an incorrect program to jq, it exited earlier than it normally would. This caused the pipe between curl and jq to close, which cause curl to become unable to write to the pipe, which caused curl to emit the error message you did provide.
Demo
https://jqplay.org/s/nolGbk3sD1
Use filter
.[] | .name, .id
I need to generate a CloudFormation parameter list (config.json) during a CodeBuild. I have a bash script in my repo that will take a couple of parameters. This will either be production, staging, or development.
Here's the generator.json. The values here will be used to generate config.json.
{
"Parameters" : {
"FargateStackSuffix" : "environment-fargate",
"VPCStackSuffix": "environment-base-vpc",
"ContainerPort" : "80",
"ContainerCpu" : "256",
"ContainerMemory" : "512",
"Path" : "*",
"productionDesiredCount" : "3",
"stagingDesiredCount" : "2",
"developmentDesiredCount" : "1",
"ELBType" : "application",
"ELBIpAddressType": "ipv4",
"productionZone": "service.example.com",
"stagingZone": "service-staging.example.com",
"devZone": "service-dev.example.com"
}
}
For example:
./generate.sh my-service production
Would generate this:
{
"Parameters" : {
"FargateStackSuffix" : "production-fargate",
"VPCStackSuffix": "production-base-vpc",
"ServiceName" : "myservice",
"EnvironmentName" : "production",
"ContainerPort" : "80",
"ContainerCpu" : "256",
"ContainerMemory" : "512",
"Path" : "*",
"DesiredCount" : "3",
"ELBType" : "application",
"ELBIpAddressType": "ipv4",
"Zone": "myservice.example.com"
}
}
As you can see there's a bit of replacement based on additions. Just iterating over the keys won't do this. Is there a way to do all fot eh transformations I need with just jq?
EDIT: I ended up using some sed to filter replace a few values. Now I need to replace the appropriate DesiredCount:
cat .codedeploy/generator.json | jq '[paths(type == "string" and contains("DesiredCount"))]'
[]
My problem is that it's returning an empty array.
You can use object construction to do this.
cat generator.json | jq --arg variable $variable '{Parameters: {DesiredCount: .Parameters.developmentDesiredCount, Zone: .Parameters.devZone, environmentName: "Production", var: $variable}}'
Here I have shown a few solutions to help you.
you can reference the parameters in generator.json by using dot notation in the value for that field in the new object.
You can simply define a string
You can use the args flag to pass in a bash variable.
Output (incomplete, but shows solution)
{
"Parameters": {
"DesiredCount": "1",
"Zone": "service-dev.example.com",
"environmentName": "Production",
"var": "VariableString"
}
}
Here's one such way you can generate that json. Perhaps the tricky part here is generating the dynamic name for the environment DesiredCount and Zone. But jq handles that rather easily.
$ jq --arg ServiceName "myservice" --arg EnvironmentName "production" '.Parameters |=
{
FargateStackSuffix,
VPCStackSuffix,
$ServiceName,
$EnvironmentName,
ContainerPort,
ContainerCpu,
ContainerMemory,
Path,
DesiredCount: ."\($EnvironmentName)DesiredCount",
ELBType,
ELBIpAddressType,
Zone: ."\($EnvironmentName)Zone"
}
' generator.json > config.json
$ cat config.json
{
"Parameters": {
"FargateStackSuffix": "environment-fargate",
"VPCStackSuffix": "environment-base-vpc",
"ServiceName": "myservice",
"EnvironmentName": "production",
"ContainerPort": "80",
"ContainerCpu": "256",
"ContainerMemory": "512",
"Path": "*",
"DesiredCount": "3",
"ELBType": "application",
"ELBIpAddressType": "ipv4",
"Zone": "service.example.com"
}
}
I have a .csv, file unfortunately one of the columns contains a dictionary that has commas in it , for example:
{"name": "Umbulharjo", "type": "Kecamatan", "level": "3", "region1": "Yogyakarta", "region2": "Yogyakarta", "region3": "Umbulharjo", "postcode": "55161"}
How can i put a " before every { and after every } in R? then i can set " as quote when i am using read.csv or read.csv2 or read.table
Your data looks to be JSON-ish. If you're doing a lot of JSON stuff, I suggest using a library that understands JSON.
paste('{"name": "Umbulharjo", "type": "Kecamatan", "level": "3", "region1": "Yogyakarta", "region2": "Yogyakarta", "region3": "Umbulharjo", "postcode": "55161"}', '')
#[1] "{\"name\": \"Umbulharjo\", \"type\": \"Kecamatan\", \"level\": \"3\", \"region1\": \"Yogyakarta\", \"region2\": \"Yogyakarta\", \"region3\": \"Umbulharjo\", \"postcode\": \"55161\"} "
I need to delete multiple keys at once from some JSON (using jq), and I'm trying to learn if there is a better way of doing this, than calling map and del every time. Here's my input data:
test.json
[
{
"label": "US : USA : English",
"Country": "USA",
"region": "US",
"Language": "English",
"locale": "en",
"currency": "USD",
"number": "USD"
},
{
"label": "AU : Australia : English",
"Country": "Australia",
"region": "AU",
"Language": "English",
"locale": "en",
"currency": "AUD",
"number": "AUD"
},
{
"label": "CA : Canada : English",
"Country": "Canada",
"region": "CA",
"Language": "English",
"locale": "en",
"currency": "CAD",
"number": "CAD"
}
]
For each item, I want to remove the number, Language, and Country keys. I can do that with this command:
$ cat test.json | jq 'map(del(.Country)) | map(del(.number)) | map(del(.Language))'
That works fine, and I get the desired output:
[
{
"label": "US : USA : English",
"region": "US",
"locale": "en",
"currency": "USD"
},
{
"label": "AU : Australia : English",
"region": "AU",
"locale": "en",
"currency": "AUD"
},
{
"label": "CA : Canada : English",
"region": "CA",
"locale": "en",
"currency": "CAD"
}
]
However, I'm trying to understand if there is a jq way of specifying multiple labels to delete, so I don't have to have multiple map(del()) directives?
You can provide a stream of paths to delete:
$ cat test.json | jq 'map(del(.Country, .number, .Language))'
Also, consider that, instead of blacklisting specific keys, you might prefer to whitelist the ones you do want:
$ cat test.json | jq 'map({label, region, locale, currency})'
There is no need to use both map and del.
You can pass multiple paths to del, separated by commas.
Here is a solution using "dot-style" path notation:
jq 'del( .[] .Country, .[] .number, .[] .Language )' test.json
doesn't require quotation marks (which you may feel makes it more readable)
doesn't group the paths (requires you to retype .[] once per path)
Here is an example using "array-style" path notation, which allows you to combine paths with a common prefix like so:
jq 'del( .[] ["Country", "number", "Language"] )' test.json
Combines subpaths under the "last-common ancestor" (which in this case is the top-level list iterator .[])
peak's answer uses map and delpaths, though it seems you can also use delpaths on its own:
jq '[.[] | delpaths( [["Country"], ["number"], ["Language"]] )]' test.json
Requires both quotation marks and array of singleton arrays
Requires you to put it back into a list (with the start and end square brackets)
Overall, here I'd go for the array-style notation for brevity, but it's always good to know multiple ways to do the same thing.
A better compromise between "array-style" and "dot-style" notation mentioned in by Louis in his answer.
del(.[] | .Country, .number, .Language)
jqplay
This form can also be used to delete a list of keys from a nested object (see russholio's answer):
del(.a | .d, .e)
Implying that you can also pick a single index to delete keys from:
del(.[1] | .Country, .number, .Language)
Or multiple:
del(.[2,3,4] | .Country,.number,.Language)
You can delete a range using the range() function (slice notation doesn't work):
del(.[range(2;5)] | .Country,.number,.Language) # same as targetting indices 2,3,4
Some side notes:
map(del(.Country,.number,.Language))
# Is by definition equivalent to
[.[] | del(.Country,.number,.Language)]
If the key contains special characters or starts with a digit, you need to surround it with double quotes like this: ."foo$", or else .["foo$"].
This question is very high in the google results, so I'd like to note that some time in the intervening years, del has apparently been altered so that you can delete multiple keys with just:
del(.key1, .key2, ...)
So don't tear your hair out trying to figure out the syntax work-arounds, assuming your version of jq is reasonably current.
In addition to #user3899165's answer, I found that to delete a list of keys from "sub-object"
example.json
{
"a": {
"b": "hello",
"c": "world",
"d": "here's",
"e": "the"
},
"f": {
"g": "song",
"h": "that",
"i": "I'm",
"j": "singing"
}
}
$ jq 'del(.a["d", "e"])' example.json
delpaths is also worth knowing about, and is perhaps a little less mysterious:
map( delpaths( [["Country"], ["number"], ["Language"]] ))
Since the argument to delpaths is simply JSON, this approach is particularly useful for programmatic deletions, e.g. if the key names are available as JSON strings.