There's "0" in a JSON giving me trouble, I'm trying to grab the value at:
response > data > sessions > 0 > user
The code I used was:
jq -r '.response.data.sessions."0".user'
The terminal returns: jq: error (at :1): Cannot index array with string "0"
I also tried:
jq -r '.response.data.sessions.0.user'
The terminal returned: jq: error: syntax error, unexpected LITERAL, expecting $end (Unix shell quoting issues?) at , line 1:
.response.data.sessions.0.user
Can use like this
https://jqplay.org/s/nYlPohfTdZ
.response.data.sessions[0].user
Related
Hi I am having a shell script ,
the shell script has a variable named RESPONSE
I want a to check if the json object has an attribute named "address" is present. Also how to get value of the attribute address . I want to use jq library
#!/bin/bash
RESPONSE={"content-length":"2","address":"192.168.123.1","path":"/hello-world"}
appreciate any help
jq '.address' will output the value of the property value, e.g. "192.168.123.1". (jq -r to get rid of the quote marks)
jq 'has(.address)' will output true/false
jq -e 'has(.address)' >/dev/null will set the process' exit status to 0/1 depending on the object having the key "address" (regardless of value)
jq -e '.address' >/dev/null will set the process' exit status to 0 if the "address" property has a truthy value (anything except null or false). A null or false valued property will set the exit code to 1.
Having the appropriate exit status allows you to do things such as:
if jq -e '.address' >/dev/null; then
echo 'object has address field'
fi
# or:
jq -e 'has(.address)' >/dev/null || echo 'object does not have an address'
in AWS CLI, the command aws quicksight describe-data-set blah blah returns a json document with the following troublesome syntax:
{
"Status": 200,
"DataSet": {
"Arn": "arn:aws:quicksight:<region>:<acct>:dataset/b7c87122-e180-47a9-a8a4-19f171e13fc8",
"DataSetId": "b7c87122-e180-47a9-a8a4-19f171e13fc8",
"Name": "MyName",
"CreatedTime": "2022-08-16T12:01:54.948000-05:00",
"LastUpdatedTime": "2022-08-19T08:47:55.553000-05:00",
"PhysicalTableMap": {
"6fac5dee-3691-4ddd-ba7a-0667168bb80c": {
"CustomSql": {
"DataSourceArn": "arn:aws:quicksight:<region>:<acct>:datasource/46f83f8b-181e-4575-8d61-84c50125f3aa",
I need to address that DataSetArn, but the key "6fac5dee-3691-4ddd-ba7a-0667168bb80c" is unknown to me at runtime. How do I address it?
I tried:
jq -r '.DataSet.PhysicalTableMap.*.CustomSql.DataSourceArn'
jq -r '.DataSet.PhysicalTableMap.\*.CustomSql.DataSourceArn'
jq -r '.DataSet.PhysicalTableMap.?.CustomSql.DataSourceArn'
jq -r '.DataSet.PhysicalTableMap.\?.CustomSql.DataSourceArn'
jq -r '.DataSet.PhysicalTableMap.%.CustomSql.DataSourceArn'
jq -r '.DataSet.PhysicalTableMap.\%.CustomSql.DataSourceArn'
All return an error similar to:
jq: error: syntax error, unexpected INVALID_CHARACTER, expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
.DataSet.PhysicalTableMap.\?.CustomSql.DataSourceArn
jq: 1 compile error
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
BrokenPipeError: [Errno 32] Broken pipe
I'm a noob, I know I'm guessing here. Does anyone have any insight on this?
Something like this:
jq -r '.DataSet.PhysicalTableMap[].CustomSql.DataSourceArn'
The part .DataSet.PhysicalTableMap returns the object as one result, the following filter [] takes that object and returns each value as one result. The following filters take each of these separate results and refines more stuff.
Note: If the object is the top-level item then the syntax is .[] .
As in Passing bash variable to jq, we should be able to use a JQ variable as $VAR in a jq expression.
projectID=$(jq -r --arg EMAILID "$EMAILID" '
.resource[]
| select(.username==$EMAILID)
| .id' file.json
)
SO to extract project_id from the json file sample.json.
{
"dev": {
"gcp": {
"project_id": "forecast-dev-1234",
"project_number": "123456789",
"endpoint_id": "6837352639743655936"
}
}
}
Run the JQ expression using a variable but did not work.
# TARGET=dev
$ jq -r --arg TARGET "${TARGET}" '.$TARGET.gcp.project_id' sample.json
-----
jq: error: syntax error, unexpected '
.$TARGET.gcp.project_id
(Unix shell quoting issues?) at <top-level>, line 1:
.$TARGET.gcp.project_id
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.$TARGET.gcp.project_id
jq: 2 compile errors
Please help understand why and how to use the variable to form an expression to extract project_id.
JQ Manual does not provide clear explanation for variable and ---arg. Is there a good resource that clearly explain JQ variable and how to use it?
Another way to set the exit status is with the halt_error builtin function.
--arg name value:
This option passes a value to the jq program as a predefined variable. If you run jq with --arg foo bar, then $foo is available in the program and has the value "bar". Note that value will be treated as a string, so --arg foo 123 will bind $foo to "123".
Workaround
Using interpolation.
$ TARGET=dev
$ jq -r --arg TARGET "${TARGET}" '."\($TARGET)".gcp.project_id' sample_interpolation.json
-----
forecast-dev-1234
Version
jq --version
---
jq-1.64l
You're using variables fine, the problem is that object-identifier syntax doesn't allow general expressions. It's a shorthand syntax for when the key you're looking up is a fixed identifier-like string, like .foo or .project_id. As noted in the manual, you can use the more general generic object index filter for arbitrary keys including those that are calculated by some expression, such as .[$TARGET]:
$ TARGET=dev
$ jq -r --arg TARGET "${TARGET}" '.[$TARGET].gcp.project_id' sample.json
forecast-dev-1234
❯ jq --version
jq-1.6
I'm using .jq file as a filter like following, it works:
❯ cat jq/script.jq
def fi(v):
v | tostring |
if test("\\.") then
"float"
else
"integer"
end;
def estype(v):
if type=="number" then
fi(v)
else
type
end;
def esprop(v):
if type=="object" then
{"properties": v | with_entries(.value |= esprop(.))}
else
{"type": estype(v)}
end;
with_entries(.value |= esprop(.))
❯ cat test.json | jq -f jq/script.jq
...(omit results)
But when I use it as library, it throw an error:
# comment the last filter, except the definitions of functions
❯ cat jq/script.jq
def fi(v):
v | tostring |
if test("\\.") then
"float"
else
"integer"
end;
def estype(v):
if type=="number" then
fi(v)
else
type
end;
def esprop(v):
if type=="object" then
{"properties": v | with_entries(.value |= esprop(.))}
else
{"type": estype(v)}
end;
# with_entries(.value |= esprop(.))
❯ cat test.json | jq -L jq/script.jq 'import script;'
jq: error: syntax error, unexpected IDENT, expecting FORMAT or QQSTRING_START (Unix shell quoting issues?) at <top-level>, line 1:
import script;
jq: 1 compile error
What it means and how could I debug and fix this?
Are .jq files as a filter or a library has different syntax(doesn't seems like that)?
1a. What does it mean?
syntax error, unexpected IDENT, expecting FORMAT or QQSTRING_START
This means the parser found an identifier where it was expecting a string. (FORMAT is the token for a formatter like #csv or #text, while QQSTRING_START is the token for a string, like "script". In practice it's useless to use a formatter here since it won't let you use a non-constant string, but the parser doesn't know that.)
1b. How to debug and fix this?
Probably easiest to refer back to the manual. It says that the form expect for "import" is
import RelativePathString as NAME;
and the form expected for "include" is
include RelativePathString;
It lacks examples to make this 100% clear, but "RelativePathString" is a placeholder - it needs to be a literal string. Try one of these:
cat test.json | jq -L jq 'include "script"; with_entries(.value |= esprop(.))'
cat test.json | jq -L jq 'import "script" as script; with_entries(.value |= script::esprop(.))'
Note that the library path should be the directory containing your script, and the difference between include and import.
2. Do .jq files used as a filter or a library have a different syntax?
They use the same syntax. The problem was with the import statement, not with the script file.
I am very new to RNAseq analysis. I am trying to run HTSeq using Galaxy as I am unfamiliar with coding and how to do this with Python.
I get the following error:
Use of uninitialized value $input_file in concatenation (.) or string at /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/fcaramia/edger/86292c2b0ba9/edger/htseq.pl line 57.
Usage: file [-bchikLlNnprsvz0] [--apple] [--mime-encoding] [--mime-type]
[-e testname] [-F separator] [-f namefile] [-m magicfiles] file ...
file -C [-m magicfiles]
file [--help]
Use of uninitialized value $input_file in concatenation (.) or string at /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/fcaramia/edger/86292c2b0ba9/edger/htseq.pl line 62.
This is followed by a long list of where the error is found. For example:
Error occured when reading first line of sam file.
Error:
[Exception type: StopIteration, raised in count.py:81]
Use of uninitialized value $sample in hash element at /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/fcaramia/edger/86292c2b0ba9/edger/htseq.pl line 67.
Use of uninitialized value $files[0] in string eq at /mnt/galaxyTools/shed_tools/toolshed.g2.bx.psu.edu/repos/fcaramia/edger/86292c2b0ba9/edger/htseq.pl line 78.