Hi I am having a shell script ,
the shell script has a variable named RESPONSE
I want a to check if the json object has an attribute named "address" is present. Also how to get value of the attribute address . I want to use jq library
#!/bin/bash
RESPONSE={"content-length":"2","address":"192.168.123.1","path":"/hello-world"}
appreciate any help
jq '.address' will output the value of the property value, e.g. "192.168.123.1". (jq -r to get rid of the quote marks)
jq 'has(.address)' will output true/false
jq -e 'has(.address)' >/dev/null will set the process' exit status to 0/1 depending on the object having the key "address" (regardless of value)
jq -e '.address' >/dev/null will set the process' exit status to 0 if the "address" property has a truthy value (anything except null or false). A null or false valued property will set the exit code to 1.
Having the appropriate exit status allows you to do things such as:
if jq -e '.address' >/dev/null; then
echo 'object has address field'
fi
# or:
jq -e 'has(.address)' >/dev/null || echo 'object does not have an address'
Related
As in Passing bash variable to jq, we should be able to use a JQ variable as $VAR in a jq expression.
projectID=$(jq -r --arg EMAILID "$EMAILID" '
.resource[]
| select(.username==$EMAILID)
| .id' file.json
)
SO to extract project_id from the json file sample.json.
{
"dev": {
"gcp": {
"project_id": "forecast-dev-1234",
"project_number": "123456789",
"endpoint_id": "6837352639743655936"
}
}
}
Run the JQ expression using a variable but did not work.
# TARGET=dev
$ jq -r --arg TARGET "${TARGET}" '.$TARGET.gcp.project_id' sample.json
-----
jq: error: syntax error, unexpected '
.$TARGET.gcp.project_id
(Unix shell quoting issues?) at <top-level>, line 1:
.$TARGET.gcp.project_id
jq: error: try .["field"] instead of .field for unusually named fields at <top-level>, line 1:
.$TARGET.gcp.project_id
jq: 2 compile errors
Please help understand why and how to use the variable to form an expression to extract project_id.
JQ Manual does not provide clear explanation for variable and ---arg. Is there a good resource that clearly explain JQ variable and how to use it?
Another way to set the exit status is with the halt_error builtin function.
--arg name value:
This option passes a value to the jq program as a predefined variable. If you run jq with --arg foo bar, then $foo is available in the program and has the value "bar". Note that value will be treated as a string, so --arg foo 123 will bind $foo to "123".
Workaround
Using interpolation.
$ TARGET=dev
$ jq -r --arg TARGET "${TARGET}" '."\($TARGET)".gcp.project_id' sample_interpolation.json
-----
forecast-dev-1234
Version
jq --version
---
jq-1.64l
You're using variables fine, the problem is that object-identifier syntax doesn't allow general expressions. It's a shorthand syntax for when the key you're looking up is a fixed identifier-like string, like .foo or .project_id. As noted in the manual, you can use the more general generic object index filter for arbitrary keys including those that are calculated by some expression, such as .[$TARGET]:
$ TARGET=dev
$ jq -r --arg TARGET "${TARGET}" '.[$TARGET].gcp.project_id' sample.json
forecast-dev-1234
As the tile says, I wonder when jq exits with code 1. In its manual, it says -e sets the exit status of jq to 0 if the last output values was neither false nor null, 1 if the last output value was either false or null. Not clear what it means by the last output values of false or null? what if I don't use -e?
The following is intended to supplement the information given about the -e option elsewhere on this page.
Assuming -e has NOT been specified, the return code is:
5 if a call to error/1 or halt_error/0 causes program termination
an integral value (*) depending on N if halt_error(N) causes program termination, where N is a number; in particular, if N is non-negative, then the status is set to (N%256).
Otherwise, but still assuming -e has NOT been specified, the return code is:
2 if a parsing error reading input causes program termination
3 or 4 if a syntax error in the jq program causes program termination
0 on normal termination.
(*) Specifically:
N % 256 | if . < 0 then 256+. else . end
This answer is based on current jq 1.6 source code from https://github.com/stedolan/jq
With --exit-status (-e), there are 6 possible exit codes:
0: jq output something and last line was neither false nor null
1: last line output was false or null
2: usage problem or system error
3: jq program compile error
4: jq didn't ouput anything at all
5: unknown (unexpected) error: any error other than 2 and 3
Without --exit-status (-e), 0 just means that jq ran successfully. Additionally, exit status 1 and 4 disappear and 0 is returned instead.
Here is (Unix Bourne shell) some ways to get 1 as an exit value:
$ echo false | jq -e .
false
$ echo '{ "foo": false }' | jq -e .foo
false
$ echo null | jq -e .
null
$ echo '{ "foo": null }' | jq -e .foo
null
$ echo '{ }' | jq -e .foo
null
$ echo '{ "foo": false }' | jq -e '.bar?'
null
Here is how to get 4:
$ echo 'false' | jq -e '.foo?'
And (I'm sure you want to know) here is one way to get 5:
$ echo false | jq .foo
jq: error (at <stdin>:1): Cannot index boolean with string "foo"
Let's pretend I'm running something like this:
jq -nr --arg target /tmp \
'(["echo","Hello, world"]|#sh)+">\($target)/sample.txt"' \
| sh
Everything is fine unless I forgot to pass variable $target:
$ jq -nr '(["echo","Hello, world"]|#sh)+">\($target)/sample.txt"'
jq: error: $target is not defined at <top-level>, line 1:
(["echo","Hello, world"]|#sh)+">\($target)/sample.txt"
jq: 1 compile error
How can I catch this and use default value?
I've tried:
$target?
($target)?
try $target catch null
$target? // null
But it seems to be parsing-time error, which obviously can't be caught at runtime. Have I've missed any dynamic syntax?
I've found that command-line arguments can be found in $ARGS.name, but there are two drawbacks:
This was introduced in version 1.6, but I have 1.5 on CentOS 7.
It doesn't catch locally defined variables.
Assuming you need to do something more useful with jq than write 'Hello World' over a text file. I propose the following,
Maybe we can learn some programming tips from Jesus:
"Give to Caesar what belongs to Caesar, and give to God what belongs to God"
Suppose that Caesar is bash shell and God is jq, bash is appropriate to work and test the existence of files, directories and environment variables, jq is appropriate to process information in json format.
#!/bin/bash
dest_folder=$1
#if param1 is not given, then the default is /tmp:
if [ -z $dest_folder ]; then dest_folder=/tmp ; fi
echo destination folder: $dest_folder
#check if destination folder exists
if [ ! -d $dest_folder ]
then
echo "_err_ folder not found"
exit 1
fi
jq -nr --arg target $dest_folder '(["echo","Hello, world"]|#sh)+">\($target)/sample.txt"' | sh
#if the file is succesfully created, return 0, if not return 1
if [ -e "$dest_folder/sample.txt" ]
then
echo "_suc_ file was created ok"
exit 0
else
echo "_err_ when creating file"
exit 1
fi
Now you can include this script as a step in a more complex batch, because it is congruent with linux style, returning 0 on success.
I am using jq to search for specific results in a large file. I do not care for duplicate entries matching this specific condition, and it takes a while to process the whole file. What I would like to do is print some details about the first match and then terminate the jq command on the file to save time.
I.e.
jq '. | if ... then "print something; exit jq" else ... end'
I looked into http://stedolan.github.io/jq/manual/?#Breakingoutofcontrolstructures but this didn't quite seem to apply
EDIT:
The file I am parsing contains multiple json objects, one after another. They are not in an array.
Here is an approach which uses a recent version of first/1 (currently in master)
def first(g): label $out | g | ., break $out;
first(inputs | if .=="100" then . else empty end)
Example:
$ seq 1000000000 | jq -M -Rn -f filter.jq
Output (followed by immediate termination)
"100"
Here I use seq in lieu of a large JSON dataset.
To do what is requested is possible using features that were added after the release of jq 1.4. The following uses foreach and inputs:
label $top
| foreach inputs as $line
# state: true means found; false means not yet found
(false;
if . then break $top
else if $line | tostring | test("goodbye") then true else false end
end;
if . then $line else empty end
)
Example:
$ cat << EOF | jq -n -f exit.jq
1
"goodbye"
3
4
EOF
Result:
"goodbye"
You can accomplish it using halt and the inputs builtin:
jq -n 'inputs | if ... then "something", halt else ... end'
Will print "something" and terminate gracefully when the condition matches.
For this to work (i.e. terminate when condition is true), jq needs the -n parameter. See this issue
I want to be able to tell if a command exists on any POSIX system from a shell script.
On Linux, I can do the following:
if which <command>; then
...snip...
fi
However, Solaris and MacOS which do not give an exit failure code when the command does not exist, they just print an error message to STDOUT.
Also, I recently discovered that the which command itself is not POSIX (see http://pubs.opengroup.org/onlinepubs/9699919799/idx/utilities.html)
Any ideas?
command -v is a POSIX specified command that does what which does.
It is defined to to return >0 when the command is not found or an error occurs.
You could read the stdout/stderr of "which" into a variable or an array (using backticks) rather than checking for an exit code.
If the system does not have a "which" or "where" command, you could also grab the contents of the $PATH variable, then loop over all the directories and search for the given executable. That's essentially what which does (although it might use some caching/optimization of $PATH results).
One which utility is available as shell script in the Git repository of debianutils package of Debian Linux. The script seems to be POSIX compatible and you could use it, if you take into account copyright and license. Note that there have been some controversy whether or not and how the which utility should be deprecated; (at time of writing) current version in Git shows deprecation message whereas an earlier version added later removed -s option to enable silent operation.
command -v as such is problematic as it may output a shell function name, an alias definition, a keyword, a builtin or a non-executable file path. On the other hand some path(s) output by which would not be executed by shell if you run the respective argument as such or as an argument for command. As an alternative for using the which script, a POSIX shell function using command -v could be something like
#!/bin/sh
# Argument $1 should be the basename of the command to be searched for.
# Outputs the absolute path of the command with that name found first in
# a directory listed in PATH environment variable, if the name is not
# shadowed by a special built-in utility, a regular built-in utility not
# associated with a PATH search, or a shell reserved word; otherwise
# outputs nothing and returns 1. If this function prints something for
# an argument, it is the path of the same executable as what 'command'
# would execute for the same argument.
executable() {
if cmd=$(unset -f -- "$1"; command -v -- "$1") \
&& [ -z "${cmd##/*}" ] && [ -x "$cmd" ]; then
printf '%s\n' "$cmd"
else
return 1
fi
}
Disclaimer: Note that the script using command -v above does not find an executable whose name equals a name of a special built-in utility, a regular built-in utility not associated with a PATH search, or a shell reserved word. It might not find an executable either in case if there is non-executable file and executable file available in PATH search.
A function_command_exists for checking if a command exists:
#!/bin/sh
set -eu
function_command_exists() {
local command="$1"
local IFS=":" # paths are delimited with a colon in $PATH
# iterate over dir paths having executables
for search_dir in $PATH
do
# seek only in dir (excluding subdirs) for a file with an exact (case sensitive) name
found_path="$(find "$search_dir" -maxdepth 1 -name "$command" -type f 2>/dev/null)"
# (positive) if a path to a command was found and it was executable
test -n "$found_path" && \
test -x "$found_path" && \
return 0
done
# (negative) if a path to an executable of a command was not found
return 1
}
# example usage
echo "example 1";
command="ls"
if function_command_exists "$command"; then
echo "Command: "\'$command\'" exists"
else
echo "Command: "\'$command\'" does not exist"
fi
command="notpresent"
if function_command_exists "$command"; then
echo "Command: "\'$command\'" exists"
else
echo "Command: "\'$command\'" does not exist"
fi
echo "example 2";
command="ls"
function_command_exists "$command" && echo "Command: "\'$command\'" exists"
command="notpresent"
function_command_exists "$command" && echo "Command: "\'$command\'" does not exist"
echo "End of the script"
output:
example 1
Command: 'ls' exists
Command: 'notpresent' does not exist
example 2
Command: 'ls' exists
End of the script
Note that even the set -eu that turns -e option for the script was used the script was executed to the last line "End of the script"
There is no Command: 'notpresent' does not exist in the example 2 because of the && operator so the execution of echo "Command: "\'$command\'" does not exist" is skipped but the execution of the script continues till the end.
Note that the function_command_exists does not check if you have a right to execute the command. This needs to be done separately.
Solution with command -v <command-to-check>
#!/bin/sh
set -eu;
# check if a command exists (Yes)
command -v echo > /dev/null && status="$?" || status="$?"
if [ "${status}" = 127 ]; then
echo "<handle not found 1>"
fi
# check if a command exists (No)
command -v command-that-does-not-exists > /dev/null && status="$?" || status="$?"
if [ "${status}" = 127 ]; then
echo "<handle not found 2>"
fi
produces:
<handle not found 2>
because echo was found at the first example.
Solution with running a command and handling errors including command not found.
#!/bin/sh
set -eu;
# check if a command exists (No)
command -v command-that-does-not-exist > /dev/null && status="$?" || status="$?"
if [ "${status}" = 127 ]; then
echo "<handle not found 2>"
fi
# run command and handle errors (no problem expected, echo exist)
echo "three" && status="$?" || status="$?"
if [ "${status}" = 127 ]; then
echo "<handle not found 3>"
elif [ "${status}" -ne 0 ]; then
echo "<handle other error 3>"
fi
# run command and handle errors (<handle not found 4> expected)
command-that-does-not-exist && status="$?" || status="$?"
if [ "${status}" = 127 ]; then
echo "<handle not found 4>"
elif [ "${status}" -ne 0 ]; then
echo "<handle other error 4>"
fi
# run command and handle errors (command exists but <handle other error 5> expected)
ls non-existing-path && status="$?" || status="$?"
if [ "${status}" = 127 ]; then
echo "<handle not found 5>"
elif [ "${status}" -ne 0 ]; then
echo "<handle other error 5>"
fi
produces:
<handle not found 2>
three
./function_command_exists.sh: 34: ./function_command_exists.sh: command-that-does-not-exist: not found
<handle not found 4>
ls: cannot access 'non-existing-path': No such file or directory
<handle other error 5>
The following works in both bash and zsh and avoids both functions and aliases.
It returns 1 if the binary is not found.
bin_path () {
if [[ -n ${ZSH_VERSION:-} ]]; then
builtin whence -cp "$1" 2> /dev/null
else
builtin type -P "$1"
fi
}