How to escape dash in jq call?
cat afile | jq -r .commit-sha1
So the dash is being interpreted in some way. How to escape it? I tried wrap with "
For such keys, you need to quote the key, but the quotes must also be quoted so that they aren't removed by the shell prior to running jq.
jq -r '."commit-sha1"'
The abbreviated from .foo is for "simple, identifier-like keys", according to the man page.
Related
I am trying to run a jq query on a windows machine and it extracts values from output on a separate line
jq -r .Accounts[].Id
Output
204359864429
224271824096
282276286062
210394168456
090161402717
How do I run the jq query so that it combines the output on a single line separated by space
This is what I need-
204359864429 224271824096 282276286062 210394168456 090161402717
Any help would be appreciated.
The usual way would be to use the #csv or #tsv operators to convert the result in the CSV or tab-delimited format. These operators need the result to be contained in an array. For your case also to have a single space delimit, we can do a simple join(" ") operation
jq -r '[.Accounts[].Id]|join(" ")'
You can use the #sh formatter:
jq -r ".Accounts[].Id | #sh"
From the jq docs:
The input is escaped suitable for use in a command-line for a POSIX shell. If the input is an array, the output will be a series of space-separated strings.
Reference:
https://stedolan.github.io/jq/manual/#Basicfilters
At first I thought the join() solution above did not work. Then I realized that I was "overfeeding" the join() filter, causing it to fail because I was providing more than a simple array as input. I had concatenated several filters with , and failed to limit the scope of my join().
Did not work:
jq -r \
'.ansible_facts |
.ansible_hostname,
.ansible_all_ipv4_addresses | join(" "),
.ansible_local."aws-public-ipv4".publicIP'
This gave the error,
jq: error (at <stdin>:0): Cannot iterate over string ("hostone")
because jq was attempting to "consume" not only ansible_all_ipv4_addresses but also the output of the preceding ansible_hostname filter (I am not certain why this is or whether it was even intended by the author of jq).
Does work:
jq -r \
'.ansible_facts |
.ansible_hostname,
(.ansible_all_ipv4_addresses | join(" ")),
.ansible_local."aws-public-ipv4".publicIP'
Here, I restrict join() to .ansible_all_ipv4_addresses only (ansible_all_ipv4_addresses is an array of IP addresses I wish to translate into a single, space-separated string).
P.S.: I found that the #sh filter produces space-separated output as desired, but in addition delimits each output item in single quotes.
P.P.S.:
Here was my workaround, until I discovered that join() works just the same as it when used properly (see above):
jq -r '.Accounts[].Id | #tsv | sub("\t";" ";"g")'
Explanation: the #tsv filter produces Tab Separated Values, then the sub() filter substitutes tabs with spaces, globally.
If you'd like jq to escape dashes, you need to put your key between square brackets like this;
jq '.["key-key"]'
and apart from that, if you'd like to include a variable in jq, you need to use double quotes instead of single quotes;
jq "."${var[i+1]}""
but my variable contains dash in it and in this case, I've tried to merge the 2 examples above but it didn't work;
var=key-key
jq ".["${var[i+1]}"]."key""
how can I get this work?
Update:
This is the final script, which I've forgot to mention;
declare -a var=(
"key-key"
"key2-key2"
"key3-key3"
)
for ((i=0; i<${#var[#]})); do
curl -s "url" | jq ".["${var}"]."something""
done
To have double-quotes in a jq command you've enclosed in double-quotes, you'd escape them with a backslash :
jq ".[\"key-key\"]"
Another problem with your final command is that ${var[i+1]} expands to the empty string, because this syntax is used to index elements of an array, and you previously defined var as a simple string.
A better way to work with variables in jq commands is to define them through the --arg name value option, after which you can refer to them with $foo in a single-quotes enclosed command :
jq --arg keyName key-key '.[$keyName]'
To fix the code included in the update, I would use the following :
declare -a var=(
"key-key"
"key2-key2"
"key3-key3"
)
json=$(curl -s "url")
for searchedKey in "${var[#]}"; do
echo $json | jq --arg keyName $searchedKey '.[$keyName].something'
done
I have a file which contains 1 line like below
VINOTH |KARTHICK |RAVI
I'm using the below command to remove the leading and trailing spaces , but it's not not working.
awk '{ gsub(/^[ \t]+|[ \t]+$/, ""); print }' Input_File
Please help.
Required Output.
VINOTH|KARTHICK|RAVI
You may use
sed 's/[ \t]*|[ \t]*/|/g;s/^[ \t]*\|[ \t]*$//g' Input_File
There are two regexps here:
s/[ \t]*|[ \t]*/|/g replaces all | enclosed with optional whitespaces with a single | (the | in the regex matches a literal | char as per BRE POSIX standard)
s/^[ \t]*\|[ \t]*$//g removes all whitespaces at the start and end of lines. Note that \| here is an OR operator (escaped because the BRE POSIX syntax is used).
See the online demo.
Could you please try following(since your sample input and expected output are not clear so didn't test it).
awk '{gsub(/^[[:space:]]+|[[:space:]]+$/,"")} 1' Input_file
I am trying to use the following command to extract a substring from a string :
propertyPath=$(echo $path|awk '{print substr($0,3,$index)}')
However the command does not pick up the value of the $index variable and therefore does not return a valid substring.
This would work:
propertyPath=$(echo $path|awk '{print substr($0,3,'$index')}')
or
propertyPath=$(echo $path|awk -v index=$index '{print substr($0,3,index)}')
propertyPath=$(echo $path|awk "{print substr(\$0,1,$index)}")
Single quotes tell the shell not to expand values, so you'll have to use double quotes around your awk command. When you do that though, you'll have to escape the $ for the variables you do not want to expand early.
I have a log file (application.log) which might contain the following string of normal & special characters on multiple lines:
*^%Q&$*&^#$&*!^#$*&^&^*&^&
I want to search for the line number(s) which contains this special character string.
grep '*^%Q&$*&^#$&*!^#$*&^&^*&^&' application.log
The above command doesn't return any results.
What would be the correct syntax to get the line numbers?
Tell grep to treat your input as fixed string using -F option.
grep -F '*^%Q&$*&^#$&*!^#$*&^&^*&^&' application.log
Option -n is required to get the line number,
grep -Fn '*^%Q&$*&^#$&*!^#$*&^&^*&^&' application.log
The one that worked for me is:
grep -e '->'
The -e means that the next argument is the pattern, and won't be interpreted as an argument.
From: http://www.linuxquestions.org/questions/programming-9/how-to-grep-for-string-769460/
A related note
To grep for carriage return, namely the \r character, or 0x0d, we can do this:
grep -F $'\r' application.log
Alternatively, use printf, or echo, for POSIX compatibility
grep -F "$(printf '\r')" application.log
And we can use hexdump, or less to see the result:
$ printf "a\rb" | grep -F $'\r' | hexdump -c
0000000 a \r b \n
Regarding the use of $'\r' and other supported characters, see Bash Manual > ANSI-C Quoting:
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the ANSI C standard
grep -n "\*\^\%\Q\&\$\&\^\#\$\&\!\^\#\$\&\^\&\^\&\^\&" test.log
1:*^%Q&$&^#$&!^#$&^&^&^&
8:*^%Q&$&^#$&!^#$&^&^&^&
14:*^%Q&$&^#$&!^#$&^&^&^&
You could try removing any alphanumeric characters and space. And then use -n will give you the line number. Try following:
grep -vn "^[a-zA-Z0-9 ]*$" application.log
Try vi with the -b option, this will show special end of line characters
(I typically use it to see windows line endings in a txt file on a unix OS)
But if you want a scripted solution obviously vi wont work so you can try the -f or -e options with grep and pipe the result into sed or awk.
From grep man page:
Matcher Selection
-E, --extended-regexp
Interpret PATTERN as an extended regular expression (ERE, see below). (-E is specified by POSIX.)
-F, --fixed-strings
Interpret PATTERN as a list of fixed strings, separated by newlines, any of which is to be matched. (-F is specified
by POSIX.)