Loop over environment variables in POSIX sh - unix

I need to loop over environment variables and get their names and values in POSIX sh (not bash). This is what I have so far.
#!/usr/bin/env sh
# Loop over each line from the env command
while read -r line; do
# Get the string before = (the var name)
name="${line%=*}"
eval value="\$$name"
echo "name: ${name}, value: ${value}"
done <<EOF
$(env)
EOF
It works most of the time, except when an environment variable contains a newline. I need it to work in that case.
I am aware of the -0 flag for env that separates variables with nul instead of newlines, but if I use that flag, how do I loop over each variable? Edit: #chepner pointed out that POSIX env doesn't support -0, so that's out.
Any solution that uses portable linux utilities is good as long as it works in POSIX sh.

There is no way to parse the output of env with complete confidence; consider this output:
bar=3
baz=9
I can produce that with two different environments:
$ env -i "bar=3" "baz=9"
bar=3
baz=9
$ env -i "bar=3
> baz=9"
bar=3
baz=9
Is that two environment variables, bar and baz, with simple numeric values, or is it one variable bar with the value $'3\nbaz=9' (to use bash's ANSI quoting style)?
You can safely access the environment with POSIX awk, however, using the ENVIRON array. For example:
awk 'END { for (name in ENVIRON) {
print "Name is "name;
print "Value is "ENVIRON[name];
}
}' < /dev/null
With this command, you can distinguish between the two environments mentioned above.
$ env -i "bar=3" "baz=9" awk 'END { for (name in ENVIRON) { print "Name is "name; print "Value is "ENVIRON[name]; }}' < /dev/null
Name is baz
Value is 9
Name is bar
Value is 3
$ env -i "bar=3
> baz=9" awk 'END { for (name in ENVIRON) { print "Name is "name; print "Value is "ENVIRON[name]; }}' < /dev/null
Name is bar
Value is 3
baz=9

Maybe this would work?
#!/usr/bin/env sh
env | while IFS= read -r line
do
name="${line%%=*}"
indirect_presence="$(eval echo "\${$name+x}")"
[ -z "$name" ] || [ -z "$indirect_presence" ] || echo "name:$name, value:$(eval echo "\$$name")"
done
It is not bullet-proof, as if the value of a variable with a newline happens to have a line beginning that looks like an assignment, it could be somewhat confused.
The expansion uses %% to remove the longest match, so if a line contains several = signs, they should all be removed to leave only the variable name from the beginning of the line.

Here an example based on the awk approach:
#!/bin/sh
for NAME in $(awk "END { for (name in ENVIRON) { print name; }}" < /dev/null)
do
VAL="$(awk "END { printf ENVIRON[\"$NAME\"]; }" < /dev/null)"
echo "$NAME=$VAL"
done

Related

Unable to use -C of grep in Unix Shell Script

I am able to use grep in normal command line.
grep "ABC" Filename -C4
This is giving me the desired output which is 4 lines above and below the matched pattern line.
But if I use the same command in a Unix shell script, I am unable to grep the lines above and below the pattern. It is giving me output as the only lines where pattern is matched and an error in the end that cannot says cannot open grep : -C4
The results are similar if I use -A4 and -B4
I'll assume you need a portable POSIX solution without the GNU extensions (-C NUM, -A NUM, and -B NUM are all GNU, as are arguments following the pattern and/or file name).
POSIX grep can't do this, but POSIX awk can. This can be invoked as e.g. grepC -C4 "ABC" Filename (assuming it is named "grepC", is executable, and is in your $PATH):
#!/bin/sh
die() { echo "$*\nUsage: $0 [-C NUMBER] PATTERN [FILE]..." >&2; exit 2; }
CONTEXT=0 # default value
case $1 in
-C ) CONTEXT="$2"; shift 2 ;; # extract "4" from "-C 4"
-C* ) CONTEXT="${1#-C}"; shift ;; # extract "4" from "-C4"
--|-) shift ;; # no args or use std input (implicit)
-* ) [ -f "$1" ] || die "Illegal option '$1'" ;; # non-option non-file
esac
[ "$CONTEXT" -ge 0 ] 2>/dev/null || die "Invalid context '$CONTEXT'"
[ "$#" = 0 ] && die "Missing PATTERN"
PATTERN="$1"
shift
awk '
/'"$PATTERN"'/ {
match='$CONTEXT'
for(i=1; i<=CONTEXT; i++) if(NR>i) print last[i];
print
next
}
match { print; match-- }
{ for(i='$CONTEXT'; i>1; i--) last[i] = last[i-1]; last[1] = $0 }
' "$#"
This sets up die as a fatal error function, then finds the desired lines of context from your arguments (either -C NUMBER or -CNUMBER), with an error for unsupported options (unless they're files).
If the context is not a number or there is no pattern, we again fatally error out.
Otherwise, we save the pattern, shift it away, and reserve the rest of the options for handing to awk as files ("$#").
There are three stanzas in this awk call:
Match the pattern itself. This requires ending the single-quote portion of the string in order to incorporate the $PATTERN variable (which may not behave correctly if imported via awk -v). Upon that match, we store the number of lines of context into the match variable, loop through the previous lines saved in the last hash (if we've gone far enough to have had them), and print them. We then skip to the next line without evaluating the other two stanzas.
If there was a match, we need the next few lines for context. As this stanza prints them, it decrements the counter. A new match (previous stanza) will reset that count.
We need to save previous lines for recalling upon a match. This loops through the number of lines of context we care about and stores them in the last hash. The current line ($0) is stored in last[1].

How can I set a default value when incorrect/invalid input is entered in Unix?

i want to set the value of inputLineNumber to 20. I tried checking if no value is given by user by [[-z "$inputLineNumber"]] and then setting the value by inputLineNumber=20. The code gives this message ./t.sh: [-z: not found as message on the console. How to resolve this? Here's my full script as well.
#!/bin/sh
cat /dev/null>copy.txt
echo "Please enter the sentence you want to search:"
read "inputVar"
echo "Please enter the name of the file in which you want to search:"
read "inputFileName"
echo "Please enter the number of lines you want to copy:"
read "inputLineNumber"
[[-z "$inputLineNumber"]] || inputLineNumber=20
for N in `grep -n $inputVar $inputFileName | cut -d ":" -f1`
do
LIMIT=`expr $N + $inputLineNumber`
sed -n $N,${LIMIT}p $inputFileName >> copy.txt
echo "-----------------------" >> copy.txt
done
cat copy.txt
Changed the script after suggestion from #Kevin. Now the error message ./t.sh: syntax error at line 11: `$' unexpected
#!/bin/sh
truncate copy.txt
echo "Please enter the sentence you want to search:"
read inputVar
echo "Please enter the name of the file in which you want to search:"
read inputFileName
echo Please enter the number of lines you want to copy:
read inputLineNumber
[ -z "$inputLineNumber" ] || inputLineNumber=20
for N in $(grep -n $inputVar $inputFileName | cut -d ":" -f1)
do
LIMIT=$((N+inputLineNumber))
sed -n $N,${LIMIT}p $inputFileName >> copy.txt
echo "-----------------------" >> copy.txt
done
cat copy.txt
Try changing this line from:
[[-z "$inputLineNumber"]] || inputLineNumber=20
To this:
if [[ -z "$inputLineNumber" ]]; then
inputLineNumber=20
fi
Hope this helps.
Where to start...
You are running as /bin/sh but trying to use [[. [[ is a bash command that sh does not recognize. Either change the shebang to /bin/bash (preferred) or use [ instead.
You do not have a space between [[-z. That causes bash to read it as a command named [[-z, which clearly doesn't exist. You need [[ -z $inputLineNumber ]] (note the space at the end too). Quoting within [[ doesn't matter, but if you change to [ (see above), you will need to keep the quotes.
Your code says [[-z but your error says [-z. Pick one.
Use $(...) instead of `...`. The backticks are deprecated, and $() handles quoting appropriately.
You don't need to cat /dev/null >copy.txt, certainly not twice without writing to it in-between. Use truncate copy.txt or just plain >copy.txt.
You seem to have inconsistent quoting. Quote or escape (\x) anything with special characters (~, `, !, #, $, &, *, ^, (), [], \, <, >, ?, ', ", ;) or whitespace and any variable that could have whitespace. You don't need to quote string literals with no special characters (e.g. ":").
Instead of LIMIT=`expr...`, use limit=$((N+inputLineNumber)).

How to quote strings in file names in zsh (passing back to other scripts)

I have a script that has a string in a file name like so:
filename_with_spaces="a file with spaces"
echo test > "$filename_with_spaces"
test_expect_success "test1: filename with spaces" "
run cat \"$filename_with_spaces\"
run grep test \"$filename_with_spaces\"
"
test_expect_success is defined as:
test_expect_success () {
echo "expecting success: $1"
eval "$2"
}
and run is defined as:
#!/bin/zsh
# make nice filename removing special characters, replace space with _
filename=`echo $# | tr ' ' _ | tr -cd 'a-zA-Z0-9_.'`.run
echo "#!/bin/zsh" > $filename
print "$#" >> $filename
chmod +x $filename
./$filename
But when I run the toplevel script test_expect_success... I get cat_a_file_with_spaces.run with:
#!/bin/zsh
cat a file with spaces
The problem is the quotes around a file with spaces in cat_a_file_with_spaces.run is missing. How do you get Z shell to keep the correct quoting?
Thanks
Try
run cat ${(q)filename_with_spaces}
. It is what (q) modifier was written for. Same for run script:
echo -E ${(q)#} >> $filename
. And it is not bash, you don't need to put quotes around variables: unless you specify some option (don't remember which exactly)
command $var
always passes exactly one argument to command no matter what is in $var. To ensure that some zsh option will not alter the behavior, put
emulate -L zsh
at the top of every script.
Note that initial variant (run cat \"$filename_with_spaces\") is not a correct quoting: filename may contain any character except NULL and / used for separating directories. ${(q)} takes care about it.
Update: I would have written test_expect_success function in the following fashion:
function test_expect_success()
{
emulate -L zsh
echo "Expecting success: $1" ; shift
$#
}
Usage:
test_expect_success "Message" run cat $filename_with_spaces

How to custom display prompt in KornShell to show hostname and current directory?

I am using KornShell (ksh) on Solaris and currently my PS1 env var is:
PS1="${HOSTNAME}:\${PWD} \$ "
And the prompt displays: hostname:/full/path/to/current/directory $
However, I would like it to display: hostname:directory $
In other words, how can I display just the hostname and the name of the current directory, i.e. tmp or ~ or public_html etc etc?
From reading the ksh man page you want
PS1="${HOSTNAME}:\${PWD##*/} \$ "
Tested on default ksh on SunOS 5.8
Okay, a little old and a little late, but this is what I use in Kornshell:
PS1='$(print -n "`logname`#`hostname`:";if [[ "${PWD#$HOME}" != "$PWD" ]] then; print -n "~${PWD#$HOME}"; else; print -n "$PWD";fi;print "\n$ ")'
This makes a prompt that's equivalent to PS1="\u#\h:\w\n$ " in BASH.
For example:
qazwart#mybook:~
$ cd bin
qazwart#mybook:~/bin
$ cd /usr/local/bin
qazwart#mybook:/usr/local/bin
$
I like a two line prompt because I sometimes have very long directory names, and they can take up a lot of the command line. If you want a one line prompt, just leave off the "\n" on the last print statement:
PS1='$(print -n "`logname`#`hostname`:";if [[ "${PWD#$HOME}" != "$PWD" ]] then; print -n "~${PWD#$HOME}"; else; print -n "$PWD";fi;print "$ ")'
That's equivalent to PS1="\u#\h:\w$ " in BASH:
qazwart#mybook:~$ cd bin
qazwart#mybook:~/bin$ cd /usr/local/bin
qazwart#mybook:/usr/local/bin$
It's not quite as easy as setting up a BASH prompt, but you get the idea. Simply write a script for PS1 and Kornshell will execute it.
For Solaris and other Older Versions of Kornshell
I found that the above does not work on Solaris. Instead, you'll have to do it the real hackish way...
In your .profile, make sure that ENV="$HOME/.kshrc"; export ENV
is set. This is probably setup correctly for you.
In your .kshrc file, you'll be doing two things
You'll be defining a function called _cd. This function will change to the directory specified, and then set your PS1 variable based upon your pwd.
You'll be setting up an alias cd to run the _cd function.
This is the relevant part of the .kshrc file:
function _cd {
logname=$(logname) #Or however you can set the login name
machine=$(hostname) #Or however you set your host name
$directory = $1
$pattern = $2 #For "cd foo bar"
#
# First cd to the directory
# We can use "\cd" to evoke the non-alias original version of the cd command
#
if [ "$pattern" ]
then
\cd "$directory" "$pattern"
elif [ "$directory" ]
then
\cd "$directory"
else
\cd
fi
#
# Now that we're in the directory, let's set our prompt
#
$directory=$PWD
shortname=${directory#$HOME} #Possible Subdir of $HOME
if [ "$shortName" = "" ] #This is the HOME directory
then
prompt="~$logname" # Or maybe just "~". Your choice
elif [ "$shortName" = "$directory" ] #Not a subdir of $HOME
then
prompt="$directory"
else
prompt="~$shortName"
fi
PS1="$logname#$hostname:$prompt$ " #You put it together the way you like
}
alias cd="_cd"
This will set your prompt as the equivelent BASH PS1="\u#\h:\w$ ". It isn't pretty, but it works.
ENV=~/.kshrc, and then in your .kshrc:
function _cd {
\cd "$#"
PS1=$(
print -n "$LOGNAME#$HOSTNAME:"
if [[ "${PWD#$HOME}" != "$PWD" ]]; then
print -n "~${PWD#$HOME}"
else
print -n "$PWD"
fi
print "$ "
)
}
alias cd=_cd
cd "$PWD"
Brad
HOST=`hostname`
PS1='$(print -n "[${USER}#${HOST%%.*} ";[[ "$HOME" == "$PWD" ]] && print -n "~" ||([[ "${PWD##*/}" == "" ]] && print -n "/" || print -n "${PWD##*/}");print "]$")'
PS1=`id -un`#`hostname -s`:'$PWD'$
and...
if you work between two shells for most of your effort [ksh and bourne sh]
and desire a directory tracking display on your command line
then PWD can be substituted easily in ksh
and if you use /usr/xpg4/bin/sh for your sh SHELL, it will work there as well
Try this:
PS1="\H:\W"
More information on: How to: Change / Setup bash custom prompt, I know you said ksh, but I am pretty sure it will work.

Quoting command-line arguments in shell scripts

The following shell script takes a list of arguments, turns Unix paths into WINE/Windows paths and invokes the given executable under WINE.
#! /bin/sh
if [ "${1+set}" != "set" ]
then
echo "Usage; winewrap EXEC [ARGS...]"
exit 1
fi
EXEC="$1"
shift
ARGS=""
for p in "$#";
do
if [ -e "$p" ]
then
p=$(winepath -w $p)
fi
ARGS="$ARGS '$p'"
done
CMD="wine '$EXEC' $ARGS"
echo $CMD
$CMD
However, there's something wrong with the quotation of command-line arguments.
$ winewrap '/home/chris/.wine/drive_c/Program Files/Microsoft Research/Z3-1.3.6/bin/z3.exe' -smt /tmp/smtlib3cee8b.smt
Executing: wine '/home/chris/.wine/drive_c/Program Files/Microsoft Research/Z3-1.3.6/bin/z3.exe' '-smt' 'Z: mp\smtlib3cee8b.smt'
wine: cannot find ''/home/chris/.wine/drive_c/Program'
Note that:
The path to the executable is being chopped off at the first space, even though it is single-quoted.
The literal "\t" in the last path is being transformed into a tab character.
Obviously, the quotations aren't being parsed the way I intended by the shell. How can I avoid these errors?
EDIT: The "\t" is being expanded through two levels of indirection: first, "$p" (and/or "$ARGS") is being expanded into Z:\tmp\smtlib3cee8b.smt; then, \t is being expanded into the tab character. This is (seemingly) equivalent to
Y='y\ty'
Z="z${Y}z"
echo $Z
which yields
zy\tyz
and not
zy yz
UPDATE: eval "$CMD" does the trick. The "\t" problem seems to be echo's fault: "If the first operand is -n, or if any of the operands contain a backslash ( '\' ) character, the results are implementation-defined." (POSIX specification of echo)
bash’s arrays are unportable but the only sane way to handle argument lists in shell
The number of arguments is in ${#}
Bad stuff will happen with your script if there are filenames starting with a dash in the current directory
If the last line of your script just runs a program, and there are no traps on exit, you should exec it
With that in mind
#! /bin/bash
# push ARRAY arg1 arg2 ...
# adds arg1, arg2, ... to the end of ARRAY
function push() {
local ARRAY_NAME="${1}"
shift
for ARG in "${#}"; do
eval "${ARRAY_NAME}[\${#${ARRAY_NAME}[#]}]=\${ARG}"
done
}
PROG="$(basename -- "${0}")"
if (( ${#} < 1 )); then
# Error messages should state the program name and go to stderr
echo "${PROG}: Usage: winewrap EXEC [ARGS...]" 1>&2
exit 1
fi
EXEC=("${1}")
shift
for p in "${#}"; do
if [ -e "${p}" ]; then
p="$(winepath -w -- "${p}")"
fi
push EXEC "${p}"
done
exec "${EXEC[#]}"
I you do want to have the assignment to CMD you should use
eval $CMD
instead of just $CMD in the last line of your script. This should solve your problem with spaces in the paths, I don't know what to do about the "\t" problem.
replace the last line from $CMD to just
wine '$EXEC' $ARGS
You'll note that the error is ''/home/chris/.wine/drive_c/Program' and not '/home/chris/.wine/drive_c/Program'
The single quotes are not being interpolated properly, and the string is being split by spaces.
You can try preceeding the spaces with \ like so:
/home/chris/.wine/drive_c/Program Files/Microsoft\ Research/Z3-1.3.6/bin/z3.exe
You can also do the same with your \t problem - replace it with \\t.

Resources