I found the following bash script in order to monitor cp progress.
#!/bin/sh
cp_p()
{
strace -q -ewrite cp -- "${1}" "${2}" 2>&1 \
| awk '{
count += $NF
if (count % 10 == 0) {
percent = count / total_size * 100
printf "%3d%% [", percent
for (i=0;i<=percent;i++)
printf "="
printf ">"
for (i=percent;i<100;i++)
printf " "
printf "]\r"
}
}
END { print "" }' total_size=$(stat -c '%s' "${1}") count=0
}
I don't understand the "-ewrite" option for the strace command. The closest thing I've found is the man page for strace which is
-e write=set Perform a full hexadecimal and ASCII dump of all the
data written to file descriptors
listed in the specified set. For
example, to see all output activity on
file descriptors 3 and 5 use -e
write=3,5. Note that this is
independent from the normal tracing of
the write(2) system call which is
controlled by the option -e
trace=write.
However I don't understand what the -ewrite option does.
-ewrite means that only the "write" system call will be traced.
-e expr A qualifying expression which modifies which events
to trace or how to trace them. The format of the
expression is:
[qualifier=][!]value1[,value2]...
where qualifier is one of trace, abbrev, verbose,
raw, signal, read, or write and value is a quali-
fier-dependent symbol or number. The default qual-
ifier is trace. Using an exclamation mark negates
the set of values. For example, -eopen means lit-
erally -e trace=open which in turn means trace only
the open system call. By contrast, -etrace=!open
means to trace every system call except open. In
addition, the special values all and none have the
obvious meanings.
Note that some shells use the exclamation point for
history expansion even inside quoted arguments. If
so, you must escape the exclamation point with a
backslash.
Related
Bash's builtin read command has a -i option, which specifies an initial input, which the user can accept as it is or edit or add to. I cannot find anything similar for Zsh's read command.
None of the options listed in the zshbuiltins man page seem relevant:
read [ -rszpqAclneE ] [ -t [ num ] ] [ -k [ num ] ] [ -d delim ]
[ -u n ] [ name[?prompt] ] [ name ... ]
Read one line and break it into fields using the characters in $IFS as separators, except as noted below. The first field is assigned to the first name, the second field to
the second name, etc., with leftover fields assigned to the last name. If name is omitted then REPLY is used for scalars and reply for arrays.
-r Raw mode: a `\' at the end of a line does not signify line continuation and backslashes in the line don't quote the following character and are not removed.
-s Don't echo back characters if reading from the terminal.
-q Read only one character from the terminal and set name to `y' if this character was `y' or `Y' and to `n' otherwise. With this flag set the return status is zero only
if the character was `y' or `Y'. This option may be used with a timeout (see -t); if the read times out, or encounters end of file, status 2 is returned. Input is
read from the terminal unless one of -u or -p is present. This option may also be used within zle widgets.
-k [ num ]
Read only one (or num) characters. All are assigned to the first name, without word splitting. This flag is ignored when -q is present. Input is read from the ter-
minal unless one of -u or -p is present. This option may also be used within zle widgets.
Note that despite the mnemonic `key' this option does read full characters, which may consist of multiple bytes if the option MULTIBYTE is set.
-z Read one entry from the editor buffer stack and assign it to the first name, without word splitting. Text is pushed onto the stack with `print -z' or with push-line
from the line editor (see zshzle(1)). This flag is ignored when the -k or -q flags are present.
-e
-E The input read is printed (echoed) to the standard output. If the -e flag is used, no input is assigned to the parameters.
-A The first name is taken as the name of an array and all words are assigned to it.
-c
-l These flags are allowed only if called inside a function used for completion (specified with the -K flag to compctl). If the -c flag is given, the words of the cur-
rent command are read. If the -l flag is given, the whole line is assigned as a scalar. If both flags are present, -l is used and -c is ignored.
-n Together with -c, the number of the word the cursor is on is read. With -l, the index of the character the cursor is on is read. Note that the command name is word
number 1, not word 0, and that when the cursor is at the end of the line, its character index is the length of the line plus one.
-u n Input is read from file descriptor n.
-p Input is read from the coprocess.
-d delim
Input is terminated by the first character of delim instead of by newline.
-t [ num ]
Test if input is available before attempting to read. If num is present, it must begin with a digit and will be evaluated to give a number of seconds, which may be a
floating point number; in this case the read times out if input is not available within this time. If num is not present, it is taken to be zero, so that read returns
immediately if no input is available. If no input is available, return status 1 and do not set any variables.
This option is not available when reading from the editor buffer with -z, when called from within completion with -c or -l, with -q which clears the input queue before
reading, or within zle where other mechanisms should be used to test for input.
Note that read does not attempt to alter the input processing mode. The default mode is canonical input, in which an entire line is read at a time, so usually `read
-t' will not read anything until an entire line has been typed. However, when reading from the terminal with -k input is processed one key at a time; in this case,
only availability of the first character is tested, so that e.g. `read -t -k 2' can still block on the second character. Use two instances of `read -t -k' if this is
not what is wanted.
If the first argument contains a `?', the remainder of this word is used as a prompt on standard error when the shell is interactive.
The value (exit status) of read is 1 when an end-of-file is encountered, or when -c or -l is present and the command is not called from a compctl function, or as described
for -q. Otherwise the value is 0.
The behavior of some combinations of the -k, -p, -q, -u and -z flags is undefined. Presently -q cancels all the others, -p cancels -u, -k cancels -z, and otherwise -z can-
cels both -p and -u.
The -c or -l flags cancel any and all of -kpquz.
How can I achieve the same goal in Zsh?
In zsh you could manually set the variable and use vared instead of read:
name=iconoclast
vared -p "Enter your name: " name
This is the equivalent of the following in bash:
read -p "Enter your name: " -e -i iconoclast name
I am trying to get line count from a file like below in a ksh script. But it returns nothing :
filerecordcount= $((`wc -l <../data/act.dat`))
I also tried these :
filerecordcount= `wc -l <../data/act.dat`
filerecordcount= $(wc -l <../data/act.dat)
When i print the variable its not printing the value in the variable.
print "Record Count in .dat file : $filerecordcount." 1>&2;
But when i try the same from command prompt it returns the count
wc -l<../data/act.dat
You don't have to give the space after the = in assigning part. Use the below one. It will work fine. But don't forget to print the variable filerecordcount.
UPDATE:
filerecordcount=$((`wc -l <../data/act.dat`))
Simplify simplify. Your backquotes are doing the command expansion, and their output is being processed by $((...)) as an arithmetic expression. It's a little redundant.
filerecordcount=$(wc -l < ../data/act.dat)
No space after the =, and just one level of command expansion.
Alternately you can use process substitution:
read filerecordcount < <(wc -l < ../data/act.dat)
Or you could even do this without a subshell, using a loop:
filerecordcount=0
while read junk; do ((filerecordcount++)); done < ../data/act.dat
What is the meaning for $! in shell or shell scripting? I am trying to understand a script which has the something like the following.
local#usr> a=1
local#usr> echo $a
1
local#usr> echo $!a
a
It is printing the variable back. Is it all for that? What are the other $x options we have? Few I know are $$, $*, $?. If anyone can point me to a good source, it will be helpful. BTW, This is in Sun OS 5.8, KSH.
The various $… variables are described in Bash manual. According to the manual $! expands to the PID of the last process launched in background. See:
$ echo "Foo"
Foo
$ echo $!
$ true&
[1] 67064
$ echo $!
67064
[1]+ Done true
In ksh it seems to do the same.
From the ksh man page on my system:
${!vname}
Expands to the name of the variable referred to by vname. This
will be vname except when vname is a name reference.
For the shell you are asking, ksh, use the the ksh manual, and read this:
Parameter Substitution
A parameter is an identifier, one or more digits, or any of
the characters *, #, #, ?, -, $, and !.
It is clear that those are the accepted options $*, $#, $#, $?, $-, $$, and $!.
More could be included in the future.
For the parameter $!, from the manual:
"!" The process number of the last background command invoked.
if you start a background process, like sleep 60 &, then there will be a process number for such process, and the parameter $! will print its number.
$ sleep 60 &
[1] 12329
$ echo "$!"
12329
If there is no background process in execution (as when the shell starts), the exansion is empty. It has a null value.
$ ksh -c 'echo $!'
If there is a background process, it will expand to the PID of such process:
$ ksh -c 'sleep 30 & echo $!'
42586
That is why echo $!a expanded to a. It is because there is no PID to report:
$ ksh -c 'echo $!a'
a
Other shells may have a different (usually pretty similar) list of expansions (a parameter with only one $ and one next character).
For example, bash recognize this *##?-$!0_ as "Special parameters". Search the Bash manual for the heading "3.4.2 Special Parameters".
Special Parameters
The shell treats several parameters specially.
It gives the Process id of last backgroundjob or background function
Please go through this link below
http://www.well.ox.ac.uk/~johnb/comp/unix/ksh.html#specvar
! is a reference operator in unix, though it is not called with that name.
It always refers to a mother process. Try typing :! in vi, it takes you to command prompt and you can execute commands as usual until exit command.
! in SQLPLUS also executes the command from the command prompt. try this in sqlplus
SQL> !ls --- this gives the list of files inthe current dir.
$! - obviously gives the process id of the current/latest process.
Is there an alternative to tee which captures standard output and standard error of the command being executed and exits with the same exit status as the processed command?
Something like the following:
eet -a some.log -- mycommand --foo --bar
Where "eet" is an imaginary alternative to "tee" :) (-a means append and -- separates the captured command). It shouldn't be hard to hack such a command, but maybe it already exists and I'm not aware of it?
This works with Bash:
(
set -o pipefail
mycommand --foo --bar | tee some.log
)
The parentheses are there to limit the effect of pipefail to just the one command.
From the bash(1) man page:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
I stumbled upon a couple of interesting solutions at Capture Exit Code Using Pipe & Tee.
There is the $PIPESTATUS variable available in Bash:
false | tee /dev/null
[ $PIPESTATUS -eq 0 ] || exit $PIPESTATUS
And the simplest prototype of "eet" in Perl may look as follows:
open MAKE, "command 2>&1 |" or die;
open (LOGFILE, ">>some.log") or die;
while (<MAKE>) {
print LOGFILE $_;
print
}
close MAKE; # To get $?
my $exit = $? >> 8;
close LOGFILE;
Here's an eet. Works with every Bash I can get my hands on, from 2.05b to 4.0.
#!/bin/bash
tee_args=()
while [[ $# > 0 && $1 != -- ]]; do
tee_args=("${tee_args[#]}" "$1")
shift
done
shift
# now ${tee_args[*]} has the arguments before --,
# and $* has the arguments after --
# redirect standard out through a pipe to tee
exec | tee "${tee_args[#]}"
# do the *real* exec of the desired program
exec "$#"
(pipefail and $PIPESTATUS are nice, but I recall them being introduced in 3.1 or thereabouts.)
This is what I consider to be the best pure-Bourne-shell solution to use as the base upon which you could build your "eet":
# You want to pipe command1 through command2:
exec 4>&1
exitstatus=`{ { command1; echo $? 1>&3; } | command2 1>&4; } 3>&1`
# $exitstatus now has command1's exit status.
I think this is best explained from the inside out – command1 will execute and print its regular output on stdout (file descriptor 1), then once it's done, echo will execute and print command1's exit code on its stdout, but that stdout is redirected to file descriptor three.
While command1 is running, its stdout is being piped to command2 (echo's output never makes it to command2 because we send it to file descriptor 3 instead of 1, which is what the pipe reads). Then we redirect command2's output to file descriptor 4, so that it also stays out of file descriptor one – because we want file descriptor one clear for when we bring the echo output on file descriptor three back down into file descriptor one so that the command substitution (the backticks) can capture it.
The final bit of magic is that first exec 4>&1 we did as a separate command – it opens file descriptor four as a copy of the external shell's stdout. Command substitution will capture whatever is written on standard out from the perspective of the commands inside it – but, since command2's output is going to file descriptor four as far as the command substitution is concerned, the command substitution doesn't capture it – however, once it gets "out" of the command substitution, it is effectively still going to the script's overall file descriptor one.
(The exec 4>&1 has to be a separate command to work with many common shells. In some shells it works if you just put it on the same line as the variable assignment, after the closing backtick of the substitution.)
(I use compound commands ({ ... }) in my example, but subshells (( ... )) would also work. The subshell will just cause a redundant forking and awaiting of a child process, since each side of a pipe and the inside of a command substitution already normally implies a fork and await of a child process, and I don't know of any shell being coded to recognize that it can skip one of those forks because it's already done or is about to do the other.)
You can look at it in a less technical and more playful way, as if the outputs of the commands are leapfrogging each other: command1 pipes to command2, then the echo's output jumps over command2 so that command2 doesn't catch it, and then command2's output jumps over and out of the command substitution just as echo lands just in time to get captured by the substitution so that it ends up in the variable, and command2's output goes on its way to the standard output, just as in a normal pipe.
Also, as I understand it, at the end of this command, $? will still contain the return code of the second command in the pipe, because variable assignments, command substitutions, and compound commands are all effectively transparent to the return code of the command inside them, so the return status of command2 should get propagated out.
A caveat is that it is possible that command1 will at some point end up using file descriptors three or four, or that command2 or any of the later commands will use file descriptor four, so to be more hygienic, we would do:
exec 4>&1
exitstatus=`{ { command1 3>&-; echo $? 1>&3; } 4>&- | command2 1>&4; } 3>&1`
exec 4>&-
Commands inherit file descriptors from the process that launches them, so the entire second line will inherit file descriptor four, and the compound command followed by 3>&1 will inherit the file descriptor three. So the 4>&- makes sure that the inner compound command will not inherit file descriptor four, and the 3>&- makes sure that command1 will not inherit file descriptor three, so command1 gets a 'cleaner', more standard environment. You could also move the inner 4>&- next to the 3>&-, but I figure why not just limit its scope as much as possible.
Almost no programs uses pre-opened file descriptor three and four directly, so you almost never have to worry about it, but the latter is probably best to keep in mind and use for general-purpose cases.
{ mycommand --foo --bar 2>&1; ret=$?; } | tee -a some.log; (exit $ret)
KornShell, all in one line:
foo; RET_VAL=$?; if test ${RET_VAL} != 0;then echo $RET_VAL; echo Error occurred!>/tmp/out.err;exit 2;fi |tee >>/tmp/out.err ; if test ${RET_VAL} != 0;then exit $RET_VAL;fi
#!/bin/sh
logfile="$1"
shift
exec 2>&1
exec "$#" | tee "$logfile"
Hopefully this works for you.
Assuming Bash or Z shell (zsh),
my_command >>my_log 2>&1
N.B. The sequence of redirection and duplication of standard error onto standard output is significant!
I didn't realise you wanted to see the output on screen as well. This will of course direct all output to the file my_log.
I want my ksh script to have different behaviors depending on whether there is something incoming through stdin or not:
(1) cat file.txt | ./script.ksh (then do "cat <&0 >./tmp.dat" and process tmp.dat)
vs. (2) ./script.ksh (then process $1 which must be a readable regular file)
Checking for stdin to see if it is a terminal[ -t 0 ] is not helpful, because my script is called from an other script.
Doing "cat <&0 >./tmp.dat" to check tmp.dat's size hangs up waiting for an EOF from stdin if stdin is "empty" (2nd case).
How to just check if stdin is "empty" or not?!
EDIT: You are running on HP-UX
Tested [ -t 0 ] on HP-UX and it appears to be working for me. I have used the following setup:
/tmp/x.ksh:
#!/bin/ksh
/tmp/y.ksh
/tmp/y.ksh:
#!/bin/ksh
test -t 0 && echo "terminal!"
Running /tmp/x.ksh prints: terminal!
Could you confirm the above on your platform, and/or provide an alternate test setup more closely reflecting your situation? Is your script ultimately spawned by cron?
EDIT 2
If desperate, and if Perl is available, define:
stdin_ready() {
TIMEOUT=$1; shift
perl -e '
my $rin = "";
vec($rin,fileno(STDIN),1) = 1;
select($rout=$rin, undef, undef, '$TIMEOUT') < 1 && exit 1;
'
}
stdin_ready 1 || 'stdin not ready in 1 second, assuming terminal'
EDIT 3
Please note that the timeout may need to be significant if your input comes from sort, ssh etc. (all these programs can spawn and establish the pipe with your script seconds or minutes before producing any data over it.) Also, using a hefty timeout may dramatically penalize your script when there is nothing on the input to begin with (e.g. terminal.)
If potentially large timeouts are a problem, and if you can influence the way in which your script is called, then you may want to force the callers to explicitly instruct your program whether stdin should be used, via a custom option or in the standard GNU or tar manner (e.g. script [options [--]] FILE ..., where FILE can be a file name, a - to denote standard input, or a combination thereof, and your script would only read from standard input if - were passed in as a parameter.)
This strategy works for bash, and would likely work for ksh. Poll 'tty':
#!/bin/bash
set -a
if [ "$( tty )" == 'not a tty' ]
then
STDIN_DATA_PRESENT=1
else
STDIN_DATA_PRESENT=0
fi
if [ ${STDIN_DATA_PRESENT} -eq 1 ]
then
echo "Input was found."
else
echo "Input was not found."
fi
Why not solve this in a more traditional way, and use the command line argument to indicate that the data will be coming from stdin?
For an example, consider the difference between:
echo foo | cat -
and
echo foo > /tmp/test.txt
cat /tmp/test.txt