I have a script called cmdtst_multi.ksh and when I run it like this:
. ./cmdtst_multi.ksh
I have a varible that gets populated like this;
treeLvls=$(td_query {})
echo "num of tree lvls:" $treeLvls
##gives back num of tree lvls: 4
treeLvlsSQL=$(for i in {1..$treeLvls}
do
echo ",NULL as LVL$i"
done)
echo "tree sql:" $treeLvlsSQl
#gives back tree sql: ,NULL as LVL1 ,NULL as LVL2 ,NULL as LVL3 ,NULL as LVL4
it runs fine
but when I run it like this:
nohup cmdtst_multi.ksh > prcsstst.log &
the treeLvlsSQl comes out like this in the log file, which is not correct:
tree sql: ,NULL as LVL{1.. ,NULL as LVL4}
why do the variables seem to run differently?
My guess is this difference has something to do with the seemingly innocuous sourcing operator(.) or the current directory operator(./) at the beginning of your script.
nohup - is no hangup to make the process not to hangup when the
user log outs
> prcsstst.log & - redirects to log file and backgrounds the
process
which rules out them as contenders for the difference in output.
Please try the nohup with sourcing operator like below:
nohup ksh93 -c ". ./cmdtst_multi.ksh" > prcsstst.log
. and ./ are important . implies sourcing the current shell which prevents spawning a separate child shell with new set of environment variables and ./ implies the script in . (current directory) is executed instead of some other script with the same name defined in $PATH.
Edit: As Jonathan Leffler points out in his comment, this could also be because KSH not being used by nohup (KSH is not the default shell perhaps?) which might explain why the KSH's FOR syntax worked initially when sourced(.) from current shell(KSH?) but not the second time around(some other type of shell).
Related
I am trying to run the following function
foo () {
sleep 1
echo "outside inotify"
(inotifywait . -e create |
while read path action file; do
echo "test"
sleep 1
done)
echo "end"
}
Until inotifywait it runs correctly; I see:
>> foo
outside inotify
Setting up watches.
Watches established.
However as soon as I create a file, I get
>>> fooo
outside inotify
Setting up watches.
Watches established.
test
foo:6: command not found: sleep
end
Any idea why? Plus do I need to spawn the subprocess ( ) around inotifywait? what are the benefits?
thank you.
Edit
I realized I am running on zsh
The read path is messing you up, because unlike POSIX-compliant shells -- which guarantee that only modification to variables with all-uppercase names can have unwanted side effects on the shell itself -- zsh also has special-cased behavior for several lower-case names, including path.
In particular, zsh presents path as an array corresponding to the values in PATH. Assigning a string to this array will overwrite your PATH as well.
TLDR;
I need a way for .zshrc to automatically be sourced each time a command is executed. PROMPT needs to be updated each time a command is executed in order to show relevant information in the prompt.
Reason
I use Watson cli for tracking time. On my previous bash setup, I prepended my prompt ($PS1) with a symbol that indicates whether the timer is running or not (red/green). I have mimicked this functionality with Oh My Zsh, as follows (in the theme file):
WATSON_DIR="$HOME/Library/Application Support/watson"
watson_status() {
local txtred="${fg_bold[red]}"
local txtgrn="${fg_bold[green]}"
local txtrst="${reset_color}"
# Started
local status_color="$txtgrn"
# Stopped
if [[ $(cat "$WATSON_DIR/state") == '{}' ]]; then
status_color="$txtred"
fi
echo -e "$status_color""◉""$txtrst"
}
PROMPT="╭── %{$(watson_status) $fg_bold[green]%}%~%{$reset_color%}$(git_prompt_info) ⌚ %{$FG[130]%}%*%{$reset_color%}
╰─➤ $ "
Current issue
The icon will indicate the color of the state at the time that .zshrc was executed. For example, if the timer is running and the icon is properly indicating green, stopping the timer will not cause the icon to turn red. In order to see the icon change color, I have to source .zshrc.
This indicates that the function watson_status() needs to be run each time a command is executed, to give the latest status at the time of the command
I recently ported some prompt code from bash to zsh - on OSX Big Sur - and these were the two big "things to know" for me:
add setopt PROMPT_SUBST to your .zshrc. This "allows for functions in the prompt"
use single quotes when defining your PS1 / PROMPT. If you use double quotes, then the whole string will be evaluated once when the terminal starts, and then that evaluated value is "re-executed" every time the command changes. But you want the functions to be re-evaluated, not the output of the function at the time when the terminal is created.
Easiest example I used to confirm I had it working:
# print_epoch() { date '+%s' }
# this is what you want
# export PS1='$(print_epoch) > '
# this is not what you want
# export PS1="$(print_epoch) > "
Also worth noting PS1 and PROMPT are interchangeable
Extra:
While we are sharing here is my fairly minimal echo my user, directory, and git branch PROMPT with some colors and a pretty leaf:
parse_git_branch() {
git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* \(.*\)/\1/'
}
setopt PROMPT_SUBST
autoload -U colors && colors
export PROMPT='%n %~ %F{blue}🌿$(parse_git_branch)%f > '
noting that:
%n prints name
%~ prints directory relative to home
%F{blue} changes text to blue
%f resets color
Documentation: http://zsh.sourceforge.net/Doc/Release/Prompt-Expansion.html#Visual-effects
What is the meaning for $! in shell or shell scripting? I am trying to understand a script which has the something like the following.
local#usr> a=1
local#usr> echo $a
1
local#usr> echo $!a
a
It is printing the variable back. Is it all for that? What are the other $x options we have? Few I know are $$, $*, $?. If anyone can point me to a good source, it will be helpful. BTW, This is in Sun OS 5.8, KSH.
The various $… variables are described in Bash manual. According to the manual $! expands to the PID of the last process launched in background. See:
$ echo "Foo"
Foo
$ echo $!
$ true&
[1] 67064
$ echo $!
67064
[1]+ Done true
In ksh it seems to do the same.
From the ksh man page on my system:
${!vname}
Expands to the name of the variable referred to by vname. This
will be vname except when vname is a name reference.
For the shell you are asking, ksh, use the the ksh manual, and read this:
Parameter Substitution
A parameter is an identifier, one or more digits, or any of
the characters *, #, #, ?, -, $, and !.
It is clear that those are the accepted options $*, $#, $#, $?, $-, $$, and $!.
More could be included in the future.
For the parameter $!, from the manual:
"!" The process number of the last background command invoked.
if you start a background process, like sleep 60 &, then there will be a process number for such process, and the parameter $! will print its number.
$ sleep 60 &
[1] 12329
$ echo "$!"
12329
If there is no background process in execution (as when the shell starts), the exansion is empty. It has a null value.
$ ksh -c 'echo $!'
If there is a background process, it will expand to the PID of such process:
$ ksh -c 'sleep 30 & echo $!'
42586
That is why echo $!a expanded to a. It is because there is no PID to report:
$ ksh -c 'echo $!a'
a
Other shells may have a different (usually pretty similar) list of expansions (a parameter with only one $ and one next character).
For example, bash recognize this *##?-$!0_ as "Special parameters". Search the Bash manual for the heading "3.4.2 Special Parameters".
Special Parameters
The shell treats several parameters specially.
It gives the Process id of last backgroundjob or background function
Please go through this link below
http://www.well.ox.ac.uk/~johnb/comp/unix/ksh.html#specvar
! is a reference operator in unix, though it is not called with that name.
It always refers to a mother process. Try typing :! in vi, it takes you to command prompt and you can execute commands as usual until exit command.
! in SQLPLUS also executes the command from the command prompt. try this in sqlplus
SQL> !ls --- this gives the list of files inthe current dir.
$! - obviously gives the process id of the current/latest process.
Is there an alternative to tee which captures standard output and standard error of the command being executed and exits with the same exit status as the processed command?
Something like the following:
eet -a some.log -- mycommand --foo --bar
Where "eet" is an imaginary alternative to "tee" :) (-a means append and -- separates the captured command). It shouldn't be hard to hack such a command, but maybe it already exists and I'm not aware of it?
This works with Bash:
(
set -o pipefail
mycommand --foo --bar | tee some.log
)
The parentheses are there to limit the effect of pipefail to just the one command.
From the bash(1) man page:
The return status of a pipeline is the exit status of the last command, unless the pipefail option is enabled. If pipefail is enabled, the pipeline's return status is the value of the last (rightmost) command to exit with a non-zero status, or zero if all commands exit successfully.
I stumbled upon a couple of interesting solutions at Capture Exit Code Using Pipe & Tee.
There is the $PIPESTATUS variable available in Bash:
false | tee /dev/null
[ $PIPESTATUS -eq 0 ] || exit $PIPESTATUS
And the simplest prototype of "eet" in Perl may look as follows:
open MAKE, "command 2>&1 |" or die;
open (LOGFILE, ">>some.log") or die;
while (<MAKE>) {
print LOGFILE $_;
print
}
close MAKE; # To get $?
my $exit = $? >> 8;
close LOGFILE;
Here's an eet. Works with every Bash I can get my hands on, from 2.05b to 4.0.
#!/bin/bash
tee_args=()
while [[ $# > 0 && $1 != -- ]]; do
tee_args=("${tee_args[#]}" "$1")
shift
done
shift
# now ${tee_args[*]} has the arguments before --,
# and $* has the arguments after --
# redirect standard out through a pipe to tee
exec | tee "${tee_args[#]}"
# do the *real* exec of the desired program
exec "$#"
(pipefail and $PIPESTATUS are nice, but I recall them being introduced in 3.1 or thereabouts.)
This is what I consider to be the best pure-Bourne-shell solution to use as the base upon which you could build your "eet":
# You want to pipe command1 through command2:
exec 4>&1
exitstatus=`{ { command1; echo $? 1>&3; } | command2 1>&4; } 3>&1`
# $exitstatus now has command1's exit status.
I think this is best explained from the inside out – command1 will execute and print its regular output on stdout (file descriptor 1), then once it's done, echo will execute and print command1's exit code on its stdout, but that stdout is redirected to file descriptor three.
While command1 is running, its stdout is being piped to command2 (echo's output never makes it to command2 because we send it to file descriptor 3 instead of 1, which is what the pipe reads). Then we redirect command2's output to file descriptor 4, so that it also stays out of file descriptor one – because we want file descriptor one clear for when we bring the echo output on file descriptor three back down into file descriptor one so that the command substitution (the backticks) can capture it.
The final bit of magic is that first exec 4>&1 we did as a separate command – it opens file descriptor four as a copy of the external shell's stdout. Command substitution will capture whatever is written on standard out from the perspective of the commands inside it – but, since command2's output is going to file descriptor four as far as the command substitution is concerned, the command substitution doesn't capture it – however, once it gets "out" of the command substitution, it is effectively still going to the script's overall file descriptor one.
(The exec 4>&1 has to be a separate command to work with many common shells. In some shells it works if you just put it on the same line as the variable assignment, after the closing backtick of the substitution.)
(I use compound commands ({ ... }) in my example, but subshells (( ... )) would also work. The subshell will just cause a redundant forking and awaiting of a child process, since each side of a pipe and the inside of a command substitution already normally implies a fork and await of a child process, and I don't know of any shell being coded to recognize that it can skip one of those forks because it's already done or is about to do the other.)
You can look at it in a less technical and more playful way, as if the outputs of the commands are leapfrogging each other: command1 pipes to command2, then the echo's output jumps over command2 so that command2 doesn't catch it, and then command2's output jumps over and out of the command substitution just as echo lands just in time to get captured by the substitution so that it ends up in the variable, and command2's output goes on its way to the standard output, just as in a normal pipe.
Also, as I understand it, at the end of this command, $? will still contain the return code of the second command in the pipe, because variable assignments, command substitutions, and compound commands are all effectively transparent to the return code of the command inside them, so the return status of command2 should get propagated out.
A caveat is that it is possible that command1 will at some point end up using file descriptors three or four, or that command2 or any of the later commands will use file descriptor four, so to be more hygienic, we would do:
exec 4>&1
exitstatus=`{ { command1 3>&-; echo $? 1>&3; } 4>&- | command2 1>&4; } 3>&1`
exec 4>&-
Commands inherit file descriptors from the process that launches them, so the entire second line will inherit file descriptor four, and the compound command followed by 3>&1 will inherit the file descriptor three. So the 4>&- makes sure that the inner compound command will not inherit file descriptor four, and the 3>&- makes sure that command1 will not inherit file descriptor three, so command1 gets a 'cleaner', more standard environment. You could also move the inner 4>&- next to the 3>&-, but I figure why not just limit its scope as much as possible.
Almost no programs uses pre-opened file descriptor three and four directly, so you almost never have to worry about it, but the latter is probably best to keep in mind and use for general-purpose cases.
{ mycommand --foo --bar 2>&1; ret=$?; } | tee -a some.log; (exit $ret)
KornShell, all in one line:
foo; RET_VAL=$?; if test ${RET_VAL} != 0;then echo $RET_VAL; echo Error occurred!>/tmp/out.err;exit 2;fi |tee >>/tmp/out.err ; if test ${RET_VAL} != 0;then exit $RET_VAL;fi
#!/bin/sh
logfile="$1"
shift
exec 2>&1
exec "$#" | tee "$logfile"
Hopefully this works for you.
Assuming Bash or Z shell (zsh),
my_command >>my_log 2>&1
N.B. The sequence of redirection and duplication of standard error onto standard output is significant!
I didn't realise you wanted to see the output on screen as well. This will of course direct all output to the file my_log.
I am running a script on a solaris Box. specifically SunOS 5.7. I am not root. I am trying to execute a script similar to the following:
newgrp thegroup <<
FOO
source .login_stuff
echo "hello world"
FOO
The Script runs. The problem is it returns back to the calling process which puts me in the old group with the source .login_stuff not being sourced. I understand this behavior. What I am looking for is a way to stay in the sub shell. Now I know I could put an xterm& (see below) in the script and that would do it, but having a new xterm is undesirable.
Passing your current pid as a parameter.
newgrp thegroup <<
FOO
source .login_stuff
xterm&
echo $1
kill -9 $1
FOO
I do not have sg available.
Also, newgrp is necessary.
The following works nicely; put the following bit at the top of the (Bourne or Bash) script:
### first become another group
group=admin
if [ $(id -gn) != $group ]; then
exec sg $group "$0 $*"
fi
### now continue with rest of the script
This works fine on Linuxen. One caveat: arguments containing spaces are broken apart. I suggest you use the
env arg1='value 1' arg2='value 2' script.sh construct to pass them in (I couldn't get it to work with $# for some reason)
The newgrp command can only meaningfully be used from an interactive shell, AFAICT. In fact, I gave up on it about ... well, let's say long enough ago that the replacement I wrote is now eligible to vote in both the UK and the USA.
Note that newgrp is a special command 'built into' the shell. Strictly, it is a command that is external to the shell, but the shell has built-in knowledge about how to handle it. The shell actually exec's the program, so you get a new shell immediately afterwards. It is also a setuid root program. On Solaris, at least, newgrp also seems to ignore the SHELL environment variable.
I have a variety of programs that work around the issue that newgrp was intended to address. Remember, the command pre-dates the ability of users to belong to multiple groups at once (see the Version 7 Unix Manuals). Since newgrp does not provide a mechanism to execute commands after it executes, unlike su or sudo, I wrote a program newgid which, like newgrp, is a setuid root program and allows you to switch from one group to another. It is fairly simple - just main() plus a set of standardized error reporting functions used. Contact me (first dot last at gmail dot com) for the source. I also have a much more dangerous command called 'asroot' that allows me (but only me - under the default compilation) to tweak user and group lists much more thoroughly.
asroot: Configured for use by jleffler only
Usage: asroot [-hnpxzV] [<uid controls>] [<gid controls>] [-m umask] [--] command [arguments]
<uid controls> = [-u usr|-U uid] [-s euser|-S euid][-i user]
<gid controls> = [-C] [-g grp|-G gid] [-a grp][-A gid] [-r egrp|-R egid]
Use -h for more help
Option summary:
-a group Add auxilliary group (by name)
-A gid Add auxilliary group (by number)
-C Cancel all auxilliary groups
-g group Run with specified real GID (by name)
-G gid Run with specified real GID (by number)
-h Print this message and exit
-i Initialize UID and GIDs as if for user (by name or number)
-m umask Set umask to given value
-n Do not run program
-p Print privileges to be set
-r euser Run with specified effective UID (by name)
-R euid Run with specified effective UID (by number)
-s egroup Run with specified effective GID (by name)
-S egid Run with specified effective GID (by number)
-u user Run with specified real UID (by name)
-U uid Run with specified real UID (by number)
-V Print version and exit
-x Trace commands that are executed
-z Do not verify the UID/GID numbers
Mnemonic for effective UID/GID:
s is second letter of user;
r is second letter of group
(This program grew: were I redoing it from scratch, I would accept user ID or user name without requiring different option letters; ditto for group ID or group name.)
It can be tricky to get permission to install setuid root programs. There are some workarounds available now because of the multi-group facilities. One technique that may work is to set the setgid bit on the directories where you want the files created. This means that regardless of who creates the file, the file will belong to the group that owns the directory. This often achieves the effect you need - though I know of few people who consistently use this.
newgrp adm << ANYNAME
# You can do more lines than just this.
echo This is running as group \$(id -gn)
ANYNAME
..will output:
This is running as group adm
Be careful -- Make sure you escape the '$' with a slash. The interactions are a little strange, because it expands even single-quotes before it executes the shell as the other group. so, if your primary group is 'users', and the group you're trying to use is 'adm', then:
newgrp adm << END
# You can do more lines than just this.
echo 'This is running as group $(id -gn)'
END
..will output:
This is running as group users
..because 'id -gn' was run by the current shell, then sent to the one running as adm.
Anyways, I know this post is ancient, but hope this is useful to someone.
This example was expanded from plinjzaad's answer; it handles a command line which contains quoted parameters that contain spaces.
#!/bin/bash
group=wg-sierra-admin
if [ $(id -gn) != $group ]
then
# Construct an array which quotes all the command-line parameters.
arr=("${#/#/\"}")
arr=("${arr[*]/%/\"}")
exec sg $group "$0 ${arr[#]}"
fi
### now continue with rest of the script
# This is a simple test to show that it works.
echo "group: $(id -gn)"
# Show all command line parameters.
for i in $(seq 1 $#)
do
eval echo "$i:\${$i}"
done
I used this to demonstrate that it works.
% ./sg.test 'a b' 'c d e' f 'g h' 'i j k' 'l m' 'n o' p q r s t 'u v' 'w x y z'
group: wg-sierra-admin
1:a b
2:c d e
3:f
4:g h
5:i j k
6:l m
7:n o
8:p
9:q
10:r
11:s
12:t
13:u v
14:w x y z
Maybe
exec $SHELL
would do the trick?
You could use sh& (or whatever shell you want to use) instead of xterm&
Or you could also look into using an alias (if your shell supports this) so that you would stay in the context of the current shell.
In a script file eg tst.ksh:
#! /bin/ksh
/bin/ksh -c "newgrp thegroup"
At the command line:
>> groups fred
oldgroup
>> tst.ksh
>> groups fred
thegroup
sudo su - [user-name] -c exit;
Should do the trick :)