This is a little complicated case for me.
I want to track if the 'script1_sparkSubmit01.sh' is completed or not which is triggered by Main.sh; if not then wait for it to complete; if completed, then proceed with the remaining script(s) in the main.sh.
Main Script: Main.sh
ksh script1_sparkSubmit01.sh 2>&1 &
pid=$!
echo $pid
while [ 1 ]
do
[ -n "$pid" ] && sleep 60 || break
done
ksh script2_sparkSubmit02.sh 2>&1 &
Another script: script1_sparkSubmit01.sh
spark-submit --jars $sqldriver_jar_path $spark_jar_path/table-load_2.11-1.0.jar >> ${log_dir}/$log_file_name1 2>&1 &
Currently, pid is giving some random value which when I lookup is not available in the current shell. However, I see the 'spark-submit' command of script1_sparkSubmit01.sh running in the current shell.
Kindly help.
Taking the PID from the script which 'script1_sparkSubmit01.sh' triggers - did work out to me.
ksh script1_sparkSubmit01.sh 2>&1 &
while [ 1 ]
do
pid=$(ps -aux | grep 'table-load_2.11-1.0.jar' | grep -v "grep" | awk '{print $2}')
[ -n "$pid" ] && sleep 30 || break
done
ksh script2_sparkSubmit01.sh 2>&1 &
Related
Case 1
RUN_LOG="run.log"
CMD="java -DprofileName=$PROFILE ..."
$CMD &
PID=$!
I'm able to run and get the pid.
Case 2
$CMD | tee $RUN_LOG
I'm able to run and log the result to a file..
Do you know how can I join both?
I want to log to the file and console, plus get the pid as an ENV variable.
Attempt 1:
(echo "Command PID is $BASHPID";exec $CMD;) | tee $RUN_LOG
It outputs to the file but the pid is not in a variable in the end.
Attempt 2:
$CMD & PID=$! >> $RUN_LOG
pid is in the ENV variable PID but the log file in empty.
Summary, I need to get the pid in a env variable PID, output the execution log from CMD both to console and file ("run.log")
Variables can't escape subshells. Write your pid via file or fifo instead:
tmp=$(mktemp -u)
mkfifo "$tmp"
( echo "$BASHPID" > "$tmp"; exec $CMD ) | tee "$RUN_LOG" &
pid=$(cat "$tmp")
rm "$tmp"
Can someone pls gimme a hint on how to apply nvprof to Kinetica ?
1) I see the name of processes of Kinetica which sits upon GPUs is gpudb_cluster_cuda, and its parent process is gpudb_host_manager. I find gpudb_host_manager is started by /etc/rc.d/init.d/gpudb_host_manager.
2) Thus I modified it as below. This should work - even for its child processes. But it doesn't. No profiling data was produced for gpudb_cluster_cuda.
# vi /etc/rc.d/init.d/gpudb_host_manager
...
# $START_PROG"$GPUDB_EXE start-host-manager 2>&1 | tee -a ${STARTUP_LOG}; ( exit \${PIPESTATUS[0]} )"
$START_PROG"/usr/local/cuda-9.2/bin/nvprof --log-file /tmp/nvprof/%p.txt --export-profile /tmp/nvprof/%p.nvvp --print-gpu-trace --profile-child-processes $GPUDB_EXE start-host-manager 2>&1 | tee -a ${STARTUP_LOG}; ( exit \${PIPESTATUS[0]} )"
...
I applied nvprof to /etc/rc.d/init.d/gpudb, and it produces some traces but it does not use GPUs at all.
# vi /etc/rc.d/init.d/gpudb
...
# $START_PROG"$GPUDB_EXE start 2>&1 | tee -a ${STARTUP_LOG}; ( exit \${PIPESTATUS[0]} )"
$START_PROG"/usr/local/cuda-9.2/bin/nvprof --log-file /tmp/nvprof/%p.txt --export-profile /tmp/nvprof/%p.nvvp --print-gpu-trace --profile-child-processes $GPUDB_EXE start 2>&1 | tee -a ${STARTUP_LOG}; ( exit \${PIPESTATUS[0]} )"
...
Of course, I stopped and restarted these. Any comment would be welcome.
I found that nvprof can be applied by editing /opt/gpudb/core/bin/gpudb as below.
[root#localhost ~]# vi /opt/gpudb/core/bin/gpudb
...
# nohup $HOST_MANAGER_CMD >> $HOST_MANAGER_LOG_FILENAME 2>&1 &
nohup /usr/local/cuda-9.2/bin/nvprof --log-file /tmp/nvprof/%p.txt --export-profile /tmp/nvprof/%p.nvvp --print-gpu-trace --profile-child-processes $HOST_MANAGER_CMD >> $HOST_MANAGER_LOG_FILENAME 2>&1 &
...
I am writing a script where I am grepping for a word and if it is true it should exit script else should continue the script. But my script is not going to else part even when grep output is false.
ps -fu rdb1 | grep humor >> humor_chk
grep humor.pl humor_chk | tail -1
if [ $? -eq 0 ]
then
echo "`date +%D' '%T` Humor is still running. Hence exiting" >> $LOG1
exit
else
ls -lrt /a/b/c/ >> files
fi
I am getting the output as:
01/19/15 05:54:03 Humor is still running. Hence exiting
I'm trying to create a Capistrano mutilstage completion for ZSH:
$ cap |
production staging
$ cap production |
deploy -- Deploy a new release
deploy:bundle -- Bundle
...
Completion code:
#compdef cap
#autoload
# /Users/pablo/.oh-my-zsh/custom/plugins/capistrano_custom/_capistrano_custom
local curcontext="$curcontext" state line ret=1
local -a _configs
_arguments -C \
'1: :->cmds' \
'2:: :->args' && ret=0
_cap_tasks() {
if [[ ! -f .cap_tasks~ ]]; then
echo "\nGenerating .cap_tasks~..." > /dev/stderr
cap -v --tasks | grep '#' | cut -d " " -f 2 > .cap_tasks~
fi
cat .cap_tasks~
}
_cap_stages() {
find config/deploy -name \*.rb | cut -d/ -f3 | sed s:.rb::g
}
case $state in
cmds)
if [[ -d config/deploy ]]; then
compadd `_cap_stages`
else
compadd `_cap_tasks`
fi
ret=0
;;
args)
compadd `_cap_tasks`
ret=0
;;
esac
return ret
The problem:
#compdef cap doesn't work. If I type cap and [TAB] it doesn't execute the completion, but with other words (i.e. shipit) works fine.
Any ideas?
Solution:
cap is really a reserved word and it seems that we can't use it with #compdef cap.
I'm wondering how cap and capistrano completions worked before (maybe an old version of ZSH).
Solution dotfiles code: capistrano_custom
Solution oh-my-zsh/PR: #2471
Both solutions use shipit instead of cap.
$ shipit |
production staging
$ shipit production |
deploy -- Deploy a new release
deploy:bundle -- Bundle
...
Yes, cap is a ZSH builtin. Quoting from zsh docs:
The zsh/cap module is used for manipulating POSIX.1e (POSIX.6)
capability sets. [...]. The builtins in this module are:
cap [ capabilities ] Change the shell’s process capability sets to the
specified capabilities, otherwise display the shell’s current
capabilities.
Consider I have a file abcde.txt which may contain one or more lines of text. I want a script that will DELETE the file if it contains single line.
Something like, if 'wc -l abscde.txt' = 1 then rm abscde.txt
My system : Solaris
Here's a simple bash script:
#!/bin/bash
LINECOUNT=`wc -l abscde.txt | cut -f1 -d' '`
if [[ $LINECOUNT == 1 ]]; then
rm -f abscde.txt
fi
delifsingleline () {
if [ $(cat $1 | wc -l) = "1" ]
then
echo "Deleting $1"
echo "rm $1"
fi
}
Lightly tested on zsh. Should work on bash as well.
This is (mostly) just a reformat of Ben's answer:
wc -l $PATH | grep '^1 ' > /dev/null && rm -f $PATH