I am building a script that notifies me of the progress of my builds. At the beginning of the build, bjam tells me "updating # targets...".
Does bjam have the functionality to notify you every time it finishes building a certain target? Ideally I want to grep the output for these and output a percentage to my screen.
After using bjam I finally realized that the answer was in front of me the whole time. It turns out that for every target being built it will output the object name to the screen. So in my case I came up with a solution that has a reasonable approximation to the number of targets, by counting the corresponding *.o and *.so lines at a given time. Hopefully this can help someone else in the future!
I created an alias called Bjam:
alias Bjam=' reset ; sudo bjam -j4 release address-model=32 toolset=gcc-arm target-os=linux threading=multi | Bjam2'
and a shell script called Bjam2:
#!/bin/bash
n=0
while read line; do
if [ $(echo $line | grep -c '^...updating.*targets...$') -eq 1 ] ; then
TOT=$(echo $line| sed 's|^\.\.\.updating \([0-9]*\) targets\.\.\.$|\1|' )
fi
if [ $(echo $line | grep -c '^gcc\.compile\.c++.*\.o$' ) -eq 1 ] || [ $(echo $line | grep -c '^.*\.so$' ) -eq 1 ]; then
n=$((n+1))
echo "$n/$TOT : $line"
else
echo $line
fi
done
Related
I am trying to source a third party script in zsh (named setup_env.sh stored in ~/), that has following lines in the beginning to guard against accidental execution:
#!/bin/sh
# Guard the script against execution - it must be sourced!
echo $0 | egrep 'setup_env.sh' > /dev/null
if [ $? -eq 0 ]; then
echo ""
echo "ERROR: the setup file must be SOURCED, NOT EXECUTED in a shell."
echo "Try (for bash) : source setup_env.sh"
echo "Or (eg. for ksh): . setup_env.sh"
exit 1
fi
# export some environment variables
...
When I source this script with source ~/setup_env.sh, I see the error message shown in the above code block.
From the script it's apparently visible that it's not written with zsh in mind. But I still want to know why zsh behaves this way, and if it's possible to source the script as it is.
I could source the script as it is without error using bash.
I could also source it in zsh after commenting out the guard block in the beginning of the script.
Can someone explain this difference in behavior for source command between zsh and bash?
zsh/bash have different ways to detect sourcing, following should work for both :
if [[ -n $ZSH_VERSION && $ZSH_EVAL_CONTEXT == toplevel ]] || \
[[ -n $BASH_VERSION && $BASH_SOURCE == $0 ]]; then
echo "Not sourced"
exit 1
fi
To explain a little more, when you run :
source setup_env.sh
# setup_env.sh containing "echo $0"
In zsh, $0 == setup_env.sh
In bash, $0 == bash
This question already has answers here:
What does $# mean in a shell script?
(8 answers)
Closed 5 years ago.
this is my piece of code:
##Checking swapspace if ignore option chosen will force creation of swap space
echo ; echo "[INFO]: Validating Swap space "
swapspace=`swapon -s|tail -1|awk '{print $3/1024}'`
if [ ${swapspace} -le 500 ]; then
echo $# |grep ignore>/dev/null
if [ `echo $?` -eq 0 ]; then
echo "[WARNING]: Swap space is below minimum requirement get the same fixed :${swapspace}
Proceeding with WorkAround. PLEASE GET IT FIXED AT THE EARLIEST"
dd if=/dev/zero of=/swapfile bs=2G count=4
chmod 0600 /swapfile; mkswap /swapfile; swapon /swapfile
export SWAPFLAG=1
else
echo "[ERROR]: Swap space is below minimum requirement get the same fixed :${swapspace}"
export SWAPFLAG=2
fi
fi
can someone please explain what is echo $# doing here?
PS: 'ignore is a hidden argument'
#Surbhi:
$# denotes all of the parameters passed to the script. eg--> ./test.ksh test1 test2 then $# will be equal to test1 test2.
If you run a shell script with arguments e.g:
./myscript goodbye cruel world
you can access those arguments within the script. In particular $# gives you them all as they were typed - e.g. if myscript.sh is
#! /bin/sh
echo $#
the above command simply prints goodbye cruel world to the terminal
I have a text file that lists a large number of file paths. I need to copy all these files from the source directory (mentioned in the path in the file one every line) to a destination directory.
Currently, the command line I tried is
while read line; do cp $ line dest_dir; done < my_file.txt
This seems to be a bit slow. Is there a way to parallelise this whole thing or speed it up ?
You could try GNU Parallel as follows:
parallel --dry-run -a fileList.txt cp {} destinationDirectory
If you like what it says, remove the --dry-run.
You could do something like the following (in your chosen shell)
#!/bin/bash
BATCHSIZE=2
# **NOTE**: check exists with -f and points at the right place. you might not need this. depends on your own taste for risk.
ln -s `which cp` /tmp/myuniquecpname
# **NOTE**: this sort of thing can have limits in some shells
for i in `cat test.txt`
do
BASENAME="`basename $i`"
echo doing /tmp/myuniquecpname $i test2/$BASENAME &
/tmp/myuniquecpname $i test2/$BASENAME &
COUNT=`ps -ef | grep /tmp/myuniquecpname | grep -v grep | wc -l`
# **NOTE**: maybe need to put a timeout on this loop
until [ $COUNT -lt $BATCHSIZE ]; do
COUNT=`ps -ef | grep /tmp/myuniquecpname | grep -v grep | wc -l`
echo waiting...
sleep 1
done
done
I'm using a shell script to help me resolve library paths so I can send out my app bundle. I don't know much about shell scripts and was hacking something together from other pieces so I really don't know how to resolve the issue. The issue revolves around lines like done << ...
Here's some code! Note, this is based off of a Qt project.
echo "Below is the list of install_name_tools that need to be added:"
while IFS= read -r -d '' file; do
baseName=`basename "$file"`
#echo "otool -L \"$file\" | grep -e \"*$baseName\""
hasUsrLocal=`otool -L "$file" | grep -v -e "*$baseName" | grep -v libgcc_s.1.dylib | grep -v libstdc++.6.dylib | grep "/usr/local\|/Users"`
if [ -n "$hasUsrLocal" ]; then
#echo "WARNING: $file has /usr/local dependencies"
#echo "\"$hasUsrLocal\""
#echo "To Fix:"
while read line; do
#Remove extra info
library=`echo "$line" | perl -pe 's/(.*?)\s\(compatibility version.*/\1/'`
libraryBaseName=`basename "$library"`
frameworkNameBase="$libraryBaseName.framework"
isframework=`echo "$library" | grep "$frameworkNameBase"`
unset fixCommand;
if [ -n "$isframework" ]; then
#Print out how to fix the framework
frameworkName=`echo $library | perl -pe "s/.*?($frameworkNameBase\/.+)/\1/"`
fixCommand=`echo "install_name_tool -change \"$library\" \"#executable_path/../Frameworks/$frameworkName\" \"$file\""`
else
#Print out how to fix the regular dylib
if [ "$baseName" != "$libraryBaseName" ]; then
fixCommand=`echo "install_name_tool -change \"$library\" \"#executable_path/../Frameworks/$libraryBaseName\" \"$file\""`
fi
fi
echo "$fixCommand"
done << (echo "$hasUsrLocal")
#echo "---------------------------------------------------------"
fi
done << (find MyProgram.app -type f -print0)
The error this prints is referring to the line done << (echo "$hasUsrLocal")
./deploy.sh: line 563: syntax error near unexpected token `('
./deploy.sh: line 563: ` done << (echo "$hasUsrLocal")'
I'll get a similar issue for done << (find MyProgram.app -type f -print0) too if I comment out some of the script. Thank you!
I believe the author intended to use a process substitution:
done < <( find ...
You could also try piping the find into the while loop:
find MyProgram ... | while IFS= read -r -d '' file; do ... done
I fixed it! Thanks to William Pursell for the keyword "process substitution." That gave me ground to "Google."
I noticed it was a spacing issue. Needs to be spaced like done < <(echo "$hasUsrLocal") for example!
Answer
The << symbol is for here-documents. What you probably want is redirected process substitution:
< <( : )
The <(command_list) construct is for process substitution, while the first less-than symbol redirects standard input of your loop to the file descriptor created by the process substitution that follows it on the line.
Pro Tip
This is a handy but confusing syntax. It really helps to read it from right to left.
When there is no files inside the folder the below script goes inside the for loop. Not sure what i can modify so that it doesn't go inside the for loop. Also when there is no files inside the directory exit status should be success. Wrapper script checks the exit status of the below script
FILESRAW ="/exp/test1/folder" .
for fspec in "$FILESRAW"/* ; do
echo "$fspec"
if [[ -f ${fspec} ]] ; then
..... processing logic
else
... processing logic
fi
done
if using bash,
you can set nullglob
shopt-s nullglob
if you have hidden files,
shopt -s dotglob
with ksh,
#!/bin/ksh
set -o noglob
for file in /path/*
do
....
done
for fspec in `dir $FILESRAW` ; do
To exit if $FILESRAW is empty:
[ $( ls "$FILESRAW" | wc -l ) -eq 0 ] && exit 0
If this test precedes the loop, it will prevent execution from reaching the for loop if $FILESRAW is empty.
When $FILESRAW is empty, "$FILESRAW"/* expands to "/exp/test1/folder/*", as ghostdog74 points out, you can change this behavior by setting nullglob with
shopt -s nullglob
If you want hidden files, set dotglob as well:
shopt -s dotglob
Alternately, you could use ls instead of globing. This has the advantage of working with very full directories (using a pipe, you won't reach the maximum argument limit):
ls "$FILESRAW" | while read file; do
echo "$file"
This becomes messier if you want hidden files, since you'll need to exclude . and .. to emulate globing behavior:
ls -a "$FILESRAW" | egrep -v '^(\.|\.\.)$' | while read file; do
echo "$file"
if you are using ksh,
try putting this in front of for loop so that it won't go inside it.
"set -noglob"
Even I have got the same problem, but I was able to resolve it by doing this.