thanks for reading & your suggestions. I'm moving files into respective directories, only some of the files use _ and some use - (underscore and hyphen) as delimiters. Is there a way to test for the different delimiters?
E.g.:
ParentDir
1897/
1898/
1994summer/
file-1897-001.txt
file-1897-002.txt
file-1898-001.txt
file-1898-002.txt
file_1994summer_001.txt
file_1994summer_002.txt
I've been processing with the following (verbose so I can understand it) shell script:
!/bin/sh
for f in *.jp2
do
base=${f%.txt}
echo "base fileName is $base"
fileName=`echo "$base" | cut -f 2 -d _`
echo "truncated fileName is $fileName"
dir=$fileName
echo "Directory is $dir"
mv -v "$f" "$dir"
sleep 1
done
When using the cut command, I'd like to be able to differentiate on the delimiter. Is that possible? Thanks in advance for your time & suggestions.
Cheers!
case "${fName}" in
*_* ) underscore_funnyFace_processing "${fName}" ;;
*-* ) hyphen_funnyFace_process "${fName}" ;;
* ) all_other_processing "${fName}" ;;
esac
I'm almost sure bourne shell supported case processing. Don't have access to one to test with.
The difference between bourne/bash/ksh/zsh would be in the shell wildcards patterns that each shell has as an extension to the basic patterns the bourne shell supported.
I hope this helps.
Related
I want to know which permission is given to a file using a shell script. So i used the below code to test for a file. But it shows nothing in output. I just wanted to know where i have made the mistake. Please help me.
The file "1.py" has all read write and execute files enabled.
ls -l 1.py | awk ' {if($1 -eq "-rwxrwxrwx")print 'True'; }'
The single quotes (') around True should be double quotes ("), and awk uses == for string comparison.
However, depending on what you're trying to do, it might be cleaner to use the Bash builtin tests:
if [ -r 1.py -a -x 1.py ]; then
echo "Yes, we can read (-r) and (-a) execute (-x) the file"
else
echo "No, we can't."
fi
This avoids having to parse ls output. For a longer list of checks, see tldp.org.
in awk, you shouldn't write shell test, e.g. [[ ... -eq ...]], you should do it in awk way:
if($1=="whatever")...
you could use
ls -l 1.py | awk '{if ($1 == "-rwxrwxrwx") print "True" }'
I need to make a unix script to read first word from a file, and if that is "Mon,Tue....Sat,Sun" then it will print echo 0 or else echo 1
I was trying with grep command but it didn't worked
This could even be done without grep or awk, using just bash builtins (assuming your shell is bash - this should also work in ksh and also zsh, and maybe in sh, but not csh, where the syntax is quite a bit different):
read firstword otherstuff < myfile.txt
case "${firstword}" in
Sun|Mon|Tue|Wed|Thu|Fri|Sat) echo 1;;
*) echo 0;;
esac
You could also use regexp matching to avoid the case statement (this is definitely bash-only, though):
if [[ "${firstword}" =~ ^(Sun|Mon|Tue|Wed|Thu|Fri|Sat)$ ]]; then
That's just a matter of preference, though...
When you need to parse input for words awk is better then grep, it still can do what grep does, but also can precess every line with simple scripts.
This is my take on solution:
awk 'NR==1{c=0} $1~/Sun|Mon|Tue|Wed|Thu|Fri|Sat/{c=1} END{print c}' test.txt
I encourage you to learn more about awk in this (short) tutorial
Try egrep command like:
head -1 myfile.txt | egrep '^[Sun|Mon|Tue|Wed|Thu|Fri|Sat] .*'; echo $?
For grep there's a fixed string option, -F (fgrep) to turn off regex interpretation of the search string.
Is there a similar facility for sed? I couldn't find anything in the man. A recommendation of another gnu/linux tool would also be fine.
I'm using sed for the find and replace functionality: sed -i "s/abc/def/g"
Do you have to use sed? If you're writing a bash script, you can do
#!/bin/bash
pattern='abc'
replace='def'
file=/path/to/file
tmpfile="${TMPDIR:-/tmp}/$( basename "$file" ).$$"
while read -r line
do
echo "${line//$pattern/$replace}"
done < "$file" > "$tmpfile" && mv "$tmpfile" "$file"
With an older Bourne shell (such as ksh88 or POSIX sh), you may not have that cool ${var/pattern/replace} structure, but you do have ${var#pattern} and ${var%pattern}, which can be used to split the string up and then reassemble it. If you need to do that, you're in for a lot more code - but it's really not too bad.
If you're not in a shell script already, you could pretty easily make the pattern, replace, and filename parameters and just call this. :)
PS: The ${TMPDIR:-/tmp} structure uses $TMPDIR if that's set in your environment, or uses /tmp if the variable isn't set. I like to stick the PID of the current process on the end of the filename in the hopes that it'll be slightly more unique. You should probably use mktemp or similar in the "real world", but this is ok for a quick example, and the mktemp binary isn't always available.
Option 1) Escape regexp characters. E.g. sed 's/\$0\.0/0/g' will replace all occurrences of $0.0 with 0.
Option 2) Use perl -p -e in conjunction with quotemeta. E.g. perl -p -e 's/\\./,/gi' will replace all occurrences of . with ,.
You can use option 2 in scripts like this:
SEARCH="C++"
REPLACE="C#"
cat $FILELIST | perl -p -e "s/\\Q$SEARCH\\E/$REPLACE/g" > $NEWLIST
If you're not opposed to Ruby or long lines, you could use this:
alias replace='ruby -e "File.write(ARGV[0], File.read(ARGV[0]).gsub(ARGV[1]) { ARGV[2] })"'
replace test3.txt abc def
This loads the whole file into memory, performs the replacements and saves it back to disk. Should probably not be used for massive files.
If you don't want to escape your string, you can reach your goal in 2 steps:
fgrep the line (getting the line number) you want to replace, and
afterwards use sed for replacing this line.
E.g.
#/bin/sh
PATTERN='foo*[)*abc' # we need it literal
LINENUMBER="$( fgrep -n "$PATTERN" "$FILE" | cut -d':' -f1 )"
NEWSTRING='my new string'
sed -i "${LINENUMBER}s/.*/$NEWSTRING/" "$FILE"
You can do this in two lines of bash code if you're OK with reading the whole file into memory. This is quite flexible -- the pattern and replacement can contain newlines to match across lines if needed. It also preserves any trailing newline or lack thereof, which a simple loop with read does not.
mapfile -d '' < file
printf '%s' "${MAPFILE//"$pat"/"$rep"}" > file
For completeness, if the file can contain null bytes (\0), we need to extend the above, and it becomes
mapfile -d '' < <(cat file; printf '\0')
last=${MAPFILE[-1]}; unset "MAPFILE[-1]"
printf '%s\0' "${MAPFILE[#]//"$pat"/"$rep"}" > file
printf '%s' "${last//"$pat"/"$rep"}" >> file
perl -i.orig -pse 'while (($i = index($_,$s)) >= 0) { substr($_,$i,length($s), $r)}'--\
-s='$_REQUEST['\'old\'']' -r='$_REQUEST['\'new\'']' sample.txt
-i.orig in-place modification with backup.
-p print lines from the input file by default
-s enable rudimentary parsing of command line arguments
-e run this script
index($_,$s) search for the $s string
substr($_,$i,length($s), $r) replace the string
while (($i = index($_,$s)) >= 0) repeat until
-- end of perl parameters
-s='$_REQUEST['\'old\'']', -r='$_REQUEST['\'new\'']' - set $s,$r
You still need to "escape" ' chars but the rest should be straight forward.
Note: this started as an answer to How to pass special character string to sed hence the $_REQUEST['old'] strings, however this question is a bit more appropriately formulated.
You should be using replace instead of sed.
From the man page:
The replace utility program changes strings in place in files or on the
standard input.
Invoke replace in one of the following ways:
shell> replace from to [from to] ... -- file_name [file_name] ...
shell> replace from to [from to] ... < file_name
from represents a string to look for and to represents its replacement.
There can be one or more pairs of strings.
I'm writing a shell script that opens a file and needs to find a tag like ##FIND_ME##. The string I'm searching for is a constant (and there is only ever one instance of it.)
Once I locate that string, I need it to start a new search for a different string from that point forward.
My *nix skills are a little rusty, should try to implement this using grep, awk, or sed?
awk '/FINDME/{f=1}f&&/NEWSEARCH/{print}' file
shell
f=0
while read -r line
do
case "$line" in
*FINDME* ) f=1;;
esac
if [ "$f" -eq 1 ] ;then
case "$line" in
*NEWSEARCH*) echo "found next tag in: $line";;
esac
fi
done <"file"
I'm trying to write a (sh -bourne shell) script that processes lines as they are written to a file. I'm attempting to do this by feeding the output of tail -f into a while read loop. This tactic seems to be proper based on my research in Google as well as this question dealing with a similar issue, but using bash.
From what I've read, it seems that I should be able to break out of the loop when the file being followed ceases to exist. It doesn't. In fact, it seems the only way I can break out of this is to kill the process in another session. tail does seem to be working fine otherwise as testing with this:
touch file
tail -f file | while read line
do
echo $line
done
Data I append to file in another session appears just file from the loop processing written above.
This is on HP-UX version B.11.23.
Thanks for any help/insight you can provide!
If you want to break out, when your file does not exist any more, just do it:
test -f file || break
Placing this in your loop, should break out.
The remaining problem is, how to break the read line, as this is blocking.
This could you do by applying a timeout, like read -t 5 line. Then every 5 second the read returns, and in case the file does not longer exist, the loop will break. Attention: Create your loop that it can handle the case, that the read times out, but the file is still present.
EDIT: Seems that with timeout read returns false, so you could combine the test with the timeout, the result would be:
tail -f test.file | while read -t 3 line || test -f test.file; do
some stuff with $line
done
I don't know about HP-UX tail but GNU tail has the --follow=name option which will follow the file by name (by re-opening the file every few seconds instead of reading from the same file descriptor which will not detect if the file is unlinked) and will exit when the filename used to open the file is unlinked:
tail --follow=name test.txt
Unless you're using GNU tail, there is no way it'll terminate of its own accord when following a file. The -f option is really only meant for interactive monitoring--indeed, I have a book that says that -f "is unlikely to be of use in shell scripts".
But for a solution to the problem, I'm not wholly sure this isn't an over-engineered way to do it, but I figured you could send the tail to a FIFO, then have a function or script that checked the file for existence and killed off the tail if it'd been unlinked.
#!/bin/sh
sentinel ()
{
while true
do
if [ ! -e $1 ]
then
kill $2
rm /tmp/$1
break
fi
done
}
touch $1
mkfifo /tmp/$1
tail -f $1 >/tmp/$1 &
sentinel $1 $! &
cat /tmp/$1 | while read line
do
echo $line
done
Did some naïve testing, and it seems to work okay, and not leave any garbage lying around.
I've never been happy with this answer but I have not found an alternative either:
kill $(ps -o pid,cmd --no-headers --ppid $$ | grep tail | awk '{print $1}')
Get all processes that are children of the current process, look for the tail, print out the first column (tail's pid), and kill it. Sin-freaking-ugly indeed, such is life.
The following approach backgrounds the tail -f file command, echos its process id plus a custom string prefix (here tailpid: ) to the while loop where the line with the custom string prefix triggers another (backgrounded) while loop that every 5 seconds checks if file is still existing. If not, tail -f file gets killed and the subshell containing the backgrounded while loop exits.
# cf. "The Heirloom Bourne Shell",
# http://heirloom.sourceforge.net/sh.html,
# http://sourceforge.net/projects/heirloom/files/heirloom-sh/ and
# http://freecode.com/projects/bournesh
/usr/local/bin/bournesh -c '
touch file
(tail -f file & echo "tailpid: ${!}" ) | while IFS="" read -r line
do
case "$line" in
tailpid:*) while sleep 5; do
#echo hello;
if [ ! -f file ]; then
IFS=" "; set -- ${line}
kill -HUP "$2"
exit
fi
done &
continue ;;
esac
echo "$line"
done
echo exiting ...
'