The idea is the write a bash script that prints the names of files in the current directory that contain their own name in their content.
e.g if a file called hello contains the string hello, we print hello, and we do this for all files in the current directory
Here's what I wrote, and I have no idea why it doesn't work.
#!/bin/bash
for file in *
do
if (cat $file | grep $file) 2> /dev/null
then
echo $file
fi
done
Fixed:
#!/bin/bash
for file in *
do
if grep $file $file 2> /dev/null
then
echo $file
fi
done
Apart from quoting issues, potential regex escaping issues, and the useless use of cat and (...), your code should work in principle.
Try this version - if it doesn't work, something else must be going on:
#!/bin/bash
for file in *
do
if grep -qF "$file" "$file" 2> /dev/null
then
echo "$file"
fi
done
-q makes grep not output matching lines (whether a match was found is implied by the exit code).
-F ensures that the search term is treated as a literal (rather than a regex).
Related
I am trying to create a script which detects if files in a directory have not UTF-8 characters and if they do, grab the file type of that particular file and perform the iconv operation on it.
The code is follows
find <directory> |sed '1d'><directory>/filelist.txt
while read filename
do
file_nm=${filename%%.*}
ext=${filename#*.}
echo $filename
q=`grep -axv '.*' $filename|wc -l`
echo $q
r=`file -i $filename|cut -d '=' -f 2`
echo $r
#file_repair=$file_nm
if [ $q -gt 0 ]; then
iconv -f $r -t utf-8 -c ${file_nm}.${ext} >${file_nm}_repaired.${ext}
mv ${file_nm}_repaired.${ext} ${file_nm}.${ext}
fi
done< <directory>/filelist.txt
While running the code, there are several files that turn into 0 byte files and .bak gets appended to the file name.
ls| grep 'bak' | wc -l
36
Where am I making a mistake?
Thanks for the help.
It's really not clear what some parts of your script are supposed to do.
Probably the error is that you are assuming file -i will output a string which always contains =; but it often doesn't.
find <directory> |
# avoid temporary file
sed '1d' |
# use IFS='' read -r
while IFS='' read -r filename
do
# indent loop body
file_nm=${filename%%.*}
ext=${filename#*.}
# quote variables, print diagnostics to stderr
echo "$filename" >&2
# use grep -q instead of useless wc -l; don't enter condition needlessly; quote variable
if grep -qaxv '.*' "$filename"; then
# indent condition body
# use modern command substitution syntax, quote variable
# check if result contains =
r=$(file -i "$filename")
case $r in
*=*)
# only perform decoding if we can establish encoding
echo "$r" >&2
iconv -f "${r#*=}" -t utf-8 -c "${file_nm}.${ext}" >"${file_nm}_repaired.${ext}"
mv "${file_nm}_repaired.${ext}" "${file_nm}.${ext}" ;;
*)
echo "$r: could not establish encoding" >&2 ;;
esac
fi
done
See also Why is testing “$?” to see if a command succeeded or not, an anti-pattern? (tangential, but probably worth reading) and useless use of wc
The grep regex is kind of mysterious. I'm guessing you want to check if the file contains non-empty lines? grep -qa . "$filename" would do that.
I have below mentioned directories in an input file and i need them to create in a loop.
data/app_rt_ws/Request/2017_06_27
data/app_rt_ws/Response/2017_06_19
data/app_rt_ws/RTWS
data/app_rt_ws/SDP
data/edge/response/9-20-2016
data/edge/response/9-22-2016
Problem is that i don't need the directories in the yyyy_mm_dd or dd-mm-yyyy format which get created at run time on the server. I need them to be discarded and have the rest of the static path of the directories created.
I am using below mentioned code but can't seem to figure out how to omit the above mentioned part
for i in `cat /tmp/inputfile.txt`
do
echo $i
cd /opt/app/app
awk '/date_year/{print $1}' (need to filter out the entries with date)
mkdir -p $i ( need to create rest of the directories)
done
You may modify your script as followed,
for i in `awk '$0 !~ (/[0-9]{1,2}-[0-9]{1,2}-20[0-9]{2}/ && /20[0-9]{2}_[0-9]{1,2}_[0-9]{1,2}/){print}' inputfile.txt`;
do
echo $i
cd /opt/app/app
mkdir -p $i
done
And the output of the awk command is like this,
$ awk '$0 !~ (/[0-9]{1,2}-[0-9]{1,2}-20[0-9]{2}/ && /20[0-9]{2}_[0-9]{1,2}_[0-9]{1,2}/){print}' inputfile.txt
data/app_rt_ws/RTWS
data/app_rt_ws/SDP
With bash for regexp matching:
$ cat tst.sh
while IFS= read -r path; do
if [[ $path =~ /([0-9]{4}(_[0-9]{2}){2}|([0-9]{1,2}-){2}[0-9]{4})$ ]]; then
path="${path%/*}"
fi
echo mkdir -p "$path"
done < "$1"
$ ./tst.sh file
mkdir -p data/app_rt_ws/Request
mkdir -p data/app_rt_ws/Response
mkdir -p data/app_rt_ws/RTWS
mkdir -p data/app_rt_ws/SDP
mkdir -p data/edge/response
mkdir -p data/edge/response
Remove the echo once you've tested and are happy with the result. Check the regexp - you said you wanted to match dd-mm-yyyy but then your input contained dates as d-mm-yyyy so idk what you really wanted and so I guessed you'd be happy with 1 or 2 digits, hence [0-9]{1,2}.
With other shells use a case statement or similar to match the date at the end of line as a globbing pattern. You do NOT need to call an external tool to do this check.
Using:
for i in `some command`
is always to be avoided because it breaks when the output of some command contains spaces and it's using deprecated backticks instead of modern notation $(some command), and doing:
echo $i
is always to be avoided because you should ALWAYS quote your shell variables (echo "$i") unless you have a specific reason not to so as to avoid accidental word splitting and file name generation.
You could use the bash string substitutions to get rid of the date.
for i in `cat /tmp/inputfile.txt`
do
echo $i
cd /opt/app/app
if [[ $i =~ [0-9] ]]; then
mkdir -p ${i%/*}
else
mkdir -p $i
fi
done
The substitution cuts off everything after the last / so the date is gone.
However if you just want all the ones without numbers then you could do:
for i in `cat /tmp/inputfile.txt`
do
echo $i
cd /opt/app/app
if [[ $i =~ [0-9] ]]; then
:
else
mkdir -p $i
fi
done
Output of the second version:
data/app_rt_ws/RTWS
data/app_rt_ws/SDP
I hope this is what you looked for :)
I'm trying to write a Unix script to create directories based on file names and move those files to the designated directories.
File pattern:
*PLAIN*nn.pdf (e.g. 4520009455604706_PLAIN_1221.pdf)
Directories to be created: Cynn (e.g. Cy21)
[NOTE: Need a step to check if directory exists, if not, then create new directory]
After creating the above directories, I need to move all files matching *PLAIN*21.pdf to the directory /Cy21.
[EDITED] Solution added below.
My solution is like this:
#!/bin/sh
for file in *.pdf
do
if test -s $file
then
cycle=`echo $file | awk -F'.' '{print $1}' | awk '{print substr($0,(length($0)-1))}'`
dir="./Cy"$cycle
if [ -d $dir ]
then
mv $file ./Cy$cycle
else
mkdir $dir
mv $file $dir
fi
else
echo "File error"
echo $file
fi
done
I have a file that fetches descriptions from the DB. The values have special characters in them.
So while writing the same into the file after converting to DOS the characters change to something else.
So as a correction i have used sed command to replace the converted characters to the original special character and it works.
I could do for all the special characters that were present in DB.
Examples :
Original :
CANDELE 50°
After conversion to dos its visible as CANDELE 50Áø
.So i used a sed command sed -e 's/Áø/°/g'
What i want to do now is a permanent fix to automatically change any special character that comes in. is there any command that automatically converts a special character to its original after conversion to dos so that I can avoid a manual addition for every character.
Kindly help me doing the same:
changeFileFormat () {
cd $1
echo "Changing file format Unix -> DOS." >> $LOG_FILE
for file in `ls *.csv`
do
mv ${file} ${file}_unix
unix2dos ${file}_unix >> ${file}_dos
sed -e 's/ÁøU/°/g' -e 's/Âû/Ó/g' -e 's/‹¨«//g' -e 's/ª/ì/g' -e 's/¸/ù/g' -e 's/Áœ/£/g' -e 's/Á¨/¿/g' -e 's/ƒâª/€/g' ${file}_dos >> ${file}
if [ $? -ne 0 ]; then
echo "Conversion failed for file: [ $file ]." >> $LOG_FILE
mv ${file}_unix ${file}
else
rm -f ${file}_dos
rm -f ${file}_unix
fi;
done
echo "Conversion finished." >> $LOG_FILE
}
DB description : CANDELE 50°
CSV file that gets created in unix : ART|M|02A_1057M5706 |CANDELE 50°
After DOS conversion : ART|M|02A_1057M5706 |CANDELE 50Áø
After SED command : ART|M|02A_1057M5706 |CANDELE 50°
Is there any way I can search through all folders in my path for a file. Something like
for f in $PATH ; do ; find "$f" -print | grep lapack ; done
So that every folder in PATH is recursively searched for lapack
This should do it, I ran a few tests, seems to be working:
echo -n $PATH | xargs -d: -i find "{}" -name "*lapack*"
The -n in echo prevents it from writing a newline in the end (otherwise the newline would be passed as part of the last directory name to find(1)).
The -d in xargs(1) says that the delimiter is :. The -i makes it replace {} with the current path.
The rest is self-explanatory, I guess.