Recursively rename folders - recursion

I have a Windows backup program that is creating a large number of sub-directories with a "$" at the end. The Linux client for that same backup program is expecting a ":" at the end of the directory name in order to restore the folders. Some folders have the "$" and others do not - to do with an internal versioning system.
If I am restoring a file from the windows client to another windows box, the backup program works great. The program is using "$" in windows because ":" is invalid in a path.
I am looking for a recursive rename in bash that will step through a path and find all of the folders with a single "$" char at the end and replace it with a ":"
My test data is this:
./a$
./b$
./c$
./white space$
./white space$/another test$
./white space$/another test$/a$
./white space$/another test$/b$
./white space$/another test$/x
./white space$/another test$/y
./white space$/test
./white space$/test 2
./white space$/testing$
I have tried :
find . -type d -name "*$"
which gives me a list of the folders that need to be renamed
./a$
./b$
./c$
./white space$
./white space$/another test$
./white space$/another test$/a$
./white space$/another test$/b$
./white space$/testing$
find . -type d -name "*$" | sed 's/$$/\:/'
gives me the end result I am looking for
./a:
./b:
./c:
./white space:
./white space$/another test:
./white space$/another test$/a:
./white space$/another test$/b:
./white space$/testing:
but I can't get it to rename in the same step. Also, I need it to work in reverse order (starting with the deepest match and working back) so that the first rename does not make the rest fail.

You can use this solution to take care of all spaces and special characters, then with printf '%d' you can get the depth for each matched directory, then perform a descending sort and build a rename command via awk to be executed with sh :
while IFS= read -r -d '' n; do
printf '%q\n' "$n"
done < <(find . -type d -name "*$" -printf '%d|' -print0) | \
sort -r -k1 | \
awk -F '|' '{ printf "rename -v -n '\''s/\\$$/:/'\'' %s \n",$2 }' | \
sh
rename -n will only print names of file to be renamed, if you are ready, do the rename operation removing the -n :
while IFS= read -r -d '' n; do
printf '%q\n' "$n"
done < <(find . -type d -name "*$" -printf '%d|' -print0) | \
sort -r -k1 | \
awk -F '|' '{ printf "rename -v '\''s/\\$$/:/'\'' %s \n",$2 }' | \
sh
Display progression
data=$(while IFS= read -r -d '' n; do
printf '%q\n' "$n"
done < <(find . -type d -name "*$" -printf '%d|' -print0) | \
sort -r -k1);
count=$(echo "$data" | awk 'END{ print NR}');
echo "$data" | awk -F '|' -v count=$count '{ printf "echo -n \""NR"/"count" file : \";rename -v -n '\''s/\\$$/:/'\'' %s \n",$2 }' | \
sh
Output :
1/8 file : ./white space$/another test$/b$ renamed as ./white space$/another test$/b:
2/8 file : ./white space$/another test$/a$ renamed as ./white space$/another test$/a:
3/8 file : ./white space$/testing$ renamed as ./white space$/testing:
4/8 file : ./white space$/another test$ renamed as ./white space$/another test:
5/8 file : ./white space$ renamed as ./white space:
6/8 file : ./c$ renamed as ./c:
7/8 file : ./b$ renamed as ./b:
8/8 file : ./a$ renamed as ./a:
Alternative
A more straightforward solution, use rnm bulk rename tool :
rnm -rs '/\$$/:/g' -dp -1 *

Related

How to calculate total size of all css,js and html pages separately uing shell script?

I have the following code in shell script
find -name "*.css" -exec -printf '%16f Size: %6s\n'
This gives me the file size of every css file. How do I modify this to get the added sum of all the file sizes ?
You could use awk:
find . -name "*.css" -type f -printf '%s\n' | awk '{ tot+=$0 } END { print tot }'
Or in pure bash:
total=0
while read -r s;
do
total=$(( total+s ))
done < <(find . -name "*.css" -type f -printf '%s\n')
echo $total
In 2 steps:
1) ll *css | tr -s " " > filename.txt
2) awk 'BEGIN {x=0} {x+=$5} END {print x}' filename.txt

Size of files in subdirectories

I would like to know if there is an easy way to compute the total size of files in subdirectories in unix? I am interested in all the .js files in a folder with subdirectories and I am trying to use du -ah and grep *.js but does not work. Any help is appreciated. thanks
find . -iname "*.js" -ls | awk '{sum += $7} END {print sum}'
I dont think there is a way with du, but you can use awk
This is for all java files:
> find . -name "*.java" | xargs du -a | awk '{sum+=$1}END{print sum}'
2774
so you can modify this to :
find . -name "*.js" | xargs du -a | awk '{sum+=$1}END{print sum}'
Try below command..... It will print the total at the end...
find . -name '*.js' -exec du {} \; | awk 'sum=sum+$1; END{print sum " total" }'
find . -name '*.js' -exec stat -c %s '{}' + | awk '{ sum += $0 } END { print sum }'

searching for part of a filename (UNIX)

On unix I have files which have been renamed as their original name follwed by _inode number (ie the file dog would be renamed dog_inodeno). I am now trying to remove the inode no so i can search for the original file name elsewhere. Does anyone know how I can do this and teh coding neccesary.
Thanks
This should do the job:
find . -type f -name "*_[0-9]*" -exec \
sh -c 'for i do
b=$(basename "$i")
r=$(basename "$i" "_$(ls -i "$i"|awk "{print \$1}")")
if [ "$b" != "$r" ]; then
echo mv "$i" "$(dirname $i)/$r"
fi
done' sh {} +
Replace echo mv by mv for the script to actually rename the files.
The solution here will do rename your files only if the inode number of a file is part of the file's name in the mentioned format, which is what the OP wants.
Solution is successfuly tested at my end.
find ./ -name "*_[0-9][0-9][0-9][0-9][0-9][0-9]" -exec sh 'rename-files.sh' {} \;
Store the below script for the find command to be successful.
#Script Name: rename-files.sh
#!/bin/bash
#Store the result of find
find_result=$1
#Get the existing file name
fname_alone=`expr ${find_result} : '.*/\(.*\)' '|' ${find_result}`
fname_with_relative_path=`expr ${find_result} : '.\(.*\)' '|' ${find_result}`
fname_with_full_path=`echo "$(pwd)${fname_with_relative_path}"`
#Get the inode number of file name
file_inode_no=`find ./ -name ${fname_alone} -printf '%i'`
#Read the end of name
end_of_name=`echo $fname_alone | awk -F "_" '{print $NF}' `
#Check if end of name contains its file's inode number
if [ $end_of_name -eq $file_inode_no ]
then
#Remove the inode number at the end of file name
new_name=`expr $find_result : '.\(.*\)_.*' '|' $find_result`
#Append the path of the file
renamed_to=`echo "$(pwd)${new_name}"`
#Rename your dog_inodeno to dog
mv $fname_with_full_path $renamed_to
fi
Hope this helps.

Unix script to recursively search a directory and sub directories to grep and print content between 2 patterns in file

I have some files in a directory and sub directories. I need to search all the files and print the file name and the content between 2 matching patterns in the file.
For e.g. lets say my file looks like below.
File1.txt:
Pattern1
ABCDEFGHI
Pattern2
dafoaf
fafaf
dfadf
afadf
File2.txt
Pattern1
XXXXXXXXX
Pattern2
kdfaf
adfdaf
fdafad
I need to get following output
File1.txt:
ABCDEGHI
File2.txt:
XXXXXXXX
and so on for all the files under directory and sub directories separated by new line.
This might work for you:
find . \
-type f \
-exec awk 'BEGING {print FILENAME ":"} /Pattern1/ { p=1 ; next } /Pattern2/ {p=0} p==1 {print $0} END {print ""}' \{\} \;
Note, this prints the FILENAME, even if Pattern1 was not found!
This will work for you :
Create this shell script as my_grep.sh
#!/bin/sh
grep -nH "Pattern" $1 >>temp
if [ `grep -c $1 temp` -eq 2 ]; then
limits=`grep $1 temp | cut -f2 -d:`
lower_limit=`echo $limits | cut -f1 -d" "`
upper_limit=`echo $limits | cut -f2 -d" "`
echo "$1:"
head -`expr $upper_limit - 1` $1 | tail -`expr $upper_limit - $lower_limit - 1`
fi
Use find command to search files and fire this schell script:
$ find ./test -type f -exec ./my_grep {} \;
./test/File1.txt:
ABCDEFGHI
./test/File2.txt:
XXXXXXXXX

KSH sort filenames

I'm searching through a number of directories for "searchstring", and then running a script on each $file:
for file in `find $dir -name ${searchstring}'*'`;
do
echo $file >> $debug
script.sh $file >> $output
done
My $debug file yields the following:
/root/0007_searchstring/out/filename_20120105_020000.log
/root/0006_searchstring/out/filename_20120105_010000.log
/root/0005_searchstring/out/filename_20120105_013000.log
(filename is _yyyymmdd_hhmmss.log)
...
Is there a way to get find to order by filename or by mktime? Should I pipe find to sort first? Make an array then sort it as per this question?
If you want to ignore the directory path and just use the file name, then you should be able to use:
for file in `find $dir -name ${searchstring}'*' | sort --field-separator=/ --key=4`;
'ls -t' if you need to regenerate the list based on timestamp.
'sort -n' if the list is fairly static?
To sort by modification time, you can use stat with find:
$ find . -exec stat {} -c '%Y %n' \; | sort -n | cut -d ' ' -f 2
You can pipe the output of find through sort to sort by filename:
find $dir -name "${searchstring}*" | sort | while read file
do
echo "$file" >> $debug
script.sh "$file" >> $output
done

Resources