How to extract the name of immediate directory along with the filename? - unix

I have a file whose complete path is like
/a/b/c/d/filename.txt
If I do a basename on it, I get filename.txt. But this filename is not too unique.
So, it would be better if I could extract the filename as d_filename.txt i.e.
{immediate directory}_{basename result}
How can I achieve this result?

file="/path/to/filename"
echo $(basename $(dirname "$file")_$(basename "$file"))
or
file="/path/to/filename"
filename="${file##*/}"
dirname="${file%/*}"
dirname="${dirname##*/}"
filename="${dirname}_${filename}"

This code will recursively search through your hierarchy starting with the directory that you run the script in. I've coded the loop in such a way that it will handle any filename you throw at it; file names with spaces, newlines etc.
*Note**: the loop is currently written to not include any files in the directory that this script resides in, it only looks at subdirs below it. This was done as it was the easiest way to make sure the script does not include itself in its processing. If for some reason you must include the directory the script resides in, it can be changed to accommodate this.
Code
#!/bin/bash
while IFS= read -r -d $'\0' file; do
dirpath="${file%/*}"
filename="${file##*/}"
temp="${dirpath}_${filename}"
parent_file="${temp##*/}"
printf "dir: %10s orig: %10s new: %10s\n" "$dirpath" "$filename" "$parent_file"
done < <(find . -mindepth 2 -type f -print0)
Test tree
$ tree -a
.
|-- a
| |-- b
| | |-- bar
| | `-- c
| | |-- baz
| | `-- d
| | `-- blah
| `-- foo
`-- parent_file.sh
Output
$ ./parent_file.sh
dir: ./a/b/c/d orig: blah new: d_blah
dir: ./a/b/c orig: baz new: c_baz
dir: ./a/b orig: bar new: b_bar
dir: ./a orig: foo new: a_foo

$ FILE=/a/b/c/d/f.txt
$ echo $FILE
/a/b/c/d/f.txt
$ echo $(basename ${FILE%%$(basename $FILE)})_$(basename $FILE)
d_f.txt

don't need to call external command
s="/a/b/c/d/filename.txt"
t=${s%/*}
t=${t##*/}
filename=${t}_${s##*/}

Take the example:
/a/1/b/c/d/file.txt
/a/2/b/c/d/file.txt
The only reliable way to qualify file.txt and avoid conflicts is to build the entire path into the new filename, e.g.
/a/1/b/c/d/file.txt -> a_1_b_c_d_file.txt
/a/2/b/c/d/file.txt -> a_2_b_c_d_file.txt
You may be able to skip part of the beginning if you know for sure that it will be common to all files, e.g if you know that all files reside somewhere underneath the directory /a above:
/a/1/b/c/d/file.txt -> 1_b_c_d_file.txt
/a/2/b/c/d/file.txt -> 2_b_c_d_file.txt
To achieve this on a per-file basis:
# file="/path/to/filename.txt"
new_file="`echo \"$file\" | sed -e 's:^/::' -e 's:/:_:g'`"
# new_file -> path_to_filename.txt
Say you want do do this recursively in a directory and its subdirectories:
# dir = /a/b
( cd "$dir" && find . | sed -e 's:^\./::' | while read file ; do
new_file="`echo \"$file\" | sed -e 's:/:_:g'`"
echo "rename $dir/$file to $new_file"
done )
Output:
rename /a/b/file.txt to file.txt
rename /a/b/c/file.txt to c_file.txt
rename /a/b/c/e/file.txt to c_e_file.txt
rename /a/b/d/e/file.txt to d_e_file.txt
...
The above is highly portable and will run on essentially any Unix system under any variant of sh (inclusing bash, ksh etc.)

Related

script to watch new files in a folder and when found, based on filename call different scripts

I am trying to design a file watcher solution in which I need to watch a particular folder for different file names everyday, once the file name is found, I need to call a script specific to the file name.
Example:
Watch Folder -
file1.txt
file2.txt
file3.txt
call script.sh abc file1
call script.sh abc file2
call script.sh abc file3
I tried to make use of the inotifywait but have not been able to get it to work. Any help would be appreciated.
sftp_home=/app/public/ent_sftp
script=/app/public/bin
curr_date=$(TZ=":US/Eastern" date '+%Y%m%d')
inotifywait -m $sftp_home -e create -e moved_to |
while read path action file; do
echo "The file '$file' appeared in directory '$path' via '$action'"
if [ "$file" = "file1${curr_date}*.txt" ]; then
echo "file1${curr_date}*.txt was found and process will be initiated"
cd $script
./script.sh file1
elif [ "$file" = "file2${curr_date}*.txt" ]; then
echo "file2${curr_date}*.txtwas found today and process will be initiated"
cd $script
./script.sh file2
fi
done
Thanks,
Kavin
If you want to do glob expansions in the match, you can do that with a case statement:
unset arg
case $file in
file1${curr_date}*.txt)
arg=file1
;;
file2${curr_date}*.txt)
arg=file2
;;
*)
echo No file found >&2
;;
esac
if test -n "$arg"; then
echo "${arg}${curr_date}*.txt was found and process will be initiated"
cd $script
./script.sh "$arg"
fi

zsh completion with virtual path

I want to create a zsh completion for a tool with a virtual file tree.
e.g. my file tree looks like the following:
/
|- foo/
| |- bar
| |- baz/
| |- qux
|- foobar
My tool mycmd has a subcommand for listing the current directory:
$ mycmd ls
foo/
foobar
$ mycmd ls foo/
bar
baz/
My actual zsh completion looks like this:
_mycmd_ls() {
if [ ! -z "$words[-1]" ]; then
dir=$(dirname /$words[-1])
lastpart=$(basename $words[-1])
items=$(mycmd ls $dir | grep "^$lastpart")
else
items=$(mycmd ls)
fi
_values -s ' ' 'items' ${(uozf)items}
}
_mycmd() {
local -a commands
commands=(
'ls:list items in directory'
)
_arguments -C -s -S -n \
'(- 1 *)'{-v,--version}"[Show program\'s version number and exit]: :->full" \
'(- 1 *)'{-h,--help}'[Show help message and exit]: :->full' \
'1:cmd:->cmds' \
'*:: :->args' \
case "$state" in
(cmds)
_describe -t commands 'commands' commands
;;
(args)
_mycmd_ls
;;
(*)
;;
esac
}
_mycmd
IMHO is _values the wrong utility function. The actual behaviour is:
$ mycmd ls<TAB>
foo/ foobar
$ mycmd ls foo/<TAB> ## <- it inserts automatically a space before <TAB> and so $words[-1] = ""
foo/ foobar
I can't use the utility function _files or _path_files because the file tree is only virtual.
I would suggest to make use of compadd rather _values to get in control of the suffix character appended. Then looking at the available choices, set an empty suffix character in case a virtual directory is part of the result:
_mycmd_ls() {
if [ ! -z "$words[-1]" ]; then
dir=$(dirname /$words[-1])
lastpart=$(basename $words[-1])
items=$(mycmd ls $dir | grep "^$lastpart")
else
items=$(mycmd ls)
fi
local suffix=' ';
# do not append space to word completed if it is a directory (ends with /)
for val in $items; do
if [ "${val: -1:1}" = '/' ]; then
suffix=''
break
fi
done
compadd -S "$suffix" -a items
}

searching for part of a filename (UNIX)

On unix I have files which have been renamed as their original name follwed by _inode number (ie the file dog would be renamed dog_inodeno). I am now trying to remove the inode no so i can search for the original file name elsewhere. Does anyone know how I can do this and teh coding neccesary.
Thanks
This should do the job:
find . -type f -name "*_[0-9]*" -exec \
sh -c 'for i do
b=$(basename "$i")
r=$(basename "$i" "_$(ls -i "$i"|awk "{print \$1}")")
if [ "$b" != "$r" ]; then
echo mv "$i" "$(dirname $i)/$r"
fi
done' sh {} +
Replace echo mv by mv for the script to actually rename the files.
The solution here will do rename your files only if the inode number of a file is part of the file's name in the mentioned format, which is what the OP wants.
Solution is successfuly tested at my end.
find ./ -name "*_[0-9][0-9][0-9][0-9][0-9][0-9]" -exec sh 'rename-files.sh' {} \;
Store the below script for the find command to be successful.
#Script Name: rename-files.sh
#!/bin/bash
#Store the result of find
find_result=$1
#Get the existing file name
fname_alone=`expr ${find_result} : '.*/\(.*\)' '|' ${find_result}`
fname_with_relative_path=`expr ${find_result} : '.\(.*\)' '|' ${find_result}`
fname_with_full_path=`echo "$(pwd)${fname_with_relative_path}"`
#Get the inode number of file name
file_inode_no=`find ./ -name ${fname_alone} -printf '%i'`
#Read the end of name
end_of_name=`echo $fname_alone | awk -F "_" '{print $NF}' `
#Check if end of name contains its file's inode number
if [ $end_of_name -eq $file_inode_no ]
then
#Remove the inode number at the end of file name
new_name=`expr $find_result : '.\(.*\)_.*' '|' $find_result`
#Append the path of the file
renamed_to=`echo "$(pwd)${new_name}"`
#Rename your dog_inodeno to dog
mv $fname_with_full_path $renamed_to
fi
Hope this helps.

KSH sort filenames

I'm searching through a number of directories for "searchstring", and then running a script on each $file:
for file in `find $dir -name ${searchstring}'*'`;
do
echo $file >> $debug
script.sh $file >> $output
done
My $debug file yields the following:
/root/0007_searchstring/out/filename_20120105_020000.log
/root/0006_searchstring/out/filename_20120105_010000.log
/root/0005_searchstring/out/filename_20120105_013000.log
(filename is _yyyymmdd_hhmmss.log)
...
Is there a way to get find to order by filename or by mktime? Should I pipe find to sort first? Make an array then sort it as per this question?
If you want to ignore the directory path and just use the file name, then you should be able to use:
for file in `find $dir -name ${searchstring}'*' | sort --field-separator=/ --key=4`;
'ls -t' if you need to regenerate the list based on timestamp.
'sort -n' if the list is fairly static?
To sort by modification time, you can use stat with find:
$ find . -exec stat {} -c '%Y %n' \; | sort -n | cut -d ' ' -f 2
You can pipe the output of find through sort to sort by filename:
find $dir -name "${searchstring}*" | sort | while read file
do
echo "$file" >> $debug
script.sh "$file" >> $output
done

Concatenate multiple files but include filename as section headers

I would like to concatenate a number of text files into one large file in terminal. I know I can do this using the cat command. However, I would like the filename of each file to precede the "data dump" for that file. Anyone know how to do this?
what I currently have:
file1.txt = bluemoongoodbeer
file2.txt = awesomepossum
file3.txt = hownowbrowncow
cat file1.txt file2.txt file3.txt
desired output:
file1
bluemoongoodbeer
file2
awesomepossum
file3
hownowbrowncow
Was looking for the same thing, and found this to suggest:
tail -n +1 file1.txt file2.txt file3.txt
Output:
==> file1.txt <==
<contents of file1.txt>
==> file2.txt <==
<contents of file2.txt>
==> file3.txt <==
<contents of file3.txt>
If there is only a single file then the header will not be printed. If using GNU utils, you can use -v to always print a header.
I used grep for something similar:
grep "" *.txt
It does not give you a 'header', but prefixes every line with the filename.
This should do the trick as well:
$ find . -type f -print -exec cat {} \;
./file1.txt
Content of file1.txt
./file2.txt
Content of file2.txt
Here is the explanation for the command-line arguments:
find = linux `find` command finds filenames, see `man find` for more info
. = in current directory
-type f = only files, not directories
-print = show found file
-exec = additionally execute another linux command
cat = linux `cat` command, see `man cat`, displays file contents
{} = placeholder for the currently found filename
\; = tell `find` command that it ends now here
You further can combine searches trough boolean operators like -and or -or. find -ls is nice, too.
When there is more than one input file, the more command concatenates them and also includes each filename as a header.
To concatenate to a file:
more *.txt > out.txt
To concatenate to the terminal:
more *.txt | cat
Example output:
::::::::::::::
file1.txt
::::::::::::::
This is
my first file.
::::::::::::::
file2.txt
::::::::::::::
And this is my
second file.
This should do the trick:
for filename in file1.txt file2.txt file3.txt; do
echo "$filename"
cat "$filename"
done > output.txt
or to do this for all text files recursively:
find . -type f -name '*.txt' -print | while read filename; do
echo "$filename"
cat "$filename"
done > output.txt
find . -type f -print0 | xargs -0 -I % sh -c 'echo %; cat %'
This will print the full filename (including path), then the contents of the file. It is also very flexible, as you can use -name "expr" for the find command, and run as many commands as you like on the files.
And the missing awk solution is:
$ awk '(FNR==1){print ">> " FILENAME " <<"}1' *
This is how I normally handle formatting like that:
for i in *; do echo "$i"; echo ; cat "$i"; echo ; done ;
I generally pipe the cat into a grep for specific information.
I like this option
for x in $(ls ./*.php); do echo $x; cat $x | grep -i 'menuItem'; done
Output looks like this:
./debug-things.php
./Facebook.Pixel.Code.php
./footer.trusted.seller.items.php
./GoogleAnalytics.php
./JivositeCode.php
./Live-Messenger.php
./mPopex.php
./NOTIFICATIONS-box.php
./reviewPopUp_Frame.php
$('#top-nav-scroller-pos-<?=$active**MenuItem**;?>').addClass('active');
gotTo**MenuItem**();
./Reviews-Frames-PopUps.php
./social.media.login.btns.php
./social-side-bar.php
./staticWalletsAlerst.php
./tmp-fix.php
./top-nav-scroller.php
$active**MenuItem** = '0';
$active**MenuItem** = '1';
$active**MenuItem** = '2';
$active**MenuItem** = '3';
./Waiting-Overlay.php
./Yandex.Metrika.php
you can use this simple command instead of using a for loop,
ls -ltr | awk '{print $9}' | xargs head
If the files all have the same name or can be matched by find, you can do (e.g.):
find . -name create.sh | xargs tail -n +1
to find, show the path of and cat each file.
If you like colors, try this:
for i in *; do echo; echo $'\e[33;1m'$i$'\e[0m'; cat $i; done | less -R
or:
tail -n +1 * | grep -e $ -e '==.*'
or: (with package 'multitail' installed)
multitail *
Here is a really simple way. You said you want to cat, which implies you want to view the entire file. But you also need the filename printed.
Try this
head -n99999999 * or head -n99999999 file1.txt file2.txt file3.txt
Hope that helps
If you want to replace those ugly ==> <== with something else
tail -n +1 *.txt | sed -e 's/==>/\n###/g' -e 's/<==/###/g' >> "files.txt"
explanation:
tail -n +1 *.txt - output all files in folder with header
sed -e 's/==>/\n###/g' -e 's/<==/###/g' - replace ==> with new line + ### and <== with just ###
>> "files.txt" - output all to a file
find . -type f -exec cat {} \; -print
AIX 7.1 ksh
... glomming onto those who've already mentioned head works for some of us:
$ r head
head file*.txt
==> file1.txt <==
xxx
111
==> file2.txt <==
yyy
222
nyuk nyuk nyuk
==> file3.txt <==
zzz
$
My need is to read the first line; as noted, if you want more than 10 lines, you'll have to add options (head -9999, etc).
Sorry for posting a derivative comment; I don't have sufficient street cred to comment/add to someone's comment.
I made a combination of:
cat /sharedpath/{unique1,unique2,unique3}/filename > newfile
and
tail -n +1 file1 file2
into this:
tail -n +1 /sharedpath/{folder1,folder2,...,folder_n}/file.extension | cat > /sharedpath/newfile
The result is a newfile that contains the content from each subfolder (unique1,unique2..) in the {} brackets, separated by subfolder name.
note unique1=folder1
In my case the file.extension has the same name in all subfolders.
If you want the result in the same format as your desired output you can try:
for file in `ls file{1..3}.txt`; \
do echo $file | cut -d '.' -f 1; \
cat $file ; done;
Result:
file1
bluemoongoodbeer
file2
awesomepossum
file3
hownowbrowncow
You can put echo -e before and after the cut so you have the spacing between the lines as well:
$ for file in `ls file{1..3}.txt`; do echo $file | cut -d '.' -f 1; echo -e; cat $file; echo -e ; done;
Result:
file1
bluemoongoodbeer
file2
awesomepossum
file3
hownowbrowncow
This method will print filename and then file contents:
tail -f file1.txt file2.txt
Output:
==> file1.txt <==
contents of file1.txt ...
contents of file1.txt ...
==> file2.txt <==
contents of file2.txt ...
contents of file2.txt ...
For solving this tasks I usually use the following command:
$ cat file{1..3}.txt >> result.txt
It's a very convenient way to concatenate files if the number of files is quite large.
First I created each file: echo 'information' > file1.txt for each file[123].txt.
Then I printed each file to makes sure information was correct:
tail file?.txt
Then I did this: tail file?.txt >> Mainfile.txt. This created the Mainfile.txt to store the information in each file into a main file.
cat Mainfile.txt confirmed it was okay.
==> file1.txt <==
bluemoongoodbeer
==> file2.txt <==
awesomepossum
==> file3.txt <==
hownowbrowncow

Resources