I am trying to find if files in a directory, output of ls, exists in a file. So I have a file called test.txt inside this file I have few filenames like, V1.txt,v2.txt, v3.txt.
Now when I do ls I find list of files in the directory, I want to search if any of these files are on my test.txt file.
Example:
Sounds like you got two tasks you want to do.
1) loop through directory listing
2) search text file for any of the names of files that were in the directory
Test Setup
echo "V1.txt" > file_list.txt
echo "V2.txt" >> file_list.txt
mkdir testdir
touch testdir/V1.txt
touch testdir/V2.txt
touch testdir/V3.txt
Code
#!/bin/bash
for f in `ls testdir`
do
if grep -q $f file_list.txt; then
echo "found $f"
else
echo "not found $f"
fi
done
Related
In Unix, WHILE command, I am trying to read a file, which is in another directory. But somehow not working, not even throwing any error.
while read line
do
echo $line
done < /tmp/myfile.txt
The file is present in /tmp folder, has all the permissions.
It is not clear why your while loop is not working. Normally loops, if condition syntax are different for different shell. Hence at the beginning of a shell script file we always define the shell where exactly this script should run and that first line is start with a #. Now do the following and check that might help you.
create a file $vi test.sh
put the below line in it
#!/usr/bin/ksh
filename="/tmp/myfile.txt"
while [ 1 ]
do
read -r line
if [ ${line:-1} -eq 1 ]; then
break
else
echo $line
fi
done < "$filename"
or
#!/usr/bin/ksh
filename="/tmp/myfile.txt"
while read -r line
do
echo $line
done < "$filename"
save the file and set the permission like below
$chmod 777 test.sh
now run the file
$./test.sh
I'm trying to write a Unix script to create directories based on file names and move those files to the designated directories.
File pattern:
*PLAIN*nn.pdf (e.g. 4520009455604706_PLAIN_1221.pdf)
Directories to be created: Cynn (e.g. Cy21)
[NOTE: Need a step to check if directory exists, if not, then create new directory]
After creating the above directories, I need to move all files matching *PLAIN*21.pdf to the directory /Cy21.
[EDITED] Solution added below.
My solution is like this:
#!/bin/sh
for file in *.pdf
do
if test -s $file
then
cycle=`echo $file | awk -F'.' '{print $1}' | awk '{print substr($0,(length($0)-1))}'`
dir="./Cy"$cycle
if [ -d $dir ]
then
mv $file ./Cy$cycle
else
mkdir $dir
mv $file $dir
fi
else
echo "File error"
echo $file
fi
done
Using awk, how can i redirect the data to all files in the directory?
like ,lets say i want to redirect to file.txt i will do:
echo "abc"|awk '{print >"file.txt"}'
how can i acheive the same thing but to multiple files(in fact all files) in the directory.
there are some .txt files in teh cirrent directory.
so i want to redirect abc to all the files in the current directory which have an extension of .txt.
You can use tee:
echo "abc" | tee files/*
If the directory with all these files are in /files/.
echo abc|awk '{print |"tee files"}'
One way using GNU awk:
awk -v str="abc" '{ print str > FILENAME; nextfile }' *.txt
This command will overwrite all the content of all files with txt extension with the string abc. You will need the GNU version because of the nextfile instruction, that closes current file and begin to process the next one.
Im new to unix,I have search a lot of info but still don not how to make it in a bash
What i know is used this command ls -tr|xargs -i ksh -c "mv {} ../tmp/" to move file by file.
Now I need to make a script that sorts all of these files by system date and moves them into a directory, The first 1000 oldest files being to be moved.
Example files r like these
KPK.AWQ07102011.66.6708.01
KPK.AWQ07102011.68.6708.01
KPK.EER07102011.561.8312.13
KPK.WWS07102011.806.3287.13
-----------This is the script tat i hv been created-------
if [ ! -d /app/RAID/Source_Files/test/testfolder ] then
echo "test directory does not exist!"
mkdir /app/RAID/Source_Files/calvin/testfolder
echo "unused_file directory created!"
fi
echo "Moving xx oldest files to test directory"
ls -tr /app/RAID/Source_Files/test/*.Z|head -1000|xargs -i ksh -c "mv {} /app/RAID/Source_Files/test/testfolder/"
the problem of this script is
1) unix prompt a syntax erro 'if'
2) The move command is working but it create a new filename testfolder instead move to directory testfolder (testfolder alredy been created in this path)
anyone can gv me a hand ? thanks
Could this help?
mv `ls -tr|head -1000` ../tmp/
head -n takes the n first lines of the previous command (here the 1000 oldest files). The backticks allow for the result of ls and head commands to be used as arguments to mv.
I have a tar.gz file about 13GB in size. It contains about 1.2 million documents. When I untar this all these files sit in one single directory & any reads from this directory takes ages. Is there any way I can split the files from the tar into multiple new folders?
e.g.: I would like to create new folders named [1,2,...] each having 1000 files.
This is a quick and dirty solution but it does the job in Bash without using any temporary files.
i=0 # file counter
dir=0 # folder name counter
mkdir $dir
tar -tzvf YOURFILE.tar.gz |
cut -d ' ' -f12 | # get the filenames contained in the archive
while read filename
do
i=$((i+1))
if [ $i == 1000 ] # new folder for every 1000 files
then
i=0 # reset the file counter
dir=$((dir+1))
mkdir $dir
fi
tar -C $dir -xvzf YOURFILE.tar.gz $filename
done
Same as a one liner:
i=0; dir=0; mkdir $dir; tar -tzvf YOURFILE.tar.gz | cut -d ' ' -f12 | while read filename; do i=$((i+1)); if [ $i == 1000 ]; then i=0; dir=$((dir+1)); mkdir $dir; fi; tar -C $dir -xvzf YOURFILE.tar.gz $filename; done
Depending on your shell settings the "cut -d ' ' -f12" part for retrieving the last column (filename) of tar's content output could cause a problem and you would have to modify that.
It worked with 1000 files but if you have 1.2 million documents in the archive, consider testing this with something smaller first.
Obtain filename list with --list
Make files containing filenames with grep
untar only these files using --files-from
Thus:
tar --list archive.tar > allfiles.txt
grep '^1' allfiles.txt > files1.txt
tar -xvf archive.tar --files-from=files1.txt
If you have GNU tar you might be able to make use of the --checkpoint and --checkpoint-action options. I have not tested this, but I'm thinking something like:
# UNTESTED
cd /base/dir
mkdir $(printf "dir%04d\n" {1..1500}) # probably more than you need
ln -s dest0 linkname
tar -C linkname ... --checkpoint=1000 \
--checkpoint-action='sleep=1' \
--checkpoint-action='exec=ln -snf dest%u linkname ...
you can look at the man page and see if there are options like that. worst comes to worst, just extract the files you need (maybe using --exclude ) and put them into your folders.
tar doesn't provide that capability directly. It only restores its files into the same structure from which it was originally generated.
Can you modify the source directory to create the desired structure there and then tar the tree? If not, you could untar the files as they are in the file and then post-process that directory using a script to move the files into the desired arrangement. Given the number of files, this will take some time but at least it can be done in the background.