I have below 3 Files at source Location ${PDR}
PHONE_ABC.txt
PHONE_EFG.txt
PHONE_XYZ.txt
I need to copy same files at 3 different locations and add time and some text to it:
Target Location 1 ${LCT1}
Target Location 2 ${LCT2}
Target Location 3 ${LCT3}
Files at Location 1 should be as below:
PHONE_ABC_LCT1_20180914.txt
PHONE_EFG_LCT1_20180914.txt
PHONE_XYZ_LCT1_20180914.txt
Files at Location 2 should be as below:
PHONE_ABC_LCT2_20180914.txt
PHONE_EFG_LCT2_20180914.txt
PHONE_XYZ_LCT2_20180914.txt
Files at Location 3 should be as below:
PHONE_ABC_LCT3_20180914.txt
PHONE_EFG_LCT3_20180914.txt
PHONE_XYZ_LCT3_20180914.txt
Code Used
#!/usr/bin/ksh
cp ${PDR}/PHONE_*.txt ${LCT1}/
cp ${PDR}/PHONE_*.txt ${LCT2}/
cp ${PDR}/PHONE_*.txt ${LCT3}/
# define list of files
LCT1=${LCT1}/PHONE_*.txt
LCT2=${LCT2}/PHONE_*.txt
LCT3=${LCT3}/PHONE_*.txt
# grab time
dtstamp=`date +%Y%m%d`
# for LCT1
for file in ${LCT1}
do
if [ ! -s ${file} ]
then
continue
fi
filebase=${file%.csv}
mv ${file} ${filebase}_LCT1_${dtstamp}.txt
done
# for LCT2
for file in ${LCT2}
do
if [ ! -s ${file} ]
then
continue
fi
filebase=${file%.csv}
mv ${file} ${filebase}_LCT2_${dtstamp}.txt
done
# for LCT3
for file in ${LCT3}
do
if [ ! -s ${file} ]
then
continue
fi
filebase=${file%.csv}
mv ${file} ${filebase}_LCT3_${dtstamp}.txt
done
This is giving me what i require. But somehow i believe this code could be made more efficient and robust which i am not able to figure it out.
Also on day 2 it keeps appending timestamp to the files at target location which i don't thing is good thing to do.
Any pointers to make this code look more efficient and good.
Something like that maybe :
#!/usr/bin/ksh
# grab time
dtstamp=$(date +"%Y%m%d")
cd ${PDR}
for file in PHONE_*.txt
do
if [ ! -s ${file} ]
then
continue
fi
cp ${file} ${LCT1}/${file%.txt}_LCT1_${dtstamp}.csv
cp ${file} ${LCT2}/${file%.txt}_LCT2_${dtstamp}.csv
cp ${file} ${LCT3}/${file%.txt}_LCT3_${dtstamp}.csv
done
Can I delete files in a directory that do NOT contain any text? These are text files with the extension '.fasta'. Initially I am running this script:
for g in `cat genenames.txt` ; do cat *${g}_*.fa > $g.fasta ; done
On a list of files that look like:
id_genename_othername.fa
But in some directories, not all the genenames from the list (genenames.txt) have files with names that match. So sometimes I will get this message:
cat: *genename_*.fa: No such file or directory
The above code still makes a '.fasta' file with the genename that doesn't exist and I would like to remove it. THANK YOU.
Assuming your script is using #!/bin/bash, I'd do
shopt -s nullglob
while IFS= read -r pattern; do
files=( *"$pattern"*.fa )
if [[ "${#files[#]}" -eq 0 ]]; then
echo "no files match pattern *$pattern*.fa"
else
cat "${files[#]}" > $pattern.fasta
fi
done < genenames.txt
Have you tried the following?
for g in `cat genenames.txt` ; do cat *${g}_*.fa 2>/dev/null > $g.fasta ; done
This should prevent the not found errors from producing files
I want to write a script that add '0' at the end of the files that doesn't have it.
This is what I wrote:
#!/bin/bash
for file in $1
do
echo $file
ls $file | grep "\0$"
if ["$?"="1"]
then
fi
done
I don't know hot to target the files in a way I can rename them
for file in *[!0]; do mv "$file" "${file}0"; done
For each name that does not end 0, rename it so it does. Note that this handles names with spaces etc in them.
I want to give the script a directory, and it will rename the files in it that do not end in 0. How can I use this in a way I can tell the script which directory to work with?
So, make the trivial necessary changes, working with a single directory (and not rejecting the command line if more than one directory is specified; just quietly ignoring the extras):
for file in "${1:?}"/*[!0]; do mv "$file" "${file}0"; done
The "${1:?}" notation ensures that $1 is set and is not empty, generating an error message if it isn't. You could alternatively write "${1:-.}" instead; that would work on the current directory instead of a remote directory. The glob then generates the list of file names in that directory that do not end with a 0 and renames them so that they do. If you have Bash, you can use shopt -s nullglob you won't run into problems if there are no files without the 0 suffix in the directory.
You can generalize to handle any number of arguments (all supposed to be directories, defaulting to the current directory if no directory is specified):
for dir in "${#:-.}"
do
for file in "$dir"/*[!0]; do mv "$file" "${file}0"; done
done
Or (forcing directories):
for dir in "${#:-.}"
do
(cd "$dir" && for file in *[!0]; do mv "$file" "${file}0"; done)
done
This has the merit of reporting which arguments are not directories, or are inaccessible directories.
There are endless variations of this sort that could be made; some of them might even be useful.
Now, I want to do the same but, instead of the file ending with '0', the script should rename files that do not end with '.0' so that they do end with '.0'?
This is slightly trickier because of the revised ending. Simply using [!.][!0] is insufficient. For example, if the list of files includes 30, x.0, x0, z.9, and z1, then echo *[!.][!0] only lists z1 (omitting 30, x0 and z.9 which do not end with .0).
I'd probably use something like this instead:
for dir in "${#:-.}"
do
(
cd "$dir" &&
for file in *
do
case "$file" in
(*.0) : skip it;;
(*) mv "$file" "${file}0";;
esac
done
)
done
The other alternative lists more glob patterns:
for dir in "${#:-.}"
do
(cd "$dir" && for file in *[!.][!0] *.[!0] *[!.]0; do mv "$file" "${file}0"; done)
done
Note that this rapidly gets a lot trickier if you want to look for files not ending .00 — there would be a 7 glob expressions (but the case variant would work equally straight-forwardly), and shopt -s nullglob becomes increasingly important (or you need [ -f "$file" ] && mv "$file" "${file}.0" instead of the simpler move command).
I would like to know how to search certain pattern of files (GunZip Files) in all Sub Directories ( Month wise / Date wise - Sub Directories created).
And then, execute a script on the found files. Also need to populate FILENAME along with output for tracking purpose and further analysis on that particular files.
Step1: For example: currently searching files on this pattern TT_DETAIL*.gz.
find /cygdrive/c/Test/ -name TT_DETAIL*.gz
output#1:
/cygdrive/c/Test/Feb2014/TT_DETAIL_20141115.csv.gz
/cygdrive/c/Test/Jan2014/TT_DETAIL_20141110.csv.gz
/cygdrive/c//Test/Mar2014/TT_DETAIL_20141120.csv.gz
Step2:
zcat TT_DETAIL*.gz | awk 'BEGIN { FS=OFS=","} { if ($11=="10") print $2,$3,$6,$10,$11,$17}' >Op_TT_Detail.txt
cat Op_TT_Detail.txt
ZZZ,AAA,ECH,1,10,XXX
ZZZ,BBB,ECH,1,10,XXX
ZZZ,CCC,ECH,1,10,XXX
ZZZ,DDD,ECH,1,10,XXX
Thanks fedorqui for below script is working fine without FILENAME.
while IFS= read -r file
do
awk 'BEGIN { FS=OFS=","} { if ($11=="10") print $2,$3,$6,$10,$11,$17}' <(zcat "$file") >>Op_TT_Detail.txt
done < <(find /cygdrive/c/Test/ -name TT_DETAIL*.gz)
Have tried below command to populate FILENAME along with output for tracking purpose :
while IFS= read -r file
do
awk 'BEGIN { FS=OFS=","} { if ($11=="10") print $2,$3,$6,$10,$11,$17,FILENAME}' <(zcat "$file") >>Op_TT_Detail.txt
done < <(find /cygdrive/c/Test/ -name TT_DETAIL*.gz)
Desired Output:
ZZZ,AAA,ECH,1,10,XXX,/cygdrive/c/Test/Feb2014/TT_DETAIL_20141115.csv.gz
ZZZ,BBB,ECH,1,10,XXX,/cygdrive/c/Test/Feb2014/TT_DETAIL_20141115.csv.gz
ZZZ,CCC,ECH,1,10,XXX,/cygdrive/c//Test/Mar2014/TT_DETAIL_20141120.csv.gz
ZZZ,DDD,ECH,1,10,XXX,/cygdrive/c//Test/Mar2014/TT_DETAIL_20141120.csv.gz
Since FILENAME is not working for *.gz files , should I write" find /cygdrive/c/Test/ -name TT_DETAIL*.gz " into another output file
then call that output file into script , I don't have a write access for source files located server.
Looking for your suggestions !!!
Nice to see you are using the snippet I wrote in the previous question!
I would use this:
while IFS= read -r file
do
awk -v file="$file" 'BEGIN { FS=OFS=","} \
{ if ($11=="10") print $2,$3,$6,$10,$11,$17, file}' \
<(zcat "$file") >>Op_TT_Detail.txt
done < <(find /cygdrive/c/Test/ -name TT_DETAIL*.gz)
That is, with -v file="$file" you give the file name as a variable to awk. And then you use it in your print command.
I have a text-file with many filenames in a single column (~4,000 lines) and a directory with ~13,000 files (including the ~4,000 files in the text-file). How do I copy only the ~4,000 files in the text-file to another directory?
This should make it:
while read file
do
file=$(echo $file | tr -d '\\r')
cp dir/$file another_dir/
done < your_file
In Bash you can also use:
for f in `cat file` ; do cp $f destination ; done
This should work....it depends on how long the filenames are.
cp `cat text-file` target_directory