Hi there I have a backup shell script executed through crontab but I have a rather large problem.
This is the particular line that scans my drive:
find $E -mtime -1 -exec cp -r --parents {} $B/$T \;
where E and B are variables holding directory paths and T holds the current date. It checks for all files that have been edited within the past day and copies them to the new directory. The folder structure is kept intact due to the --parents argument. The problem I have is that this seems to also check the mtime of all folders, meaning that if I were to change a single file in a very large folder, the entire folder would be copied across during backup, taking up an unnecessary amount of disk space. Is there any way I could remove folder mtime from the equation? I guess it might be possible to exclude folders themselves (not their contents) from the search as long as the --parents argument still takes effect.
I'm guessing you want to apply this only to regular files -
find $E -type f -mtime -1 -exec cp -r --parents {} $B/$T \;
otherwise
find $E ! -type d -mtime -1 -exec cp -r --parents {} $B/$T \;
to get other types of files as well, skipping the evaluation of age on directories.
Related
It's a simple question that I can't seem to figure out. I'm on a Mac with Big Sur with all the latest updates, and I'm going through Terminal to get these commands to run. If there's a better way please let me know.
This is, in basic terms, what I'm trying to do--I want RSYNC to recursively go through a source directory (which in this case would ideally be an entire drive), find any files modified within the last 24 hours, and copy those to another drive, while preserving the folder structure. So if I have:
/Volumes/Drive1/Folder1/File1.file
/Volumes/Drive1/Folder1/File2.file
/Volumes/Drive1/Folder1/File3.file
And File1 has been modified in the last 24 hours, but the other two haven't, I want it to copy that file, so that on the second drive I wind up with:
/Volumes/Drive2/Folder1/File1.file
But without copying File2 and File3.
I've tried a lot of different solutions and strings, but I'm running into problems. The closest I've been able to get is this:
find /Volumes/Drive1/ -type f -mtime -1 -exec cp -a "{}" /Volumes/Drive2/ \;
The problem is that while this one does go through Drive1 and find all the files newer than a day like I want, when it copies them it just dumps them all into the root of Drive2.
This one also seems to come close:
rsync --progress --files-from=<(find /Volumes/Drive1/ -mtime -1 -type f -exec basename {} \;) /Volumes/Drive1/ /Volumes/Drive2/
This one also identifies all the files modified in the last 24 hours, but instead of copying them it gives an error, "link_stat (filename and path) failed: no such file or directory (2)."
I've spent several days trying to figure out what I'm doing wrong but I can't figure it out. Help please!
I think this'll work:
srcDir=/Volumes/Drive1
destDir=/Volumes/Drive2
(cd "$srcDir" && find . -type f -mtime -1 -print0) |
while IFS= read -r -d $'\0' filepath; do
mkdir -p "$(dirname "$destDir/$filepath")"
cp -a "$srcDir/$filepath" "$destDir/$filepath"
done
Explanation:
Using cd "$srcDir"; find . -whatever will generate relative paths (starting with "./") from the source directory to the found files; that means appending the results to $srcDir and $destDir will give the full source and destination paths for each file.
Putting it in parentheses makes it run in a subshell, so the cd won't affect other commands. Coupling cd and find with && means that if cd fails, it won't run find (which would run in the wrong place, generate a list of the wrong file file, and generally cause trouble).
Using -print0 and while IFS= read -r -d $'\0' is a standard weird-filename-safe way of iterating over found files (see BashFAQ #20). Note that if anything in the loop reads from standard input (e.g. cp -i asking for confirmation), it'll steal part of the file list; if this is a worry, use this variant (instead of the pipe) to send the file list over file descriptor #3 instead of standard input:
while IFS= read -r -d $'\0' filepath <&3; do
...
done 3< <(cd "$srcDir" && find . -type f -mtime -1 -print0)
Finally, mkdir -p is used to make sure the destination directory exists, and then cp to copy the file.
I was trying to remove all git files from a repository with this:
find . -name ".git*" -exec rm -rf {} \;
However, rm warns that files could not be deleted (because their parent directory has already been deleted).
Is there a way to get find to stop recursing when it finds a matching directory?
E.g. find...
/.gitmodules
/.git/stuff
/.git/.gitfile
... produces
/.gitmodules
/.git
Use -depth:
find . -depth -name ".git*" -exec rm -rf {} \;
This would allow you to process the files or subdirectories first before their parent directories.
I'd like to copy a content of directory 1 to directory 2.
However, I'd like to only copy files (and not directories) from my directory 1. How can I do that ?
cp dir1/* dir2/*
then I still have the directories issue.
you can also use this in dir1
find . -type f -exec cp{} dir2/ \;
You may try this one
cp dir1/*.* dir2/*
try this one
find dir1 -type f -exec cp {} dir2/ \;
The currently approved solution will work, however if sub-directories exist, this will also copy the files from the subfolders as well, but instead of putting the copied files in sub folders, it will copy them to dir1.
/dir1/dir1a/test.txt will end up as dir1/test.txt
-maxdepth can be used to only copy files in dir1:
find dir1 -maxdepth 1 -type f -exec cp {} dir2/ \;
I need a script file for backup (zip or tar or gz) of old log files in our unix server (causing the space problem). Could you please help me to create the zip or gz files for each log files in current directory and sub-directories also?
I found one command which is to create gz file for the older files, but it creates only one gz file for all older file. But I need individual gz file for each log file.
find /tmp/log/ -mtime +180 | xargs tar -czvPf /tmp/older_log_$(date +%F).tar.gz
Thanking you in advance.
Best way is
find . -mtime +3 -print -exec gzip {} \;
Where +3 means zip all files which is older than 3 days.
Thanks a lot for your reply.
I got it.
files=($(find /tmp/mallik3/ -mtime +"$days"))
for files in ${files[*]}
do
echo $files
zip $files-$(date --date="- "$days"days" +%F)_.zip $files
# tar cvfz $(files)_$(date --date='-6months' +%F).tar.gz $files
# rm $files
done
First, the -mtime argument does not get you files that are "older" than a certain amount. Rather, it checks the last time the file was modified. The creation date of files is not kept in most file systems. Often, the last modified time is sufficient, but it is not the same as the age of the file.
If you just want to create a single tar file for each archive, use -exec instead of passing the data to xargs:
find /tmp/log/ -mtime +180 -type f -exec sh -c \
'tar -czvPf /tmp/older_log_$(basename $0)_$(date +%F).tar.gz $0' {} \;
I have hundreds of directories and files in one directory.
What is the best way deleting only directories (no matter if the directories have anything in it or not, just delete them all)
Currently I use ls -1 -d */, and record them in a file, and do sed, and then run it. It rather long way. I'm looking for better way deleting only directories
To delete all directories and subdirectories and leave only files in the working directory, I have found this concise command works for me:
rm -r */
It makes use of bash wildcard */ where star followed by slash will match only directories and subdirectories.
find . -maxdepth 1 -mindepth 1 -type d
then
find . -maxdepth 1 -mindepth 1 -type d -exec rm -rf '{}' \;
To add an explanation:
find starts in the current directory due to . and stays within the current directory only with -maxdepth and -mindepth both set to 1. -type d tells find to only match on things that are directories.
find also has an -exec flag that can pass its results to another function, in this case rm. the '{}' \; is the way these results are passed. See this answer for a more complete explanation of what {} and \; do
First, run:
find /path -d -type d
to make sure the output looks sane, then:
find /path -d -type d -exec rm -rf '{}' \;
-type d looks only for directories, then -d makes sure to put child directories before the parent.
Simple way :-
rm -rf `ls -d */`
find command only (it support file deletion)\
find /path -depth -type d -delete
-type d looks only for directories, then -depth makes sure to put child directories before the parent. -delete removing filtered files/folders
In one line:
rm -R `ls -1 -d */`
(backquotes)