Rsync with delete option but excluding subfolders - rsync

I need to sync a local directory (destination) from the remote server directory (source) with the --delete option.
I've got this:
Local directory is /home/user/config and into it
removeit.txt
bar.csv
foo.h
config.conf
scripts <- Is a directory
logs <- Is adirectory
The remote directory is config and into it
bar.csv
foo.h
I want in the local directory after rsync
bar.csv
foo.h
config.conf
scripts <- Is a directory
logs <- Is adirectory
I've tested with multiple options but I can't get a entire rsync command with my needs.
rsync -avz --min-size=1 --delete -e "ssh" user#example.com:./config/ ./config --exclude ./config/scripts --exclude ./config/logs --exclude ./config/device.conf --dry-run
With a list of include files
rsync -avz --min-size=1 --include-from=list --exclude=* --delete-excluded -e "ssh" user#example.com:.config/ config/
But nothing works as I expected. The subfolders in the destination are deleted.

The man page for rsync has a section on "Include/Exclude Pattern Rules" which you should review. In particular, it looks like you are using ./ to refer to the directory relative of your shell's working directory which won't work the way you want.
Here is a relevant section of the documentation:
if the pattern starts with a / then it is anchored to a particular
spot in the hierarchy of files, otherwise it is matched against the
end of the pathname. This is similar to a leading ^ in regular
expressions. Thus /foo would match a name of "foo" at either the "root
of the transfer" (for a global rule) or in the merge-file's directory
(for a per-directory rule). An unqualified foo would match a name of
"foo" anywhere in the tree because the algorithm is applied
recursively from the top down; it behaves as if each path component
gets a turn at being the end of the filename. Even the unanchored
"sub/foo" would match at any point in the hierarchy where a "foo" was
found within a directory named "sub". See the section on ANCHORING
INCLUDE/EXCLUDE PATTERNS for a full discussion of how to specify a
pattern that matches at the root of the transfer.
The pattern you are using might be causing rsync to look for a folder literally called . which contains a file or folder called scripts or whatever.
Try replacing the relative-ish ./config/ part with / in your exclude patterns. That will anchor the pattern in the "root of transfer" which is the directory you are synchronizing. In other words, try this:
rsync -avz --min-size=1 --delete -e "ssh" user#example.com:./config/ ./config --exclude /scripts --exclude /logs --exclude /device.conf --dry-run
I hope that works. If it's not too late to check, please let me know if this worked for you.

Related

MacOS: xargs cp does not copy subdirectories

I am on Mac OS.
I have a directory with round about 3000 files and several subdirectories (wordpress installation)
Now I have to find all the files in a similar directory (have to separate master and child installation) that are additional files and have to copy them away into another directory.
I use this command:
$ diff -rq dt-the7 dt-the7-master-from-Yana|grep 'Only in dt-the7'|awk {' print $3 $4 '}|sed 's/:/\//g'|xargs -J {} rsync -av {} neu/
but somehow a certain file 3d.png and a list of other that should be in a subdir of the destination dir are copied into the root dir of the destination.
Any idea why that might be?
It makes no difference whether I use cp, rsync or ditto
You need the -R relative option on your rsync command.
Without this rsync just copies the item referenced rather than the path referenced, so items at the root level are copied as you expected but items in sub-directories are also copied to the root, which is not what you wanted.
With the option rsync takes account of the relative path and recreates it at the destination.
An example with another command might help, consider:
cp A/B.txt C/
that will copy B.txt into C, it does not create a folder A in C which in turn contains the file B.txt. rsync without -R behaves like that cp command, with -R it creates the A directory in C.
HTH

How to exclude wildcard directory but include wildcard files using rdiff-backup?

I am using rdiff-backup. Really awesome simple powerful backup tool. However I am fighting with wildcard glob patterns. I have this directory structure:
/data/aaa/cache
/data/bbb/cache
/data/ccc/cache
etc....
In each cache directory are original files and cache files. Original files are named simply 1.jpg, 2.png, 3.gif, and so on. Cache files have some string attached to the original filename.
So I want to backup all the /data/*/cache directories, but to include only original files, not the cache files.
I am using this command:
rdiff-backup --exclude **/cache --include **/cache/+([0-9]).+([a-z]) /data /backup
But rdiff-backup returns this and I am lost:
Found interrupted initial backup. Removing...
Fatal Error: Last selection expression:
Command-line include glob: **/cache/+([0-9]).+([a-z])
only specifies that files be included. Because the default is to
include all files, the expression is redundant. Exiting because this
probably isn't what you meant.
You might want to do a two step process:
create a list of all files you want to exclude e.g. with find . -name "**/cache" > excludes.lst
use the list with --exclude-filelist excludes.lst
This way you can avoid fighting with the glob option and you have full control over your excludes
From http://rdiff-backup.nongnu.org/rdiff-backup.1.html :
A given file is excluded by the file selection system exactly
when the first matching file selection condition
specifies that the file be excluded; otherwise the file is included.
...
For instance,
rdiff-backup --include /usr --exclude /usr /usr /backup
is exactly the same as
rdiff-backup /usr /backup
because the include and exclude directives match exactly the same
files, and the --include comes first, giving it precedence.
So, in your case, it is complaining about the final --include because if a file gets there (i.e. it isn't matched by the previous --exclude) it will be included whether or not it matches the --include. That's what the error message was trying to say.
As for how to accomplish your goal...
Assuming you do want to exclude only paths of the form: /data/*/cache/[0-9]*.[a-z][a-z][a-z]?* just specify that:
rdiff-backup --exclude '/data/*/cache/[0-9]*.[a-z][a-z][a-z]?*' --exclude '*' /data /backup
This should work (I haven't tested it).

Delete over rsync using ssh is not working

I'm doing an rsync trial where I have two files in the current folder:
share_2014_09_08.tar.gz share_2014_10_08.tar.gz
I want to rsync to a remote folder that contains three older files. I use the command:
rsync -avz --del ./*.tar.gz backups#pc01:/home/backups/monthly/
And the result int the destination folder is:
share_2014_03_05.tar.gz share_2014_09_08.tar.gz share_2014_10_08.tar.gz
As I understand it, this file:
share_2014_03_05.tar.gz
should have been deleted, so my question is what am I doing wrong.
You're passing to rsync a list of files that you want to synchronize. Not existing files are not passed, so delete option has no effects.
If you want to delete files, you'll have to synchronize the parent directory that contained removed files. You can use a include mask to only sync tarballs:
rsync -avz --include "*.tar.gz" --exclude "*" --del . backups#pc01:/home/backups/monthly/

How to exclude symbolic links on the web server?

When pushing to the server from our local machines, we'd like to exclude (or not overwrite) directories used by awstats.
Is it possible to get rsync to replace everything on the server but ignore two symbolic links and a directory?
awstats-icon -> icon
awstatsicons -> icon
/icon
Our current rsync call looks like this where we're excluding .git from our local machines:
rsync -vzrP --delete --exclude '.git' ~/Sites/path/to/sitename/ user#server.com:/path/to/sitename/
Any pointers in the right direction would be much appreciated.
Would it work for you to also exclude the symbolic links and the directory?
rsync -vzrP --delete --exclude '.git' --exclude 'awstats-icon' --exclude 'awstatsicons' --exclude 'icon' ~/Sites/path/to/sitename/ user#server.com:/path/to/sitename/
This would prevent pushing the symbolic links and directory, while preserving (not deleting) the remote symbolic links or directories with the same names.

How to preserve file permissions with cmake "install directory" directive?

Prolog: I'm an idiot for missing this in the documentation
cmake-2.8.10.2
How do you make cmake preserve the original file permissions when installing a directory? For the project at hand, I'd like it to essentially copy some directories from my source tree to the install tree. To wit:
install(
DIRECTORY config runp
DESTINATION ${CMAKE_INSTALL_PREFIX}
PATTERN ".svn" EXCLUDE
PATTERN ".git" EXCLUDE
PATTERN "start_collection.snl" EXCLUDE
)
All works as expected -- except that executable scripts are getting copied in with incorrect file permissions. In fact, none of the original file permissions are preserved. Globally setting permissions using FILE_PERMISSIONS and DIRECTORY_PERMISSIONS is something I do not want to do, and frankly, would be a hack in this context.
In the shell-scripting world, I'd do something simple like this:
for i in config runp ; do
tar cf - $i | tar -C $CMAKE_INSTALL_PREFIX -xf -
done
Documentation suggests using USE_SOURCE_PERMISSIONS when calling install():
install(
DIRECTORY config runp
DESTINATION ${CMAKE_INSTALL_PREFIX}
USE_SOURCE_PERMISSIONS
PATTERN ".svn" EXCLUDE
PATTERN ".git" EXCLUDE
PATTERN "start_collection.snl" EXCLUDE
)
Alternatively, you can use install(PROGRAMS signature of this command. See docs for more info.

Resources