How to use rsync selecting no file - rsync

I would like to use the --delete capability of rsync to erase everyting on a destination directory.
What I am currently doing is:
rsync -zavr --delete ./input_dir_1/ user#host:dest_dir/
rsync -zavr ./input_dir_2/ user#host:dest_dir/
rsync -zavr ./input_dir_3/ user#host:dest_dir/
But in order to use rsync in a loop, is there a way to first select no input file (and use --delete to just delete everything)? Ideally I would like something like:
# Delete everything on destination dir
rsync -zavr --delete /dev/null user#host:dest_dir/
# Now iterate over all input directories (could use a for loop)
rsync -zavr ./input_dir_1/ user#host:dest_dir/
rsync -zavr ./input_dir_2/ user#host:dest_dir/
rsync -zavr ./input_dir_3/ user#host:dest_dir/


rsync uses ssh. Why not delete directly? (Be careful not to delete the wrong directory!)
ssh user#host 'rm -rf dest_dir; mkdir dest_dir'
You can use the exclude feature of rsync:
rsync -r --exclude='*' --delete-excluded / user#host:dest_dir/
Any folder will work for the source.

Related

ignore subdirectories timestamps when syncing from shell

i want to write a shell command to sync current directory to backup directory with some requirments. the command i'm using is:
rsync -ptvHS --progress --delete-after --exclude /backup $pwd ~/backup
i want the directory timestamps to be ignored, eventhough i use -t to preserve the file timestamps.
Any idea?
thank you in advance

From the man page:
-t, --times preserve modification times
-O, --omit-dir-times omit directories from --times
-J, --omit-link-times omit symlinks from --times
Seems like you need to add -O to your command.
This is from rsync 3.1.2; you might find your version is too old.

How to rsync web repo

I want to rsync everything in mirrors.kernel.org/centos/6/updates/x86_64/Packages/ to a directory on my server. I do NOT want to wind up with a directory structure like ~/mirrors.kernel.org/centos/6/updates/x86_64/Packages/
[joliver#lake ~]$ rsync mirrors.kernel.org/centos/6/updates/x86_64/Packages/ CentOS6/
rsync: change_dir "/home/joliver/mirrors.kernel.org/centos/6/updates/x86_64/Packages" failed: No such file or directory (2)
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1039) [sender=3.0.6]
[joliver#lake ~]$ mkdir -p mirrors.kernel.org/centos/6/updates/x86_64/Packages
[joliver#lake ~]$ rsync mirrors.kernel.org/centos/6/updates/x86_64/Packages/ mirrors.kernel.org/centos/6/updates/x86_64/Packages
skipping directory .[joliver#lake ~]$ rsync mirrors.kernel.org/centos/6/updates/x86_64/Packages/ mirrors.kernel.org/centos/6/updates/x86_64/Packages/
skipping directory .
After I have that, I'd like to be able to rsync just the deltas. I suppose just redoing the rsync and then finding all files with a ctime of less that since the last run would suffice, but it would be nice if there was a neater way to grab new / changed files.

rsync -avSHP --delete --exclude "local*" --exclude "isos" \
mirrors.kernel.org::centos/6/updates/x86_64/Packages/ CentOS6/

Prevent rsync from deleting destination files that match a given pattern

I'm using rsync to sync files from a source to a destination:
rsync -av --delete source destination
I have a single directory on the destination side which is not on the source side. I'd like to prevent rsync from deleting this directory. Is there an option I can pass to rsync to prevent this directory from being deleted upon sync?

You can exclude files/directories with --exclude. This will prevent the somedir directory from being synced/deleted:
rsync -avrc --delete --exclude somedir source destination

As mentioned in a similar question, this can be accomplished by using the --filter option with protect rule:
$ rsync ... --filter 'protect /remote-directory-to-keep/' ...
Unlike the currently accepted answer, using --filter is useful, for instance, if you also wish to use --exclude with --delete-excluded.

rsync - create all missing parent directories?

I'm looking for an rsync-like program which will create any missing parent directories on the remote side.
For example, if I have /top/a/b/c/d on one server and only /top/a exists on the remote server, I want to copy d to the remote server and have the b and c directories created as well.
The command:
rsync /top/a/b/c/d remote:/top/a/b/c
won't work because /tmp/a/b doesn't exist on the remote server. And if it did exist then the file d would get copied to the path /top/a/b/c.
This is possible to do with rsync using --include and --exclude switches, but it is very involved, e.g.:
rsync -v -r a dest:dir \
--include 'a/b' \
--include 'a/b/c' \
--include 'a/b/c/d' \
--include 'a/b/c/d/e' \
--exclude 'a/*' \
--exclude 'a/b/*' \
--exclude 'a/b/c/*' \
--exclude 'a/b/c/d/*'
will only copy a/b/c/d/e to dest:dir/a/b/c/d/e even if the intermediate directories have files. (Note - the includes must precede the excludes.)
Are there any other options?

You may be looking for
rsync -aR
for example:
rsync -a --relative /top/a/b/c/d remote:/
See also this trick in other question.

rsync -aq --rsync-path='mkdir -p /tmp/imaginary/ && rsync' file user#remote:/tmp/imaginary/
From http://www.schwertly.com/2013/07/forcing-rsync-to-create-a-remote-path-using-rsync-path/, but don't copy and paste from there, his syntax is butchered.
it lets you execute arbitrary command to setup the path for rsync executables.

As of version 3.2.3 (6 Aug 2020), rynsc has a flag for this purpose.
From the rsync manual page (man rsync):
--mkpath create the destination's path component

i suggest that you enforce the existence manually:
ssh user#remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
this creates the target folder if it does not exists already.

According to https://unix.stackexchange.com/a/496181/5783, since rsync 2.6.7, --relative works if you use . to anchor the starting parent directory to create at the destination:
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top1/a/b/c/d
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top2/a
derek#DESKTOP-2F2F59O:~/projects/rsync$ rsync --recursive --relative --verbose top1/a/./b/c/d top2/a/
sending incremental file list
b/
b/c/
b/c/d/
sent 99 bytes received 28 bytes 254.00 bytes/sec
total size is 0 speedup is 0.00

--relative does not work for me since I had different setup.
Maybe I just didn't understood how --relative works, but I found that the
ssh remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
is easy to understand and does the job.

I was looking for a better solution, but mine seems to be better suited when you have too many sub-directories to create them manually.
Simply use cp as an intermediate step with the --parents option
cp --parents /your/path/sub/dir/ /tmp/localcopy
rsync [options] /tmp/localcopy/* remote:/destination/path/
cp --parents will create the structure for you.
You can call it from any subfolder if you want only one subset of the parent folders to be copied.

A shorter way in Linux to create rsync destination paths is to use the '$_' Special Variable. (I think, but cannot confirm, that it is also the same in OSX).
'$_' holds the value of the last argument of the previous command executed. So the question could be answered with:
ssh remote mkdir -p /top/a/b/c/ && rsync -avz /top/a/b/c/d remote:$_

scp to rsync conversion

The following is working as expected.
scp -o IdentityFile=/home/companyuser/.ssh/id_dsa_fner {} companyuser#14.140.100.189:/home/fner/
But the rsync version of the same command does not work:
rsync -av -o IdentityFile=/home/companyuser/.ssh/id_dsa_fner /home/companyuser/ companycuser#14.140.100.189:/home/fner/
I use find -exec >> scp to copy files in the first example. I can copy all the files in one location.
But rsync will allow me to have the same directory structure on destination exactly like master host.

you might try specifying the transfer type with -e
rsync -avze "ssh -o IdentityFile=/path/to/file" /sync/here/ root#remotehost:/to/here

Resources