When using rsync, it is possible to create the target directory on the server using the --rsync-path trick as follows:
rsync -av -e "ssh" --rsync-path "mkdir -p /home/user/new/new && rsync" ./file.txt user#10.0.2.60:/home/user/new/new
This however does not seem to work when using an ssh tunnel. The following command just hangs:
rsync -av -e "ssh -A user#10.0.2.61 ssh" --rsync-path "mkdir -p /home/user/new/new && rsync" ./file.txt user#10.0.2.60:/home/user
I have verified the last command works if I remove the --rsync-path argument and create the directory manually on the target device. But how to make rsync create the missing directory when using ssh tunneling?
Managed to solve it. The command inside --rsync-path must be wrapped in another level of quotes:
rsync -av -e "ssh -A user#10.0.2.61 ssh" --rsync-path "'mkdir -p /home/user/new/new && rsync'" ./file.txt user#10.0.2.60:/home/user
I want to exclude one special hidden file in just one special folder.
The command I used is:
rsync -a --delete \
--exclude='/absolute/path/to/webpage/folder1' \
--exclude='/absolute/path/to/webpage/backups' \
--exclude='/absolute/path/to/webpage/.htaccess' \
/absolute/path/to/webpage/ \
/absolute/path/to/copy_of_webpage &>/dev/null
rsync always overwrites my .htaccess.
Also I want to keep my .htpasswd and I thought about using wildcards like:
rsync -a --delete \
--exclude='/absolute/path/to/webpage/folder1' \
--exclude='/absolute/path/to/webpage/backups' \
--exclude='/absolute/path/to/webpage/.ht*' \
/absolute/path/to/webpage/ \
/absolute/path/to/copy_of_webpage &>/dev/null
But that doesn't work either.
You could exclude all .htaccess with --exclude '.htaccess'
Exclude the path relative to the source folder, not the absolute path.
If your root (as above) is:
/absolute/path/to/webpage/
and you wish to exclude:
/absolute/path/to/webpage/.htaccess
/absolute/path/to/webpage/backups
then you'll need to say:
--exclude='/.htaccess' --exclude='/backups'
Per the docs:
"/foo" would match a file called "foo" at... the "root of the transfer"
I'm looking for an rsync-like program which will create any missing parent directories on the remote side.
For example, if I have /top/a/b/c/d on one server and only /top/a exists on the remote server, I want to copy d to the remote server and have the b and c directories created as well.
The command:
rsync /top/a/b/c/d remote:/top/a/b/c
won't work because /tmp/a/b doesn't exist on the remote server. And if it did exist then the file d would get copied to the path /top/a/b/c.
This is possible to do with rsync using --include and --exclude switches, but it is very involved, e.g.:
rsync -v -r a dest:dir \
--include 'a/b' \
--include 'a/b/c' \
--include 'a/b/c/d' \
--include 'a/b/c/d/e' \
--exclude 'a/*' \
--exclude 'a/b/*' \
--exclude 'a/b/c/*' \
--exclude 'a/b/c/d/*'
will only copy a/b/c/d/e to dest:dir/a/b/c/d/e even if the intermediate directories have files. (Note - the includes must precede the excludes.)
Are there any other options?
You may be looking for
rsync -aR
for example:
rsync -a --relative /top/a/b/c/d remote:/
See also this trick in other question.
rsync -aq --rsync-path='mkdir -p /tmp/imaginary/ && rsync' file user#remote:/tmp/imaginary/
From http://www.schwertly.com/2013/07/forcing-rsync-to-create-a-remote-path-using-rsync-path/, but don't copy and paste from there, his syntax is butchered.
it lets you execute arbitrary command to setup the path for rsync executables.
As of version 3.2.3 (6 Aug 2020), rynsc has a flag for this purpose.
From the rsync manual page (man rsync):
--mkpath create the destination's path component
i suggest that you enforce the existence manually:
ssh user#remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
this creates the target folder if it does not exists already.
According to https://unix.stackexchange.com/a/496181/5783, since rsync 2.6.7, --relative works if you use . to anchor the starting parent directory to create at the destination:
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top1/a/b/c/d
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top2/a
derek#DESKTOP-2F2F59O:~/projects/rsync$ rsync --recursive --relative --verbose top1/a/./b/c/d top2/a/
sending incremental file list
b/
b/c/
b/c/d/
sent 99 bytes received 28 bytes 254.00 bytes/sec
total size is 0 speedup is 0.00
--relative does not work for me since I had different setup.
Maybe I just didn't understood how --relative works, but I found that the
ssh remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
is easy to understand and does the job.
I was looking for a better solution, but mine seems to be better suited when you have too many sub-directories to create them manually.
Simply use cp as an intermediate step with the --parents option
cp --parents /your/path/sub/dir/ /tmp/localcopy
rsync [options] /tmp/localcopy/* remote:/destination/path/
cp --parents will create the structure for you.
You can call it from any subfolder if you want only one subset of the parent folders to be copied.
A shorter way in Linux to create rsync destination paths is to use the '$_' Special Variable. (I think, but cannot confirm, that it is also the same in OSX).
'$_' holds the value of the last argument of the previous command executed. So the question could be answered with:
ssh remote mkdir -p /top/a/b/c/ && rsync -avz /top/a/b/c/d remote:$_
Suppose I want to rsync file foo.txt on my local machine to file /home/me/somedirectory/bar.txt on a remote computer, and that somedirectory/ doesn't yet exist. How do I do this?
I tried rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/bar.txt, but I get a rsync: push_dir#3 "/home/me/somedirectory" failed: No such file or directory (2) error.
(Copying the file without renaming it works, though. That is, this runs fine: rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/`)
Just put a trailing slash on your target dir. Something like this:
rsync foo.txt remotemachine:somedirectory/
Assuming that "/home/me" is your home dir on the remote machine, there is no need to specify it in the command line. Also, you don't need to clutter up your rsync with the -e unless you just like to do that.
You can do this process successfully in 2 stepes:-
1] rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/
this will copy the foo.txt and create directory somedirectory on destination.
then
2] rsync -e ssh -z --delete-after foo.txt remotemachine:/home/me/somedirectory/bar.txt
and here you can delete foo.txt on destination by using --delete-after option.
you can see it's usage from man pages. This option must be used with -r option
This serves your purpose.
or if second command doesn't work then use :-
rsync -e ssh -z foo.txt remotemachine:/home/me/somedirectory/bar.txt
and delete foo.txt manually.
The following is working as expected.
scp -o IdentityFile=/home/companyuser/.ssh/id_dsa_fner {} companyuser#14.140.100.189:/home/fner/
But the rsync version of the same command does not work:
rsync -av -o IdentityFile=/home/companyuser/.ssh/id_dsa_fner /home/companyuser/ companycuser#14.140.100.189:/home/fner/
I use find -exec >> scp to copy files in the first example. I can copy all the files in one location.
But rsync will allow me to have the same directory structure on destination exactly like master host.
you might try specifying the transfer type with -e
rsync -avze "ssh -o IdentityFile=/path/to/file" /sync/here/ root#remotehost:/to/here