I'm using rsync to sync files from a source to a destination:
rsync -av --delete source destination
I have a single directory on the destination side which is not on the source side. I'd like to prevent rsync from deleting this directory. Is there an option I can pass to rsync to prevent this directory from being deleted upon sync?
You can exclude files/directories with --exclude. This will prevent the somedir directory from being synced/deleted:
rsync -avrc --delete --exclude somedir source destination
As mentioned in a similar question, this can be accomplished by using the --filter option with protect rule:
$ rsync ... --filter 'protect /remote-directory-to-keep/' ...
Unlike the currently accepted answer, using --filter is useful, for instance, if you also wish to use --exclude with --delete-excluded.
Related
I want to develop a Rsync script to recursively copy the list of files/folders that are created on one server to another server. I'm using the below command to copy a particular file. However whether we can include a list of files that should be ignored while copying the files from source to destination.
rsync -av --include /sourcepath/* --exclude filename user#destinationserver:/destinationpath
We can refer [1] [1]: https://www.tutorialspoint.com/unix_commands/rsync.html
rsync -lapv --include /sourcecpath/* --exclude-from exclude-list-one user#destinationserver:/destinationpath
-l, --links copy symlinks as symlinks
-a, --archive archive mode; same as -rlptgoD (no -H)
-v, --verbose increase verbosity
-p, --perms preserve permissions
--exclude-from=FILE read exclude patterns from FILE
i want to write a shell command to sync current directory to backup directory with some requirments. the command i'm using is:
rsync -ptvHS --progress --delete-after --exclude /backup $pwd ~/backup
i want the directory timestamps to be ignored, eventhough i use -t to preserve the file timestamps.
Any idea?
thank you in advance
From the man page:
-t, --times preserve modification times
-O, --omit-dir-times omit directories from --times
-J, --omit-link-times omit symlinks from --times
Seems like you need to add -O to your command.
This is from rsync 3.1.2; you might find your version is too old.
I'm looking for an rsync-like program which will create any missing parent directories on the remote side.
For example, if I have /top/a/b/c/d on one server and only /top/a exists on the remote server, I want to copy d to the remote server and have the b and c directories created as well.
The command:
rsync /top/a/b/c/d remote:/top/a/b/c
won't work because /tmp/a/b doesn't exist on the remote server. And if it did exist then the file d would get copied to the path /top/a/b/c.
This is possible to do with rsync using --include and --exclude switches, but it is very involved, e.g.:
rsync -v -r a dest:dir \
--include 'a/b' \
--include 'a/b/c' \
--include 'a/b/c/d' \
--include 'a/b/c/d/e' \
--exclude 'a/*' \
--exclude 'a/b/*' \
--exclude 'a/b/c/*' \
--exclude 'a/b/c/d/*'
will only copy a/b/c/d/e to dest:dir/a/b/c/d/e even if the intermediate directories have files. (Note - the includes must precede the excludes.)
Are there any other options?
You may be looking for
rsync -aR
for example:
rsync -a --relative /top/a/b/c/d remote:/
See also this trick in other question.
rsync -aq --rsync-path='mkdir -p /tmp/imaginary/ && rsync' file user#remote:/tmp/imaginary/
From http://www.schwertly.com/2013/07/forcing-rsync-to-create-a-remote-path-using-rsync-path/, but don't copy and paste from there, his syntax is butchered.
it lets you execute arbitrary command to setup the path for rsync executables.
As of version 3.2.3 (6 Aug 2020), rynsc has a flag for this purpose.
From the rsync manual page (man rsync):
--mkpath create the destination's path component
i suggest that you enforce the existence manually:
ssh user#remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
this creates the target folder if it does not exists already.
According to https://unix.stackexchange.com/a/496181/5783, since rsync 2.6.7, --relative works if you use . to anchor the starting parent directory to create at the destination:
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top1/a/b/c/d
derek#DESKTOP-2F2F59O:~/projects/rsync$ mkdir --parents top2/a
derek#DESKTOP-2F2F59O:~/projects/rsync$ rsync --recursive --relative --verbose top1/a/./b/c/d top2/a/
sending incremental file list
b/
b/c/
b/c/d/
sent 99 bytes received 28 bytes 254.00 bytes/sec
total size is 0 speedup is 0.00
--relative does not work for me since I had different setup.
Maybe I just didn't understood how --relative works, but I found that the
ssh remote mkdir -p /top/a/b/c
rsync /top/a/b/c/d remote:/top/a/b/c
is easy to understand and does the job.
I was looking for a better solution, but mine seems to be better suited when you have too many sub-directories to create them manually.
Simply use cp as an intermediate step with the --parents option
cp --parents /your/path/sub/dir/ /tmp/localcopy
rsync [options] /tmp/localcopy/* remote:/destination/path/
cp --parents will create the structure for you.
You can call it from any subfolder if you want only one subset of the parent folders to be copied.
A shorter way in Linux to create rsync destination paths is to use the '$_' Special Variable. (I think, but cannot confirm, that it is also the same in OSX).
'$_' holds the value of the last argument of the previous command executed. So the question could be answered with:
ssh remote mkdir -p /top/a/b/c/ && rsync -avz /top/a/b/c/d remote:$_
I would like to rsync from local computer to server. On a directory that does not exist, and I want rsync to create that directory on the server first.
How can I do that?
If you have more than the last leaf directory to be created, you can either run a separate ssh ... mkdir -p first, or use the --rsync-path trick as explained here :
rsync -a --rsync-path="mkdir -p /tmp/x/y/z/ && rsync" $source user#remote:/tmp/x/y/z/
Or use the --relative option as suggested by Tony. In that case, you only specify the root of the destination, which must exist, and not the directory structure of the source, which will be created:
rsync -a --relative /new/x/y/z/ user#remote:/pre_existing/dir/
This way, you will end up with /pre_existing/dir/new/x/y/z/
And if you want to have "y/z/" created, but not inside "new/x/", you can add ./ where you want --relativeto begin:
rsync -a --relative /new/x/./y/z/ user#remote:/pre_existing/dir/
would create /pre_existing/dir/y/z/.
From the rsync manual page (man rsync):
--mkpath create the destination's path component
--mkpath was added in rsync 3.2.3 (6 Aug 2020).
Assuming you are using ssh to connect rsync, what about to send a ssh command before:
ssh user#server mkdir -p existingdir/newdir
if it already exists, nothing happens
The -R, --relative option will do this.
For example: if you want to backup /var/named/chroot and create the same directory structure on the remote server then -R will do just that.
this worked for me:
rsync /dev/null node:existing-dir/new-dir/
I do get this message :
skipping non-regular file "null"
but I don't have to worry about having an empty directory hanging around.
I don't think you can do it with one rsync command, but you can 'pre-create' the extra directory first like this:
rsync --recursive emptydir/ destination/newdir
where 'emptydir' is a local empty directory (which you might have to create as a temporary directory first).
It's a bit of a hack, but it works for me.
cheers
Chris
This answer uses bits of other answers, but hopefully it'll be a bit clearer as to the circumstances. You never specified what you were rsyncing - a single directory entry or multiple files.
So let's assume you are moving a source directory entry across, and not just moving the files contained in it.
Let's say you have a directory locally called data/myappdata/ and you have a load of subdirectories underneath this.
You have data/ on your target machine but no data/myappdata/ - this is easy enough:
rsync -rvv /path/to/data/myappdata/ user#host:/remote/path/to/data/myappdata
You can even use a different name for the remote directory:
rsync -rvv --recursive /path/to/data/myappdata user#host:/remote/path/to/data/newdirname
If you're just moving some files and not moving the directory entry that contains them then you would do:
rsync -rvv /path/to/data/myappdata/*.txt user#host:/remote/path/to/data/myappdata/
and it will create the myappdata directory for you on the remote machine to place your files in. Again, the data/ directory must exist on the remote machine.
Incidentally, my use of -rvv flag is to get doubly verbose output so it is clear about what it does, as well as the necessary recursive behaviour.
Just to show you what I get when using rsync (3.0.9 on Ubuntu 12.04)
$ rsync -rvv *.txt user#remote.machine:/tmp/newdir/
opening connection using: ssh -l user remote.machine rsync --server -vvre.iLsf . /tmp/newdir/
user#remote.machine's password:
sending incremental file list
created directory /tmp/newdir
delta-transmission enabled
bar.txt
foo.txt
total: matches=0 hash_hits=0 false_alarms=0 data=0
Hope this clears this up a little bit.
eg:
from: /xxx/a/b/c/d/e/1.html
to: user#remote:/pre_existing/dir/b/c/d/e/1.html
rsync:
cd /xxx/a/ && rsync -auvR b/c/d/e/ user#remote:/pre_existing/dir/
rsync source.pdf user1#192.168.56.100:~/not-created/target.pdf
If the target file is fully specified, the directory ~/not-created is not created.
rsync source.pdf user1#192.168.56.100:~/will-be-created/
But the target is specified with only a directory, the directory ~/will-be-created is created. / must be followed to let rsync know will-be-created is a directory.
use rsync twice~
1: tranfer a temp file, make sure remote relative directories has been created.
tempfile=/Users/temp/Dir0/Dir1/Dir2/temp.txt
# Dir0/Dir1/Dir2/ is directory that wanted.
rsync -aq /Users/temp/ rsync://remote
2: then you can specify the remote directory for transfer files/directory
tempfile|dir=/Users/XX/data|/Users/XX/data/
rsync -avc /Users/XX/data rsync://remote/Dir0/Dir1/Dir2
# Tips: [SRC] with/without '/' is different
This creates the dir tree /usr/local/bin in the destination and then syncs all containing files and folders recursively:
rsync --archive --include="/usr" --include="/usr/local" --include="/usr/local/bin" --include="/usr/local/bin/**" --exclude="*" user#remote:/ /home/user
Compared to mkdir -p, the dir tree even has the same perms as the source.
If you are using a version or rsync that doesn't have 'mkpath', then --files-from can help. Suppose you need to create 'mysubdir' in the target directory
Create 'filelist.txt' to contain
mysubdir/dummy
mkdir -p source_dir/mysubdir/
touch source_dir/mysubdir/dummy
rsync --files-from='filelist.txt' source_dir target_dir
rsync will copy mysubdir/dummy to target_dir, creating mysubdir in the process. Tested with rsync 3.1.3 on Raspberry Pi OS (debian).
File foo.txt exists on the remote machine at: /home/user/foo.txt
It doesn't exist on the local machine.
I want to delete foo.txt using rsync.
I do not know (and assume for the purposes of this question that I cannot find out) what other files are in /home/user on either the local or remote machines, so I can't just sync the whole directory.
What rsync command can I use to delete foo.txt on the remote machine?
Try this:
rsync -rv --delete --include=foo.txt '--exclude=*' /home/user/ user#remote:/home/user/
(highly recommend running with --dry-run first to test it) Although it seems like it would be easier to use ssh...
ssh user#remote "rm /home/user/foo.txt"
That's a bit trivial, but if, like me, you came to this page looking for a way to delete the content of a directory from remote server using rsync, this is how I did it:
Create an empty mock folder:
mkdir mock
Sync with it:
rsync -arv --delete --dry-run ~/mock/ remote_server:~/dir_to_clean/
Remove --dry-run from the line above to actually do the thing.
As suggested above, use --dry-run to test prior. --delete deletes files on the remote location per the rsync man page.
rsync -rv --delete user#hostname.local:full/path/to/foo.txt
Comment below stating this will list only is incorrect. To list only use --list-only and remove --delete.
Just came across the same problem, needed to use rsync to delete a remote file, as only rsync and no other SSH commands were allowed. The --remove-source-files option (formerly known as --remove-sender-files) did exactly that:
rsync -avPn --remove-source-files remote:/home/user/foo.txt .
rm foo.txt
As always, remove the -n option to really execute this.