selecting files from directory using rsync - rsync

I want to download some files from a server using rsync. I am able to download (e.g. "00/")a complete directory using the code: `
rsync -rlptzv --include '00' --exclude '*/' --delete --port=33444 rsync.wwpdb.org::ftp/data/biounit/PDB/divided/ /scratch/
I am getting stuck when I try to download a file (e.g. 100d.pdb1.gz) from the "00" folder. I have tried adding the file name as full to the path on the server but that did not work:
rsync -rlptzv --include '00/100d.pdb1.gz' --exclude '*/' --delete --port=33444 rsync.wwpdb.org::ftp/data/biounit/PDB/divided/ /scratch/pdb_download/datasets/pdb/rcsb/

Related

I want to develop a Rsync script to recursively copy the list of files/folders that are created on one server to another server

I want to develop a Rsync script to recursively copy the list of files/folders that are created on one server to another server. I'm using the below command to copy a particular file. However whether we can include a list of files that should be ignored while copying the files from source to destination.
rsync -av --include /sourcepath/* --exclude filename user#destinationserver:/destinationpath
We can refer [1] [1]: https://www.tutorialspoint.com/unix_commands/rsync.html
rsync -lapv --include /sourcecpath/* --exclude-from exclude-list-one user#destinationserver:/destinationpath
-l, --links copy symlinks as symlinks 
-a, --archive archive mode; same as -rlptgoD (no -H) 
-v, --verbose increase verbosity 
-p, --perms preserve permissions 
--exclude-from=FILE read exclude patterns from FILE 

Rsync exclude correct syntax

I want to copy a directory from remote machine to local using rsync, but without some inner folders.
I'm using this command:
rsync -rave --exclude 'js' --exclude 'css' --exclude 'fonts' root#{IP}:/rem_dir1/rem_dir2/public /local_dir1/local_dir2/public
But result of it is:
Unexpected remote arg: root#{IP}:/rem_dir1/rem_dir2/public
rsync error: syntax or usage error (code 1) at main.c(1361) [sender=3.1.2]
I'm sure remote root is correct. So the problem is in rsync command syntax.
What is the correct way to exclude several folders using rsync?
For example we have /public folder which contains dir1, dir2, dir3, dir4 and dir5. How to copy only dir1 and dir2 from /public?
As with your other question, the -rave makes no sense. You want just -av.
You can get fancy with include and exclude commands, but the easiest way to copy just two directories is just to list them:
rsync -av \
root#{IP}:/rem_dir1/rem_dir2/public/dir1 \
root#{IP}:/rem_dir1/rem_dir2/public/dir2 \
/local_dir1/local_dir2/public/
where \ is just line-continuation (so I can wrap the long line), and I deliberately only added / to the end of the destination path, not the source paths.

ignore subdirectories timestamps when syncing from shell

i want to write a shell command to sync current directory to backup directory with some requirments. the command i'm using is:
rsync -ptvHS --progress --delete-after --exclude /backup $pwd ~/backup
i want the directory timestamps to be ignored, eventhough i use -t to preserve the file timestamps.
Any idea?
thank you in advance
From the man page:
-t, --times preserve modification times
-O, --omit-dir-times omit directories from --times
-J, --omit-link-times omit symlinks from --times
Seems like you need to add -O to your command.
This is from rsync 3.1.2; you might find your version is too old.

Prevent rsync from deleting destination files that match a given pattern

I'm using rsync to sync files from a source to a destination:
rsync -av --delete source destination
I have a single directory on the destination side which is not on the source side. I'd like to prevent rsync from deleting this directory. Is there an option I can pass to rsync to prevent this directory from being deleted upon sync?
You can exclude files/directories with --exclude. This will prevent the somedir directory from being synced/deleted:
rsync -avrc --delete --exclude somedir source destination
As mentioned in a similar question, this can be accomplished by using the --filter option with protect rule:
$ rsync ... --filter 'protect /remote-directory-to-keep/' ...
Unlike the currently accepted answer, using --filter is useful, for instance, if you also wish to use --exclude with --delete-excluded.

Local rsync inclusion/exclusion

I can't seem to get the command to backup /etc/php5, /etc/apache2 and /etc/mysql right. I'm using two since I couldn't figure out how to do both in one. The first one works:
rsync -vahtl --dry-run --log-file=$LOGFILE --exclude="wp-includes/" --exclude="wp-admin/" --exclude="wp-*.php" /var/www $DROPBOX_FOLDER
But when I run the second, I tried a bunch of --include and --exclude directives variation and nothing works:
rsync -vahtl --dry-run --log-file=$LOGFILE --include="php5" --exclude="*" /etc $DROPBOX_FOLDER
rsync -vahtl --dry-run --log-file=$LOGFILE --include="*/" --include="php5/" --exclude="*" /etc $DROPBOX_FOLDER
etc..
The quickest way is to run it as a bash script using something like this.Would need to be adjusted to your Linux flavor
#!/bin/sh
# rsync backup script
rsync -avz --delete-excluded --exclude-from=backup.lst / /home/USERNAME/Dropbox/backup
Then make a file called backup.lst in the directory in which you will run the script from
# Include
+ /etc/php5
+ /etc/apache2
+ /etc/mysql
+ /var/www
# Exclude
- /var/www/wp-admin/*
- /var/www/wp-*.php
- /var/www/wp-includes/*
- /etc/*
- /run/*
- /proc/*
- /sys/*
- /tmp/*
- lost+found/
- /media/*
- /mnt/*
Here are some exclude/include examples:
# --exclude "*.o" would exclude all filenames matching *.o
# --exclude "/foo" would exclude a file in the base directory called foo
# --exclude "foo/" would exclude any directory called foo.
# --exclude "/foo/*/bar" would exclude any file called bar two levels below a
base directory called foo.
# --exclude "/foo/**/bar" would exclude any file called bar two or more levels below
a base directory called foo.
# --include "*/" --include "*.c" --exclude "*"
would include all directories
and C source files
# --include "foo/" --include "foo/bar.c" --exclude "*"
would include only foo/bar.c (the foo/ directory must be
explicitly included or it would be excluded by the "*")

Resources