Uhoh... issued this command:
rsync -avv --progress --delete "/media/Drive1_/" "/media/Drive2_/Backup/"
Previously, it had been working great and was:
rsync -avv --progress --delete "/media/Drive1_" "/media/Drive2_/Backup/"
But I added a trailing slash. Now, I'm seeing lots of output like this:
deleting /media/Drive1_/pics/pics/somepic.jpg
or
deleting /media/Drive1_/Drive2_backup/pics/somepic.jpg
I believe there should only be one /pic/ directory, but it lists two. Navigating to that path in samba shows the jpg serves up just fine.
I want the trailing slash to be there. Does this output though mean that Rsync deleted content from /media/Drive1_ ?
I believe you have pasted the "previous" command exactly the same as the "new" one but I will assume the previous one did not have the trailing slash.
When you put a trailing slash at the end of a source directory, rsync treats it as a "contents of this directory". So if you do "rsync pics/ otherdir" you put contents of pics into otherdir. If you do "rsync pics otherdir", you would put the directory pics into otherdir.
In the destination directory there is no difference whether you put trailing slash or not. If the directory doesn't exist, it will get created. Depending on the existance of trailing slash after source directory, it will either have contents of source in it or the source directory in it.
Example:
rsync -avv /media/drive/backup/ /media/drive2/backup
is the same as
rsync -avv /media/drive/backup /media/drive2
Related
I am trying to use rsync to do backups. I have an include file called /etc/daily.rsync and it contains the following:
+ /home/demo
- *
Then I run the command below:
$ sudo rsync -acvv --delete --include-from=/etc/daily.rsync /mnt/offsite_backup/home/
sending incremental file list
delta-transmission disabled for local transfer or --whole-file
drwxrwxr-x 6 2021/02/22 14:09:13 .
total: matches=0 hash_hits=0 false_alarms=0 data=0
sent 52 bytes received 131 bytes 366.00 bytes/sec
total size is 0 speedup is 0.00
When I go look in the directory I see nothing. What I think is that it is trying to rsync from the current directory which btw is empty. So this leaves me to believe that it is not getting the data form the include file.
This command runs as expected:
sudo rsync -acvv --delete /home/demo /mnt/offsite_backup/home/
The different posts made many suggestions, and I have tried them. I am just stuck. Any thoughts would be very welcome.
I think you're misunderstanding what a filter file (like the one you specified with --include-from) does. It does not specify where to sync files from; it specifies which files within the source directory to sync.
You need to specify both the source and destination as part of the command line. In the command:
sudo rsync -acvv --delete --include-from=/etc/daily.rsync /mnt/offsite_backup/home/
You only specified one directory, /mnt/offsite_backup/home/, so rsync has assumed it's the source, and there is no destination. According to the rsync man page:
As a special case, if a single source arg is specified without a
destination, the files are listed in an output format similar to "ls -l".
So, basically, it's listing the contents of /mnt/offsite_backup/home/, and apparently that's empty.
The second command you gave specifies both the source and destination, which is why it works correctly. If you want to add a filter file to, be aware that the paths in the filter will be relative to the source. So if you used
sudo rsync -acvv --delete --include-from=/etc/daily.rsync /home/demo /mnt/offsite_backup/home/
...it's going to try to include the file/directory /home/demo/home/demo, which probably doesn't exist. Except it actually won't do that, because the - * line will exclude /home/demo/home, so if it did exist, it and its contents will be excluded. You need to include the parent directories of anything you want to include in the sync operation. Again, from the man page:
The concept path exclusion is particularly important when using a
trailing '*' rule. For instance, this won't work:
+ /some/path/this-file-will-not-be-found
+ /file-is-included
- *
This fails because the parent directory "some" is excluded by the '*' rule, so rsync never visits any of the files in the "some" or
"some/path" directories. One solution is to ask for all directories in
the hierarchy to be included by using a single rule: "+ */" (put it
somewhere before the "- *" rule), and perhaps use the
--prune-empty-dirs option. Another solution is to add specific include rules for all the parent dirs that need to be visited. For instance,
this set of rules works fine:
+ /some/
+ /some/path/
+ /some/path/this-file-is-found
+ /file-also-included
- *
ok, so after walking away from the problem I realized that, I never specified what actual directory I wanted to sync. The include can't work from thin air. so the command is:
sudo rsync -acv --delete /home/ --include-from=/etc/weekly.rsync /mnt/offline_backup/home/
The include file had to change as well.
+ demo/***
+ truenorth/***
- *
To have it decend into the directory structure, I needed the ***. I hope this can help someone else out.
I need to sync a local directory (destination) from the remote server directory (source) with the --delete option.
I've got this:
Local directory is /home/user/config and into it
removeit.txt
bar.csv
foo.h
config.conf
scripts <- Is a directory
logs <- Is adirectory
The remote directory is config and into it
bar.csv
foo.h
I want in the local directory after rsync
bar.csv
foo.h
config.conf
scripts <- Is a directory
logs <- Is adirectory
I've tested with multiple options but I can't get a entire rsync command with my needs.
rsync -avz --min-size=1 --delete -e "ssh" user#example.com:./config/ ./config --exclude ./config/scripts --exclude ./config/logs --exclude ./config/device.conf --dry-run
With a list of include files
rsync -avz --min-size=1 --include-from=list --exclude=* --delete-excluded -e "ssh" user#example.com:.config/ config/
But nothing works as I expected. The subfolders in the destination are deleted.
The man page for rsync has a section on "Include/Exclude Pattern Rules" which you should review. In particular, it looks like you are using ./ to refer to the directory relative of your shell's working directory which won't work the way you want.
Here is a relevant section of the documentation:
if the pattern starts with a / then it is anchored to a particular
spot in the hierarchy of files, otherwise it is matched against the
end of the pathname. This is similar to a leading ^ in regular
expressions. Thus /foo would match a name of "foo" at either the "root
of the transfer" (for a global rule) or in the merge-file's directory
(for a per-directory rule). An unqualified foo would match a name of
"foo" anywhere in the tree because the algorithm is applied
recursively from the top down; it behaves as if each path component
gets a turn at being the end of the filename. Even the unanchored
"sub/foo" would match at any point in the hierarchy where a "foo" was
found within a directory named "sub". See the section on ANCHORING
INCLUDE/EXCLUDE PATTERNS for a full discussion of how to specify a
pattern that matches at the root of the transfer.
The pattern you are using might be causing rsync to look for a folder literally called . which contains a file or folder called scripts or whatever.
Try replacing the relative-ish ./config/ part with / in your exclude patterns. That will anchor the pattern in the "root of transfer" which is the directory you are synchronizing. In other words, try this:
rsync -avz --min-size=1 --delete -e "ssh" user#example.com:./config/ ./config --exclude /scripts --exclude /logs --exclude /device.conf --dry-run
I hope that works. If it's not too late to check, please let me know if this worked for you.
I have this rsync command:
'rsync -va --delete --exclude-from=rsync_exclusions.txt SourceA DestinationB'
rsync_exclusions.txt contains a single line:
#script.sh#
The command runs fine, except the #script.sh# file gets added to the destination.
I tried escaping the file name in the exclusion file so:
/#script.sh/#
which also did not work. Any idea how to exclude this file, without resorting to renaming it without the sharps? Note that other file names in the the same dir get excluded fine with the same command when added to the exclusion file.
Try using [#]script.sh[#] in the excludes file.
Suddenly grep command stopped working. When I did the ls -l ~/grep showing the one file in my home directory.But this file has been present for ages. If I give command which grep --> pointing to /bin/grep and with /bin/grep it is working fine. Can anyone please suggest.
Thanks,
Regards,
Shiv
You can delete the zero-byte file in your home directory. It's not doing anything. (I don't know how it got there.) The problem is that the first entry in PATH, ".", points to whatever directory you're in. So when you're in your home directory, the shell (bash, I assume) looks for grep in the current directory, and finds the file that's there, which can't do anything.
I consider it a bad idea to have "." in your path. It's convenient, and natural if you're coming from the Windows world, but it means that what gets executed can change depending on what directory you're in (as you have now seen). It also means that if you're on a multiuser system, someone can put an executable in one of their directories, and then when you cd into their directory, all of a sudden you're executing their code, which might not be what you want, and could be dangerous.
Instead, remove ".:" (dot colon) from your PATH. When you need to run a script in the current directory, add "./" to its name to execute it. "/bin" and "/usr/bin" should usually be at the front of the list. Some people prefer to put "/usr/local/bin" at the front of the list, or something else.
You can change your PATH by editing .profile or .bash_profile or .bashrc. It depends on how you have your shell set up. Be careful to separate each directory path in PATH with one ":" character.
I am attempting to use rsync to copy files, but I want to not copy hidden files and folders, and there is one ordinary file I want excluded from the file transfer. I believe I am eliminating the hidden folders with the --exclude="./" and I believe I am excluding the hidden file with the --exclude file path option. If I eliminate the --exclude file path option, I don't get any errors, but that file is copied, which I do not want. If I eliminate the --excluude="./" the hidden files are copied, which I do not want either. What am I doing wrong?
mbp:~ username $ rsync —-exclude /Users/username/work/java/textsearch/settings/search_config.properties --exclude=".*/" -avz /Users/username/work/java/ root#remote.local:/usr/local/java/ -n
building file list ... rsync: link_stat "/Users/username/?\#200\#224-exclude" failed: No such file or directory (2)
done
sent 9560 bytes received 20 bytes 6386.67 bytes/sec
total size is 17461760 speedup is 1822.73
rsync error: some files could not be transferred (code 23) at /SourceCache/rsync/rsync-42/rsync/main.c(992) [sender=2.6.9]
1) What is /Users/username/?#200#224-exclude and why is rsync looking for it?
2) How do I get rsync to copy everything except the hidden folders/files and the specified file?
If this is an exact copy of the command line, the "--" in front of exclude is not using the correct characters. Delete this and replace with double minus. What happens is, that rsync doesn't recognize the option, instead searching the user directory for the file "—-exclude"