Using the --delete flag in rsync with verbose mode and I see when i run it is says 'deleting file xxxxx' because the file no longer exists in the source. However when I list the folder, the file is still there but just with a ~ at the end of the filename?
I'd think those are backup files. From man rsync:
-b, --backup
With this option, preexisting destination files are renamed as each file is transferred or deleted. You can control where the backup file goes and what (if any) suffix gets appended using the
--backup-dir and --suffix options.
--suffix=SUFFIX
This option allows you to override the default backup suffix used with the --backup (-b) option. The default suffix is a ~ if no --backup-dir was specified, otherwise it is an empty string.
I ran into this same problem before and there's essentially an error in the rsync documentation. If you are using the -a archive option like I was, which I bet you are, the documentation says -a is equivalent to -rlptgoD it's actually equivalent to -rlptgoDb so just use -rlptgoD instead of -a so you aren't using the -b (backup) option inadvertantly
Related
My zsh has some completion features I don't understand and can't find where to change. I have two issues where I suspect they have a similar "fix" for my problem. I initialize the zsh completion system with
autoload -Uz compinit
compinit
to get advanced completion features, but I also get the following problems that I don't have without compinit.
First: I happen to have a directory called mydir in my home directory and unfortunately, there is also a user called mydir. When I want to change into my directory and then use tab completion, i.e.
cd mydir/<TAB>
I get the content directories of ~myusername/mydir/ along with all directories available for ~mydir/. I already tried to put
zstyle ':completion:*' users myusername
in my .zlogin file, but it does only change the completion of the username itself and not subsequent directories. Is there some similar switch to turn off completion of other users' home directories? Alternatively, it would already be good if the current directory completion would appear first in the completion menu.
Second: I wrote a script called setup-file-with-a-long-name.sh that resides in my home directory. When I want to execute it via
source setup-file-with-a-long-name.sh
I start with the first few characters, I press <TAB> and I get a list of lots of executable files that are probably somewhere on my $PATH installed by the system, but I don't care about all those files, I just want my file (which is the only match in the current directory) to be displayed either first in the menu and accessible via <TAB> <TAB> or better yet, be accepted after the first <TAB>. (If I select any of them, they don't work anyway because source needs the absolute path, not the filename. Therefore this is a behavior I don't understand and can't see how this is useful as a default for anybody.)
Possible workarounds:
1. Write ~/ explicitly - this is what I want to avoid, because I have to ssh into a new shell pretty often and want to start navigating without thinking about whether I am in $HOME or not.
2. Don't use compinit - well, I like the context-aware completion in principle, I just want to adapt it to my needs.
The following works in bash,
man source -
source filename [arguments]
Read and execute commands from filename in the current shell environment and return the exit status of the last command executed from filename. If filename does not contain a slash, file
names in PATH are used to find the directory containing filename. The file searched for in PATH need not be executable. When bash is not in posix mode, the current directory is searched if
no file is found in PATH. If the sourcepath option to the shopt builtin command is turned off, the PATH is not searched.
to disable the flag instructions are a little above the description of sourcepath
shopt [-pqsu] [-o] [optname ...]
Toggle the values of variables controlling optional shell behavior. With no options, or with the -p option, a list of all settable options is displayed, with an indication of
whether or not each is set. The -p option causes output to be displayed in a form that may be reused as input. Other options have the following meanings:
-s Enable (set) each optname.
-u Disable (unset) each optname.
-q Suppresses normal output (quiet mode); the return status indicates whether the optname is set or unset. If multiple optname arguments are given with -q, the return
status is zero if all optnames are enabled; non-zero otherwise.
-o Restricts the values of optname to be those defined for the -o option to the set builtin.
...
sourcepath
If set, the source (.) builtin uses the value of PATH to find the directory containing the file supplied as an argument. This option is enabled by default.
so executing the following should remove path from your tab completion...
shopt -u sourcepath
I have to copy a big directory to my NAS using rsync, I would like to say to rsync only copy the files when source and destination are different to avoid to copy a files already copied.
Skipping identical files is the whole purpose why people use rsync. This is default behavior of rsync. Most of the time the only option you want to use is -a:
rsync -a -P <source> <dest>
The -P just means show progress and the -a means "archive" and that means "when copying files, try to make copy as identical as possible" (try to keep permissions, ownership, timestamps, etc.) but is also means "Only update files if you have to". It's like saying "make sure <dest> is an up-to-date backup of <source>".
However, by default rsync will already consider two files identical, if they have same file size and same last modification date. Of course, two files may also have same size and same last modification date and not be identical. So when running that command for the very first time and you are not sure which files may need update and which ones don't, try this:
rsync -a -c -P <source> <dest>
-c means don't rely just upon size and date, checksum every file and compare the checksums. Only if checkums are identical, consider files as identical. Note that rsync will not necessary checksum the whole file, big files are broken into smaller chunks and every chunk is checksumed separately as only chunks that have changed are transferred.
So even with checksuming you can save you a lot of time when copying over a network connection. It won't save you any time when copying locally because just copying everything is probably faster than checksuming everything. So a plain copy will always beat a checksuming rsync in speed when both, source and destination, are local drives. In that case use
cp -a -v <source> <dest>
or if your system doesn't know -a, use
cp -pPR -v <source> <dest>
that's identical to -a. Again, the -v is just to see some progress.
And I'd only use -c for the very first sync, after that, relying on file size and last modification date usually works very well for updating and it is a whole lot faster. It will work because if a file has been altered since the last sync, it will have a different last modification date and so by just comparing the dates rysnc will know that the file must be updated at the destination. Of course, that only works if your systems all have the correct date/time set and if you don't manipulate the last modification date of files and also don't forbid your system to update them.
If you want to skip files solely on presence, use this:
rsync -a -P --ignore-existing <source> <dest>
That's like telling rsync "If you see a file with the same name at the destination, always consider it to be identical and never update it".
Please note that if -a detects a file in <source> is different than a files in <dist>, whether this is determined by size and modification date or by checksumming, it will always update the file at <dest> to match then file at <source>. If multiple sources are syncing to the same destination, you might also want to add -u which means "in case two files are different, only update if the file at <source> has a newer last modification date than then file at <dest>"
Just as a general tip, if you type
man <command>
in a terminal, you will get a nice help page on most systems (Linux, MacOS X and UNIX systems), explaining you all the options in all detail. You can scroll up/down using arrow keys or page up/down and you can leave that view by hitting "q" for quit. E.g.
man rsync
My problem is in two parts:
My team and I are using an Test Design Studio to write .vbs files in a Accurev Workspace. The problem is that Accurev recognize them as binaries instead text/ptext files... which causes problems when merging. Is there a setting in Accurev I can change to force it to recognize .vbs files as text/ptext?
All those binaries that are already in the stream, I need solution to convert them all into text/ptext. I've given up on the Client UI, because it means I'd have to go in the Workspace explorer and go through every single folder, one by one, and keep those binaries. Then I thought of the commands. I tried
2.1. accurev keep -c "keep ptext" -n -E ptext -R target_folder
2.2. accurev keep -c "keep ptext" -n -E ptext -R .
2.3. But I get a No Element Selected. That's because the "-n" flag is required for recursive, but it means it'll ignore non-modified files... and most of my files are backed and not modified... otherwise I can't even select the directory for keeping (I'll report "can't keep a directory"). I could create a file-list, but it would take as long as manually keeping all the files one by one. I also tried if I could work directly in the stream (since it has another empty stream above, it lists all it's files as outgoing), but I do not have the keep option in the stream. Is there an easy way to convert all files in stream/workspace as text/ptext?
Yes, you will need to enable a pre-create-trigger using the elem_type.pl script found in "accurev install dir/examples" on your server. Inside the elem_type file, you will see the directions for setting this trigger.
Yes, run the following command to generate a list of all the files in your workspace.
"accurev stat -a -ffl > list.txt"
Then run the this command to convert the files to ptext:
"accurev keep -c "ptext conversion" -E ptext -l list.txt"
Then you can promote those files.
Check the files with a hex editor to see if there are any non-ASCII characters.
If there's binary content in the file AccuRev will see those files as binary.
Overwrite the keep as jstanley suggested to change the type.
On the add use "accurev add -E ptext -c "your favorite comment" file.vbs
I found rsync behaves differently in the following two situations:
(1) All the files are copied by using rsync, then using rsync again will be fast (skip all the files);
(2) Use cp to copy files, then using rsync will be slow (or may be run freshly?)
So my confusion is "Does rsync generate any internal things on the files so that it can refer to avoid duplicate checking?"
rsync -a (in archive mode, which I presume you ran) retains all attributes of a file, including creation/modification time. cp does not. I suppose something in the file attributes that's different when you use cp, probably a later modification time, in the destination files, made rsync think they are newer files, so it either recopied them or had to check the contents.
I use an rsync command to sync two directories remote >local
the command is (used in python script)
os.system('rsync --verbose --progress --stats --recursive\
--copy-links --times --include="*/" --include="*good_name*.good_ext*"\
--exclude-from "/myhome/mydir/src/rsync.exclude"\
%s %s'%(remotepath,localpath))
I want to exclude certain directories that has the same files that I also want to include.
I want to include recursively
any_dir_name/any_file_name.good
but I want to exclude any and all files that are in
bad_dir_name/
I used --exclude-from and here is my exclude from file
*
/*.bad_dir_name/
Unfortunately it doesn't work. I suspect it may have something to do with --include="*/" but if I remove it the command doesn't sync any files at all.
I got it. I used -vv to find according to which rule the directory was showing up in the sync list and since rsync supports regular expressions,
I changed my include statement from "*/" to
--include="*[^.bad_dir_name]/"
and all works fine now.