I'm trying to backup a machine using rsync and after reading numerous SO QAs and the man pages, I'm still failing to understand how include/exclude precedence works so I can transfer the right set of files. Obfuscating the specific details, I'm attempting the following:
Include rescursively:
/home/erik/foo
/home/erik/bar
/home/erik/baz
Include /git rescursively, but exclude some specific subdirs, like /git/src/github.com/foo and /git/src/github.com/bar.
So far, the rsync command I think should accomplish this. It does not, and I've tried a number of variations that fail in different ways:
rsync -am \
--include='*/' \
--include='/home/erik/foo' \
--include='/home/erik/bar' \
--include='/home/erik/baz' \
--include='/git' \
--exclude='/git/bin' \
--exclude='/git/src/github.com/foo' \
--exclude='/git/src/github.com/bar' \
--exclude='*' \
/ nfs.example.com:/data/pool/backup/laptop
Some specific questions:
I have seen it suggested many times that the initial --include='*/ is necessary, although I'm not entirely sure why. I gather it has something to do with ensuring directories are expanded and followed(?). I also assume the final exclude is what excludes any file that doesn't match on the higher statements? Could someone elaborate on whether or not these are both necessary, and their positions are significant?
I am unsure whether or not the directories need leading /. I have seen hints that these paths are relative to the requested transfer root of /, which would suggest it should be something like home/erik, but I have not had success with this either. Could someone expand on how this works?
I'm unsure if a suffix / is required in the paths if I want to include the directory and all subcontents?
Could someone elaborate on whether or not the position of the params is actually significant, i.e., the first in the list to match will be applied?
Is there any reason I should prefer --filter='+ X' over --include? Same for exclude?
I'm unsure if there's a more native way to cross post an answer, but I asked on Stack Exchange and got this excellent answer that solved what I was trying to do perfectly, with context.
Related
Someone sent me a patch and told me to apply it using the command:
patch -Np1 -i file.patch
Out of curiosity, I tried to find out what the -Np1 option does but the patch man page is curiously opaque on this. Can anyone explain what this does? Thanks.
It's two flags. -N and -p1.
-N or --forward
Ignore patches that seem to be reversed or already applied. See
also -R.
and
-pnum or --strip=num
Strip the smallest prefix containing num leading slashes from each
file name found in the patch file. A sequence of one or more adja-
cent slashes is counted as a single slash. This controls how file
names found in the patch file are treated, in case you keep your
files in a different directory than the person who sent out the
patch. For example, supposing the file name in the patch file was
/u/howard/src/blurfl/blurfl.c
setting -p0 gives the entire file name unmodified, -p1 gives
u/howard/src/blurfl/blurfl.c
without the leading slash, -p4 gives
blurfl/blurfl.c
and not specifying -p at all just gives you blurfl.c. Whatever you
end up with is looked for either in the current directory, or the
directory specified by the -d option.
I would like to issue a patch command which is somewhat dumber than the default, but I cannot find the right flags (if they exist at all).
I don't want it to create .rej or .orig files, not even when the patch fails. If the patch fails I'd like the original files to remain unchanged.
I don't want it to try guessing if the patch is reversed or not, or try matching the lines before or after those given in the patch. If the lines at the given line numbers do not match, it should fail.
I've tried with -f -N -V never -r - --no-backup-if-mismatch, but still backup files are created and "fuzzy" matching is tried.
Run it with --dry-run -s and only apply if it doesn't report any problems (you may be able to key off the return code).
For disabling the fuzz, you need -F0
I have a query regarding the execution of a complex command in the makefile of the current system.
I am currently using shell command in the makefile to execute the command. However my command fails as it is a combination of a many commands and execution collects a huge amount of data. The makefile content is something like this:
variable=$(shell ls -lart | grep name | cut -d/ -f2- )
However the make execution fails with execvp failure, since the file listing is huge and I need to parse all of them.
Please suggest me any ways to overcome this issue. Basically I would like to execute a complex command and assign that output to a makefile variable which I want to use later in the program.
(This may take a few iterations.)
This looks like a limitation of the architecture, not a Make limitation. There are several ways to address it, but you must show us how you use variable, otherwise even if you succeed in constructing it, you might not be able to use it as you intend. Please show us the exact operations you intend to perform on variable.
For now I suggest you do a couple of experiments and tell us the results. First, try the assignment with a short list of files (e.g. three) to verify that the assignment does what you intend. Second, in the directory with many files, try:
variable=$(shell ls -lart | grep name)
to see whether the problem is in grep or cut.
Rather than store the list of files in a variable you can easily use shell functionality to get the same result. It's a bit odd that you're flattening a recursive ls to only get the leaves, and then running mkdir -p which is really only useful if the parent directory doesn't exist, but if you know which depths you want to (for example the current directory and all subdirectories one level down) you can do something like this:
directories:
for path in ./*name* ./*/*name*; do \
mkdir "/some/path/$(basename "$path")" || exit 1; \
done
or even
find . -name '*name*' -exec mkdir "/some/path/$(basename {})" \;
For my developer work I reside in the *nix shell environment pretty much all day, but still can't seem to memorize the name and argument specifics of programs I don't use daily. I wonder how other 'casual amnesiacs' handle this. Do you maintain an big cheat sheet? Do you rehearse the emacs shortcuts when you take your weekly shower? Or is your desk covered under sticky notes?
Using bash_completion is one way of not having to remember the precise syntax of program arguments.
> svn [tab][tab]
--help checkout delete lock pdel propget revert
--version ci diff log pedit proplist rm
-h cleanup export ls pget propset status
add co help merge plist pset switch
annotate commit import mkdir praise remove unlock
blame copy info move propdel rename update
cat cp list mv propedit resolved
If I don't use a command regularly enough to remember what I want, I tend to just use --help or the man pages when I need to.
Or, if I'm lucky, I use CTRL+R and let bash's history search find when I last used it.
Eventually you just remember them, well the set that you use anyway. I used to maintain a README in my home directory when I was starting out but that disappeared many years ago.
One useful command is man -k which you pass a word to and it will return a list of all commands whose man page summary contains that word.
'apropos' is also a very useful command. It will list all commands whose man pages contain the keyword.
It seems like the only way to do this is to pass the -i parameter in when you initially run less. Does anyone know of some secret hack to make something like this work
/something to search for/i
You can also type command -I while less is running. It toggles case sensitivity for searches.
You can also set the environment variable LESS
I use LESS=-Ri, so that I can pump colorized output from grep into it, and maintain the ANSI colour sequences.
Another little used feature of less that I found is starting it with +F as an argument (or hitting SHIFT+F while in less). This causes it to follow the file you've opened, in the same way that tail -f <file> will. Very handy if you're watching log files from an application, and are likely to want to page back up (if it's generating 100's of lines of logging every second, for instance).
Add-on to what #Juha said: Actually -i turns on Case-insensitive with SmartCasing, i.e if your search contains an uppercase letter, then the search will be case-sensitive, otherwise, it will be case-insensitive. Think of it as :set smartcase in Vim.
E.g.: with -i, a search for 'log' in 'Log,..' will match, whereas 'Log' in 'log,..' will not match.
It appears that you can summon this feature on a per search basis like so:
less prompt> /search string/-i
This option is in less's interactive help which you access via h:
less prompt> h
...
-i ........ --ignore-case
Ignore case in searches that do not contain uppercase.
-I ........ --IGNORE-CASE
Ignore case in all searches.
...
I've not extensively checked but the help in less version 487 on MacOS as well as other Linux distros lists this option as being available.
On MacOS you can also install a newer version of less via brew:
$ brew install less
$ less --version
less 530 (POSIX regular expressions)
Copyright (C) 1984-2017 Mark Nudelman
References
less is always case-insensitive
When using -i flag, be sure to enter the search string completely in lower case, because if any letter is upper case, then its an exact match.
See also: the -I (capital i) flag of less(1) to change this behavior.