I've been using this utility successfully for many years, in many environemnts. But I'm noticing that on one particular environment, it produces very unexpected results.
grep -r 'search-term1' . | grep 'search-term2'
The above code greps recursively for all instances of search-term1, in the current-dir. The results are then piped to another grep, which selects only those lines that also contain search-term2. This works exactly as I would expect.
grep -r 'search-term1' . | grep -r 'search-term2'
The only difference in the above code is that the -r recursive flag in specified in both grep commands. I would expect the behavior to not change for this particular case. After all, the input to the 2nd grep is a pipe-input, and there's nothing further to be found recursively.
I have been using the command successfully, for many years, in many different environments (both unix and mac-os). However, the most recent environment that I started working in (unix), breaks the above behavior. The second piped grep searches for all instances of search-term2, not only in the piped-input, but also all files in my current directory. Because of this, instead of getting only results that contain both search-terms, I get all results in current-dir that contain the 2nd search term.
Is there any reason why this one particular environment produces this odd behavior? Is there any way I can avoid this, while still preserving the -r flag?
FAQ:
Q: Why am I using the -r flag on a piped input?
Ans: I actually have grep saved as an alias, with many different options and flags that I always want to use as a default. The recursive flag is one of them. I would like to always use this alias, instead of having to type out all the flags every time.
Q: If you want to search for all instances matching both search terms, why not do (insert-superior-method-here) instead?
Ans: You're probably right. I'm sure there are things I can change in my usual habits that would workaround this issue. However, as intellectual curiosity, I would like to find out why recursive-greps-on-pipes work as intended on most environments, but not all, and if that can somehow be resolved.
The -r flag to grep changed in grep version 2.11 (release notes to implicitly use the working directory as the input if no file arguments are given.
If no file operand is given, and a command-line -r or equivalent
option is given, grep now searches the working directory.
You aren't giving the second grep any file arguments so it defaults to the current directory despite there being pipe input.
Try grep -r 'search-term1' . | grep -r 'search-term2' - as a workaround.
grep -r 'search-term1' . | grep -r -d skip 'search-term2' may also work around the problem.
Related
real beginner in Unix commands so not sure if the following is actually possible but here goes.
Is it possible to highlight just one item in a ls output?
I.e.: in a directory I use the following
ls -l --color=auto
this lists 4 items in green
file1.xls
file2.xls
file3.xls
file4.xls
But I want to highlight a specific item, in this case file2.
Is this possible?
The ls program will not do this for you. But you could filter the results from ls through a custom script which modifies the text to highlight just one item. It would be simpler if no color was originally given; then you could match on the given filename (for example as the pattern in an awk script, or in a sed script) and modify just that one item, adding colors.
That is, certainly it is possible. Writing a sample script is a different question.
How you approach the problem depends on what you want from the output. If that is (literally) the output from ls with a single filename in color, then a script would be the normal approach. You could use grep as suggested in the other answer, which raises a few issues:
commenting on ls -l --color=auto makes it sound as if you are using GNU ls, hence likely using Linux. An appropriate tag for the question would be linux rather than unix. If you ask for unix, the answers should differ.
supposing that you are using Linux. Then likely you have GNU grep, which can do colors. That would let you do something like this:
ls -l | grep --color=always file2 |less -R
however, there is a known bug in GNU grep's use of color (see xterm FAQ "grep --color" does not show the right output).
using grep like this shows only the matching lines. For ls that might be a good choice. For matches in a manual page -- definitely not.
Alternatively, less (which is found more often on Unix systems than GNU grep) also can highlight matches (not in color) and would show the file you are looking for in context. You could do this:
ls -l | less -p file2
(Both grep and less use patterns aka regular expressions, but I left the example simple — read the documentation to learn more).
If you're a beginner I would strongly suggest you learn the grep command if you want to filter results - A Unix users best friend (mine anyway)
Use grep to only display the list items you want to see...
ls- l | grep "file2"
NOTE: This is no different to typing ls -l file2 by the way but your pattern could be expanded based on what you actually want displayed on the screen.
So if you had a directory full of files ".txt", ".xls", ".doc" and you wanted to only see ".doc" with the word "work" in the name (work1.doc) you could write:
ls -ls | grep "work" | grep "txt"
This would list work1.txt, work2.txt, work3.txt and so on.
This is a very basic example but I use grep extensively whilst in the unix shell and would advise using this to filter all results instead of colours.
A little side note using grep -v will show you everything but the pattern you give it
ls -l | grep -v ".txt" will show everything BUT .txt files.
In Terminal/shell script, you can list all files in the current directory with ls -l, and then pipe it to execute an additional command. For example, ls -l | grep -i "calc" returns all files whose filename includes calc. In R, you can list all objects currently stored in the workspace, with ls() command.
However, I want to do narrow down the list returned by ls() with something like the grep feature in R, where the input is the returned list by ls() and the output is the list narrowed down by grep (or something), much like the UNIX pipe feature I mentioned above. Is it feasible to do it in R?
Also, is it also feasible to narrow down the list by xargs-like functionality in R? So I like to get only the objects on which the literal includes if, so that if a function on the list returned by ls() includes the if-else condition inside it, I want to display the function in console. You can do it in Terminal with find . | xargs grep "if" (of course those are files in the current directory, not an R object in workspace, but I showed it just the purpose of illustration).
Note that this is not a post on how to call shell commands from within R. It's not what I want to do.
I use OS X 10.9.3 and R 3.1.0.
ls() has a pattern parameter that might be what you need:
pattern an optional regular expression. Only names matching pattern
are returned. glob2rx can be used to convert wildcard patterns
to regular expressions.
For the second part of your question, you could use capture.output(getAnywhere()) and grep to look inside function source. You'll need to pass in the functions to that and I'd make that whole operation a function to keep the implementation clean.
You can do
grep("calc",list.files(),value=TRUE)
which should "emulate" ls -l | grep -i "calc". See ?list.files and grep.
I have a query regarding the execution of a complex command in the makefile of the current system.
I am currently using shell command in the makefile to execute the command. However my command fails as it is a combination of a many commands and execution collects a huge amount of data. The makefile content is something like this:
variable=$(shell ls -lart | grep name | cut -d/ -f2- )
However the make execution fails with execvp failure, since the file listing is huge and I need to parse all of them.
Please suggest me any ways to overcome this issue. Basically I would like to execute a complex command and assign that output to a makefile variable which I want to use later in the program.
(This may take a few iterations.)
This looks like a limitation of the architecture, not a Make limitation. There are several ways to address it, but you must show us how you use variable, otherwise even if you succeed in constructing it, you might not be able to use it as you intend. Please show us the exact operations you intend to perform on variable.
For now I suggest you do a couple of experiments and tell us the results. First, try the assignment with a short list of files (e.g. three) to verify that the assignment does what you intend. Second, in the directory with many files, try:
variable=$(shell ls -lart | grep name)
to see whether the problem is in grep or cut.
Rather than store the list of files in a variable you can easily use shell functionality to get the same result. It's a bit odd that you're flattening a recursive ls to only get the leaves, and then running mkdir -p which is really only useful if the parent directory doesn't exist, but if you know which depths you want to (for example the current directory and all subdirectories one level down) you can do something like this:
directories:
for path in ./*name* ./*/*name*; do \
mkdir "/some/path/$(basename "$path")" || exit 1; \
done
or even
find . -name '*name*' -exec mkdir "/some/path/$(basename {})" \;
Trying to rename a set of files in a directory with various filetypes, all with one common word, say 'foo', to another word, say 'bar' on a MacBook Pro.
E.g.:
foo.txt
form_foo.plist
home_foo.png
images_foo.zip
->
bar.txt
form_bar.plist
home_bar.png
images_bar.zip
Any ideas?
Use with care:
ls | grep foo | while read -r name; do echo mv "$name" "${name//foo/bar}"; done
That will report the commands it will run when you omit "echo". Inspect
the results, then rerun with "echo" omitted. This makes no attempt to work
on files with newlines in the name, nor does it recurse into subdirectories. If you want to work with files whose name begins with ., add -a to the invocation of ls. For safety's sake, you may want to add -i to the invocation of mv. Certainly make a backup first.
I don't have access to a Mac, but under Ubuntu you can use the rename command for this. Here's the man page in case that command is available
I always use tar czf file.tar.gz dirname instead of tar -czf file.tar.gz dirname (without -) because faster
I know... from the documentation, what I'm doing is wrong.
But what is the effect of my command?
(so far, I dont see any problem of my command... sometime I think why the documentation suggest tar -czf file.tar.gz dirname instead of tar czf file.tar.gz dirname?)
It's all in the info pages:
“Like short options, "old options" are single letters. However, old options must be written together as a single clumped set[...] this old style syntax makes it difficult to match option letters with their corresponding arguments, and is often confusing. In the command tar cvbf 20 /dev/rmt0, for example, 20 is the argument for -b, /dev/rmt0 is the argument for -f, and -v does not have a corresponding argument. Even using short options like in tar -c -v -b 20 -f /dev/rmt0 is clearer, putting all arguments next to the option they pertain to.”
And you don't want to confuse your users, do you?
$ man tar
The bundled-arguments format is supported for compatibility with historic
implementations. It consists of an initial word (with no leading - character)
in which each character indicates an option. Arguments follow as separate
words. The order of the arguments must match the order of the corresponding
characters in the bundled command word. For example,
tar tbf 32 file.tar
specifies three flags t, b, and f. The b and f flags both require arguments,
so there must be two additional items on the command line. The 32 is the
argument to the b flag, and file.tar is the argument to the f flag.
while #user502515 is right, I always use tar cvf and I hope #Captain is still using it as well. It's cleaner and if some users find it confusing they can go man tar and learn it. I accept that the new syntax is more explicit, but the old one is beautiful and besides, it's fun to use syntax from 1979.