I have the following unix command, which I'm using to try finding a date in the format yyyy-mm-dd in a file:
grep -i -w [\d]{4}-[\d]{2}-[\d]{2}? <filename>
but for some reason I'm getting an empty answer. Am I matching the regex correctly for grep?
The following is working, using bash extended regex (-E, --extended-regexp):
grep -E -i -w "[0-9]{4}-[0-9]{2}-[0-9]{2}" <filename>
But, in this case you should use [0-9] instead of \d.
If you want to use \d, you need to specify the PERL regex (-P, --perl-regexp):
grep -P -i -w "\d{4}-\d{2}-\d{2}" <filename>
You could try the below grep commands,
grep -oP '\d{4}-\d{2}-\d{2}' file
OR
$ grep -oE '[0-9]{4}-[0-9]{2}-[0-9]{2}' file
OR
grep -o '[0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}' file
Related
I wrote a script in R that has several arguments. I want to iterate over 20 directories and execute my script on each while passing in a substring from the file path as my -n argument using sed. I ran the following:
find . -name 'xray_data' -exec sh -c 'Rscript /Users/Caitlin/Desktop/DeMMO_Pubs/DeMMO_NativeRock/DeMMO_NativeRock/R/scipts/dataStitchR.R -f {} -b "{}/SEM_images" -c "{}/../coordinates.txt" -z ".tif" -m ".tif" -a "Unknown|SEM|Os" -d "overview" -y "overview" --overview "overview.*tif" -p FALSE -n "`sed -e 's/.*DeMMO.*[/]\(.*\)_.*[/]xray_data/\1/' "{}"`"' sh {} \;
which results in this error:
ubs/DeMMO_NativeRock/DeMMO_NativeRock/R/scipts/dataStitchR.R -f {} -b "{}/SEM_images" -c "{}/../coordinates.txt" -z ".tif" -m ".tif" -a "Unknown|SEM|Os" -d "overview" -y "overview" --overview "overview.*tif" -p FALSE -n "`sed -e 's/.*DeMMO.*[/]\(.*\)_.*[/]xray_data/\1/' "{}"`"' sh {} \;
sh: command substitution: line 0: syntax error near unexpected token `('
sh: command substitution: line 0: `sed -e s/.*DeMMO.*[/](.*)_.*[/]xray_data/1/ "./DeMMO1/D1T3rep_Dec2019_Ellison/xray_data"'
When I try to use sed with my pattern on an example file path, it works:
echo "./DeMMO1/D1T1exp_Dec2019_Poorman/xray_data" | sed -e 's/.*DeMMO.*[/]\(.*\)_.*[/]xray_data/\1/'
which produces the correct substring:
D1T1exp_Dec2019
I think there's an issue with trying to use single quotes inside the interpreted string but I don't know how to deal with this. I have tried replacing the single quotes around the sed pattern with double quotes as well as removing the single quotes, both result in this error:
sed: RE error: illegal byte sequence
How should I extract the substring from the file path dynamically in this case?
To loop through the output of find.
while IFS= read -ru "$fd" -d '' files; do
echo "$files" ##: do whatever you want to do with the files here.
done {fd}< <(find . -type f -name 'xray_data' -print0)
No embedded commands in quotes.
It uses a random fd just in case something inside the loop is eating/slurping stdin
Also -print0 delimits the files with null bytes, so it should be safe enough to handle spaces tabs and newlines on the path and file names.
A good start is always put an echo in front of every commands you want to do with the files, so you have an idea what's going to be executed/happen just in case...
This is the solution that ultimately worked for me due to issues with quotes in sed:
for dir in `find . -name 'xray_data'`;
do sampleID="`basename $(dirname $dir) | cut -f1 -d'_'`";
Rscript /Users/Caitlin/Desktop/DeMMO_Pubs/DeMMO_NativeRock/DeMMO_NativeRock/R/scipts/dataStitchR.R -f "$dir" -b "$dir/SEM_images" -c "$dir/../coordinates.txt" -z ".tif" -m ".tif" -a "Unknown|SEM|Os" -d "overview" -y "overview" --overview "overview.*tif" -p FALSE -n "$sampleID";
done
I want to remove all files in a directory except some in UNIX. Part I desired files have a known name and for the other part, I'm using ls|grep command. But the ls | grep is working when there is only one occurrence and not when there is more than one. it is the same with find|grep. here are my commands:
rm -v !("R1.r"|"R2.r"|"r2.par"|$(ls|grep nario)|"sh.sh")
rm -v !("R1.r"|"R2.r"|"r2.par"|$(find|grep nario)|"sh.sh")
Is there any problem with my commands???
It looks like you're trying to use BASH specific extglob syntax. ls|grep won't work in the middle of the glob, because each pattern needs to be separated by a |, which won't happen with ls|grep. The easier way to do what you want is to use the shell globbing to find the files under the directory you're looking for instead. Make sure you do the following:
Are using BASH
Have extglob enabled: shopt -s extglob
Have globstar enabled: shopt -s globstar
Use file globbing rather than ls | grep
Then try again:
rm -v !("R1.r"|"R2.r"|"r2.par"|**/*nario*|"sh.sh")
Note: globstar requires BASH version 4 or higher.
I'm using tcsh, and I'm trying to grep a path from a file with several ID, I'm doing:
grep I241149 $ENV_CASTRO/ALL_CMD_LINES.BAK | grep -o \$"ENV_CASTRO.*.asm"
that gets me:
$ENV_CASTRO/central/WS678/test_do_all.asm
but if I try
cp `grep I241149 $ENV_CASTRO/ALL_CMD_LINES.BAK | grep -o \$"ENV_CASTRO.*.asm"` .
it prompts
cp: cannot stat `$ENV_CASTRO/central/WS678/test_do_all.asm': No such file or directory
How do I tell tcsh that the output of grep contains a $ that means it is an environment variable and is not plain text?
Thanks in advance.
eval is your friend ....
eval cp `grep I241149 $ENV_CASTRO/ALL_CMD_LINES.BAK | grep -o \$"ENV_CASTRO.*.asm"` .
I don't have the time to create files to test this.
I hope this helps.
The problem is that the output of the grep command is not being evaluated by the shell, and so variable substitution is not happening.
One way to solve this would be to execute the desired command within another shell, for example,
sh -c "cp `grep I241149 $ENV_CASTRO/ALL_CMD_LINES.BAK | grep -o '$ENV_CASTRO.*.asm'` ."
I want to find the lines in which atleast one of the strings among string1 and string2 are present in the file.
grep 'string1' 'string2' file;-this is an error even though!!
how could i do the above on unix command line?
If your version of grep supports the -E flag (e.g. the GNU version), you can use extended regular expressions, which allows you to perform queries like this:
$ grep -E 'string1|string2'
or
$ grep -E 'string[12]'
See pcresyntax(3) and pcrepattern(3) for further information on PCRE (Perl Compatible Regular Expressions).
Found it!!
grep -e 'string1' -e 'string2' file;
I usually use the following pipeline to grep for a particular search string and yet ignore certain other patterns:
grep -Ri 64 src/install/ | grep -v \.svn | grep -v "file"| grep -v "2\.5" | grep -v "2\.6"
Can this be achieved in a succinct manner? I am using GNU grep 2.5.3.
Just pipe your unfiltered output into a single instance of grep and use an extended regexp to declare what you want to ignore:
grep -Ri 64 src/install/ | grep -v -E '(\.svn|file|2\.5|2\.6)'
Edit: To search multiple files maybe try
find ./src/install -type f -print |\
grep -v -E '(\.svn|file|2\.5|2\.6)' | xargs grep -i 64
Edit: Ooh. I forgot to add the simple trick to stop a cringeable use of multiple grep instances, namely
ps -ef | grep something | grep -v grep
Replacing that with
ps -ef | grep "[s]omething"
removes the need of the second grep.
Use the -e option to specify multiple patterns:
grep -Ri 64 src/install/ | grep -v -e '\.svn' -e file -e '2\.5' -e '2\.6'
You might also be interested in the -F flag, which indicates that patterns are fixed strings instead of regular expressions. Now you don't have to escape the dot:
grep -Ri 64 src/install/ | grep -vF -e .svn -e file -e 2.5 -e 2.6
I noticed you were grepping out ".svn". You probably want to skip any directories named ".svn" in your initial recursive grep. If I were you, I would do this instead:
grep -Ri 64 src/install/ --exclude-dir .svn | grep -vF -e file -e 2.5 -e 2.6
you can use awk instead of grep
awk '/64/&&!/(\.svn|file|2\.[56])/' file
You maybe want to use ack-grep which allow to exclude with perl regexp as well and avoid all the VC directories, great for grepping source code.
The following script will remove all files except a list of files:
echo cleanup_all $#
if [[ $# -eq 0 ]]; then
FILES=`find . -type f`
else
EXCLUDE_FILES_EXP="("
for EXCLUDED_FILE in $#
do
EXCLUDE_FILES_EXP="$EXCLUDE_FILES_EXP./$EXCLUDED_FILE|"
done
# strip last char
EXCLUDE_FILES_EXP="${EXCLUDE_FILES_EXP%?}"
EXCLUDE_FILES_EXP="$EXCLUDE_FILES_EXP)"
echo exluded files expression : $EXCLUDE_FILES_EXP
FILES=`find . -type f | egrep -v $EXCLUDE_FILES_EXP`
fi
echo removing $FILES
for FILE in $FILES
do
echo "cleanup: removing file $FILE"
rm $FILE
done