I need a fast and efficient approach to the following problem (I am working with many files.) But for example:
I have two files: file2
Hello
Goodbye
Salut
Bonjour
and file1
Hello, is it Me you're looking for?
I would like to find any word in file 2 that exists in file 2, and then convert that word to lower case.
I can grep the words in a file by doing:
grep -f file2.txt file1.txt
and returns
Hello
So now I want to convert to
hello
so that the final output is
hello, is it Me you're looking for?
Where if I match multiple files:
grep -f file2.txt *_infile.txt
The output will be stored in respective separate outfiles.
I know I can convert to lower case using something like tr, but I only know how to do this on every instance of an uppercase letter. I only want to convert words common between two files from uppercase to lowercase.
Thanks.
I would solve the problem a bit differently.
First, I would mark matches in grep. --color=always works well, although it's somewhat cumbersome and potentially unreliable in detection. Then I would change marked matches with sed or perl:
grep --color=always -F -f file2.txt file1.txt | \
perl -p -e 's/\x1b.*?\[K(.*?)\x1b.*?\[K/\L\1/g'
The cryptic RE matches the coloring escape sequence before the match, de-coloring escape sequence right after the match and captures everything in between into group 1. Then it applies lowercase \L conversion to the capture. Likely GNU sed can do the same, but probably perl is more portable.
Related
This might be the worst example ever given on StackOverflow, but my purpose is to remove everything in File1 against File2. Whilst ignoring case sensitivity and matching the entire line. For example Cats#123:bob would be removed from File2 as the word Cat appears in File1. So regardless of case sensitivty, if a matching word is found it should eradicate the entirety of the line.
Input (File1):
Cat
Dog
Horse
Wheel
MainFile (File2)
Cats#123:bob
dog#1:truth
Horse-1:fairytale
Wheel:tremendous
Divination:maximus
Desired output
Divination:maximus
As the output shows, only "Divination:maximus" should be outputted as no matching words were found in File1. I prefer to use Sed or Awk generally as I use Cygwin. But any suggestions are welcomed, I can answer all questions you may have, thanks.
Here's what I've tried so far, but it's not working unfortunately, as my output is incorrect. To add to this, simply the wrong lines are being outputted. I'm fairly inexperienced so I don't know how to develop upon this syntax below, and maybe it's completely irrelevant to the job at hand.
grep -avf file1.txt file2.txt > output.txt
The grep command can do that for you:
grep -v -i -f file1 file2
The -f file1 tells grep to use the patterns in file1
The -i flag means case insensitive
The -v flag means to search lines that do not contain those patterns
I'm searching through text files using grep and sed commands and I also want the file names displayed before my results. However, I'm trying to remove part of the file name when it is displayed.
The file names are formatted like this: aja_EPL_1999_03_01.txt
I want to have only the date without the beginning letters and without the .txt extension.
I've been searching for an answer and it seems like it's possible to do that with a sed or a grep command by using something like this to look forward and back and extract between _ and .txt:
(?<=_)\d+(?=\.)
But I must be doing something wrong, because it hasn't worked for me and I possibly have to add something as well, so that it doesn't extract only the first number, but the whole date. Thanks in advance.
Edit: Adding also the working command I've used just in case. I imagine whatever command is needed would have to go at the beginning?
sed '/^$/d' *.txt | grep -P '(^([A-ZÖÄÜÕŠŽ].*)?[Pp][Aa][Ll]{2}.*[^\.]$)' *.txt --colour -A 1
The results look like this:
aja_EPL_1999_03_02.txt:PALLILENNUD : korraga üritavad ümbermaailmalendu kaks meeskonda
A desired output would be this:
1999_03_02:PALLILENNUD : korraga üritavad ümbermaailmalendu kaks meeskonda
First off, you might want to think about your regular expression. While the one you have you say works, I wonder if it could be simplified. You told us:
(^([A-ZÖÄÜÕŠŽ].*)?[Pp][Aa][Ll]{2}.*[^\.]$)
It looks to me as if this is intended to match lines that start with a case insensitive "PALL", possibly preceded by any number of other characters that start with a capital letter, and that lines must not end in a backslash or a dot. So valid lines might be any of:
PALLILENNUD : korraga üritavad etc etc
Õlu on kena. Do I have appalling speling?
Peeter Pall is a limnologist at EMU!
If you'd care to narrow down this description a little and perhaps provide some examples of lines that should be matched or skipped, we may be able to do better. For instance, your outer parentheses are probably unnecessary.
Now, let's clarify what your pipe isn't doing.
sed '/^$/d' *.txt
This reads all your .txt files as an input stream, deletes any empty lines, and prints the output to stdout.
grep -P 'regex' *.txt --otheroptions
This reads all your .txt files, and prints any lines that match regex. It does not read stdin.
So .. in the command line you're using right now, your sed command is utterly ignored, as sed's output is not being read by grep. You COULD instruct grep to read from both files and stdin:
$ echo "hello" > x.txt
$ echo "world" | grep "o" x.txt -
x.txt:hello
(standard input):world
But that's not what you're doing.
By default, when grep reads from multiple files, it will precede each match with the name of the file from whence that match originated. That's also what you're seeing in my example above -- two inputs, one x.txt and the other - a.k.a. stdin, separated by a colon from the match they supplied.
While grep does include the most minuscule capability for filtering (with -o, or GNU grep's \K with optional Perl compatible RE), it does NOT provide you with any options for formatting the filename. Since you can'd do anything with the output of grep, you're limited to either parsing the output you've got, or using some other tool.
Parsing is easy, if your filenames are predictably structured as they seem to be from the two examples you've provided.
For this, we can ignore that these lines contain a file and data. For the purpose of the filter, they are a stream which follows a pattern. It looks like you want to strip off all characters from the beginning of each line up to and not including the first digit. You can do this by piping through sed:
sed 's/^[^0-9]*//'
Or you can achieve the same effect by using grep's minimal filtering to return every match starting from the first digit:
grep -o '[0-9].*'
If this kind of pipe-fitting is not to your liking, you may want to replace your entire grep with something in awk that combines functionality:
$ awk '
/[\.]$/ {next} # skip lines ending in backslash or dot
/^([A-ZÖÄÜÕŠŽ].*)?PALL/ { # lines to match
f=FILENAME
sub(/^[^0-9]*/,"",f) # strip unwanted part of filename, like sed
printf "%s:%s\n", f, $0
getline # simulate the "-A 1" from grep
printf "%s:%s\n", f, $0
}' *.txt
Note that I haven't tested this, because I don't have your data to work with.
Also, awk doesn't include any of the fancy terminal-dependent colourization that GNU grep provides through the --colour option.
This question might have been asked a million times before, but did’t see my exaxt case.
Suppose a text file contains:
a
ab
bac
Now I want to grep on ‘a’ and have a hit only on the 1st line. After the ‘a’ there’s always a [tab] character.
Anyone any ideas?
Thanks!
Ronald
Try this:
head -1 *.txt | grep -P "a\t"
head will give you specified amount of lines of each file (all txt files in my example) , grep -P use the regular expressions as defined by perl (perl has \t as tab)
I want to search Exact word pattern in Unix.
Example: Log.txt file contains following text:
aaa
bbb
cccaaa ---> this should not be counted in grep output looking for aaa
I am using following code:
count=$?
count=$(grep -c aaa $EAT_Setup_BJ3/Log.txt)
Here output should be ==> 1 not 2, using above code I am getting 2 as output.
Something is missing, so can any one help me for the this please?
Use whole word option:
grep -c -w aaa $EAT_Setup_BJ3/Log.txt
From the grep manual:
-w, --word-regexp
Select only those lines containing matches that form whole words. The test is that the matching substring must
either be at the beginning of the line, or preceded by a non-word constituent character.
As noted in the comment -w is a GNU extension. With a non GNU grep you can use the word boundaries:
grep -c "\<aaa\>" $EAT_Setup_BJ3/Log.txt
Word boundary matching is an extension to the standard POSIX grep utility. It might be available or not. If you want to search for words portably, I suggest you look into perl instead, where you would use
perl -ne 'print if /\baaa\b/' $EAT_Setup_BJ3/Log.txt
You can use a word boundary (\b) in regex to match an exact word. To enable extended regex, use the -E flag with grep.
Solution:
grep -E "\baaa\b" $EAT_Setup_BJ3/Log.txt
I have a file and i want to sort it according to a word and to remove the special characters.
The grep command is used to search for the characters
-b Display the block number at the beginning of each line.
-c Display the number of matched lines.
-h Display the matched lines, but do not display the filenames.
-i Ignore case sensitivity.
-l Display the filenames, but do not display the matched lines.
-n Display the matched lines and their line numbers.
-s Silent mode.
-v Display all lines that do NOT match.
-w Match whole word
but
How to use the grep command to do the file sort and remove the special character and number.
grep searches inside all the files to find matching text. It doesn't really sort and it doesn't really chop and change output. What you want is probably to use the sort command
sort <filename>
and the output sent to either the awk command or the sed command, which are common tools for manipulating text.
sort <filename> | sed 's/REPLACE/NEW_TEXT/g'
something like above I'd imagine.
The following command would do it.
sort FILE | tr -d 'LIST OF SPECIAL CHARS' > NEW_FILE