Say, I have two files:
file1:
0
0
-3.44785
-2.15069
5.70183
17.8715
and file2:
31.9812
50.5646
72.361
96.8705
119.893
144.409
Two combine them side by side, I use :
paste -d" " file1 file2
or I use awk command to do such thing.
If I want to combine these two files one after another, what should I write? I know how to do this using "cat". I have tried different things to modify the "paste" command but they don't give desired output.
Could you please help? Thanks.
cat (short for concatenate) is your friend:
cat file1 file2
That's pretty basic; most people are aware of cat long before they learn to deal with awk, so kudos for mastering the latter!
Normally I would use cat file1 file2, but you could do it like:
awk '{print $0}' file1 file2
or
awk '1' file1 file2
(Note: the '1' does the the same thing as print)
Related
This might be the worst example ever given on StackOverflow, but my purpose is to remove everything in File1 against File2. Whilst ignoring case sensitivity and matching the entire line. For example Cats#123:bob would be removed from File2 as the word Cat appears in File1. So regardless of case sensitivty, if a matching word is found it should eradicate the entirety of the line.
Input (File1):
Cat
Dog
Horse
Wheel
MainFile (File2)
Cats#123:bob
dog#1:truth
Horse-1:fairytale
Wheel:tremendous
Divination:maximus
Desired output
Divination:maximus
As the output shows, only "Divination:maximus" should be outputted as no matching words were found in File1. I prefer to use Sed or Awk generally as I use Cygwin. But any suggestions are welcomed, I can answer all questions you may have, thanks.
Here's what I've tried so far, but it's not working unfortunately, as my output is incorrect. To add to this, simply the wrong lines are being outputted. I'm fairly inexperienced so I don't know how to develop upon this syntax below, and maybe it's completely irrelevant to the job at hand.
grep -avf file1.txt file2.txt > output.txt
The grep command can do that for you:
grep -v -i -f file1 file2
The -f file1 tells grep to use the patterns in file1
The -i flag means case insensitive
The -v flag means to search lines that do not contain those patterns
I have a file with only one string per row, "File1", and a file that has two strings per row, "File2". I wan't go through File1 and check for every row/string in how many of the rows in File2 it appear.
I would like to use something like this:
grep '{sed -n 1p File1}' File2
But it doesn't work. Why and what should I do?
//Thanks!
It seems to me that you're asking two completely different questions: one in the title and one in the body. I'll try to summarize them and answer both.
First question:
How to use the output from sed command as a string and use that string in the grep command?
First answer:
The unix way is to pipe a command's output as input to another program. If you instead want to use the first program's output as e.g. a parameter to the subsequent command (not as its input), there's a handy program that allows you to do just that: xargs.
Example of piping the sed output as input to grep:
sed -n 1p File1 | grep 'needle'
Example of using the sed output as a search string for grep:
sed -n 1p File1 | xargs -I{} grep {} File2
Second question:
I have a file with only one string per row, "File1", and a file that has two strings per row, "File2". I wan't go through File1 and check for every row/string in how many of the rows in File2 it appear.
Second answer:
awk 'NR==FNR{!a[$0]++;next} $1 in a{a[$1]++} END{for(i in a){print i" "a[i]-1}}' File1 File2
Test files:
==> File1 <==
one
two
three
==> File2 <==
one apple
two bananas
two strawberries
three kiwis
three pomegrenades
three lychees
Test run output:
three 3
two 2
one 1
That's only if you mean the string from File1 to appear as the first column of File2. For a more general approach, where you want to count every File2 row containing the string from File1:
awk 'NR==FNR{!a[$0]++;next} {for(i in a){if(match($0,i)){a[i]++}}} END{for(i in a){print i" "a[i]-1}}' File1 File2
Test files:
==> File1 <==
one
two
three
==> File2 <==
one one
two one
two two
three three
three two
three one
Test run output:
three 3
two 3
one 3
I need a one-liner (that I can put in a dos batch file), preferably using a unix command like AWK or JOIN. The function I need is essentially a more elaborate version of the following JOIN command:
join -j 1 -a 1 file1.txt file2.txt -t "^" > output.txt
[walkthrough: field separators are "^", join key is 1st field of both fields, and not exactly sure what the "-a 1" is doing exactly but it is sticking the bit-to-be joined on the end of the row of the other file, which is what I want.
Now, this one-liner works fine where both files are sorted and there is only one matching line in the 2nd file ... but I need it to try to match up to 4 lines in the 2nd file.
E.g.
file1:
12^blahblah
13^blahblahblahblah
14^blahblahblahblahblahblahblahblah
file2:
12^banana
12^orange
12^apple
13^potato
14^tomato
So I want the output like this:
12^blahblah^banana,orange,apple
13^blahblahblahblah^potato
14^blahblahblahblahblahblahblahblah^tomato
[Doesn't have to be a comma separating the new items]
You can try this awk command:
awk -F'^' 'NR==FNR{if($1 in a){a[$1]=a[$1]","$2} else {a[$1]=$2}} NR>FNR{print $0 "^" a[$1]}' file2 file1
The script fills an array a with the content of file2 and and append the content of the array when parsing file1
$ awk -F'^' 'NR==FNR{a[$1]=$0 FS;next} {a[$1] = a[$1] s[$1] $2; s[$1]=","} END{for (i in a) print a[i]}' file1 file2
12^blahblah^banana,orange,apple
13^blahblahblahblah^potato
14^blahblahblahblahblahblahblahblah^tomato
I am new to unix. I want to grep the unmatched pattern from a file1 provided that the patterns are in the file2. The real files are having more than 1000 lines.
Example:
File1:
Hi(Everyone)
How(u)people(are)doing?
ThanksInadvance
File2:
Hi(Every
ThanksI
Required Result:
How(u)people(are)doing?
I want only the pattern to be used like ("Hi(Every") for the grep.It should return the unmatched line from file1.
this line works for given example:
grep -Fvf file2 file1
The 3 options used above:
-F makes grep do fixed-string match
-v invert matching
-f get patterns from file
the Grep-Flag -v inverts the Grep-Command.
cat File1 |grep -v ("Hi(Every")
should return all Lines from File1 where ("Hi(Every") doesnt contains.
best regards,
Jan
I would like to compare two files [ unsorted ]
file1 and file2. I would like to do file2 - file1 [ the difference ] irrespective of the line number?
diff is not working.
I got the solution by using comm
comm -23 file1 file2
will give you the desired output.
The files need to be sorted first anyway.
Well, you can just sort the files first, and diff the sorted files.
sort file1 > file1.sorted
sort file2 > file2.sorted
diff file1.sorted file2.sorted
You can also filter the output to report lines in file2 which are absent from file1:
diff -u file1.sorted file2.sorted | grep "^+"
As indicated in comments, you in fact do not need to sort the files. Instead, you can use a process substitution and say:
diff <(sort file1) <(sort file2)
There are 3 basic commands to compare files in unix:
cmp : This command is used to compare two files byte by byte and as any mismatch occurs,it echoes it on the screen.if no mismatch occurs i gives no response.
syntax:$cmp file1 file2.
comm : This command is used to find out the records available in one but not in another
diff
Most easy way: sort files with sort(1) and then use diff(1).