I have a code which searches IN string in a file then puts those records in a different file for which I am using below:
cat ${ALL_CARDS} | grep -v ' IN ' | awk '{print $1}' > ${NO_EXEC_CARDS}
But now I have to include multiple search string along with IN like NE & ON so how to include those also in this code line
cat | grep | awk is an anti-pattern. Also, just using awk makes your question trivial:
awk '! /IN/ && ! /ON/ && ! /NE/ {print $1}' "$ALL_CARDS" > "$NO_EXEC_CARDS"
You have some additional whitespace in your grep, so perhaps you want:
awk '! / IN / && ! / ON / && ! / NE / {print $1}'
awk is well designed for this. It simply reads each line, and if the expression does not match " IN " or " ON " or " NE ", then the action is executed. You could also write this as:
awk '! (/ IN / || / ON / || / NE /) {print $1}'
Related
I would like to modify several shell variables within awk:
echo "$LINE_IN" | awk '/pattern1/ {print $0; WRITTEN=1; REC=$REC+1}' >> $FILE1
I tried to put eval, but still does not work:
eval $( echo "$LINE_IN" | awk '/pattern1/ {print $0; WRITTEN=1; REC=$REC+1}' >> $FILE1 )
Any suggestion?
I would like to use k-shell script, thanks!
Count the hits when you are finished:
echo "${LINE_IN}" | grep -E 'pattern1' > "${FILE1}"
REC=$(wc -l < "${FILE1}")
if (( REC > 0 )); then
WRITTEN=1
fi
When you really want to use awk, you must let awk write the results to stdout and parse stdout:
echo "${LINE_IN}" | awk '/echo/ {print $0 > "x3"; WRITTEN=1; REC++}
END { print "WRITTEN=" WRITTEN; print "REC=" REC}'
WRITTEN=1
REC=6
And when you want the variables really set, wrap it:
source (echo "${LINE_IN}" | awk '/echo/ {print $0 > "x3"; WRITTEN=1; REC++}
END { print "WRITTEN=" WRITTEN; print "REC=" REC}')
Note: Get used to using lowercase variable names like written, file and rec.
I wanted to merge a string ABC to a line that I have extracted from a file.
The following command is used to extract the lines 20-25 in file_ABC, take only the first column, which is then transposed to become a row (or line).
sed -n '20,25p' < file_ABC | awk '{print $1}' | paste -s
This is the result:
2727778 14734 0 0 0 2713044
I would like to add at the first position of this line the string ABC.
ABC 2727778 14734 0 0 0 2713044
Any suggestion on how to do that?
A quick hack would be to use something like
printf 'ABC\t%s\n' "$(sed -n '20,25p' < file_ABC | awk '{print $1}' | paste -s)"
You could modify your initial command instead to use awk for everything, though:
awk '
BEGIN {printf "ABC"}
NR>=20 && NR<=25 {printf "\t%s", $1}
END {print ""}
' file_ABC
This might work for you (GNU sed):
sed '20,25{s/\s.*//;H};$!d;x;s/^/ABC/;s/\n/ /g' file
Gather up the first column fields by appending them to the hold space for rows 20 to 25 only. At the end of the file prepend ABC and replace the introduced newlines by spaces.
For fun, bash only
filename=file_ABC
words=("${filename##*_}")
i=0
while read -r word rest_of_line; do
((++i < 20 )) && continue
(( i > 25 )) && break
words+=("$word")
done < "$filename"
join() { local IFS=$1; shift; echo "$*"; }
join $'\t' "${words[#]}"
But this will be much slower than a single awk call.
if you want to keep all in one script
$ awk 'BEGIN {line="ABC"}
NR>=20 && NR<=25 {line=line FS $1}
NR==25 {print line; exit}' file
improved version as suggested by #EdMorton
$awk 'NR>=20 {line=line OFS $1}
NR==25 {print "ABC" line; exit}' file
I have this file :
933|Mahinda|Perera|male|1989-12-03|2010-03-17T13:32:10.447+0000|192.248.2.123|Firefox
1129|Carmen|Lepland|female|1984-02-18|2010-02-28T04:39:58.781+0000|81.25.252.111|Internet Explorer
4194|Hồ ChÃ|Do|male|1988-10-14|2010-03-17T22:46:17.657+0000|103.10.89.118|Internet Explorer
8333|Chen|Wang|female|1980-02-02|2010-03-15T10:21:43.365+0000|1.4.16.148|Internet Explorer
8698|Chen|Liu|female|1982-05-29|2010-02-21T08:44:41.479+0000|14.103.81.196|Firefox
8853|Albin|Monteno|male|1986-04-09|2010-03-19T21:52:36.860+0000|178.209.14.40|Internet Explorer
10027|Ning|Chen|female|1982-12-08|2010-02-22T17:59:59.221+0000|1.2.9.86|Firefox
and with this order
./tool.sh --browsers -f <file>
i want to count the number of the browsers in specific order , for example :
Chrome 143
Firefox 251
Internet Explorer 67
i use this command :
if [ "$1" == "--browsers" -a "$2" == "-f" -a "$4" == "" ]
then
awk -F'|' '{print $8}' $3 | sort | uniq -c | awk ' {print $2 , $3 , $1} '
fi
but it works only for 3 arguments. How to make it work for many arguments? for example a browser with 4 words or more
Seems like an awk one-liner to count your browsers:
$ awk -F'|' '{a[$8]++} END{for(i in a){printf("%s %d\n",i,a[i])}}' inputfile
Firefox 3
Internet Explorer 4
This increments elements of an array, then at the end of the file steps through the array and prints the totals. If you want the output sorted, you can just pipe it through sort. I don't see a problem with multiple words in a browser name.
try this:
awk -F"|" '{print $8}' in | sort | uniq -c | awk '{print $2,$1}'
where in is the input file.
output
[myShell] ➤ awk -F"|" '{print $8}' in | sort | uniq -c | awk '{print $2,$1}'
Firefox 3
Internet 4
also for parsing argument is better to use getopts
i.e.
#!/bin/bash
function usage {
echo "usage: ..."
}
while getopts b:o:h opt; do
case $opt in
b)
fileName=$OPTARG
echo "filename[$fileName]"
awk -F"|" '{print $8}' $fileName | sort | uniq -c | awk '{print $2,$1}'
;;
o)
otherargs=$OPTARG
echo "otherargs[$otherargs]"
;;
h)
usage && exit 0
;;
?)
usage && exit 2
;;
esac
done
output
[myShell] ➤ ./arg -b in
filename[in]
Firefox 3
Internet 4
Your final Awk hard-codes two fields; just continue with $4, $5, $6 etc to print more fields. However, this will add a spurious space for each comma.
Better yet, since the first field is fixed width (because that's the output format from uniq -c), you can do print substr($0,8), $1
I'd do it in perl:
#!/bin/perl
use strict;
use warnings;
use Data::Dumper;
my %count_of;
while ( <> ) {
chomp;
$count_of{(split /\|/)[7]}++;
}
print Dumper \%count_of;
This can be cut down to a one liner:
perl -F'\|' -lane '$c{$F[7]++}; END{ print "$_ => $c{$_}" for keys %c }'
I am missing something subtle. I tried running below command but it didn't work. Can you please help .
ls | awk '{ split($1,a,".gz")} {cp " " $1 " " a[1]".gz"}'
Although when i am trying to print it is showing copy command.
ls | awk '{ split($1,a,".gz")} {print "cp" " " $1 " " a[1]".gz"}'
Not sure where the problem is. Any pointers will be helpful
To summarize some of the comments and point out what's wrong with the first example:
ls | awk '{ split($1,a,".gz")} {cp " " $1 " " a[1]".gz"}'
^ unassigned variable
The cp defaults to "" and is not treated as the program cp. If you do the following in a directory with one file, test.gz_monkey, you'll see why:
ls | awk '{split($1,a,".gz"); cmd=cp " " $1 " " a[1] ".gz"; print ">>" cmd "<<" }'
results in
>> test.gz_monkey test.gz<<
^ the space here is because cp was "" when cmd was assigned
Notice that you can separate statements with a ; instead of having two action blocks. Awk does support running commands in a subshell - one of which is system, another is getline. With the following changes, your concept can work:
ls | awk '{split($1,a,".gz"); cmd="cp "$1" "a[1]".gz"; system(cmd) }'
^ notice cp has moved inside a string
Another thing to notice - ls isn't a good choice for only finding files in the current directory. Instead, try find:
find . -type f -name "*.gz_*" | awk '{split($1,a,".gz"); cmd="cp "$1" "a[1]".gz"; system(cmd) }'
while personally, I think something like the following is more readable:
find . -type f -name "*.gz_*" | awk '{split($1,a,".gz"); system(sprintf( "cp %s %s.gz", $1, a[1])) }'
Why are you using awk at all? Try:
for f in *; do cp "$f" "${f%.gz*}.gz"; done
Here is a query:
grep bar 'foo.txt' | awk '{print $3}'
The field name emitted by the 'awk' query are mangled C++ symbol names. I want to pass each to dem and finally output the output of 'dem'- i.e the demangled symbols.
Assume that the field separator is a ' ' (space).
awk is a pattern matching language. The grep is totally unnecessary.
awk '/bar/{print $3}' foot.txt
does what your example does.
Edit Fixed up a bit after reading the comments on the precedeing answer (I don't know a thing about dem...):
You can make use of the system call in awk with something like:
awk '/bar/{cline="dem " $3; system(cline)}' foot.txt
but this would spawn an instance of dem for each symbol processed. Very inefficient.
So lets get more clever:
awk '/bar/{list = list " " $3;}END{cline="dem " list; system(cline)}' foot.txt
BTW-- Untested as I don't have dem or your input.
Another thought: if you're going to use the xargs formulation offered by other posters, cut might well be more efficient than awk. At that point, however, you would need grep again.
How about
grep bar 'foo.txt' | awk '{ print $3 }' | xargs dem | awk '{ print $3 }'
This will print the demangled symbols, complete with argument lists in the case of methods:
awk '/bar/ { print $3 }' foo.txt | xargs dem | sed -e 's:.* == ::'
This will print the demangled symbols, without argument lists in the case of methods:
awk '/bar/ { print $3 }' foo.txt | xargs dem | sed -e 's:.* == \([^(]*\).*:\1:'
Cheers,
V.