Hello i have this file.txt
a=a
b=b
c=c
d=d
e=e
f=f
.
etc
(about 150 rows)
I need the output to be:
a b c d e f ....
a b c d e f ....
I already tried using paste -d " " - - < file.txt but i need something to work with huge number of rows to columns.
Thank you in advance.
Try this :
awk -F= '{
arr1[NR]=$1
arr2[NR]=$2
}
END{
for (i in arr1) {
printf("%s ", arr1[i])
}
print""
for (i in arr2) {
printf("%s ", arr2[i])
}
print ""
}' file
Output:
a b c d e f
a b c d e f
You can separate the file using the internal field separator:
while IFS== read -r left right; do echo $left; done < "test.txt" | xargs
This gives you the left side. For the right side, you could do
while IFS== read -r left right; do echo $right; done < "test.txt" | xargs
If you are talking about only 150 rows, scanning the file twice should be finde.
mash of echo, cut and tr
$ cat ip.txt
a=1
b=2
c=3
d=4
$ echo $(cut -d= -f1 ip.txt | tr '\n' ' ') ; echo $(cut -d= -f2 ip.txt | tr '\n' ' ')
a b c d
1 2 3 4
Related
I have one file that looks like below
1234|A|B|C|10|11|12
2345|F|G|H|13|14|15
3456|K|L|M|16|17|18
I want the output as
1234|A
1234|B
1234|C
2345|F
2345|G
2345|H
3456|K
3456|L
3456|M
I have tried with the below script.
awk -F"|" '{print $1","$2","$3","$4"}' file.dat | awk -F"," '{OFS=RS;$1=$1}1'
But the output is generated as below.
1234
A
B
C
2345
F
G
H
3456
K
L
M
Any help is appreciated.
What about a single simple awk process such as this:
$ awk -F\| '{print $1 "|" $2 "\n" $1 "|" $3 "\n" $1 "|" $4}' file.dat
1234|A
1234|B
1234|C
2345|F
2345|G
2345|H
3456|K
3456|L
3456|M
No messing with RS and OFS.
If you want to do this dynamically, then you could pass in the number of fields that you want, and then use a loop starting from the second field.
In the script, you might first check if the number of fields is equal or greater than the number you pass into the script (in this case n=4)
awk -F\| -v n=4 '
NF >= n {
for(i=2; i<=n; i++) print $1 "|" $i
}
' file
Output
1234|A
1234|B
1234|C
2345|F
2345|G
2345|H
3456|K
3456|L
3456|M
# perl -lne'($a,#b)=((split/\|/)[0..3]);foreach (#b){print join"|",$a,$_}' file.dat
1234|A
1234|B
1234|C
2345|F
2345|G
2345|H
3456|K
3456|L
3456|M
I have two files suppose 1st.dat
a
b
c
d
and another file sppose 2nd.dat
d
e
f
g
my output should be like
a
b
c
e
f
g
I have tried using diff and sdiff but I am not getting the output as I mentioned. Please Help
You can use grep
grep -vf 2nd.dat 1st.dat > out.dat && grep -vf 1st.dat 2nd.dat >> out.dat
try diff like this:-
diff 1st.dat 2nd.dat|grep -e '<' -e '>'|sed -e 's/< \+//g' -e 's/> \+//g' > output.txt
cat output.txt
a
b
c
e
f
g
I think your version of uniq doesn't support the option --unique.
Can't harm to try:
cat 1st.dat 2nd.dat | sort | uniq --unique
This would be much easier than
cat 1st.dat 2nd.dat |grep -vf <(sort 1st.dat 2nd.dat | uniq -d)
A solution with grep -vf will need some more attention (matching substrings, empty lines). When you go for grep -vf the solution of #Daniel is easier to understand and modify, and Daniel doesn't use a slow sort.
You can use comm command in linux, like this comm -3 1st.dat 2nd.dat.
It will give you the desired output.
diff -c file1 file2 | grep '^- \|^+ ' | tr '+' ' ' | tr '-' ' '
file 1:
a
b
c
d
file 2:
d
e
f
g
output
a
b
c
e
f
g
I have file like this
01 10 a
11 20 b
21 30 c
31 40 d
41 50 e
I want to input a number and compare with 1st and 2nd column and to print corresponding 3rd column
For example if I enter 23 it should display c, if I enter 45 it should display e
egrep "^${DIGIT_1}[0-9] ${DIGIT_2}[0-9]" file | awk '{print $3}'
DIGIT_1 is 2 and DIGIT_2 is 3 in your example
Use this simple script
#!/bin/sh
echo "Enter the number"
read num
while read line
do
set -- $line
if [ $num -ge $1 ] && [ $num -le $2 ] ;then
echo $3
exit 1
fi
done < filename
echo "not found"
another awk approach
awk -v d=<yournumber> '{dt=int(d/10);du=d-dt*10;c1=int($1/10);c2=int($2/10);if(dt==c1&&du==c2)print $3}' <yourfile>
In a simple awk script:
% awk -vfirst=2 -vsecond=3 '
$1 ~ first && $2 ~ second { print $3 }
' file-like-this
c
% awk -vfirst=4 -vsecond=5 '
$1 ~ first && $2 ~ second { print $3 }
' file-like-this
e
You can get awk to determine the first and second digits of a number like so:
% awk -vnumber=45 '
BEGIN { first = int(number / 10); second = number % 10 }
$1 ~ first && $2 ~ second { print $3 }
' file-like-this
e
Simple awk approach
cat file
01 10 a
11 20 b
21 30 c
31 40 d
41 50 e
i=40
awk 'inp<=$2 {f=$3;exit} END {print $3}' inp=$i file
d
have this text file:
name, age
joe,42
jim,20
bob,15
mike,24
mike,15
mike,54
bob,21
Trying to get this (count):
joe 1
jim 1
bob 2
mike 3
Thanks,
$ awk -F, 'NR>1{arr[$1]++}END{for (a in arr) print a, arr[a]}' file.txt
joe 1
jim 1
mike 3
bob 2
EXPLANATIONS
-F, splits on ,
NR>1 treat lines after line 1
arr[$1]++ increment array arr (split with ,) with first column as key
END{} block is executed at the end of processing the file
for (a in arr) iterating over arr with a key
print a print key , arr[a] array with a key
Strip the header row, drop the age field, group the same names together (sort), count identical runs, output in desired format.
tail -n +2 txt.txt | cut -d',' -f 1 | sort | uniq -c | awk '{ print $2, $1 }'
output
bob 2
jim 1
joe 1
mike 3
It looks like you want sorted output. You could simply pipe or print into sort -nk 2:
awk -F, 'NR>1 { a[$1]++ } END { for (i in a) print i, a[i] | "sort -nk 2" }' file
Results:
jim 1
joe 1
bob 2
mike 3
However, if you have GNU awk installed, you can perform the sorting without coreutils. Here's the single process solution that will sort the array by it's values. The solution should still be quite quick. Run like:
awk -f script.awk file
Contents of script.awk:
BEGIN {
FS=","
}
NR>1 {
a[$1]++
}
END {
for (i in a) {
b[a[i],i] = i
}
n = asorti(b)
for (i=1;i<=n;i++) {
split (b[i], c, SUBSEP)
d[++x] = c[2]
}
for (j=1;j<=n;j++) {
print d[j], a[d[j]]
}
}
Results:
jim 1
joe 1
bob 2
mike 3
Alternatively, here's the one-liner:
awk -F, 'NR>1 { a[$1]++ } END { for (i in a) b[a[i],i] = i; n = asorti(b); for (i=1;i<=n;i++) { split (b[i], c, SUBSEP); d[++x] = c[2] } for (j=1;j<=n;j++) print d[j], a[d[j]] }' file
A strictly awk solution...
BEGIN { FS = "," }
{ ++x[$1] }
END { for(i in x) print i, x[i] }
If name, age is really in the file, you could adjust the awk program to ignore it...
BEGIN { FS = "," }
/[0-9]/ { ++x[$1] }
END { for(i in x) print i, x[i] }
I come up with two functions based on the answers here:
topcpu() {
top -b -n1 \
| tail -n +8 \
| awk '{ print $12, $9, $10 }' \
| awk '{ CPU[$1] += $2; MEM[$1] += $3 } END { for (k in CPU) print k, CPU[k], MEM[k] }' \
| sort -k3 -n \
| tail -n 10 \
| column -t \
| tac
}
topmem() {
top -b -n1 \
| tail -n +8 \
| awk '{ print $12, $9, $10 }' \
| awk '{ CPU[$1] += $2; MEM[$1] += $3 } END { for (k in CPU) print k, CPU[k], MEM[k] }' \
| sort -k2 -n \
| tail -n 10 \
| column -t \
| tac
}
$ topcpu
chrome 0 75.6
gnome-shell 6.2 7
mysqld 0 4.2
zsh 0 2.2
deluge-gtk 0 2.1
Xorg 0 1.6
scrcpy 0 1.6
gnome-session-b 0 0.8
systemd-journal 0 0.7
ibus-x11 6.2 0.7
$ topmem
top 12.5 0
Xorg 6.2 1.6
ibus-x11 6.2 0.7
gnome-shell 6.2 7
chrome 6.2 74.6
adb 6.2 0.1
zsh 0 2.2
xdg-permission- 0 0.2
xdg-document-po 0 0.1
xdg-desktop-por 0 0.4
enjoy!
cut -d',' -f 1 file.txt |
sort | uniq -c
2 bob
1 jim
1 joe
3 mike
I need distinct values from the below columns:
AA|BB|CC
a#gmail.com,c#yahoo.co.in|a#gmail.com|a#gmail.com
y#gmail.com|x#yahoo.in,z#redhat.com|z#redhat.com
c#gmail.com|b#yahoo.co.in|c#uix.xo.in
Here records are '|' seperated and in the 1st column, we can two email id's which are ',' seperated. so, I want to consider that also. I want distinct email id's in the AA,BB,CC column, whether it is '|' seperated or ',' seperated.
Expected output:
c#yahoo.co.in|a#gmail.com|
y#gmail.com|x#yahoo.in|z#redhat.com
c#gmail.com|b#yahoo.co.in|c#uix.xo.in
is awk unix enough for you?
{
for(i=1; i < NF; i++) {
if ($i ~ /#/) {
mail[$i]++
}
}
}
END {
for (x in mail) {
print mail[x], x
}
}
output:
$ awk -F'[|,]' -f v.awk f1
2 z#redhat.com
3 a#gmail.com
1 x#yahoo.in
1 c#yahoo.co.in
1 c#gmail.com
1 y#gmail.com
1 b#yahoo.co.in
Using awk :
cat file | tr ',' '|' | awk -F '|' '{ line=""; for (i=1; i<=NF; i++) {if ($i != "" && list[NR"#"$i] != 1){line=line $i "|"}; list[NR"#"$i]=1 }; print line}'
Prints :
a#gmail.com|c#yahoo.co.in|
y#gmail.com|x#yahoo.in|z#redhat.com|
c#gmail.com|b#yahoo.co.in|c#uix.xo.in|
Edit :
Now works properly with inputs such as :
a#gmail.com|c#yahoo.co.in|
y#gmail.com|x#yahoo.in|a#gmail.com|
c#gmail.com|c#yahoo.co.in|c#uix.xo.in|
Prints :
a#gmail.com|c#yahoo.co.in|
y#gmail.com|x#yahoo.in|a#gmail.com|
c#gmail.com|c#yahoo.co.in|c#uix.xo.in|
The following python code will solve your problem:
#!/usr/bin/env python
while True:
try:
addrs = raw_input()
except EOFError:
break
print '|'.join(set(addrs.replace(',', '|').split('|')))
In Bash only:
while read s; do
IFS='|,'
for e in $s; do
echo "$e"
done | sort | uniq
unset IFS
done
This seems to work, although I'm not sure what to do if there are more than three unique mails. Run with awk -f filename.awk dataname.dat
BEGIN {IFS=/[,|]/}
NF {
delete uniqmails;
for (i=1; i<=NF; i++)
uniqmails[$i] = 1;
sep="";
n=0;
for (m in uniqmails) {
printf "%s%s", sep, m;
sep="|";
n++;
}
for (;n<3;n++) printf "|";
print ""; // EOL
}
There's also this "one-liner" that doesn't need awk:
while read line; do
echo $line | tr ",|" "\n" | sort -u |\
paste <( seq 3) - | cut -f 2 |\
tr "\n" "|" |\
rev | cut -c 2- | rev;
done
With perl:
perl -lane '$s{$_}++ for split /[|,]/; END { print for keys %s;}' input
I have edited this post, Hope it will work
while read line
do
val1=`echo $line|awk -F"|" '{print $1}'`
val2=`echo $line|awk -F"|" '{print $2}'`
val3=`echo $line|awk -F"|" '{print $3}'`
a=`echo $line|awk -F"|" '{print $2,"|",$3}'|sed 's/'$val1'//g'`
aa=`echo "$val1|$a"`
b=`echo $aa|awk -F"|" '{print $1,"|",$3}'|sed 's/'$val2'//g'`
b1=`echo $b|awk -F"|" '{print $1}'`
b2=`echo $b|awk -F"|" '{print $2}'`
bb=`echo "$b1|$val2|$b2"`
c=`echo $bb|awk -F"|" '{print $1,"|",$2}'|sed 's/'$val3'//g'`
cc=`echo "$c|$val3"|sed 's/,,/,/;s/,|/|/;s/|,/|/;s/^,//;s/ //g'`
echo "$cc">>abcd
done<ab.dat
cat abcd
c#yahoo.co.in||a#gmail.com
y#gmail.com|x#yahoo.in|z#redhat.com
c#gmail.com|b#yahoo.co.in|c#uix.xo.in
You can subtract all "," separated values and parse in the same way...if your all values are having "," separated.