How to print number of particular columns in shell script? - unix

Question

I have a text file temp1 and say it has more than 20 columns and it has numerical values like as follows,
1,0,3,0,5........,
1,0,5,0,8........,
3,0,6,0,3........,
5,0,6,0,4........,
.................,
I want to remove the columns which has the total(sum) of zero and i need to redirect remaining columns to the new file
ie : for example as above 2nd and 4th columns have the total of zero so i need to remove 2nd and 4 th column and redirect it to separate file .
can any one help me pls?

Answer 1

$ cat file
1,0,3,0,5
1,0,5,0,8
3,0,6,0,3
5,0,6,0,4
$ awk -f tst.awk file
1,3,5
1,5,8
3,6,3
5,6,4
$ cat tst.awk
BEGIN{ FS="," }
{
for (j=1;j<=NF;j++) {
val[NR,j] = $j
sum[j] += val[NR,j]
}
}
END {
for (i=1;i<=NR;i++) {
ofs = ""
for (j=1;j<=NF;j++) {
if (sum[j]) {
printf "%s%s",ofs,val[i,j]
ofs = FS
}
}
print ""
}
}

Answer 2

You can use awk: (the following is ugly but I hope readable. That's the goal. I let better awkist enhance/reduce it further)
If the data is in file /path/to/zefile:
awk -F',' '
FNR==NR { for (col=1;col<=NF;col++)
{ if ($col != 0)
{wewantthiscolumn[col]=1 }
}
next
}
{ for (col=1;col<=NF;col++)
{ if (wewantthiscolumn[col]==1)
{ printf ("%s,",$col) }
}
print ""
}' /path/to/zefile /path/to/zefile | sed -e 's/,$//'
The idea: we launch awk on /path/to/zefile /path/to/zefile (hence, it read is twice).
On the first pass, we create a "wewantthiscolumn" array. This array contains "1" as soon as that column has something different from 0. The "next" ensure we only do this bit when FNR (=Number of Rows in the CURRENT file) == NR (=total number of rows), which is true only on the first pass.
On the second pass (hence we go directly to the 2nd { } as now NR>FNR) : we only display the column value $col which has a corresponding wewantthiscolumn(col)==1, and followed by a "," (so there is a little problem: the last col will have a "," after it)
Then we pass this through sed to get rid of the ",$" bit.
I am not sure there is not a very better way : can awk delete a field? so it could delete field col on the 2nd pass? Then it would be much easier to print the resulting $0, setting OFS=',' to have them separated with , ...
This would make the 2nd pass:
awk -F',' '
FNR==NR { for (col=1;col<=NF;col++)
{ if ($col != 0)
{wewantthiscolumn[col]=1 }
}
next
}
{ for (col=1;col<=NF;col++)
{ if (wewantthiscolumn[col]==0)
$col="DELETETHIS"
}
gensub(",DELETETHIS","",g)
gensub("DELETETHIS,","",g)
print $0
}' /path/to/zefile /path/to/zefile
I didn't want to assume no columns could be empty, hence I use "DELETETHIS" to make sure I only delete relevant fields... But this means the 1st way is in fact simpler ^^ : only print the fields you need, and then get rid of the "," at the end of line.

Answer 3

Here's one way using awk. Run like:
awk -f ./script.awk file{,}
Contents of script.awk:
BEGIN {
FS=","
}
FNR==NR {
for(i=1;i<=NF;i++) {
if ($i != 0) {
a[i]
}
}
next
}
{
for(j=1;j<=NF;j++) {
if (j in a) {
printf "%s%s", $j, (j==NF ? RS : FS)
}
}
}
Alternatively, here's the one-liner:
awk -F, 'FNR==NR { for(i=1;i<=NF;i++) if ($i != 0) a[i]; next } { for(j=1;j<=NF;j++) if (j in a) printf "%s%s", $j, (j==NF ? RS : FS) }' filex{,}
Contents of file:
1,0,3,0,5,0
1,0,5,0,8,1
3,0,6,0,3,2
5,0,6,0,4,5
Results:
1,3,5,0
1,5,8,1
3,6,3,2
5,6,4,5

Answer 4

A solution using python:
#!/usr/bin/env python
def transpose(grid):
return zip(*grid)
def removeBlankRows(grid):
return [list(row) for row in grid if any(map(int,row))]
grid = []
with open("input.csv") as fd:
for line in fd:
grid.append(line.strip().split(','))
data = removeBlankRows(transpose(removeBlankRows(transpose(grid))))
for i in data:
print ",".join(i)
input:
1,0,3,0,5
1,0,5,0,8
3,0,6,0,3
5,0,6,0,4
output:
1,3,5
1,5,8
3,6,3
5,6,4
input:
1,0,3,0,5
1,0,5,0,8
3,0,6,0,3
5,0,6,1,4
output:
1,3,0,5
1,5,0,8
3,6,0,3
5,6,1,4