R: reading and manipulating a strangely formatted file - r

Question

I have a file that is formatted in a slightly weird way, like so:
Cluster 1 Score:3.96
Category Term Count
GOTERM_BP_FAT GO:0006412 34
KEGG_PATHWAY hsa00970 9
GOTERM_BP_FAT GO:0043038 9
GOTERM_BP_FAT GO:0043039 9
Cluster 2 Score:3.94
Category Term Count
GOTERM_BP_FAT GO:0006414 21
KEGG_PATHWAY hsa03010 20
GOTERM_BP_FAT GO:0034660 16
GOTERM_BP_FAT GO:0006399 11
GOTERM_BP_FAT GO:0042254 10
GOTERM_BP_FAT GO:0022613 12
... and several more "sub-data frames" (including space in-between) and additional (here omitted) columns for the rows after the Cluster X rows.
What I want to do is to somehow read each separate cluster, get it as a data frame (i.e. a data frame with the names Category, Term, Count), manipulate the data frame a bit (adding columns based on calculations, mostly) and then write the manipulated data frame AND the the Cluster X row to a new file on the very same format as it started.
I've racked my brain for some smart way to do this, but I haven't really come up with anything other than reading each row separately and doing different things depending on the type of row, like this:
con <- file('test.txt', open="r")
# Read file line for line
while ( length(currentLine <- readLines(con, n=1, warn=FALSE)) > 0 ) {
line = strsplit(currentLine, '\t')[[1]]
# Save previous data, initiate new cluster name/score
if ( grepl('Annotation Cluster', line[1]) ) {
# Save previous data if available
if ( exists('currentData') ) {
## save the current data somehow
}
# Initiate new
clusterInfo = line
}
# Initiate new, empty data frame
else if ( grepl('Category', line[1]) ) {
currentData = data.frame(t(rep(NA, length(line))))
names(currentData) = line
}
# Add data to data frame
else if ( grepl('GOTERM', line[1]) || grepl('KEGG', line[1]) ) {
currentData = rbind(currentData, line)
# Delete NAs if line row
if ( nrow(currentData) == 2 ) {
currentData = na.omit(currentData)
}
}
}
The above is obviously not finished (I'm not sure how to save the clusterInfo together with currentData to the same format), but I hope I get across my idea. I'm not really too fond of this approach, though... It seems very odd, to me, to create data frames row-by-row like this, and try and save the data at the same time as you're initiating the start of the next block of data.
Is there some better way of doing this?

Answer 1

You can try read.mtable from my GitHub-only "SOfun" package.
Usage would be something like:
library(SOfun)
read.mtable(x, "Cluster", header = TRUE) ## Replace "x" with your file name
# $`Cluster 1 Score:3.96`
# Category Term Count
# 1 GOTERM_BP_FAT GO:0006412 34
# 2 KEGG_PATHWAY hsa00970 9
# 3 GOTERM_BP_FAT GO:0043038 9
# 4 GOTERM_BP_FAT GO:0043039 9
#
# $`Cluster 2 Score:3.94`
# Category Term Count
# 1 GOTERM_BP_FAT GO:0006414 21
# 2 KEGG_PATHWAY hsa03010 20
# 3 GOTERM_BP_FAT GO:0034660 16
# 4 GOTERM_BP_FAT GO:0006399 11
# 5 GOTERM_BP_FAT GO:0042254 10
# 6 GOTERM_BP_FAT GO:0022613 12
As you acn see, the "cluster" information is retained as the list names. Thus, you can go ahead and use lapply to do whatever calculations you need to do, and then re-write the data in whatever form you need to.
Reproducible sample data:
x <- tempfile()
writeLines("Cluster 1 Score:3.96
Category Term Count
GOTERM_BP_FAT GO:0006412 34
KEGG_PATHWAY hsa00970 9
GOTERM_BP_FAT GO:0043038 9
GOTERM_BP_FAT GO:0043039 9
Cluster 2 Score:3.94
Category Term Count
GOTERM_BP_FAT GO:0006414 21
KEGG_PATHWAY hsa03010 20
GOTERM_BP_FAT GO:0034660 16
GOTERM_BP_FAT GO:0006399 11
GOTERM_BP_FAT GO:0042254 10
GOTERM_BP_FAT GO:0022613 12", con = x, sep = "\n")

Answer 2

You could read the file with readLines and split it with a numeric index ('indx') created based on the lines having 'Cluster'. Read the list elements with read.table, create two new columns ('Cluster' and 'Score') and rbind the list elements to create a single dataset.
lines <- readLines('Clusterfile.txt')
indx <- cumsum(grepl('^Cluster', lines))
res <- do.call(rbind,lapply(split(lines, indx), function(x) {
d1 <-read.table(text=x[-1], header=TRUE, stringsAsFactors=FALSE)
d2 <- read.table(text=gsub('[^0-9.]+', ' ', x[1]),
col.names=c('Cluster', 'Score'))
cbind(d1, d2)}))
row.names(res) <- NULL
head(res,3)
# Category Term Count Cluster Score
#1 GOTERM_BP_FAT GO:0006412 34 1 3.96
#2 KEGG_PATHWAY hsa00970 9 1 3.96
#3 GOTERM_BP_FAT GO:0043038 9 1 3.96