creating a data frame with lapply(and plyr package) - r

Question

I have multiple data files in which im interested in cleaning up then obtaining means from to run repeated measures ANOVA on.
Here's example data, in real data theres 4500 rows and another line called Actresponse which sometimes contains a 9 which I trim around : https://docs.google.com/file/d/0B20HmmYd0lsFVGhTQ0EzRFFmYXc/edit?pli=1
I have just discovered plyr and how awesome it is for manipulating data, but the way I'm using it right now looks rather stupid to me. I have 4 different things I"m interested in that I want to read into a data frame. I've read them in to 4 separate data frames to start, I'm wondering if there is a way I can combine this and read all the means into one data frame (a row for each reqresponse of each file) with less lines of code. Basically, can I achieve what I've done here without rewriting a lot of the same code 4 times?
PMScoreframe <- lapply(list.files(pattern='^[2-3].txt'),function(ff){
data <- read.table(ff, header=T, quote="\"")
data <- data[-c(seq(from = 1, to = 4001, by=500), seq(from = 2, to = 4002, by=500)), ]
ddply(data[data$Reqresponse==9,],.(Condition,Reqresponse),summarise,Score=mean(Score))
})
PMRTframe <- lapply(list.files(pattern='^[2-3].txt'),function(ff){
data <- read.table(ff, header=T, quote="\"")
data <- data[data$RT>200,]
data <- ddply(data,.(Condition),function(x) x[!abs(scale(x$RT)) > 3,])
ddply(data[data$Reqresponse==9,],.(Condition,Reqresponse,Score),summarise,RT=mean(RT))
})
OtherScoreframe <- lapply(list.files(pattern='^[2-3].txt'),function(ff){
data <- read.table(ff, header=T, quote="\"")
data <- data[-c(seq(from = 1, to = 4001, by=500), seq(from = 2, to = 4002, by=500)), ]
select <- rep(TRUE, nrow(data))
index <- which(data$Reqresponse==9|data$Actresponse==9|data$controlrepeatedcue==1)
select[unique(c(index,index+1,index+2))] <- FALSE
data <- data[select,]
ddply(data[data$Reqresponse=="a"|data$Reqresponse=="b",],. (Condition,Reqresponse),summarise,Score=mean(Score))
})
OtherRTframe <- lapply(list.files(pattern='^[2-3].txt'),function(ff){
data <- read.table(ff, header=T, quote="\"")
data <- data[-c(seq(from = 1, to = 4001, by=500), seq(from = 2, to = 4002, by=500)), ]
select <- rep(TRUE, nrow(data))
index <- which(data$Reqresponse==9|data$Actresponse==9|data$controlrepeatedcue==1)
select[unique(c(index,index+1,index+2))] <- FALSE
data <- data[select,]
data <- data[data$RT>200,]
data <- ddply(data,.(Condition),function(x) x[!abs(scale(x$RT)) > 3,])
ddply(data[data$Reqresponse=="a"|data$Reqresponse=="b",],.(Condition,Reqresponse,Score),summarise,RT=mean(RT))
})

Answer 1

I think this deals with what you're trying to do. Basically, I think you need to read all the data in once, then deal with that data.frame. There are several questions dealing with how to read it all in, here is how I would do it so I maintain a record of which file each row in the data.frame comes from, which can also be used for grouping:
filenames <- list.files(".", pattern="^[2-3].txt")
import <- mdply(filenames, read.table, header = T, quote = "\"")
import$file <- filenames[import$X1]
Now import is a big dataframe with all your files in it (I'm assuming your pattern recognition etc for reading in files is correct). You can then do summaries based on whatever criteria you like.
I'm not sure what you're trying to achieve in line 3 of your code above, but for the ddply below that, you just need to do:
ddply(import[import$Reqresponse==9,],.(Condition,Reqresponse,file),summarise,Score=mean(Score))
There's so much going on in the rest of your code that it's hard to make out exactly what you want.
I think the important thing is that to make this efficient, and easier to follow, you need to read your data in once, then work on that dataset - making subsets if necessary, doing summary stats or whatever else it is.
As an example of how you can work with this, here's an attempt to deal with your problem of dealing with trials (rows?) that have reqresponse == 9 and the following two. There are probably ways of doing this more efficiently, but this is slightly based on how you were doing it to show you briefly how to work with the larger dataframe. Now modified to remove the first two trials of each file:
import.clean <- ddply(import, .(file), function(x) {
index <- which(x$reqresponse == 9)
if(length(index) > 0) {
index <- unique(c(index, index + 1, index + 2, 1, 2))
}
else index <- c(1,2)
x <- x[-index,]
return(x)
})