Looping and storing results over many data frames - r

Question

I want to to perform at least six looping steps in R. My data sets are 28 files stored in one folder. Each file has 22 rows (21 individual cases and one row for column names) and columns as follows: Id, id, PC1, PC2….PC20.
I intend to:
read each file into R as a data frame
delete first column named “Id” in the each data frame
arrange each data frame as follows:
first column should be “id” and
next ten columns should be first ten PCs (PC1, PC2, …PC10)
sort each data frame according to “id” (data frames should have the same order of individuals and their respective PC’s scores)
perform pairwise comparison by protest function in the vegan package among all possible pair’s combinations (378 combinations)
store result of each pair’s comparison in a symmetric (28*28) matrix which will be used in further analysis
At the moment I am able to do it manually for each pair of data (code is below):
## 1. step
## read files into R as a data frame
c_2d_hand_1a<-read.table("https://googledrive.com/host/0B90n5RdIvP6qbkNaUG1rTXN5OFE/PC scores, c_2d_hand-1a, Symmetric component.txt",header=T)
c_2d_hand_1b<-read.table("https://googledrive.com/host/0B90n5RdIvP6qbkNaUG1rTXN5OFE/PC scores, c_2d_hand-1b, Symmetric component.txt",header=T)
## 2. step
## delete first column named “Id” in the each data frame
c_2d_hand_1a[,1]<-NULL
c_2d_hand_1b[,1]<-NULL
## 3. step
## arrange each data frame that have 21 rows and 11 columns (id,PC1,PC2..PC10)
c_2d_hand_1a<-c_2d_hand_1a[,1:11]
c_2d_hand_1b<-c_2d_hand_1b[,1:11]
## 4. step
## sort each data frame according to “id”
c_2d_hand_1a<-c_2d_hand_1a[order(c_2d_hand_1a$id),]
c_2d_hand_1b<-c_2d_hand_1b[order(c_2d_hand_1b$id),]
## 5. step
## perform pairwise comparison by protest function
library(permute)
library(vegan)
c_2d_hand_1a_c_2d_hand_1b<-protest(c_2d_hand_1a[,2:ncol(c_2d_hand_1a)],c_2d_hand_1b[,2:ncol(c_2d_hand_1b)],permutations=10000)
summary(c_2d_hand_1a_c_2d_hand_1b)[2] ## or c_2d_hand_1a_c_2d_hand_1b[3]
Since I am a newbie in data handling/manipulation in R, my self-learning skills are suitable to perform respective steps manually, typing codes for each data set and perform each pairwise comparisons at the time. Since I need to perform those six steps 378 times, manual typing would be exhaustive and time consuming.
I tried to import files as a list and tried several operations, but I was unsuccessful. Specifically, using list.files(), I made the list, called “probe”. I was able to select certain data frame using e.g. probe[2]. Also I could assess column “Id” by e.g. probe[2][1], and deleted it by probe[2][1]<-NULL. But when I tried to work with for loop, I was stuck.

Answer 1

This code is untested, but with some luck, it should work. The summary of the protest() results are stored in a matrix of lists.
# develop a way to easily reference all of the URLs
url.begin <- "https://googledrive.com/host/0B90n5RdIvP6qbkNaUG1rTXN5OFE/PC scores, "
url.middle <- c("c_2d_hand-1a", "c_2d_hand-1b")
url.end <- ", Symmetric component.txt"
L <- length(url.middle)
# read in all of the data and save it to a list of data frames
mybiglist <- lapply(url.middle, function(mid) read.table(paste0(url.begin, mid, url.end), header=TRUE))
# save columns 2 to 12 in each data frame and order by id
mybiglist11cols <- lapply(mybiglist, function(df) df[order(df$id), 2:12])
# get needed packages
library(permute)
library(vegan)
# create empty matrix of lists to store results
results <- matrix(vector("list", L*L), nrow=L, ncol=L)
# perform pairwise comparison by protest function
for(i in 1:L) {
for(j in 1:L) {
df1 <- mybiglist11cols[[i]]
df2 <- mybiglist11cols[[j]]
results[i, j] <- list(summary(protest(df1[, -1], df2[, -1], permutations=10000)))
}}