R data frame column comparison without for loops, complex case - r

Question

Let's suppose I have a data frame with the following parameters
DATA <- data.frame(ROWID, ID1, NAME1, ...IDn, NAMEn)
Sample of what the data might look like:
ROWID | ID1 | NAME1 | ID2 | NAME2 | IDn | NAMEn
001 | 001 | FAS | 002 | MAS | 999 | ZOO
002 | 003 | BIN | 004 | DUN | 998 | SOO
Where I have 201 columns by 10k+ rows. What I would like to do is to reshape this data such that for each row in the original DATA, I produce a set of rows in a subsequent data frame. Each row would consist of the originating ROWID, IDa, NAMEa, IDb, NAMEb pairs such that the first is matched with all others (99 pairs containing ID1, 98 with ID2, and so on). This would occur for each row producing a large data frame of all possible combinations within rows for every row. The result would look like:
ROWID1 | ID1 | NAME1 | ID2 | NAME2
ROWID1 | ID1 | NAME1 | ID3 | NAME3
...
ROWID1 | ID2 | NAME2 | ID3 | NAME3
...
ROWID2 | ID1 | NAME1 | ID2 | NAME2
ROWID2 | ID1 | NAME1 | ID3 | NAME3
...
The code I produced to do this is as follows. It works great, but only on smaller data frames. The full data frame is painfully slow, and I am hoping to have alternatives to speed it up using functions or something else of which I am unaware. Thanks in advance!!
DATA <- data.frame(as described above)
META <- data.frame(ROWID=numeric(0),ID1=numeric(0),
BUS1=character(0),ID2=numeric(0),BUS2=character(0))
for (i in 1:length(DATA$ROWID)) {
SET <- data.frame(ROWID=numeric(0),ID1=numeric(0),
BUS1=character(0),ID2=numeric(0),BUS2=character(0))
ROWID <- DATA[i,1]
for (x in seq(3,ncol(DATA),2)) {
for (y in seq(x,ncol(DATA),2)) {
ID1 <- DATA[i,x-2]
BUS1 <- DATA[i,x]
ID2 <- DATA[i,y-2]
BUS2 <- DATA[i,y]
if (!is.na(BUS1) && !is.na(BUS2)) {
NEW <- cbind(ROWID, ID1, BUS1, ID2, BUS2)
SET <- rbind(SET, NEW)
}
}
}
META <- rbind(META, SET)
}

Answer 1

Here is my way to write it, which includes all the 3 optimizations I wrote as comments. Also, be careful! your code had some bugs in the addressing the columns... which I hopefully also fixed.
require('compiler')
enableJIT(3)
DATA2 = as.matrix(DATA)
META2 <- matrix(character(),ncol=5,nrow=(nrow(DATA2)*(ncol(DATA2)-2)^2/2)) # you want a matrix instead of a data.frame, and you want to pre-allocate its size
colnames(META2) = c("ROWID","ID1","BUS1","ID2","BUS2")
k=0
for (i in 1:nrow(DATA2)) {
for (x in seq(3,ncol(DATA2)-2,2)) {
for (y in seq(x+2,ncol(DATA2),2)) {
k=k+1
META2[k,] = c(DATA2[i,1],DATA2[i,x-1], DATA2[i,x], DATA2[i,y-1], DATA2[i,y]) # no need to use temporary variables
}
}
}
META2 = as.data.frame(META2) # converting back to data.frame
META2$BUS1 = as.numeric(META2$BUS1)
META2$BUS2 = as.numeric(META2$BUS2)
I will let you handle yourself the case in which BUS1 or BUS2 is NA - basically, you need to not add these lines (and not increment the variable k), and after the loops you need to crop your matrix to remove the trailing empty rows.