Lists to Dataframes with NA or 0 - r

Question

I have a list with 138 tables in it (prop.table). Each table can have up to 20 variables in it (numerical categories ranging from 11-95 as the colnames). I need to convert this list to a master dataframe. The first three tables look like this:
[[1]]
x
21 41 42 43 52 71 81 82
0.02007456 0.58158876 0.22483510 0.09349011 0.05248064 0.01204474 0.00544881 0.01003728
[[2]]
x
21 41 42 43 52 71 90
0.01175122 0.36973345 0.34107194 0.03066781 0.08655775 0.01633706 0.14388077
[[3]]
x
21 22 23 41 42
0.043254082 0.008307075 0.016614151 0.930392438 0.001432254
I need to convert this to a matrix so it looks like this, with NAs or 0 when the categorical variable is not available:
x<-matrix (nrow=3, ncol=11 )
colnames(x) <-c('21', '22', '23', '41', '42', '43', '52', '71', '81', '82', '90' )
I have tried using this line from a previous similar question but the table is not correct:
df <- data.frame(matrix(unlist(prop.table), nrow=138, byrow=T))
Any suggestions on how to resolve this issue and get the table I need?

Answer 1

Is this is what you want?
x1 <- c(1, 5, 7)
names(x1) <- 1:3
x2 <- c(1, 2, 7)
names(x2) <- c(1,3,5)
l <- list(x1, x2)
m <- matrix(nrow=length(l), ncol=5)
colnames(m) <- 1:5
for (i in 1:length(l)) {
m[i, names(l[[i]])] <- l[[i]]
}
Maybe one can replace the loop with an apply function, but I'm not sure...Basically, I loop through the list and set in every row of the matrix those columns that match with the names of the vector in the list.
Sorry for not using your data set, but you didn't have the code at hand and I was too lazy to type it out.

Answer 2

rbind.fill from the plyr package will do just this for you:
# make an example `prop.table`:
tbl <- 1:10
names(tbl) <- letters[1:10]
tbl <- as.matrix(tbl)
# make sure some of the columns are missing
prop.table <- list(tbl[sample(10, size=8),], tbl[sample(10, size=7),], tbl[sample(10, size=9),])
# [[1]]
# d b g c h f e i
# 4 2 7 3 8 6 5 9
# [[2]]
# h g d a j f c
# 8 7 4 1 10 6 3
# [[3]]
# c i b d j a h g e
# 3 9 2 4 10 1 8 7 5
You can use the rbind.fill function from plyr, which is just rbind but it fills missing columns out with NA. It can take in a list of data frames to rbind together, so I convert each element of prop.table into a dataframe first (needed the t to ensure each prop.table[[i]] was treated as a row, not a column)
rbind.fill(lapply(prop.table, function (x) as.data.frame(t(x))))
# d b g c h f e i a j
# 1 4 2 7 3 8 6 5 9 NA NA
# 2 4 NA 7 3 8 6 NA NA 1 10
# 3 4 2 7 3 8 NA 5 9 1 10
(Note - you can sort the columns of the output dataframe with x[, order(colnames(x))])

Answer 3

Here is simple way to using lapply, rbind and do.call
ptl
## [[1]]
## x
## 21 41 42 43 52 71 81 82
## 0.02007456 0.58158876 0.22483510 0.09349011 0.05248064 0.01204474 0.00544881 0.01003728
##
## [[2]]
## x
## 21 41 42 43 52 71 90
## 0.01175122 0.36973345 0.34107194 0.03066781 0.08655775 0.01633706 0.14388077
##
## [[3]]
## x
## 21 22 23 41 42
## 0.043254082 0.008307075 0.016614151 0.930392438 0.001432254
##
## [[4]]
## x
## 21 22 31 41 42 43 81
## 0.10028653 0.03123209 0.00487106 0.66103152 0.03037249 0.01604585 0.15616046
##
## [[5]]
## x
## 21 41 42 43 81
## 0.0662080825 0.8291774147 0.0005732302 0.0865577529 0.0174835196
##
## [[6]]
## x
## 21 22 31 41 42 43 81
## 0.081948424 0.002292264 0.006303725 0.825501433 0.029226361 0.020630372 0.034097421
##
# Get unique names of all columns in tables in the list
resCol <- unique(unlist(lapply(ptl, names)))
# Get dimensions of desired result
nresCol <- length(resCol)
nresRow <- length(ptl)
# Create 'Template' data.frame row
DF <- as.data.frame(matrix(rep(0, nresCol), nrow = 1, dimnames = list(1, resCol)))
# for every table in list, create copy of DF, fill it appropriately, then rbind result together using do.call
result <- do.call(rbind, lapply(ptl, function(x) {
retDF <- DF
retDF[, names(x)] <- x
return(retDF)
}))
# rename rows(optional)
rownames(result) <- 1:nrow(result)
result
## 21 41 42 43 52 71 81 82 90 22 23 31
## 1 0.02007456 0.5815888 0.2248351018 0.09349011 0.05248064 0.01204474 0.00544881 0.01003728 0.0000000 0.000000000 0.00000000 0.000000000
## 2 0.01175122 0.3697334 0.3410719404 0.03066781 0.08655775 0.01633706 0.00000000 0.00000000 0.1438808 0.000000000 0.00000000 0.000000000
## 3 0.04325408 0.9303924 0.0014322544 0.00000000 0.00000000 0.00000000 0.00000000 0.00000000 0.0000000 0.008307075 0.01661415 0.000000000
## 4 0.10028653 0.6610315 0.0303724928 0.01604585 0.00000000 0.00000000 0.15616046 0.00000000 0.0000000 0.031232092 0.00000000 0.004871060
## 5 0.06620808 0.8291774 0.0005732302 0.08655775 0.00000000 0.00000000 0.01748352 0.00000000 0.0000000 0.000000000 0.00000000 0.000000000
## 6 0.08194842 0.8255014 0.0292263610 0.02063037 0.00000000 0.00000000 0.03409742 0.00000000 0.0000000 0.002292264 0.00000000 0.006303725

Answer 4

I'm just going to suggest one solution. How about you just concatenate all of the lists in one. So you would have
MyDataFrame
variable1 1 1 1 1 1 1 1 1
variable2 21 41 42 43 52 71 81 82
variable30.02007456 0.58158876 0.22483510 0.09349011 0.05248064 0.01204474 0.00544881 0.01003728
variable1 2 2 2 2 2 2 2
variable2 21 41 42 43 52 71 90
variable30.01175122 0.36973345 0.34107194 0.03066781 0.08655775 0.01633706 0.14388077
variable1 3 3 3 3 3
variable2 21 22 23 41 42
variable30.043254082 0.008307075 0.016614151 0.930392438 0.001432254
And once you have only one data frame. You can use the reshape function. like
install.packages('reshape')
library('reshape')
cast(MyDataFrame, variable1~variable2)

Answer 5

This won't be the most efficient, but using plyr and reshape2, and assuming your list of prop.tables is called foo
library(plyr)
library(reshape2)
allData <- dcast(ldply(lapply(seq_along(foo), function(x) data.frame(foo[[x]], id = x))),
id ~ x, value.var = 'Freq')
or more straight forwardly
ff <- c('21', '22', '23', '41', '42', '43', '52', '71', '81', '82', '90' )
t(sapply(foo, function(x,y) {x[ff]} ))