Assume a table as below:
X =
col1 col2 col3
row1 "A" "0" "1"
row2 "B" "2" "NA"
row3 "C" "1" "2"
I select combinations of two rows, using the code below:
pair <- apply(X, 2, combn, m=2)
This returns a matrix of the form:
pair =
[,1] [,2] [,3]
[1,] "A" "0" "1"
[2,] "B" "2" NA
[3,] "A" "0" "1"
[4,] "C" "1" "2"
[5,] "B" "2" NA
[6,] "C" "1" "2"
I wish to iterate over pair, taking two rows at a time, i.e. first isolate [1,] and [2,], then [3,] and [4,] and finaly, [5,] and [6,]. These rows will then be passed as arguments to regression models, i.e. lm(Y ~ row[i]*row[j]).
I am dealing with a large dataset. Can anybody advise how to iterate over a matrix two rows at a time, assign those rows to variables and pass as arguments to a function?
Thanks,
S ;-)
It is unnecessary to multiply the rows of your matrix like that, and if you have a large data set it is might get problematic. In stead just pick out the relevant rows for each instance. But it is convenient to create the selection beforehand, something like this perhaps:
xselect <- combn(1:nrow(X),2)
To illustrate with your data (assuming you only use columns 2 and 3):
X <- matrix(c("A", "B", "C", 0,2,1,1,NA,2),3,3)
Y <- rnorm(2, 4, 2)
for (i in 1:ncol(xselect))
{
x1 <- as.numeric(X[xselect[1,i], c(2,3)])
x2 <- as.numeric(X[xselect[2,i], c(2,3)])
print(lm(Y ~ x1 * x2))
}
I'm not sure exactly what you're trying to do with the linear models, but to iterate over X, a pair of rows at a time, make a factor for each pair, and then use by
fac <- as.factor(sort(rep(1:(nrow(X)/2), 2)))
by(X, fac, FUN)
where FUN is whatever function you want to apply over the pairs of rows in X.
Related
Im trying to convert a data set in a long format panel structure to an adjacency matrix or edge list to make network graphs. The data set contains articles each identified by an ID-number. Each article can appear several times under a number of categories. Hence I have a long format structure at the moment:
ID <- c(1,1,1,2,2,2,3,3)
Category <- c("A","B","C","B","E","H","C","E")
dat <- data.frame(ID,Category)
I want to convert this into an adjacency matrix or edge list. Where the edge list such look something like this
A B
A C
B C
B E
B H
E H
C E
Edit: I have tried dat <- merge(ID, Category, by="Category") but it returns the error message Error in fix.by(by.x, x) : 'by' must specify a uniquely valid column
Thanks in advance
Update: I ended up using the crossprod(table(dat)) from the comments, but the solution suggested by Navy Cheng below works just as well
This code will work
do.call(rbind,lapply(split(dat, dat$ID), function(x){
t(combn(as.vector(x$Category), 2))
}))
Update
As #Parfait 's suggestion, you can have by instead of split+lapply.
1) Use by to group nodes ("A", "B", "C" ...) by Category;
2) Use combn to create edge between nodes in each group, and t to transform the matrix for further rbind
> edge.list <- by(dat, dat$ID, function(x) t(combn(as.vector(x$Category), 2)))
dat$ID: 1
[,1] [,2]
[1,] "A" "B"
[2,] "A" "C"
[3,] "B" "C"
------------------------------------------------------------
dat$ID: 2
[,1] [,2]
[1,] "B" "E"
[2,] "B" "H"
[3,] "E" "H"
------------------------------------------------------------
dat$ID: 3
[,1] [,2]
[1,] "C" "E"
3) Then merge the list
> do.call(rbind, edge.list)
[,1] [,2]
[1,] "A" "B"
[2,] "A" "C"
[3,] "B" "C"
[4,] "B" "E"
[5,] "B" "H"
[6,] "E" "H"
[7,] "C" "E"
So if you are willing to convert your data.frame to a data.table this problem can be solved pretty efficiently and cleanly and if you have many rows will be much faster.
library(data.table)
dat<-data.table(dat)
Basically you can apply functions to columns of the data.table in the j cell and group in the k cell. So you want all the combinations of categories taken two at a time for each ID which looks like this:
dat[,combn(Categories,2),by=ID]
However stopping at this point will keep the ID column and by default create a column called V1 that basically concatenates the array returned by combn into a vector of the categories and not the two-column adjacency matrix that you need. But by chaining another call to this you can create the matrix easily as you would with any single vector. In one line of code this will look like:
dat[,combn(Category,2),by=ID][,matrix(V1,ncol=2,byrow = T)]
Remember that the vector column we wish to convert to a matrix is called V1 by default and also we want the 2-column matrix to be created by row instead of the default which is by column. Hope that helps and let me know if I need to add anything to my explanation. Good luck!
I have a list of lists of strings as follows:
> ll
[[1]]
[1] "2" "1"
[[2]]
character(0)
[[3]]
[1] "1"
[[4]]
[1] "1" "8"
The longest list is of length 2, and I want to build a data frame with 2 columns from this list. Bonus points for also converting each item in the list to a number or NA for character(0). I have tried using mapply() and data.frame to convert to a data frame and fill with NA's as follows.
# Find length of each list element
len = sapply(awards2, length)
# Number of NAs to fill for column shorter than longest
len = 2 - len
df = data.frame(mapply( function(x,y) c( x , rep( NA , y ) ) , ll , len))
However, I do not get a data frame with 2 columns (and NA's as fillers) using the code above.
Thanks for the help.
We can use stri_list2matrix from stringi. As the list elements are all character vectors, it seems okay to use this function
library(stringi)
t(stri_list2matrix(ll))
# [,1] [,2]
#[1,] "2" "1"
#[2,] NA NA
#[3,] "1" NA
#[4,] "1" "8"
If we need to convert to data.frame, wrap it with as.data.frame
This question already has answers here:
Finding ALL duplicate rows, including "elements with smaller subscripts"
(9 answers)
Closed 8 years ago.
I am trying to look at protein sequence homology using R, and I'd like to go through a data frame looking for identical pairs of Position and Letter. The data look similar to the frame below:
Letter <- c("A", "B", "C", "D", "D", "E", "G", "L")
Position <- c(1, 2, 3, 4, 4, 5, 6, 7)
data.set <- cbind(Position, Letter)
Which yields:
Position Letter
[1,] "1" "A"
[2,] "2" "B"
[3,] "3" "C"
[4,] "4" "D"
[5,] "4" "D"
[6,] "5" "E"
[7,] "6" "G"
[8,] "7" "L"
I'd like to loop through and find all identical observations (in this case, observations 4 and 5), but I'm having difficulty in discovering the best way to do it.
I'd like the resultant data frame to look like:
Position Letter
[1,] "4" "D"
[2,] "4" "D"
The ways I've tried to do this ended up yielding this code, but unfortunately it returns one value of TRUE because I realized that I am comparing two identical data frames:
> identical(data.set[1:nrow(data.set),1:2], data.set[1:nrow(data.set),1:2])
[1] TRUE
I'm not sure if looping through using the identical() function would be the best way? I'm sure there's a more elegant solution that I am missing.
Thanks for any help!
Try the unique function:
unique(data.set)
...
You can use duplicated using fromLast to go in two directions:
data.set[(duplicated(data.set)==T | duplicated(data.set, fromLast = TRUE) == T),]
# Position Letter
#[1,] "4" "D"
#[2,] "4" "D"
UPDATE: FIXED
This is fixed in the upcoming release of R 3.1.0. From the CHANGELOG:
combn(x, simplify = TRUE) now gives a factor result for factor input
x (previously user error).
Related to PR#15442
I just noticed a curious thing. Why does combn appear to unclass factor variables to their underlying numeric values for all except the first combination?
x <- as.factor( letters[1:3] )
combn( x , 2 )
# [,1] [,2] [,3]
#[1,] "a" "1" "2"
#[2,] "b" "3" "3"
This doesn't occur when x is a character:
x <- as.character( letters[1:3] )
combn( x , 2 )
# [,1] [,2] [,3]
#[1,] "a" "a" "b"
#[2,] "b" "c" "c"
Reproducible on R64 on OS X 10.7.5 and Windows 7.
I think it is due to the conversion to matrix done by the simplify parameter. If you don't use it you get:
combn( x , 2 , simplify=FALSE)
[[1]]
[1] a b
Levels: a b c
[[2]]
[1] a c
Levels: a b c
[[3]]
[1] b c
Levels: a b c
The fact that the first column is OK is due to the way combn works: the first column is specified separately and the other columns are then changed from the existing matrix using [<-. Consider:
m <- matrix(x,3,3)
m[,2] <- sample(x)
m
[,1] [,2] [,3]
[1,] "a" "1" "a"
[2,] "b" "3" "b"
[3,] "c" "2" "c"
I think the offending function is therefore [<-.
As Konrad said, the treatment of factors is often odd, or at least inconsistent. In this case I think the behaviour is weird enough to constitute a bug. Try submitting it, and see what the response is.
Since the result is a matrix, and there is no factor matrix type, I think that the correct behaviour would be to convert factor inputs to character somewhere near the start of the function.
I had the same problem. Coercing back to a character vector inside the combn command seems to work:
> combn(as.character(x),2)
[,1] [,2] [,3]
[1,] "a" "a" "b"
[2,] "b" "c" "c"
I am trying to find a way to get a list in R of all the possible unique permutations of A,A,A,A,B,B,B,B,B.
Combinations was what was originally thought to be the method for obtaining a solution, hence the combinations answers.
I think this is what you're after. #bill was on the ball with the recommendation of combining unique and combn. We'll also use the apply family to generate ALL of the combinations. Since unique removes duplicate rows, we need to transpose the results from combn before uniqueing them. We then transpose them back before returning to the screen so that each column represents a unique answer.
#Daters
x <- c(rep("A", 4), rep("B",5))
#Generates a list with ALL of the combinations
zz <- sapply(seq_along(x), function(y) combn(x,y))
#Filter out all the duplicates
sapply(zz, function(z) t(unique(t(z))))
Which returns:
[[1]]
[,1] [,2]
[1,] "A" "B"
[[2]]
[,1] [,2] [,3]
[1,] "A" "A" "B"
[2,] "A" "B" "B"
[[3]]
[,1] [,2] [,3] [,4]
[1,] "A" "A" "A" "B"
[2,] "A" "A" "B" "B"
[3,] "A" "B" "B" "B"
...
EDIT Since the question is about permuations and not combinations, the answer above is not that useful. This post outlines a function to generate the unique permutations given a set of parameters. I have no idea if it could be improved upon, but here's one approach using that function:
fn_perm_list <-
function (n, r, v = 1:n)
{
if (r == 1)
matrix(v, n, 1)
else if (n == 1)
matrix(v, 1, r)
else {
X <- NULL
for (i in 1:n) X <- rbind(X, cbind(v[i], fn_perm_list(n -
1, r - 1, v[-i])))
X
}
}
zz <- fn_perm_list(9, 9)
#Turn into character matrix. This currently does not generalize well, but gets the job done
zz <- ifelse(zz <= 4, "A", "B")
#Returns 126 rows as indicated in comments
unique(zz)
There's no need to generate permutations and then pick out the unique ones.
Here's a much simpler way (and much, much faster as well): To generate all permutations of 4 A's and 5 B's, we just need to enumerate all possible ways of placing 4 A's among 9 possible locations. This is simply a combinations problem. Here's how we can do this:
x <- rep('B',9) # vector of 9 B's
a_pos <- combn(9,4) # all possible ways to place 4 A's among 9 positions
perms <- apply(a_pos, 2, function(p) replace(x,p,'A')) # all desired permutations
Each column of the 9x126 matrix perms is a unique permutation 4 A's and 5 B's:
> dim(perms)
[1] 9 126
> perms[,1:4] ## look at first few columns
[,1] [,2] [,3] [,4]
[1,] "A" "A" "A" "A"
[2,] "A" "A" "A" "A"
[3,] "A" "A" "A" "A"
[4,] "A" "B" "B" "B"
[5,] "B" "A" "B" "B"
[6,] "B" "B" "A" "B"
[7,] "B" "B" "B" "A"
[8,] "B" "B" "B" "B"
[9,] "B" "B" "B" "B"