(edit note: I changed the Title to "R: enumerate column combinations of a matrix", from "R grep: matching a matrix of strings to a list" to better reflect the solution)
I am trying to match a matrix of strings to a list: so that i can ultimately use the matrix as a map in later operations on a data.frame.
This first part works as intended, returning a list of all the possible pairs, triples and quad combinations (though perhaps this approach has created my bind?):
priceList <- data.frame(aaa = rnorm(100, 100, 10), bbb = rnorm(100, 100, 10),
ccc = rnorm(100, 100, 10), ddd = rnorm(100, 100, 10),
eee = rnorm(100, 100, 10), fff = rnorm(100, 100, 10),
ggg = rnorm(100, 100, 10))
getTrades <- function(dd, Maxleg=3)
{
nodes <- colnames(dd)
tradeList <- list()
for (i in 2:Maxleg){
tradeLeg <- paste0('legs',i)
tradeList[[tradeLeg]] <- combn(nodes, i)
}
return(tradeList)
}
tradeCombos <- getTrades(priceList, 4)
I'd now like to turn this list of possible combinations into trades. For example:
> tradeCombos[[1]][,1]
[1] "aaa" "bbb"
Needs to eventually become priceList[,2] - priceList[,1], and so forth.
I have tried a few approaches with grep and similar commands, and feel that i've come close with the following:
LocList <- sapply(tradeCombos[[1]], regexpr, colnames(priceList))
However the format is not quite suitable for the next step.
Ideally, LocList[1] would return something like: 1 2
Assuming that the tradeCombos[[1]][,1] == "aaa" "bbb".
Can someone please help?
__
With help from all of the answers below, i've now got:
colDiff <- function(x)
{
Reduce('-', rev(x))
}
getTrades <- function(dd, Maxleg=3)
{
tradeList <- list()
for (i in 2:Maxleg){
tradeLeg <- paste0('legs',i)
tradeLegsList <- combn(names(dd), i,
function(x) dd[x], simplify = FALSE)
nameMtx <- combn(names(dd), i)
names(tradeLegsList) <- apply(nameMtx, MARGIN=2,
FUN=function(x) paste(rev(x), collapse='*'))
tradeList[[tradeLeg]] <- lapply(tradeLegsList, colDiff)
}
return(tradeList)
}
tradeCombos <- getTrades(priceList, 4)
This retains the names of the constitutent parts, and is everything I was trying to achieve.
Many thanks to all for the help.
Whoa... ignore everything below and jump to the update
As mentioned in my comment, you can just use combn. This solution doesn't take you to your very last step, but instead, creates a list of data.frames. From there, it is easy to use lapply to get to whatever your final step would be.
Here's the simplified function:
TradeCombos <- function(dd, MaxLeg) {
combos = combn(names(dd), MaxLeg)
apply(combos, 2, function(x) dd[x])
}
To use it, just specify your dataset and the number of combinations you're looking for.
TradeCombos(priceList, 3)
TradeCombos(priceList, 4)
Moving on: #mplourde has shown you how to use Reduce to successively subtract. A similar approach would be taken here:
cumDiff <- function(x) Reduce("-", rev(x))
lapply(TradeCombos(priceList, 3), cumDiff)
By keeping the output of the TradeCombos function as a list of data.frames, you'll be leaving more room for flexibility. For instance, if you wanted row sums, you can simply use lapply(TradeCombos(priceList, 3), rowSums); similar approaches can be taken for whatever function you want to apply.
Update
I'm not sure why #GSee didn't add this as an answer, but I think it's pretty awesome:
Get your list of data.frames as follows:
combn(names(priceList), 3, function(x) priceList[x], simplify = FALSE)
Advance as needed. (For example, using the cumDiff function we created: combn(names(priceList), 2, function(x) cumDiff(priceList[x]), simplify = FALSE).)
This gets your eventual aim using lapply, apply, and Reduce.
lapply(tradeCombos,
function(combos)
apply(combos, MARGIN=2, FUN=function(combo) Reduce('-', priceList[rev(combo)])))
combo is a column from one of the combo matrices in tradeCombos. rev(combo) reverses the column so the last value is first. The R syntax for selecting a subset of columns from a data.frame is DF[col.names], so priceList[rev(combo)] is a subset of priceList with just the columns in combo, in reverse order. data.frames are actually just lists of columns, so any function that's designed to iterate over lists can be used to iterate over the columns in a data.frame. Reduce is one such function. Reduce takes a function (in this case the subtract function -) and a list of arguments and then successively calls the function on the arguments in the list with the results of the previous call, e.g., (((arg1 - arg2) - arg3) - arg4).
You rename the columns in tradeCombos so that the final column names reflect their source with:
tradeCombos <- lapply(tradeCombos,
function(combos) {
dimnames(combos)[[2]] <- apply(combos,
MARGIN=2,
FUN=function(combo) paste(rev(combo), collapse='-')
)
return(combos)
}
)
tradeCombos is a list with matrix elements. Therefore, tradeCombos[[1]] is a matrix for which apply is more suitable.
apply(tradeCombos[[1]],1,function(x) match(x,names(priceList)))
[,1] [,2]
[1,] 1 2
[2,] 1 3
[3,] 1 4
[4,] 1 5
[5,] 1 6
[6,] 1 7
[7,] 2 3
[8,] 2 4
[9,] 2 5
[10,] 2 6
[11,] 2 7
[12,] 3 4
[13,] 3 5
[14,] 3 6
[15,] 3 7
[16,] 4 5
[17,] 4 6
[18,] 4 7
[19,] 5 6
[20,] 5 7
[21,] 6 7
Incidentally, you can subset using the string form anyway, eg priceList[,"aaa"]
Related
I'm trying to learn how to use the apply() functions.
Suppose we have a 3 row, 2 column matrix of test <- matrix(c(1,2,3,4,5,6), ncol = 2), and we would like the maximum value of each element in the first column (1, 2, 3) to not exceed 2 for example, so we end up with a matrix of (1,2,2,4,5,6).
How would one write an apply() function to do this?
Here's my latest attempt: test1 <- apply(test[,1], 2, function(x) {if(x > 2){return(x = 2)} else {return(x)}})
We may use pmin on the first column with value 2 as the second argument, so that it does elementwise checking with the recycled 2 and gets the minimum for each value from the first column
test[,1] <- pmin(test[,1], 2)
-output
> test
[,1] [,2]
[1,] 1 4
[2,] 2 5
[3,] 2 6
Note that apply needs the 'X' as an array/matrix or one with dimensions, when we subset only a single column/row, it drops the dimensions because drop = TRUE by default
If you really want to use the apply() function, I guess you're looking for something like this:
t(apply(test, 1, function(x) c(min(x[1], 2), x[2])))
## [,1] [,2]
## [1,] 1 4
## [2,] 2 5
## [3,] 2 6
But if you want my opinion, akrun's suggestion is definitely better.
Suppose we have a "test" matrix that looks like this: (1,2,3, 4,5,6, 7,8,9, 10,11,12) generated by running test <- matrix(1:12, ncol = 4). A simple 3 x 4 (rows x columns) matrix of numbers running from 1 to 12.
Now suppose we'd like to add a value of 1 to each element in each odd-numbered matrix column, so we end up with a matrix of the following values: (2,3,4, 4,5,6, 8,9,10, 10,11,12). How would we use an apply() function to do this?
Note that this is a simplified example. In the more complete code I'm working with, the matrix dynamically expands/contracts based on user inputs so I need an apply() function that counts the actual number of matrix columns, rather than using a fixed assumption of 4 columns per the above example. (And I'm not adding a value of 1 to the elements; I'm running the parallel minima function test[,1] <- pmin(test1[,1], 5) to say limit each value to a max of 5).
With my current limited understanding of the apply() family of functions, all I can so far do is apply(test, 2, function(x) {return(x+1)}) but this is adding a value of 1 to all elements in all columns rather than only the odd-numbered columns.
You may simply subset the input data frame to access only odd or even numbered columns. Consider:
test[c(TRUE, FALSE)] <- apply(test[c(TRUE, FALSE)], 2, function(x) f(x))
test[c(FALSE, TRUE)] <- apply(test[c(FALSE, TRUE)], 2, function(x) f(x))
This works because the recycling rules in R will cause e.g. c(TRUE, FALSE) to be repeated however many times is needed to cover all columns in the input test data frame.
For a matrix, we need to use the drop=FALSE flag when subsetting the matrix in order to keep it in matrix form when using apply():
test <- matrix(1:12, ncol = 4)
test[,c(TRUE, FALSE)] <- apply(test[,c(TRUE, FALSE),drop=FALSE], 2, function(x) x+1)
test
[,1] [,2] [,3] [,4]
[1,] 2 4 8 10
[2,] 3 5 9 11
[3,] 4 6 10 12
^ ^ ... these columns incremented by 1
You may use modulo %% 2.
odd <- !seq(ncol(test)) %% 2 == 0
test[, odd] <- apply(test[, odd], 2, function(x) {return(x + 1)})
# [,1] [,2] [,3] [,4]
# [1,] 2 4 8 10
# [2,] 3 5 9 11
# [3,] 4 6 10 12
I want to assure that the result of which(..., arr.ind = TRUE) is always ordered, specifically: arranged ascending by (col, row). I do not see such a remark in the which function documentation, whereas it seems to be the case based on some experiments I made. How I can check / learn if it is the case?
Example. When I run the code below, the output is a matrix in which the results are arranged ascending by (col, row) columns.
> set.seed(1)
> vals <- rnorm(10)
> valsall <- sample(as.numeric(replicate(10, vals)))
> mat <- matrix(valsall, 10, 10)
> which(mat == max(mat), arr.ind = TRUE)
row col
[1,] 1 1
[2,] 3 1
[3,] 1 2
[4,] 2 2
[5,] 10 2
[6,] 1 6
[7,] 2 8
[8,] 4 8
[9,] 1 9
[10,] 6 9
Part1:
Answering a part of your question on how to understand functions on a deeper level, if the documentation is not enough, without going into the detail of function which().
As match() is not a primitive function (which are written in C), i.e. written using the basic building blocks of R, we can check what's going on behind the scenes by printing the function itself. Note that using the backticks allows to check functions that have reserved names, e.g. +, and is therefore optional in this example. This dense R code can be extremely tiresome to read, but I've found it very educational and it does solve some mental knots every once in a while.
> print(`which`)
function (x, arr.ind = FALSE, useNames = TRUE)
{
wh <- .Internal(which(x))
if (arr.ind && !is.null(d <- dim(x)))
arrayInd(wh, d, dimnames(x), useNames = useNames)
else wh
}
<bytecode: 0x00000000058673e0>
<environment: namespace:base>
Part2:
So after giving up on trying to understand the which and arrayInd function in the way described above, I'm trying it with common sense. The most efficient way to check each value of a matrix/array that makes sense to me, is to at some point convert it to a one-dimensional object. Coercion from matrix to atomic vector, or any reduction of dimensions will always result in concatenating the complete columns of each dimension, so to me it is natural that higher-level functions will also follow this fundamental rule.
> testmat <- matrix(1:10, nrow = 2, ncol = 5)
> testmat
[,1] [,2] [,3] [,4] [,5]
[1,] 1 3 5 7 9
[2,] 2 4 6 8 10
> as.numeric(testmat)
[1] 1 2 3 4 5 6 7 8 9 10
I found Hadley Wickham's Advanced R an extremely valuable resource in answering your question, especially the chapters about functions and data structures.
[http://adv-r.had.co.nz/][1]
I have 2 variables, a and b. a is potentially very large. b is always a vector of functions that can be applied to each column in the data frame a.
a <- data.frame(col1=c(1, 2, 3), col2=c(4, 5, 6))
b <- c(as.double, function(x) {1+x})
So the result that I want is that function b[1] is applied to col1, b[2] is applied to col2, and so on for all columns. I feel lapply should be used here but documentation seems to say that it can only have one function. I could use a loop I suppose but a "vectorised" way would be nice.
This is one way to do it:
results <- mapply(function(i,j) b[[i]](a[[j]]), i=1:length(b), j=1:length(a))
It gives you:
> results
[,1] [,2]
[1,] 1 5
[2,] 2 6
[3,] 3 7
How would one use rollapply (or some other R function) to grow the window size as the function progresses though the data. To phrase it another way, the first apply works with the first element, the second with the first two elements, the third with the first three elements etc.
If you are looking to apply min , max, sum or prod, these functions already have their cumulative counterparts as:
cummin, cummax, cumsum and cumprod
To apply more exotic functions on a growing / expanding window, you can simply use sapply
eg
# your vector of interest
x <- c(1,2,3,4,5)
sapply(seq_along(x), function(y,n) yourfunction(y[seq_len(n)]), y = x)
For a basic zoo object
x.Date <- as.Date("2003-02-01") + c(1, 3, 7, 9, 14) - 1
x <- zoo(rnorm(5), x.Date)
# cumsum etc will work and return a zoo object
cs.zoo <- cumsum(x)
# convert back to zoo for the `sapply` solution
# here `sum`
foo.zoo <- zoo(sapply(seq_along(x), function(n,y) sum(y[seq_len(n)]), y= x), index(x))
identical(cs.zoo, foo.zoo)
## [1] TRUE
From peering at the documentation at ?zooapply I think this will do what you want, where a is your matrix and sum can be any function:
a <- cbind(1:5,1:5)
# [,1] [,2]
# [1,] 1 1
# [2,] 2 2
# [3,] 3 3
# [4,] 4 4
# [5,] 5 5
rollapply(a,width=seq_len(nrow(a)),sum,align="right")
# [,1] [,2]
# [1,] 1 1
# [2,] 3 3
# [3,] 6 6
# [4,] 10 10
# [5,] 15 15
But mnel's answer seems sufficient and more generalizable.
in addition to #mnel's answer:
For more exotic functions you can simply use sapply
and if the sapply approach takes too long, you may be better off formulating your function iteratively.