I used this code to calculate relative abundance (cell/total of column) of a table I had. I don't understand how the . and ~ functions work.
The construct ~./sum(x) is technically a special type of R object called a formula
class(~./sum(x))
#> [1] "formula"
However, in tidyverse functions such as mutate_all, this formula is taken and converted into a lambda function, which is an anonymous function (i.e. a function that isn't named and is written in place as a parameter passed in a call to another function).
Internally, the formula is converted into a function with rlang::as_function. Suppose we wanted to write a function that just adds two to a variable. In base R we might write
add_two <- function(var){
return(var + 2)
}
add_two(5)
#> [1] 7
In the tidyverse, we can use a formula as shorthand for this function, where the . becomes a shorthand for "the variable that was passed as a first argument to the function":
add_two <- rlang::as_function(~ . + 2)
add_two(5)
#> [1] 7
In functions such as mutate_all, the formula will automatically be passed through rlang::as_function, so if we wanted to add two to each column in our data frame, instead of writing:
mutate_all(.funs= function(var) {return(var + 2);})
we could write
mutate_all(.funs=~.+2)
In your case, the formula ~./sum(x) is effectively transformed into
function(var) {
return(var / sum(x))
}
where x has to exist either as a column in your data frame or a variable in the calling environment.
The reasons for having it this way are that it saves typing and shortens lines of code. Inserting a function within a call to another function often leads to messy and poorly formatted code. This shorthand method helps to prevent that.
You can read more about anonymous functions and how they are used in the tidyverse here
Suppose we have this dataset:
dataset <- data.frame(a = c(1,2,3,4),
b = c(2,3,4,5),
c = c(3,4,5,6))
And you want to divide all vectors by the total (ie. for vector a = 1/10, 2/10, 3/10, 4/10). To avoid writing for all variables, you can use mutate_all, and then a lambda using .funswhich says make a function that divides all values in each vector represented by the dot by the sum of all values in that vector.
dataset %>% mutate_all(.funs = ~./sum(.))
Hope this helps.
mutate_all applies the function in .funs to all values. Each value (.) is divided by the sum(x), to get you the "relative abundance" which is essentially the fraction of the total value, which is the sum(x). You can think of ~ as a "function of". So you are saying each cell in the dataframe is a function of itself divided by the overall sum.
In my code there is a situation where I conditionally want to use one accessor function or another throughout the code. Instead of having an if-else statement for every time I want to pick which accessor to use and coding it explicitly, I tried to conditionally assign either of the accessor functions to a new function called accessor_fun and use it throughout the code, but this returns an error when I use the accessor function to reassign the values it accesses. Here is a simplified example of the problem I am having:
#reassigning the base r function names to a new function name
alt_names_fun <- names
example_list <- list(cat = 7, dog = 8, fish = 33)
other_example_list <- list(table = 44, chair = 101, desk = 35)
#works
alt_names_fun(example_list)
#throws error
alt_names_fun(example_list) <- alt_names_fun(other_example_list)
#still throws error
access_and_assign <- function(x, y, accessor) {
accessor(x) <- accessor(y)
}
access_and_assign(x = example_list, y = other_example_list, accessor = alt_names_fun)
#still throws error
alt_names_fun_2 <- function(x){names(x)}
alt_names_fun_2(example_list) <- alt_names_fun_2(other_example_list)
#works
names(example_list) <- names(other_example_list)
As you see if you try the code above, an example of the kind of error I am getting is
Error in alt_names_fun(example_list) <- alt_names_fun(other_example_list) :
could not find function "alt_names_fun<-"
So my question is, is there a way to do the reassignment of R accessor functions and use them in a way like I am trying to in the example above?
Accessor functions are really pairs of functions. One for retrieval and one for assignment. If you want to replicate that, you need to replicate both parts
alt_names_fun <- names
`alt_names_fun<-` <- `names<-`
The assignment versions have <- in their name. This is a special naming convection that R uses to find them. Since these are character normally not allowed in basic symbol names, you need to use the back ticks to enclose the function names.
In functions such as plotmeans there is an argument that specifies the data frame to use, data=. I would like to construct the name of the data frame to be used using paste0 or something similar, df <- paste0("results", i), where i is a number to get (say) "results04". If I then use data=df, I get an error saying that data= expects a variable, not a string. Is there any way to convert the string into a form that data= will accept? data=results04 without the quotes, of course, works.
Thanks for any suggestions or pointers.
The answer would have been obvious to one with more R experience, but let me put it here for others: use the get() function, so for instance
df <- paste0("results", i)
plotmeans(a ~ b, data=get(df))
I have a general function that calls an expression that uses a formula and I would like to pass this functions to various environments that store some specific variables and modify parts of a formula designated by a specific pattern.
Here is an example:
# Let's assume I have an environment storing a variable
env <- new.env()
env$..M.. <- "Sepal.Length"
# And a function that calls an expression
func <- function() summary(lm(..M.. ~ Species, data = iris))$r.squared
# And let's assume I am trying to evaluate it within the environment
environment(func) <- env
# And I would like to have some method that makes it evaluate as:
summary(lm(Sepal.Length ~ Species, data = iris))$r.squared
So far I came up with a very dirty solution based on deparsing the function down to string, greping and then parsing it back. It goes like this:
tfunc <- paste(deparse(func), collapse = "")
tfunc <- gsub("\\.\\.M\\.\\.", ..M.., tfunc, perl = TRUE)
tfunc <- eval(parse(text = tfunc))
So yes, it works, but I would like to find a cleaner method, that would somewhat magically substitute this ..M.. pattern into Sepal.Length without a need for all this parsing and deparsing.
So I would really appreciate some help and hints for that problem.
I have two lists of lists. humanSplit and ratSplit. humanSplit has element of the form::
> humanSplit[1]
$Fetal_Brain_408_AGTCAA_L001_R1_report.txt
humanGene humanReplicate alignment RNAtype
66 DGKI Fetal_Brain_408_AGTCAA_L001_R1_report.txt 6 reg
68 ARFGEF2 Fetal_Brain_408_AGTCAA_L001_R1_report.txt 5 reg
If you type humanSplit[[1]], it gives the data without name $Fetal_Brain_408_AGTCAA_L001_R1_report.txt
RatSplit is also essentially similar to humanSplit with difference in column order. I want to apply fisher's test to every possible pairing of replicates from humanSplit and ratSplit. Now I defined the following empty vector which I will use to store the informations of my fisher's test
humanReplicate <- vector(mode = 'character', length = 0)
ratReplicate <- vector(mode = 'character', length = 0)
pvalue <- vector(mode = 'numeric', length = 0)
For fisher's test between two replicates of humanSplit and ratSplit, I define the following function. In the function I use `geneList' which is a data.frame made by reading a file and has form:
> head(geneList)
human rat
1 5S_rRNA 5S_rRNA
2 5S_rRNA 5S_rRNA
Now here is the main function, where I use a function getGenetype which I already defined in other part of the code. Also x and y are integers :
fishertest <-function(x,y) {
ratReplicateName <- names(ratSplit[x])
humanReplicateName <- names(humanSplit[y])
## merging above two based on the one-to-one gene mapping as in geneList
## defined above.
mergedHumanData <-merge(geneList,humanSplit[[y]], by.x = "human", by.y = "humanGene")
mergedRatData <- merge(geneList, ratSplit[[x]], by.x = "rat", by.y = "ratGene")
## [here i do other manipulation with using already defined function
## getGenetype that is defined outside of this function and make things
## necessary to define following contingency table]
contingencyTable <- matrix(c(HnRn,HnRy,HyRn,HyRy), nrow = 2)
fisherTest <- fisher.test(contingencyTable)
humanReplicate <- c(humanReplicate,humanReplicateName )
ratReplicate <- c(ratReplicate,ratReplicateName )
pvalue <- c(pvalue , fisherTest$p)
}
After doing all this I do the make matrix eg to use in apply. Here I am basically trying to do something similar to double for loop and then using fisher
eg <- expand.grid(i = 1:length(ratSplit),j = 1:length(humanSplit))
junk = apply(eg, 1, fishertest(eg$i,eg$j))
Now the problem is, when I try to run, it gives the following error when it tries to use function fishertest in apply
Error in humanSplit[[y]] : recursive indexing failed at level 3
Rstudio points out problem in following line:
mergedHumanData <-merge(geneList,humanSplit[[y]], by.x = "human", by.y = "humanGene")
Ultimately, I want to do the following:
result <- data.frame(humanReplicate,ratReplicate, pvalue ,alternative, Conf.int1, Conf.int2, oddratio)
I am struggling with these questions:
In defining fishertest function, how should I pass ratSplit and humanSplit and already defined function getGenetype?
And how I should use apply here?
Any help would be much appreciated.
Up front: read ?apply. Additionally, the first three hits on google when searching for "R apply tutorial" are helpful snippets: one, two, and three.
Errors in fishertest()
The error message itself has nothing to do with apply. The reason it got as far as it did is because the arguments you provided actually resolved. Try to do eg$i by itself, and you'll see that it is returning a vector: the corresponding column in the eg data.frame. You are passing this vector as an index in the i argument. The primary reason your function erred out is because double-bracket indexing ([[) only works with singles, not vectors of length greater than 1. This is a great example of where production/deployed functions would need type-checking to ensure that each argument is a numeric of length 1; often not required for quick code but would have caught this mistake. Had it not been for the [[ limit, your function may have returned incorrect results. (I've been bitten by that many times!)
BTW: your code is also incorrect in its scoped access to pvalue, et al. If you make your function return just the numbers you need and the aggregate it outside of the function, your life will simplify. (pvalue <- c(pvalue, ...) will find pvalue assigned outside the function but will not update it as you want. You are defeating one purpose of writing this into a function. When thinking about writing this function, try to answer only this question: "how do I compare a single rat record with a single human record?" Only after that works correctly and simply without having to overwrite variables in the parent environment should you try to answer the question "how do I apply this function to all pairs and aggregate it?" Try very hard to have your function not change anything outside of its own environment.
Errors in apply()
Had your function worked properly despite these errors, you would have received the following error from apply:
apply(eg, 1, fishertest(eg$i, eg$j))
## Error in match.fun(FUN) :
## 'fishertest(eg$i, eg$j)' is not a function, character or symbol
When you call apply in this sense, it it parsing the third argument and, in this example, evaluates it. Since it is simply a call to fishertest(eg$i, eg$j) which is intended to return a data.frame row (inferred from your previous question), it resolves to such, and apply then sees something akin to:
apply(eg, 1, data.frame(...))
Now that you see that apply is being handed a data.frame and not a function.
The third argument (FUN) needs to be a function itself that takes as its first argument a vector containing the elements of the row (1) or column (2) of the matrix/data.frame. As an example, consider the following contrived example:
eg <- data.frame(aa = 1:5, bb = 11:15)
apply(eg, 1, mean)
## [1] 6 7 8 9 10
# similar to your use, will not work; this error comes from mean not getting
# any arguments, your error above is because
apply(eg, 1, mean())
## Error in mean.default() : argument "x" is missing, with no default
Realize that mean is a function itself, not the return value from a function (there is more to it, but this definition works). Because we're iterating over the rows of eg (because of the 1), the first iteration takes the first row and calls mean(c(1, 11)), which returns 6. The equivalent of your code here is mean()(c(1, 11)) will fail for a couple of reasons: (1) because mean requires an argument and is not getting, and (2) regardless, it does not return a function itself (in a "functional programming" paradigm, easy in R but uncommon for most programmers).
In the example here, mean will accept a single argument which is typically a vector of numerics. In your case, your function fishertest requires two arguments (templated by my previous answer to your question), which does not work. You have two options here:
Change your fishertest function to accept a single vector as an argument and parse the index numbers from it. Bothing of the following options do this:
fishertest <- function(v) {
x <- v[1]
y <- v[2]
ratReplicateName <- names(ratSplit[x])
## ...
}
or
fishertest <- function(x, y) {
if (missing(y)) {
y <- x[2]
x <- x[1]
}
ratReplicateName <- names(ratSplit[x])
## ...
}
The second version allows you to continue using the manual form of fishertest(1, 57) while also allowing you to do apply(eg, 1, fishertest) verbatim. Very readable, IMHO. (Better error checking and reporting can be used here, I'm just providing a MWE.)
Write an anonymous function to take the vector and split it up appropriately. This anonymous function could look something like function(ii) fishertest(ii[1], ii[2]). This is typically how it is done for functions that either do not transform as easily as in #1 above, or for functions you cannot or do not want to modify. You can either assign this intermediary function to a variable (which makes it no longer anonymous, figure that) and pass that intermediary to apply, or just pass it directly to apply, ala:
.func <- function(ii) fishertest(ii[1], ii[2])
apply(eg, 1, .func)
## equivalently
apply(eg, 1, function(ii) fishertest(ii[1], ii[2]))
There are two reasons why many people opt to name the function: (1) if the function is used multiple times, better to define once and reuse; (2) it makes the apply line easier to read than if it contained a complex multi-line function definition.
As a side note, there are some gotchas with using apply and family that, if you don't understand, will be confusing. Not the least of which is that when your function returns vectors, the matrix returned from apply will need to be transposed (with t()), after which you'll still need to rbind or otherwise aggregrate.
This is one area where using ddply may provide a more readable solution. There are several tutorials showing it off. For a quick intro, read this; for a more in depth discussion on the bigger picture in which ddply plays a part, read Hadley's Split, Apply, Combine Strategy for Data Analysis paper from JSS.