In my dataset I have 6 variables(x1,x2,x3,x4,x5,x6), i wish to create a function that allows me to input one variable and it will do the formula with the rest of the variables in the data set.
For instance,
fitRegression <- function(data, dependentVariable) {
fit = lm(formula = x1 ~., data = data1)
return(fit)
}
fitRegression(x2)
However, this function only returns me with results of x1. My desire result will be inputting whatever variables and will automatically do the formula with the rest of the variables.
For Example:
fitRegression(x2)
should subtract x2 from the variable list therefore we only compare x2 with x1,x3,x4,x5,x6.
and if:
fitRegression(x3)
should subtract x3 from the comparable list, therefore we compare x3 with x1,x2,x4,x5,x6.
Is there any ways to express this into my function, or even a better function.
You can do it like this:
# sample data
sampleData <- data.frame(matrix(rnorm(500),100,5))
colnames(sampleData) <- c("A","B","C","D","E")
# function
fitRegression <- function(mydata, dependentVariable) {
# select your independent and dependent variables
dependentVariableIndex<-which(colnames(mydata)==dependentVariable)
independentVariableIndices<-which(colnames(mydata)!=dependentVariable)
fit = lm(formula = as.formula(paste(colnames(mydata)[dependentVariableIndex], "~", paste(colnames(mydata)[independentVariableIndices], collapse = "+"), sep = "" )), data = mydata)
return(fit)
}
# ground truth
lm(formula = A~B+C+D+E, data = sampleData)
# reconcile results
fitRegression(sampleData, "A")
You want to select the Y variable in your argument. The main difficulty is to pass this argument without any quotes in your function (it is apparently the expected result in your code). Therefore you can use this method, using the combination deparse(substitute(...)):
fitRegression <- function(data, dependentVariable) {
formula <- as.formula(paste0(deparse(substitute(dependentVariable)), "~."))
return(lm(formula, data) )
}
fitRegression(mtcars, disp)
That will return the model.
The below function uses "purrr" and "caret" it produces a list of models.
df <-mtcars
library(purrr);library(caret)
#create training set
vect <- createDataPartition(1:nrow(df), p=0.8, list = FALSE)
#build model list
ModList <- 1:length(df) %>%
map(function(col) train(y= df[vect,col], x= df[vect,-col], method="lm"))
Related
I want to be able to input the variable name that I'll be using in the "weights" option in the lmer function. So then I can change the dataset, and cycle through the "weights" and pull the correct variable.
I want to pull the correct column for weights within the for loop.
So for y, the equation would be:
lmer(y~x+(1|study), weights = weight.var)
And y1:
lmer(y1~x+(1|study),weights = weight.var1)
So I named the weighting variables (weight.opt), then want to use them in the formula within the for loop. I can use "as.formula" to get the formula working and connected to the dataset, but I'm not sure how to do something similar with the weights.
x <- rnorm(300,0,1)
y <- x*rnorm(300,2,0.5)
y1 <- x*rnorm(300,0.1,0.1)
study <- rep(c("a","b","c"),each = 100)
weight.var <- rep(c(0.5,2,4),each = 100)
weight.var1 <- rep(c(0.1,.2,.15),each = 100)
library(lme4)
dataset <- data.frame(x,y,y1,study,weight.var,weight.var1)
resp1 <- c("y","y1")
weight.opt <- c("weight.var","weight.var1")
for(i in 1:2){
lmer(as.formula(paste(resp1[i],"~x+(1|study)")),weights = weight.opt[i],data = dataset)
}
This seems to work fine:
res_list <- list()
for(i in 1:2){
res_list[[i]] <- lmer(as.formula(paste(resp1[i],"~x+(1|study)")),
weights = dataset[[weight.opt[i]]],data = dataset)
}
I wanted to model my snps array. I can do this one by one using the following code.
Data$DX=as.factor(Data$DX)
univariate=glm(relevel(DX, "CON") ~ relevel(rs6693065_D,"AA"), family = binomial, data = Data)
summary(univariate)
exp(cbind(OR = coef(univariate), confint(univariate)))
How can I do this for all other snps using a loop or apply? The snps are rs6693065_D, rs6693065_A and hundreds of them. From the above code only "rs6693065_D" will be replaced by all other snps.
Best Regards
Zillur
Consider developing a generalized method to handle any snps. Then call it iteratively passing every snps column using lapply or sapply:
# GENERALIZED METHOD
proc_glm <- function(snps) {
univariate <- glm(relevel(data$DX, "CON") ~ relevel(snps, "AA"), family = binomial)
return(exp(cbind(OR = coef(univariate), confint(univariate))))
}
# BUILD LIST OF FUNCTION OUTPUT
glm_list <- lapply(Data[3:426], proc_glm)
Use tryCatch in case of errors like relevel:
# BUILD LIST OF FUNCTION OUTPUT
glm_list <- lapply(Data[3:426], function(col)
tryCatch(proc_glm(col), error = function(e) e))
For building a data frame, adjust method and lapply call followed with a do.call + rbind:
proc_glm <- function(col){
# BUILD FORMULA BY STRING
univariate <- glm(as.formula(paste("y ~", col)), family = binomial, data = Data)
# RETURN DATA FRAME OF COLUMN AND ESTIMATES
cbind.data.frame(COL = col,
exp(cbind(OR = coef(univariate), confint(univariate)))
)
}
# BUILD LIST OF DFs, PASSING COLUMN NAMES
glm_list <- lapply(names(Data)[3:426],
tryCatch(proc_glm(col), error = function(e) NA))
# APPEND ALL DFs FOR SINGLE MASTER DF
final_df <- do.call(rbind, glm_list)
I want to perform a certain number of statistical models based on selection criteria specified in a dataframe. So using a basic example, say I had 2 responses variables and 2 explanatory variables:
#######################Data Input############################
Responses <- as.data.frame(matrix(sample(0:10, 1*100, replace=TRUE), ncol=2))
colnames(Responses) <- c("A","B")
Explanatories <- as.data.frame(matrix(sample(20:30, 1*100, replace=TRUE), ncol=2))
colnames(Explanatories) <- c("x","y")
I then define which statistical models that I would like to run, which can include different combinations of Response / Explanatory variables and different statistical functions:
###################Model selection#########################
Function <- c("LIN","LOG","EXP") ##Linear, Logarithmic (base 10) and exponential - see the formula for these below
Respo <- c("A","B","B")
Explan <- c("x","x","y")
Model_selection <- data.frame(Function,Respo,Explan)
How do I then perform a list of models based on these selection criteria? Here is an example of the models I would like to create based on the inputs from the Model_selection data frame.
####################Model creation#########################
Models <- list(
lm(Responses$A ~ Explanatories$x),
lm(Responses$B ~ log10(Explanatories$x)),
lm(Responses$B ~ exp(Explanatories$y))
)
I would guess that some kind of loop function would be required and after looking around perhaps paste too? Thanks in advance for any help with this
This isn't the prettiest solution, but it seems to work for your example:
Models <- list()
idx <- 1L
for (row in 1:nrow(Model_selection)){
if (Model_selection$Function[row]=='LOG'){
expl <- paste0('LOG', Model_selection$Explan[row])
Explanatories[[expl]] <- log10(Explanatories[[Model_selection$Explan[row]]])
Models[[idx]] <- lm(Responses[[Model_selection$Respo[row]]] ~ Explanatories[[expl]])
}
if (Model_selection$Function[row]=='EXP'){
expl <- paste0('EXP', Model_selection$Explan[row])
Explanatories[[expl]] <- exp(Explanatories[[Model_selection$Explan[row]]])
Models[[idx]] <- lm(Responses[[Model_selection$Respo[row]]] ~ Explanatories[[expl]])
}
if (Model_selection$Function[row]=='LIN'){
expl <- paste0('LIN', Model_selection$Explan[row])
Explanatories[[expl]] <- Explanatories[[Model_selection$Explan[row]]]
Models[[idx]] <- lm(Responses[[Model_selection$Respo[row]]] ~ Explanatories[[expl]])
}
names(Models)[idx] <- paste(Model_selection$Respo[row], '~', expl)
idx <- idx+1L
}
Models
This is a perfect use-case for the tidyverse
library(tidyverse)
## cbind both data sets into one
my_data <- cbind(Responses, Explanatories)
## use 'mutate' to change function names to the existing function names
## mutate_all to transform implicit factors to characters
## NB this step could be ommitted if Function would already use the proper names
model_params <- Model_selection %>%
mutate(Function = case_when(Function == "LIN" ~ "identity",
Function == "LOG" ~ "log10",
Function == "EXP" ~ "exp")) %>%
mutate_all(as.character)
## create a function which estimates the model given the parameters
## NB: function params must be named exactly like columns
## in the model_selection df
make_model <- function(Function, Respo, Explan) {
my_formula <- formula(paste0(Respo, "~", Function, "(", Explan, ")"))
my_mod <- lm(my_formula, data = my_data)
## syntactic sugar: such that we see the value of the formula in the print
my_mod$call$formula <- my_formula
my_mod
}
## use purrr::pmap to loop over the model params
## creates a list with all the models
pmap(model_params, make_model)
I am looking for an elegant (and safe!) way to evaluate an amended call in the parent frame. By "amended" I mean I modified the call in such a way that it refers to something not included in parent frame but in another frame. I guess one could also say: "send something up but only for evaluation".
It is clarified what I want by the example below which works in some circumstances, but not all. The update function (stats:::update.default) uses eval and I added the weights argument with something (res) that is not in the same environment as the evaluation takes place. So I used get("res", pos = -1L) and I hope it is a safe way to refer to the environment res lives in. For models estimated with a variable as formula, both defined methods fail:
mod <- lm(mpg ~ cyl, data = mtcars)
form <- mpg ~ cyl
mod2 <- lm(form, data = mtcars)
wls1 <- function(x) {
res <- residuals(x)^2 # example
result <- update(x, weights = 1/get("res", pos = -1L))
return(result)
}
wls2 <- function(x) {
res <- residuals(x)^2 # example
result <- update(x, weights = 1/res)
return(result)
}
wls3 <- function(x) {
data(ChickWeight)
ChickWeight$cyl <- ChickWeight$weight
ChickWeight$mpg <- ChickWeight$Time
result <- update(x, data = ChickWeight)
return(result)
}
wls1(mod) # works
wls1(mod2) # errors
wls2(mod) # works
wls2(mod2) # erros
wls3(mod) # works
wls3(mod2) # works
How can this be solved in general in a safe way?
I was looking for a function that gives the current environment (something like a fictious this.environment() function) so avoid the pos argument and use the envir of get (I know I can create my own temporary environment and have res associated to it to use something like envir = my.eny).
We can do this by creating a quoted 'language' object for the formula and then update the call of the model
form <- quote(mpg ~ cyl)
wlsN <- function(x, formula) {
x$call$formula <- formula
res <- residuals(x)^2
update(x, weights = 1/res) # it is in the same environment. No need for get
}
wlsN(mod2, form)
#Call:
#lm(formula = mpg ~ cyl, data = mtcars, weights = 1/res)
#Coefficients:
#(Intercept) cyl
# 37.705 -2.841
-checking with other formula
form1 <- quote(disp ~ cyl + vs)
form2 <- quote(mpq ~ gear + carb)
mod1 <- lm(form1, data = mtcars)
mod2 <- lm(form2, data = mtcars)
wlsN(mod1, form1) # works
wlsN(mod2, form2) # works
It's hard to work around the fact that R looks for the value of weights in either data or the environment of the formula - which in the case of the variable named form in your example, is the global environment.
An alternative that riffs on the same theme as akrun's answer:
wls3 <- function(x) {
environment(x$call$formula) <- environment()
res <- residuals(x)^2
result <- update(x, weights=1/res)
}
I can see how this could get ugly in less trivial uses of this workaround such as when the formula of x already has an environment that does not enclose (potentially wrong use of the term) the environment in the call to wls3().
Another alternative (not recommended) is to use assign, e.g.
wls4 <- function(x) {
assign('res', residuals(x)^2, envir=environment(formula(x)))
result <- update(x, weights=1/res)
}
however this has the unintended consequence of leaving the variable res in the global environment.
I'm new to loops and I have a problem with calling variable from i'th data frame.
I'm able to call each data frame correctly, but when I should call a specified variable inside each data frame problems come:
Example:
for (i in 1:15) {
assign(
paste("model", i, sep = ""),
(lm(response ~ variable, data = eval(parse(text = paste("data", i, sep = "")))))
)
plot(data[i]$response, predict.lm(eval(parse(text = paste("model", i, sep = ""))))) #plot obs vs preds
}
Here I'm doing a simple one variable linear model 15 times, which works just fine. Problems come when I try to plot the results. How should I call data[i] response?
Let's say there are multiple dataframes with names: data1 ...data15 and that there are no other data-objects that begin with the letters: d,a,t,a. Lets also assume that in each of those dataframes are columns named 'response' and 'variable'. The this would gather the dataframes into a list and draw separate plots for the linear regression lines.
dlist <- lapply ( ls(patt='^data'), get)
lapply(dlist, function(df)
plot(NA, xlim=range(df$variable), ylim=range(df$response)
abline( coef( lm(response ~ variable, data=df) ) )
)
If you wanted to name the dataframes in that list, you could use your paste code to supply names:
names(dlist) <- paste("data", i, sep = "")
There are many other assignments you could make in the context of this loop, but you would need to describe the desired results better than with failed efforts.
Here's modified code that should work. It does one variable lm-model and calculates correlation of predicted and observed values and stores it into an empty matrix. It also plots these values.
Thanks Thomas for help.
par(mfrow=c(4,5))
results.matrix <- matrix(NA, nrow = 20, ncol = 2)
colnames(results.matrix) <- c("Subset","Correlation")
for (i in 1:length(datalist)) {
model <- lm(response ~ variable, data = datalist[[i]])
pred <- predict.lm(model)
cor <- (cor.test(pred, datalist[[i]]$response))
plot(pred, datalist[[i]]$response, xlab="pred", ylab="obs")
results.matrix[i, 1] <- i
results.matrix[i, 2] <- cor$estimate
}