I have a simple example below (which doesn't work) that attempts to do multivariate fit using the default algorithm (Gauss-Newton). I get the error: "Error in nlsModel(formula, mf, start, wts) : singular gradient matrix at initial parameter estimates".
## Defining the two independent x variables, and the one dependent y variable.
x1 = 1:100*.01
x2 = (1:100*.01)^2
y1 = 2*x1 + 0.5*x2
## Putting into a data.frame for nls() funcion.
df = data.frame(x1, x2, y1)
## Starting parameters: a = 2.1, b = 0.4 (and taking c = 0)
fit_results <-nls(y1 ~ x1*a + x2*b +c, data=df, start=c(a=2.1, b=0.4, c=0))
Note: even when I set a = 2, and b = 0.5 above, I still get the same error message.
Thanks Brian, not sure how to make a comment the selected answer. Here is code that works... turns out I needed to add more randomness in the y1 dependent variable.
## Defining the two independent x variables, and the one dependent y variable.
x1 = 1:100*0.1
x2 = runif(100,0,10)
y1 = 2*x1 + 0.5*x2*runif(100,0.9,1.1)
## Putting into a data.frame for nls() funcion.
df = data.frame(x1, x2, y1)
fit_results <-nls(y1 ~ x1*a + x2*b +c, data=df, start=c(a=2.1, b=0.4, c=0))
Related
Out of curiosity, I am trying to figure out why the PLS regression coefficients obtained with pls differ from the coefficients obtained with plsRglm, ropls, or plsdepot which all provide the same results.
Here is some code to start with. I have tried to play with the scale, center, and method arguments of the plsr function... but no success so far.
library(pls)
library(plsRglm)
library(ropls)
library(plsdepot)
data(Cornell)
pls.plsr <- plsr(
Y~X1+X2+X3+X4+X5+X6+X7,
data = Cornell,
ncomp = 3,
scale = TRUE,
center = TRUE
)
plsRglm.plsr <- plsR(
Y~X1+X2+X3+X4+X5+X6+X7,
data = Cornell,
nt = 3,
scaleX = TRUE
)
ropls.plsr <- opls(
as.matrix(Cornell[, grep("X", colnames(Cornell))]),
Cornell[, "Y"],
scaleC = "standard"
)
plsdepot.plsr <- plsreg1(
as.matrix(Cornell[, grep("X", colnames(Cornell))]),
Cornell[, "Y"],
comps = 3
)
## extract PLS regression coefficients for the PLS model with three components
coef(pls.plsr) # a
coef(plsRglm.plsr, type = "original") # b
coef(plsRglm.plsr, type = "scaled") # c
coef(ropls.plsr) # c
plsdepot.plsr$std.coefs # c
plsdepot.plsr$reg.coefs # b
Firstly, just for re-formatting purposes, we write:
library(pls)
library(plsRglm)
library(ropls)
library(plsdepot)
data(Cornell)
pls.plsr <- plsr(Y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7,
data = Cornell,
ncomp = 3, scale = T, center = T)
plsRglm.plsr <- plsR(Y ~ X1 + X2 + X3 + X4 + X5 + X6 + X7,
data = Cornell,
nt = 3, scaleX = TRUE)
ropls.plsr <- opls(as.matrix(Cornell[, grep("X", colnames(Cornell))]),
Cornell[, "Y"], scaleC = "standard")
plsdepot.plsr <- plsreg1(as.matrix(Cornell[, grep("X", colnames(Cornell))]),
Cornell[, "Y"], comps = 3)
That done, you may extract the coefficients in the original scale:
### ORIGINAL SCALE - plsRglm, plsdepot
coef(plsRglm.plsr, type = "original")
plsdepot.plsr$reg.coefs
Or you can have them scaled:
### SCALED - plsRglm, ropls, plsdepot
coef(plsRglm.plsr, type = "scaled")
coef(ropls.plsr)
plsdepot.plsr$std.coefs
Therefore all methods now give rise to the same coefficients... Except for pls::plsr. Why? You may ask. The key is in the command. When you run:
coef(pls.plsr) # , , 3 comps
You see ", , 3". That is characteristic of a tensor object. What is this? Coefficients should be simply a vector. The reason is that coef is a generic function and it is not working properly for pls::plsr models. To see what it is actually extracting:
pls.plsr$coefficients
matrix(pls.plsr$coefficients, ncol = 3) # or in matrix form. coef simply extracts the third column (it should not)
But you can see the same fit for all models if you examine the equivalent object in each R-package as in:
matrix(pls.plsr$projection, ncol = 3)
plsRglm.plsr$wwetoile
plsdepot.plsr$mod.wgs
ropls.plsr#weightStarMN
Therefore, for pls::plsr you were simply not extracting the coefficients.
My question is similar to the unanswered here: working with SSasymp in r
For a simple SSmicmen:
x1 = seq (0,10,1)
y1 = SSmicmen(x1, Vm=10, K=0.5)
plot(y1 ~ x1, type="l")
the value of K is easily identified in the point (5, 0.5), the value of half the maximum growth.
Given a simple SSasympOrig:
x2 = seq (0,10,1)
y2 = SSasympOrig(x2, Asym=10, lrc=0.1)
# Asym*(1 - exp(-exp(lrc)*input))
plot(y2 ~ x2, type="l")
is there a way to represent and/or identify the meaning and/or effect of the parameter "lcr" on the resulting graph, in a similar way as the example above?
Sure, you can visualize this:
x2 = seq (0,10,0.01)
y2 = SSasympOrig(x2, Asym=10, lrc=0.1)
# Asym*(1 - exp(-exp(lrc)*input))
plot(y2 ~ x2, type="n")
for (lrc in (10^((-5):1))) {
y2 = SSasympOrig(x2, Asym=10, lrc=lrc)
# Asym*(1 - exp(-exp(lrc)*input))
lines(y2 ~ x2, type="l", col = 6+log10(lrc))
}
This parameter controls how fast the asymptote is approached. Getting this from studying the equation requires highschool-level maths skills. Or you could try reading the Wikipedia entry about the half-life:
y2 = SSasympOrig(x2, Asym=10, lrc=0.1)
# Asym*(1 - exp(-exp(lrc)*input))
plot(y2 ~ x2, type="l")
points(x = log(2) / exp(0.1), y = 0.5 * 10)
I'm using the gamlss package in R to implement wormplots for the residuals study.
The function wp() has an argument xvar which is used for bucketing.
Assume I have a "numeric" vector x1 which if passed as "xvar = x1" behaves differently than "xvar = ~x1". Basically the second case is treated as a formula. The buckets created for both cases will be different from each other.
Code :-
library(gamlss)
glc<-gamlss.control(n.cyc = 200)
myseed <- 12345
set.seed(myseed) #this will make results reproducible
# generate data
N<-10000 # this is the sample size
dd<-data.frame(x1=rpois(N,1)
,x2=rnorm(N,.7,.3)
,x3=log(rgamma(N,shape=6,scale=10))
,x4=sample(letters[1:3], N, replace = T)
,x5=sample(letters[3:6], N, replace = T)
,ind = rbinom(N,size=1,prob=0.5)
)
#Generate distributions
dd$y_wei1<-rweibull(N,scale=exp(.3*dd$x1+.8*dd$x3),shape=5)
m1 <- gamlss(formula = y_wei1 ~ x1 + x3 + x4 + x5,
data = dd ,
family = "WEI" ,
K = 2,
control = glc
)
# Case 1.
wp(object = m1, xvar = x1, n.iter = 4)
# Case 2.
wp(object = m1, xvar = ~x1, n.iter = 4)
Edit :
I do observed that this happens only when the overlap argument is set to 0. Because when overlap=0 then internally another function( check.overlap) is called. Why is this function called?
the function has been written such that xvar = ~x1 indicated x1 is a factor/char variable and so grouping occurs based on its unique values. When user calls with xvar = x1 then bins are created based on the range and that is used to generate the wormplots.
The difference is because internally there is a check.overlap fucntion written which is impemented only if x1 is numeric. Incase of overlapping, it clips it to have non-overlapping intervals. This is missing if user calls it as xvar = ~x1.
Using a dataset I built a model as below:
fit <- lm(y ~ as.numeric(X1) + as.factor(x2) + log(1 + x3) + as.numeric(X4) , dataset)
Then I build new data:
X1 <- 1
X2 <- 10
X3 <- 15
X4 <- 0.5
new <- data.frame(X1, X2, X3, X4)
predict(fit, new , se.fit=TRUE)
Then I get the Error below:
Error in data.frame(state_today, daily_creat, last1yr_min_hosp_icu_MDRD, :
object 'X2' is not found
What am I doing wrong? Is this because of logarithm in the model?
A great way of looking at your problem another way is by constructing a self contained reproducible example. With no copy/pasting. This often gives you a fresh perspective and often teases out the weirdest bugs imaginable.
As flodel and Ben have pointed out, your problem is probably due to bad choice of variable names. I'm guessing you're using Rstudio, which in my opinion uses a terrible default font exactly for this reason. I can't tell x and X apart (easily).
Here is something similar to what you're trying to do, with all variable names correctly (un)capitalized.
xy <- data.frame(y = runif(20), x1 = runif(20), x2 = sample(1:5, 20, replace = TRUE), x3 = runif(20))
fit <- lm(y ~ as.numeric(x1) + as.factor(x2) + log(1+x3), data = xy)
predict(fit, newdata = data.frame(x1 = 1, x2 = as.factor(3), x3 = 15))
1
0.05015187
I'm trying to use linear regression to figure out the best weighting for 3 models to predict an outcome. So there are 3 variables (x1, x2, x3) that are the predictions of the dependent variable, y. My question is, how do I run a regression with the constraint that the sum of the coefficients sum to 1. For example:
this is good:
y = .2(x1) + .4(x2) + .4(x3)
since .2 + .4 + .4 = 1
this is no good:
y = 1.2(x1) + .4(x2) + .3(x3)
since 1.2 + .4 + .3 > 1
I'm looking to do this in R if possible. Thanks. Let me know if this needs to get moved to the stats area ('Cross-Validated').
EDIT:
The problem is to classify each row as 1 or 0. y is the actual values ( 0 or 1 ) from the training set, x1 is the predicted values from a kNN model, x2 is from a randomForest, x3 is from a gbm model. I'm trying to get the best weightings for each model, so each coefficient is <=1 and the sum of the coefficients == 1.
Would look something like this:
y/Actual value knnPred RfPred gbmPred
0 .1111 .0546 .03325
1 .7778 .6245 .60985
0 .3354 .1293 .33255
0 .2235 .9987 .10393
1 .9888 .6753 .88933
... ... ... ...
The measure for success is AUC. So I'm trying to set the coefficients to maximize AUC while making sure they sum to 1.
There's very likely a better way that someone else will share, but you're looking for two parameters such that
b1 * x1 + b2 * x2 + (1 - b1 - b2) * x3
is close to y. To do that, I'd write an error function to minimize
minimizeMe <- function(b, x, y) { ## Calculates MSE
mean((b[1] * x[, 1] + b[2] * x[, 2] + (1 - sum(b)) * x[, 3] - y) ^ 2)
}
and throw it to optim
fit <- optim(par = c(.2, .4), fn = minimizeMe, x = cbind(x1, x2, x3), y = y)
No data to test on:
mod1 <- lm(y ~ 0+x1+x2+x3, data=dat)
mod2 <- lm(y/I(sum(coef(mod1))) ~ 0+x1+x2+x3, data=dat)
And now that I think about it some more, skip mod2, just:
coef(mod1)/sum(coef(mod1))
For the five rows shown either of round(knnPred) or round(gbmPred) give perfect predictions so there is some question whether more than one predictor is needed.
At any rate, to solve the given question as stated the following will give nonnegative coefficients that sum to 1 (except possibly for tiny differences due to computer arithmetic). a is the dependent variable and b is a matrix of independent variables. c and d define the equality constraint (coeffs sum to 1) and e and f define the inequality constraints (coeffs are nonnegative).
library(lsei)
a <- cbind(x1, x2, x3)
b <- y
c <- matrix(c(1, 1, 1), 1)
d <- 1
e <- diag(3)
f <- c(0, 0, 0)
lsei(a, b, c, d, e, f)