Is lme4:::profile.merMod() supposed to work with glmer models? What about Negative Binomial models?
I have a negative binomial model that throws this error:
Error in names(opt) <- profnames(fm, signames) :
'names' attribute [2] must be the same length as the vector [1]
When I try and run the profile function on my model profile(model12) to get standard errors for my random effects.
Am I missing something or is this a problem with lme4?
I should mention that I'm using glmer(..., family = negative.binomial(theta = lme4:::est_theta(poissonmodel))) not glmer.nb() because I had issues with the update() function in using glmer.nb().
I can reproduce your error with the CRAN version (1.1-8). There has been some improvement in glmer.nb in the most recent development version, so if you have compilation tools installed I would definitely do devtools::install_github("lme4/lme4") and try again. In addition, update() works better with NB models now, so you might not need your workaround.
This works fine with version 1.1-9:
library("lme4")
m1 <- glmer.nb(TICKS~cHEIGHT+(1|BROOD),data=grouseticks)
pp <- profile(m1)
lattice::xyplot(pp)
Note by the way that your solution with est_theta only does the initial step or two of an iterative solution where the theta value and the other parameters are optimized in alternation ...
m0 <- glmer(TICKS~cHEIGHT+(1|BROOD),data=grouseticks,family=poisson)
m2 <- update(m0,
family = negative.binomial(theta = lme4:::est_theta(m0)))
cbind(glmer.nb=fixef(m1),pois=fixef(m0),fakenb=fixef(m2))
## glmer.nb pois fakenb
## (Intercept) 0.58573085 0.56835340 0.57759498
## cHEIGHT -0.02520326 -0.02521386 -0.02520702
profile() works OK on this model too, at least in the devel version ...
Related
I have estimated several glms with cluster robust standard errors using the function glm.cluster() from the miceadds package in R.
Unfortunately, the function does not automatically calculate a pseudo R-squared.
Moreover, I am unable to find a package for calculating a pseudo R-squared that is compatible with glm.cluster. So far, I have tried rcompanion's nagelkerke(), fmsb’s NagelkerkeR2() and even psfmi's rsq_nagel().
Did anyone else face this problem before and do you know how to resolve it without writing one's own function?
glm.cluster returns a two-element list, the first element of which (called $glm_res) is the actual glm fit. I used performance::r2_nagelkerke() because that was what I had handy, but any Nagelkerke R2 function will probably work if you apply it to the $glm_res component.
Set up example:
data(data.ma01, package = "miceadds")
dat <- data.ma01
dat$highmath <- 1 * ( dat$math > 600 ) # create dummy variable
mod2 <- miceadds::glm.cluster( data=dat, formula=highmath ~ hisei + female,
cluster="idschool", family="binomial")
library(performance)
r2_nagelkerke(mod2$glm_res)
## Nagelkerke's R2
## 0.05126081
I have discovered some heteroscedasticity in my model that I would like to compensate for with more robust standard errors. I have tried to use the Huber-White robust standard errors from the merDeriv package in R but I beleive these only work for a GLMM with a binomial distribution. Is there a way I could achieve the same thing for a Negative Binomial distribition?
Model:
library(lme4)
model <- glmer.nb(Jobs ~ 1 + Month + Year + (1|Region), data = df)
Huber-White robust standard errors:
library(merDeriv)
bread.glmerMod(model)
Error:
Error in vcov.lmerMod(object, full = full) : estfun.lmerMod() only works for lmer() models.
Thank you for any help!
This looks like a bug in the package, as far as I can tell (the bread.glmerMod function was calling estfun.lmerMod rather than estfun.glmerMod; there's a broader question here about the design of the generic functions, but never mind ...)
You should be able to install a fixed version from my fork via remotes::install_github("bbolker/merDeriv"), then reload the package and try again.
Alternately, download the tarball, change vcov.lmerMod to vcov.glmerMod in the last line of R/bread.glmerMod.R, and re-install the package ...
Try something like this:
library(lme4)
model <- glmer.nb(Jobs ~ 1 + Month + Year + (1|Region), data = df)
cov <- vcovHC(model, type = "HC1", sandwich = T)
se <- sqrt(diag(cov_m1))
(Can't confirm if it works since this there isn't a reproducible example)
I am working on a negative binomial model using the glmer.nb function within the lme4 package of R. The actual model itself is somewhat complicated, but should be (at least I believe) statistically sound. My question at the moment arises because the model is having difficult converging and returns this warning:
In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge with max|grad| = 0.00753068 (tol = 0.001, component 1)
Most of the time, I work within the standard glmer function, and there, when I get this warning, I add this argument to the glmer function:
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 100000))
That usually solves the problem. Now, looking into the help file for glmer.nb, it appears the analogous argument for glmer.nb is nb.control. However, when I just change glmerControl to nb.control, R returns an error that it can't find that function. Ok, that's fine. From the given syntax in the help file, it looks like nb.control is supposed to be set equal to a list of whatever your desired control arguments are. I have tried various ways to get my two desired changes, and R just keeps dropping nb.control with the warning "extra argument(s) ‘nb.control’ disregarded"
I have tried searching the vast resource that is the internet for an example of someone that has used the nb.control argument. Most things that I have found (and I haven't been able to find much, hence this question) seem to just recommend the use of the glmerControl argument from glmer. When I put that argument in, it doesn't seem to solve the problem.
Essentially, I am just wondering how to use the nb.control argument to change the optimizer to 'bobyqa' and change the number of iterations to a higher number than default. What is the syntax for using the nb.control argument when it is not the defauilt value of NULL? Any thoughts would be appreciated. Thanks!
It's a little counterintuitive, but you should use control=glmerControl(...) for this, just as you would for the analogous glmer fit - this will get passed through to the inner loop.
Set up data etc:
library(lme4)
dd <- expand.grid(f1 = factor(1:3),
f2 = LETTERS[1:2], g=1:9, rep=1:15)
dd$y <- simulate(~f1+f2+(1|g),
newparams=list(beta=rep(1,4),
theta=1),
newdata=dd,
seed=101,
family=negative.binomial(theta=1.5))[[1]]
Fit "vanilla":
m.nb <- glmer.nb(y ~ f1+f2 + (1|g), data=dd)
Check optimization info:
m.nb#optinfo[c("optimizer","control")]
## $optimizer
## [1] "Nelder_Mead"
##
## $control
## $control$verbose
## [1] 0
Fit with alternative optimizer/etc.:
m.nb2 <- glmer.nb(y ~ f1+f2 + (1|g), data=dd,
control=glmerControl(optimizer="bobyqa",
optCtrl=list(maxfun=1e5)))
Check that we actually changed something:
m.nb2#optinfo[c("optimizer","control")]
## $optimizer
## [1] "bobyqa"
##
## $control
## $control$maxfun
## [1] 1e+05
##
## $control$iprint
## [1] 0
I'm constructing a piecewise structural equation model using the piecewiseSEM package in R (Lefcheck - https://cran.r-project.org/web/packages/piecewiseSEM/vignettes/piecewiseSEM.html)
I already created the model set and I could evaluate the model fit, so the model itself works. Also, the data fits the model (p = 0.528).
But I do not succeed in extracting the path coefficients.
This is the error i get: Error in cbind(Xlarge, Xsmall) : number of rows of matrices must match (see arg 2)
I already tried (but this did not work):
standardising my data because of the warning: Some predictor variables are on very different scales: consider rescaling
adapted my data (threw some NA values away)
This is my modellist:
predatielijst = list(
lmer(plantgrootte ~ gapfraction + olsen_P + (1|plot_ID), data = d),
glmer(piek1 ~ gapfraction + olsen_P + plantgrootte + (1|plot_ID),
family = poisson, data = d),
glmer(predatie ~ piek1 + (1|plot_ID), family = binomial, data = d)
)
with "predatie" being a binary variable (yes or no) and all the rest continuous variables (gapfraction, plantgrootte, olsen_P & piek1)
Thanks in advance!
Try installing the development version:
library(devtools)
install_github("jslefche/piecewiseSEM#2.0")
Replace list with psem and run the coefs or summary function. It will likely get rid of your error. If not, open a bug on Github!
WARNING: this will overwrite your current version from CRAN. You will need to reinstall from CRAN to get version 1.4 back.
try to use lme (out of the nlme library) ilstead of glmer. As far as I understand, the fact that lmer does not provide p-values (while lme does) seems to be the problem here.
Hope this works.
I am getting an error while running naive bayes classifier in R. I am using the following code-
mod1 <- naiveBayes(factor(X20) ~ factor(X1) + factor(X2) +factor(X3) +factor(X4)+factor(X5)+factor(X6)+factor(X7)
+factor(X8)+factor(X9)
+factor(X10)+factor(X11)+ factor(X12)+factor(X13)+factor(X14)
+factor(X15)
+factor(X16)+factor(X17)
+factor(X18)+factor(X19),data=intent.test)
res1 <- predict(mod1)$posterior
First part of this code runs fine. But when it try to predict the posterior probability it throws following error-
**Error in as.data.frame(newdata) :
argument "newdata" is missing, with no default**
I tried running something like
res1 <- predict(mod1,new_data=intent.test)$posterior
but this also gives the same error.
You seem to be using the e1071::naiveBayes algorithm, which expects a newdata argument for prediction, hence the two errors raised when running your code. (You can check the source code of the predict.naiveBayes function on CRAN; the second line in the code is expecting a newdata, as newdata <- as.data.frame(newdata).) Also as pointed out by #Vincent, you're better off converting your variables to factor before calling the NB algorithm, although this has certainly nothing to do with the above errors.
Using NaiveBayes from the klar package, no such problem would happen. E.g.,
data(spam, package="ElemStatLearn")
library(klaR)
# set up a training sample
train.ind <- sample(1:nrow(spam), ceiling(nrow(spam)*2/3), replace=FALSE)
# apply NB classifier
nb.res <- NaiveBayes(spam ~ ., data=spam[train.ind,])
# predict on holdout units
nb.pred <- predict(nb.res, spam[-train.ind,])
# but this also works on the training sample, i.e. without using a `newdata`
head(predict(nb.res))