I am working on a negative binomial model using the glmer.nb function within the lme4 package of R. The actual model itself is somewhat complicated, but should be (at least I believe) statistically sound. My question at the moment arises because the model is having difficult converging and returns this warning:
In checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
Model failed to converge with max|grad| = 0.00753068 (tol = 0.001, component 1)
Most of the time, I work within the standard glmer function, and there, when I get this warning, I add this argument to the glmer function:
glmerControl(optimizer="bobyqa", optCtrl = list(maxfun = 100000))
That usually solves the problem. Now, looking into the help file for glmer.nb, it appears the analogous argument for glmer.nb is nb.control. However, when I just change glmerControl to nb.control, R returns an error that it can't find that function. Ok, that's fine. From the given syntax in the help file, it looks like nb.control is supposed to be set equal to a list of whatever your desired control arguments are. I have tried various ways to get my two desired changes, and R just keeps dropping nb.control with the warning "extra argument(s) ‘nb.control’ disregarded"
I have tried searching the vast resource that is the internet for an example of someone that has used the nb.control argument. Most things that I have found (and I haven't been able to find much, hence this question) seem to just recommend the use of the glmerControl argument from glmer. When I put that argument in, it doesn't seem to solve the problem.
Essentially, I am just wondering how to use the nb.control argument to change the optimizer to 'bobyqa' and change the number of iterations to a higher number than default. What is the syntax for using the nb.control argument when it is not the defauilt value of NULL? Any thoughts would be appreciated. Thanks!
It's a little counterintuitive, but you should use control=glmerControl(...) for this, just as you would for the analogous glmer fit - this will get passed through to the inner loop.
Set up data etc:
library(lme4)
dd <- expand.grid(f1 = factor(1:3),
f2 = LETTERS[1:2], g=1:9, rep=1:15)
dd$y <- simulate(~f1+f2+(1|g),
newparams=list(beta=rep(1,4),
theta=1),
newdata=dd,
seed=101,
family=negative.binomial(theta=1.5))[[1]]
Fit "vanilla":
m.nb <- glmer.nb(y ~ f1+f2 + (1|g), data=dd)
Check optimization info:
m.nb#optinfo[c("optimizer","control")]
## $optimizer
## [1] "Nelder_Mead"
##
## $control
## $control$verbose
## [1] 0
Fit with alternative optimizer/etc.:
m.nb2 <- glmer.nb(y ~ f1+f2 + (1|g), data=dd,
control=glmerControl(optimizer="bobyqa",
optCtrl=list(maxfun=1e5)))
Check that we actually changed something:
m.nb2#optinfo[c("optimizer","control")]
## $optimizer
## [1] "bobyqa"
##
## $control
## $control$maxfun
## [1] 1e+05
##
## $control$iprint
## [1] 0
Related
When trying to graph the conditional fixed effects of a glmmTMB model with two random intercepts in GGally I get the error:
There was an error calling "tidy_fun()". Most likely, this is because the
function supplied in "tidy_fun=" was misspelled, does not exist, is not
compatible with your object, or was missing necessary arguments (e.g. "conf.level=" or "conf.int="). See error message below.
Error: Error in "stop_vctrs()":
! Can't recycle "..1" (size 3) to match "..2" (size 2).`
I have tinkered with figuring out the issue and it seems to be related to the two random intercepts included in the model. I have also tried extracting the coefficient and standard error information separately through broom.mixed::tidy and then feeding the data frame into GGally:ggcoef() with no avail. Any suggestions?
# Example with built-in randu data set
data(randu)
randu$A <- factor(rep(c(1,2), 200))
randu$B <- factor(rep(c(1,2,3,4), 100))
# Model
test <- glmmTMB(y ~ x + z + (0 +x|A) + (1|B), family="gaussian", data=randu)
# A few of my attempts at graphing--works fine when only one random effects term is in model
ggcoef_model(test)
ggcoef_model(test, tidy_fun = broom.mixed::tidy)
ggcoef_model(test, tidy_fun = broom.mixed::tidy, conf.int = T, intercept=F)
ggcoef_model(test, tidy_fun = broom.mixed::tidy(test, effects="fixed", component = "cond", conf.int = TRUE))
There are some (old!) bugs that have recently been fixed (here, here) that would make confidence interval reporting on RE parameters break for any model with multiple random terms (I think). I believe that if you are able to install updated versions of both glmmTMB and broom.mixed:
remotes::install_github("glmmTMB/glmmTMB/glmmTMB#ci_tweaks")
remotes::install_github("bbolker/broom.mixed")
then ggcoef_model(test) will work.
I am estimating a mixed model with glmer and experience the error
Error in zeta(shiftpar, start = opt[seqpar1][-w]) : profiling detected new, lower deviance
I found a solution by "boosting" the devtol parameter. However, I don't know how and I can't find the solution.
Here is my model:
m3.glmer = glmer(binExap ~ (1|id) + Lag1 + Lag2 + Lag5 + BroadQ,
data = CLnMD,
family = binomial(link="logit"),
nAGQ=1, control = glmerControl(optimizer = 'bobyqa',
optCtrl=list(maxfun=100000)))
This is the code I am using for estimating the CIs:
KIsBoot <- confint.merMod(m3.glmer, method = "profile", nsim = 250)
Now where do I boost/how would I boost "devtol"?
This is admittedly a bit obscure. confint.merMod() takes a ... argument that gets passed to profile.merMod. ?profile.merMod says:
devtol: tolerance for fitted deviances less than baseline (supposedly
minimum) deviance.
So, if you want to ignore this check completely,
confint(m3.glmer, devtol = Inf)
should work. (You don't need .merMod, R figures that out automatically; "profile" is the default setting; and nsim is ignored unless method = "boot" [we should add a warning!])
However, I would also say a little bit pessimistically that if you're getting this error your profile CIs might not be very reliable ... try visualizing the profile as well (pp <- profile(m3.glmer, devtol = Inf); lattice::xyplot(pp)) to make sure it looks reasonable (i.e. at least monotonic!)
I'm trying to bag conditional inference trees following the advice of Kuhn et al in 'Applied Predictive Modeling', Ch.8:
Conditional inference trees can also be bagged using the cforest function > in the party package if the argument mtry is equal to the number of
predictors:
library(party)
The mtry parameter should be the number of predictors (the
number of columns minus 1 for the outcome).
bagCtrl <- cforest_control(mtry = ncol(trainData) - 1)
baggedTree <- cforest(y ~ ., data = trainData, controls = bagCtrl)
Note there may be a typo in the above code (and also in the package's help file), as discussed here:
R package 'partykit' unused argument in ctree_control
However when I try to replicate this code using a dataframe (and trainData in above code is also a dataframe) such that there is more than one independent/predictor variable, I'm getting an error though it works for just one independent variable:
Some dummy code for simulations:
library(party)
df = data.frame(y = runif(5000), x = runif(5000), z = runif(5000))
bagCtrl <- cforest_control(mtry = ncol(df) - 1)
baggedTree_cforest <- cforest(y ~ ., data = df, control = bagCtrl)
The error message is:
Error: $ operator not defined for this S4 class
Thanks for any help.
As suggested, posting my comment from above as an answer as a general R 'trick' if something expected doesn't work and the program has several libraries loaded:
but what solved it was adding the party namespace explicitly to the function > call, so party::cforest() instead of just cforest(). I've also got
library(partykit) loaded in my actual program which too has a cforest()
function and the error could be stemming from there though both functions are > essentially the same
caret::train() is another example where this often pops up
I'm very new to R (and stackoverflow). I've been trying to conduct a simple slopes analysis for my continuous x dichotomous regression model using lmres, and simpleSlope from the pequod package.
My variables:
SLS - continuous DV
csibdiff - continuous predictor (I already manually centered the variable with another code)
culture - dichotomous moderator
newmod<-lmres(SLS ~ csibdiff*culture, data=sibdat2)
newmodss <-simpleSlope(newmod, pred="csibdiff", mod1="culture")
However, after running the simpleSlope function, I get this error message:
Error in if (nomZ %in% coded) { : argument is of length zero
I don't understand the nomZ part but I assume something was wrong with my variables. What does this mean? I don't have a nomZ named thing in my data at all. None of my variables are null class (I checked them with the is.null() function), and I didn't seem to have accidentally deleted the contents of the variable (I checked with the table() function).
If anyone else can suggest another function/package that I can do a simple slope analysis in, as well, I'd appreciate it. I've been stuck on this problem for a few days now.
EDIT: I subsetted the relevant variables into a csv file.
https://www.dropbox.com/s/6j82ky457ctepkz/sibdat2.csv?dl=0
tl;dr it looks like the authors of the package were thinking primarily about continuous moderators; if you specify mod1="cultureEuropean" (i.e. to match the name of the corresponding parameter in the output) the function returns an answer (I have no idea if it's sensible or not ...)
It would be a service to the community to let the maintainers of the pequod package (maintainer("pequod")) know about this issue ...
Read data and replicate error:
sibdat2 <- read.csv("sibdat2.csv")
library(pequod)
newmod <- lmres(SLS ~ csibdiff*culture, data=sibdat2)
newmodss <- simpleSlope(newmod, pred="csibdiff", mod1="culture")
Check the data:
summary(sibdat2)
We do have some NA values in csibdiff, so try removing these ...
sibdat2B <- na.omit(sibdat2)
But that doesn't actually help (same error as before).
Plot the data to check for other strangeness
library(ggplot2); theme_set(theme_bw())
ggplot(sibdat2B,aes(csibdiff,SLS,colour=culture))+
stat_sum(aes(size=factor(..n..))) +
geom_smooth(method="lm")
There's not much going on here, but nothing obviously wrong either ...
Use traceback() to see approximately where the problem is:
traceback()
3: simple.slope(object, pred, mod1, mod2, coded)
2: simpleSlope.default(newmod, pred = "csibdiff", mod1 = "culture")
1: simpleSlope(newmod, pred = "csibdiff", mod1 = "culture")
We could use options(error=recover) to jump right to the scene of the crime, but let's try step-by-step debugging instead ...
debug(pequod:::simple.slope)
As we go through we can see this:
nomZ <- names(regr$coef)[pos_mod]
nomZ ## character(0)
And looking a bit farther back we can see that pos_mod is also a zero-length integer. Farther back, we see that the code is looking through the parameter names (row names of the variance-covariance matrix) for the name of the modifier ... but it's not there.
debug: pos_pred_mod1 <- fI + grep(paste0("\\b", mod1, "\\b"), jj[(fI +
1):(fI + fII)])
Browse[2]> pos_mod
## integer(0)
Browse[2]> jj[1:fI]
## [[1]]
## [1] "(Intercept)"
##
## [[2]]
## [1] "csibdiff"
##
## [[3]]
## [1] "cultureEuropean"
Browse[2]> mod1
## [1] "culture"
The solution is to tell simpleSlope to look for a variable that is there ...
(newmodss <- simpleSlope(newmod, pred="csibdiff", mod1="cultureEuropean"))
## Simple Slope:
## simple slope standard error t-value p.value
## Low cultureEuropean (-1 SD) -0.2720128 0.2264635 -1.201133 0.2336911
## High cultureEuropean (+1 SD) 0.2149291 0.1668690 1.288011 0.2019241
We do get some warnings about NaNs produced -- you'll have to dig farther yourself to see if you need to worry about them.
Is lme4:::profile.merMod() supposed to work with glmer models? What about Negative Binomial models?
I have a negative binomial model that throws this error:
Error in names(opt) <- profnames(fm, signames) :
'names' attribute [2] must be the same length as the vector [1]
When I try and run the profile function on my model profile(model12) to get standard errors for my random effects.
Am I missing something or is this a problem with lme4?
I should mention that I'm using glmer(..., family = negative.binomial(theta = lme4:::est_theta(poissonmodel))) not glmer.nb() because I had issues with the update() function in using glmer.nb().
I can reproduce your error with the CRAN version (1.1-8). There has been some improvement in glmer.nb in the most recent development version, so if you have compilation tools installed I would definitely do devtools::install_github("lme4/lme4") and try again. In addition, update() works better with NB models now, so you might not need your workaround.
This works fine with version 1.1-9:
library("lme4")
m1 <- glmer.nb(TICKS~cHEIGHT+(1|BROOD),data=grouseticks)
pp <- profile(m1)
lattice::xyplot(pp)
Note by the way that your solution with est_theta only does the initial step or two of an iterative solution where the theta value and the other parameters are optimized in alternation ...
m0 <- glmer(TICKS~cHEIGHT+(1|BROOD),data=grouseticks,family=poisson)
m2 <- update(m0,
family = negative.binomial(theta = lme4:::est_theta(m0)))
cbind(glmer.nb=fixef(m1),pois=fixef(m0),fakenb=fixef(m2))
## glmer.nb pois fakenb
## (Intercept) 0.58573085 0.56835340 0.57759498
## cHEIGHT -0.02520326 -0.02521386 -0.02520702
profile() works OK on this model too, at least in the devel version ...