R modelsummary output: robust standard errors from coxph model - r

Im running a series of coxph models in R and compiling the output into latex tables using the modelsummary package and command. The coxph provides SE and robust se as outputs and the p-value is based on the robust se. Here is a quick example to illustrate the output:
test_data <- list(time=c(4,3,1,1,2,2,3)
, status=c(1,1,1,0,1,1,0)
, x=c(0,2,1,1,1,0,0)
, sex=c(0,0,0,0,1,1,1))
model <- coxph(Surv(time, status) ~ x + cluster(sex), test_data)
Call:
coxph(formula = Surv(time, status) ~ x, data = test1, cluster = sex)
coef exp(coef) se(coef) robust se z p
x 0.460778 1.585306 0.562800 0.001052 437.9 <2e-16
Likelihood ratio test=0.66 on 1 df, p=0.4176
n= 7, number of events= 5
Next, Im trying to create latex tables from these models, displaying the robust se.
modelsummary(model, output = "markdown", fmt = 3, estimate = "{estimate}{stars}", statistic = "std.error")
| | Model 1 |
|:---------------------|:----------:|
|x | 0.461*** |
| | (0.563) |
|Num.Obs. | 7 |
|R2 | 0.090 |
|AIC | 13.4 |
As we can see, only the non-adjusted se is displayed. I could not find any alternative for this statistic = "std.error" parameter that fits and also something like vcov="robust" does not work.
How can I display any kind of robust standard errors using modelsummary for coxph models?
Thanks for reading and any help is appreciated.

The next version of modelsummary (now available on Github) will produce a more informative error message in those cases:
library(survival)
library(modelsummary)
test_data <- list(time=c(4,3,1,1,2,2,3)
, status=c(1,1,1,0,1,1,0)
, x=c(0,2,1,1,1,0,0)
, sex=c(0,0,0,0,1,1,1))
model <- coxph(Surv(time, status) ~ x + cluster(sex), test_data)
modelsummary(model, vcov = "robust")
# Error: Unable to extract a variance-covariance matrix for model object of class
# `coxph`. Different values of the `vcov` argument trigger calls to the `sandwich`
# or `clubSandwich` packages in order to extract the matrix (see
# `?insight::get_varcov`). Your model or the requested estimation type may not be
# supported by one or both of those packages, or you were missing one or more
# required arguments in `vcov_args` (like `cluster`).
As you can see, the problem is that modelsummary does not compute robust standard errors itself. Instead, it delegates this task to the sandwich or clubSandwich packages. Unfortunately, this coxph model does not appear appear to be supported by those packages:
sandwich::vcovHC(model)
#> Error in apply(abs(ef) < .Machine$double.eps, 1L, all): dim(X) must have a positive length
sandwich is the main package in the R ecosystem to compute robust standard errors. AFAICT, all the other table-making packages available (e.g., stargazer, texreg) also use sandwich, so you are unlikely to have success by looking at those. If you find another package which can compute robust standard errors for Cox models, please file a report on the modelsummary Github repository. I will investigate to see if it’s possible to add support then.
If the info you want is available in the summary object, you can add this information by following the instructions here:
https://vincentarelbundock.github.io/modelsummary/articles/modelsummary.html#adding-new-information-to-existing-models
tidy_custom.coxph <- function(x, ...) {
s <- summary(x)$coefficients
data.frame(
term = row.names(s),
robust.se = s[, "robust se", drop = FALSE])
}
modelsummary(model, statistic = "robust.se")
Model 1
x
0.461
(0.001)
Num.Obs.
7
AIC
13.4
BIC
13.4
RMSE
0.61

Related

Marginal effects plot of PLM

I’ve run an individual-fixed effects panel model in R using the plm-package. I now want to plot the marginal effects.
However, neither plot_model() nor effect_plot() work for plm-objects. plot_model() works for type = “est” but not for type = “pred”.
My online search so far only suggests using ggplot (which however only displays OLS-regressions, not fixed effects) or outdated functions (i.e, sjp.lm())
Does anyone have any recommendations how I can visualize effects of plm-objects?
IFE_Aut_uc <- plm(LoC_Authorities_rec ~ Compassion_rec, index = c("id","wave"), model = "within", effect = "individual", data = D3_long2)
summary(IFE_Aut_uc)
plot_model(IFE_Aut_uc, type = "pred”)
Error in data.frame(..., check.names = FALSE) :
arguments imply differing number of rows: 50238, 82308
and:
effect_plot(IFE_Pol_uc, pred = Compassion_rec)
Error in `stop_wrap()`:
! ~does not appear to be a one- or two-sided formula.
LoC_Politicians_recdoes not appear to be a one- or two-sided formula.
Compassion_recdoes not appear to be a one- or two-sided formula.
Backtrace:
1. jtools::effect_plot(IFE_Pol_uc, pred = Compassion_rec)
2. jtools::get_data(model, warn = FALSE)
4. jtools:::get_lhs(formula)
Edit 2022-08-20: The latest version of plm on CRAN now includes a predict() method for within models. In principle, the commands illustrated below using fixest should now work with plm as well.
In my experience, plm models are kind of tricky to deal with, and many of the packages which specialize in “post-processing” fail to handle these objects properly.
One alternative would be to estimate your “within” model using the fixest package and to plot the results using the marginaleffects package. (Disclaimer: I am the marginaleffects author.)
Note that many of the models estimated by plm are officially supported and tested with marginaleffects (e.g., random effects, Amemiya, Swaymy-Arora). However, this is not the case of this specific "within" model, which is even trickier than the others to support.
First, we estimate two models to show that the plm and fixest versions are equivalent:
library(plm)
library(fixest)
library(marginaleffects)
library(modelsummary)
data("EmplUK")
mod1 <- plm(
emp ~ wage * capital,
index = c("firm", "year"),
model = "within",
effect = "individual",
data = EmplUK)
mod2 <- feols(
emp ~ wage * capital | firm,
se = "standard",
data = EmplUK)
models <- list("PLM" = mod1, "FIXEST" = mod2)
modelsummary(models)
PLM
FIXEST
wage
0.000
0.000
(0.034)
(0.034)
capital
2.014
2.014
(0.126)
(0.126)
wage × capital
-0.043
-0.043
(0.004)
(0.004)
Num.Obs.
1031
1031
R2
0.263
0.986
R2 Adj.
0.145
0.984
R2 Within
0.263
R2 Within Adj.
0.260
AIC
4253.9
4253.9
BIC
4273.7
4273.7
RMSE
1.90
1.90
Std.Errors
IID
FE: firm
X
Now, we use the marginaleffects package to plot the results. There are two main functions for this:
plot_cap(): plot conditional adjusted predictions. How does my predicted outcome change as a function of a covariate?
plot_cme(): plot conditional marginal effects. How does the slope of my model with respect to one variable (i.e., a derivative or “marginal effect”) change with respect to another variable?
See the website for definitions and details: https://vincentarelbundock.github.io/marginaleffects/
plot_cap(mod2, condition = "capital")
plot_cme(mod2, effect = "wage", condition = "capital")

Delta method for fixed effects regression using feols function

I have the following regression in R using the 'fixest' package. Yields are a function of N, N^2, P, K and S with producer by year fixed effects.
yield<- feols(yield ~ N + N_square + P + K + S |producer*year, data=data, se="hetero")
I need to use the delta method from the 'car' package to estimate the optimal rate of N and obtain the standard error. In the below example, using the marginal effect of nitrogen from my regression, I am finding the optimal rate of N at an input:output price ratio = 4.
deltaMethod(yield, "(4 - b1)/(2*b2)", parameterNames= paste("b", 0:2, sep=""))
My issue is I am unable to run the deltaMethod with the feols regression. I am given the following error:
Warning: In vcov.fixest(object, complete = FALSE):'complete' is not a valid argument of
function vcov.fixest (fyi, some of its
main arguments are 'vcov' and 'ssc').
Error in eval(g., envir) : object 'b1' not found
The deltaMethod works with lm functions. This is an issue for me, as I cannot run my regression instead as an lm function with the fixed effects as factors. This is because with my chosen data set and fixed effect variables it is extremely slow to run.
Is there any alternatives to the deltaMethod function that works with feols regressions?
#g-grothendieck's answer covers the main issue. (Which is to say that car::deltaMethod does work with fixest objects; you just have to specify the coefficient names in a particular way.) I'd also recommend updating your version of fixest, since you appear to be using an old release.
But for posterity, let me quickly tackle the subsidiary question:
Is there any alternatives to the deltaMethod function that works with feols regressions?
You can use the "hypothesis" functionality of the (excellent) marginaleffects package. Please note that this is a relatively new feature, so you'll need to install the development version of marginaleffects at the time of writing this comment.
Here's an example that replicates Gabor's one above.
library(fixest)
fm <- feols(conc ~ uptake + Treatment | Type, CO2, vcov = "hetero")
# remotes::install_github("vincentarelbundock/marginaleffects") # dev version
library(marginaleffects)
marginaleffects(
fm,
newdata = "mean",
hypothesis = "(4 - uptake)/(2 * Treatment) = 0"
) |>
summary() ## optional
#> Average marginal effects
#> Term Effect Std. Error z value Pr(>|z|) 2.5 % 97.5 %
#> 1 hypothesis -0.06078 0.01845 -3.295 0.00098423 -0.09693 -0.02463
#>
#> Model type: fixest
#> Prediction type: response
PS. For anyone else reading this, marginaleffects is something of a spiritual successor to margins, but is basically superior in every way (speed, model coverage, etc.)
In the absence of a reproducible example we will use the built-in CO2 data frame. (In the future please provide a reproducible example that can be used in answers -- see the info at the top of the r tag page.)
1) The default method of deltaMethod does not support the parameterNames argument so use the original names.
library(fixest)
library(car)
fm <- feols(conc ~ uptake + Treatment | Type, CO2, vcov = "hetero")
deltaMethod(fm, "(4 - uptake)/(2 * Treatmentchilled)")
## Estimate SE 2.5 % 97.5 %
## (4 - uptake)/(2 * Treatmentchilled) -0.060780 0.018446 -0.096934 -0.0246
2) Alternately it can work with just the coefficients and variance matrix so try this:
co <- setNames(coef(fm), c("b1", "b2"))
deltaMethod(co, "(4 - b1)/(2*b2)", vcov(fm))
## Estimate SE 2.5 % 97.5 %
## (4 - b1)/(2 * b2) -0.060780 0.018446 -0.096934 -0.0246

Driscoll and Kraay standard errors in FE regression: reproducing Stata xtscc output in R

I am trying to replicate the results provided by the Stata command xtscc in R with package plm but I am having some trouble to see the same standard errors
I am using a dataset from the package plm also in Stata for replication purposes.
# code to obtain dataset
library(lmtest)
library(car)
library(tidyverse)
data("Produc", package="plm")
write.dta(Produc,"test.dta")
My aim is to run a two way-fixed effect panel model estimation with Driscoll and Kraay standard errors. The routine in Stata is the following
use "test.dta", clear \\ to import data
** i declare the panel
xtset state year
* create the dummies for the time fixed effects
quietly tab year, gen(yeardum)
* run a two way fixed effect regression model with Driscoll and Kraay standard errors
xi: xtscc gsp pcap emp unemp yeardum*,fe
* results are the following
Coef. Std. Err. t P>|t| [95% Conf. Interval]
pcap | -.1769881 .265713 -0.67 0.515 -.7402745 .3862983
emp | 40.61522 2.238392 18.14 0.000 35.87004 45.3604
unemp | 23.59849 85.10647 0.28 0.785 -156.8192 204.0161
In R I use the following routine:
# I declare the panel
Produc <- pdata.frame(Produc, index = c("state","year"), drop.index = FALSE)
# run a two way fixed effect model
femodel <- plm(gsp~pcap+emp+unemp, data=Produc,effect = "twoway",
index = c("iso3c","year"), model="within")
# compute Driscoll and Kraay standard errors using vcovSCC
coeftest(femodel, vcovSCC(femodel))
pcap -0.17699 0.25476 -0.6947 0.4874
emp 40.61522 2.14610 18.9252 <2e-16 ***
unemp 23.59849 81.59730 0.2892 0.7725
While point estimates are the same that in Stata, standard errors are different.
To check whether I am using the "wrong" small sample adjustment for standard errors, I also tryed running the coeftest with all available adjustments, but none yields the same values as xtscc.
library(purrr)
results <- map(c("HC0", "sss", "HC1", "HC2", "HC3", "HC4"),~coeftest(femodel, vcovSCC(femodel,type = .x)))
walk(results,print)
# none of the estimated standard errors is the same as xtscc
Does anyone know how I can replicate the results of Stata in R?
Since plm version 2.4, its function within_intercept(., return.model = TRUE) can return the full model of a within model with the intercept as in Stata. With this, it is possible to exactly replicate the result of Stata's user contributed command xtscc.
The way xtscc seems to work is by estimating the twoway FE model as a one-way FE model + dummies for the time dimension. So let's replicate that with plm:
data("Produc", package="plm")
Produc <- pdata.frame(Produc, index = c("state","year"), drop.index = FALSE)
femodel <- plm(gsp ~ pcap + emp + unemp + factor(year), data = Produc, model="within")
femodelint <- within_intercept(femodel, return.model = TRUE)
lmtest::coeftest(femodelint, vcov. = function(x) vcovSCC(x, type = "sss"))
# Estimate Std. Error t value Pr(>|t|)
# (Intercept) -6547.68816 3427.47163 -1.9104 0.0564466 .
# pcap -0.17699 0.26571 -0.6661 0.5055481
# emp 40.61522 2.23839 18.1448 < 0.00000000000000022 ***
# unemp 23.59849 85.10647 0.2773 0.7816356
# [...]

robust standard errors for mixed-effects models in lme4 package of R

I am using the lme4 package for linear mixed effect modeling
the mixed-effect model is below:
fm01 <- lmer(sublat <- goal + (1|userid))
the above command returns an S4 object called fm01
this object includes fixed effects and their OLS standard errors (below)
Fixed effects:
Estimate Std. Error t value
(Intercept) 31.644 3.320 9.530
goaltypeF1 -4.075 3.243 -1.257
goaltypeF2 -9.187 5.609 -1.638
goaltypeF3 -13.935 9.455 -1.474
goaltypeF4 -20.219 8.196 -2.467
goaltypeF5 -12.134 8.797 -1.379"
however, i need to provide robust standard errors
How can I do this with an S4 object such as returned by lme4?
It looks like robust SEs for lmerMod objects are available via the merDeriv and clubSandwich packages:
library(lme4)
library(clubSandwich)
m <- lmer(Reaction ~ Days + (Days|Subject), sleepstudy)
using merDeriv
(From the replication materials of the merDeriv JSS paper, thanks to #AchimZeileis for the tip)
library(merDeriv)
sand <- sandwich(m, bread = bread(m, full = TRUE),
mean = meat(m, level = 2))
clubSandwich
(all possible types: I don't know enough to know which is 'best' in any given case)
cstypes <- paste0("CR", c("0", "1", "1p", "1S", "2", "3"))
rob_se_fun <- function(type) sqrt(diag(vcovCR(m, type = type)))
rob_se <- sapply(cstypes, rob_se_fun)
combine results
std_se <- sqrt(diag(vcov(m)))
cbind(std = std_se, rob_se,
merDeriv = sqrt(diag(sand)[1:2]))\
std CR0 CR1 CR1p CR1S CR2 CR3
(Intercept) 6.824597 6.632277 6.824557 7.034592 6.843700 6.824557 7.022411
Days 1.545790 1.502237 1.545789 1.593363 1.550125 1.545789 1.590604
merDeriv
(Intercept) 6.632277
Days 1.502237
merDeriv's results match type="CR0" (merDeriv provides robust Wald estimates for all components, including the random effect parameters; it's up to you to decide if Wald estimates for RE parameters are reliable enough)
I think this is what you're looking for: https://cran.r-project.org/web/packages/robustlmm/vignettes/rlmer.pdf
It's the robustlmm package, which has the rlmer function.
"The structure of the objects and the methods are implemented to be as similar as possible to the ones of lme4 with robustness specific extensions where needed."
fm01_rob <- rlmer(sublat <- goal + (1|userid))

lme4 and languageR compatibility error: "input model is not a mer object”

I have a dataset dat2 on which I want to fit a linear mixed-effects model. I used lmer() (package lme4) in the past in complement of pvals.fnc to compute the associated p-values.
However, since I reinstalled R 3.0.2 version with the new lme4 (1.0.4) and languageR (1.4) packages I obtain an error about the output of lmer function. It says that the output is not a mer object. Indeed its class is lmeRmod.
Here is the code I use:
names(dat2)<-c("auc","subj","decod","soa","vis")
attach(dat2)
mod1 <- lmer(auc ~ decod + (1 | subj))
mod2 <- lmer(auc ~ vis+ (1 | subj))
mod3 <- lmer(auc ~ decod + vis + (1 | subj))
mod4 <- lmer(auc ~ decod + vis + decod*vis + (1 | subj))
pvals.fnc(mod1)
And I get this error:
> pvals.fnc(mod1)
the input model is not a mer object
NULL
Indeed, when I look at mod1, I find it is a lmeRmod object and not a mer object.
> mod1
Linear mixed model fit by REML ['lmerMod']
Formula: auc ~ decod + (1 | subj)
REML criterion at convergence: -213.3884
Random effects:
Groups Name Std.Dev.
subj (Intercept) 0.04187
Residual 0.11087
Number of obs: 155, groups: subj, 6
Fixed Effects:
(Intercept) decod2 decod3 decod4
0.9798 -0.1141 -0.3599 -0.3090
This problem is very similar to the one discribed here. Any ideas of 1/ what the problem might be (why do I do not output a mer object) and 2/ how to work around it (I tried reinstalling older versions but I have compatibility problems between packages) ?
Any help would be great !
thanks !
I confirm that the pvals.fnc function doesn't work in the new languageR -- this is essentially because mcmcsamp wasn't implemented in the new version of lme4, which in turn because it was found to be unreliable in many cases.
We (the lme4 authors) are sorry to have left languageR users in the lurch this way, but it was somewhat unavoidable.
https://github.com/lme4/lme4/blob/master/man/pvalues.Rd offers some alternative suggestions for what to do about computing p-values.
https://github.com/lme4/lme4/blob/master/man/drop1.merMod.Rd gives a particular recipe (for the development version of lme4) about how to use pbkrtest::KRmodcomp to get p-values for all of the predictors in a model:
fm1 <- lmer(Reaction~Days+(Days|Subject),sleepstudy)
## Likelihood ratio test
drop1(fm1,test="Chisq")
if (require(pbkrtest)) {
KRSumFun <- function(object, objectDrop, ...) {
krnames <- c("ndf","ddf","Fstat","p.value","F.scaling")
r <- if (missing(objectDrop)) {
setNames(rep(NA,5),krnames)
} else {
krtest <- KRmodcomp(object,objectDrop)
unlist(krtest$stats[krnames])
}
attr(r,"method") <- c("Kenward-Roger via pbkrtest package")
r
}
drop1(fm1,test="user",sumFun=KRSumFun)
}
This example produces:
Single term deletions
Model:
Reaction ~ Days + (Days | Subject)
Method:
Kenward-Roger via pbkrtest package
ndf ddf Fstat p.value F.scaling
<none>
Days 1 17 45.853 3.2638e-06 1
You can use Package ‘lmerTest’ to get the p_values.
See the example below:
#import lme4 package and lmerTest package
library(lmerTest)
# an object of class merModLmerTest
m <- lmer(Informed.liking ~ Gender+Information+Product +(1|Consumer), data=ham)
# gives summary of lmer object. The same as of class merMod but with
# additional p-values calculated based on Satterthwate's approximations
summary(m)
More about Package ‘lmerTest’, please see the link below:
http://cran.r-project.org/web/packages/lmerTest/lmerTest.pdf
You might look into the mixed function in the afex package found here. It uses Kenward-Rogers for df.

Resources