R: post-hoc comparisons using estimated marginal means but not assuming equal variances - r

I am trying to use R to run post-hoc comparisons following a significant interaction for a mixed-method Anova. I would like to do the post-hoc similar to SPSS [EMMEANS=TABLES(Group*time) COMPARE(Group) ADJ(BONFERRONI)], using estimated marginal means but not assuming equality of variance.
Dependent variable = 'depvar'. I have 3 groups ('group') and 3 time points ('timept'), which are repeated measures over subjects ('id');
aov_car(depvar ~ group*timept + Error(id/(timept)), data=myData)
If I use pairwise.t.test I can compare groups separately for each time point, but the R uses observed means and I do not know how to force using the estimated marginal means of my model:
for (itimept in unique(myData$timept)){
idx=myData$timept==itimept
pairwise.t.test(myData$depvar[idx],myData$group[idx],p.adj="bonferroni")
}
If I use emmeans or lsmeans then R uses estimated marginal means, but assumes variances are the same (the SE in the results are all the same).
myfit=lme(depvar ~ group*timept, random = ~1|id/timept, data=myData)
emmeans(myfit, pairwise~group|timept, adjust="bonferroni")
How do I run post-hoc comparisons between groups for each time point, using estimated marginal means but not assuming equal variances, similar to SPSS?
Thanks!
Cristina

It isn’t emmeans that assumes equal variances. It is the model that you fitted and subsequently handed to emmeans for further analysis. Fit a different model using, I think, the weights argument, that specifies unequal variances.
I believe that this model will do the trick:
myfit = lme(depvar ~ group*timept,
random = ~1|id/timept,
weights = varFunc(~ group*timept),
data = myData)

Related

Longitudinal analysis using sampling weigths in R

I have longitudinal data from two surveys and I want to do a pre-post analysis. Normally, I would use survey::svyglm() or svyVGAM::svy_vglm (for multinomial family) to include sampling weights, but these functions don't account for the random effects. On the other hand, lme4::lmer accounts for the repeated measures, but not the sampling weights.
For continuous outcomes, I understand that I can do
w_data_wide <- svydesign(ids = ~1, data = data_wide, weights = data_wide$weight)
svyglm((post-pre) ~ group, w_data_wide)
and get the same estimates that I would get if I could use lmer(outcome ~ group*time + (1|id), data_long) with weights [please correct me if I'm wrong].
However, for categorical variables, I don't know how to do the analyses. WeMix::mix() has a parameter weights, but I'm not sure if it treats them as sampling weights. Still, this function can't support multinomial family.
So, to resume: can you enlighten me on how to do a pre-post test analysis of categorical outcomes with 2 or more levels? Any tips about packages/functions in R and how to use/write them would be appreciated.
I give below some data sets with binomial and multinomial outcomes:
library(data.table)
set.seed(1)
data_long <- data.table(
id=rep(1:5,2),
time=c(rep("Pre",5),rep("Post",5)),
outcome1=sample(c("Yes","No"),10,replace=T),
outcome2=sample(c("Low","Medium","High"),10,replace=T),
outcome3=rnorm(10),
group=rep(sample(c("Man","Woman"),5,replace=T),2),
weight=rep(c(1,0.5,1.5,0.75,1.25),2)
)
data_wide <- dcast(data_long, id~time, value.var = c('outcome1','outcome2','outcome3','group','weight'))[, `:=` (weight_Post = NULL, group_Post = NULL)]
EDIT
As I said below in the comments, I've been using lmer and glmer with variables used to calculate the weights as predictors. It happens that glmer returns a lot of problems (convergence, high eigenvalues...), so I give another look at #ThomasLumley answer in this post and others (https://stat.ethz.ch/pipermail/r-help/2012-June/315529.html | https://stats.stackexchange.com/questions/89204/fitting-multilevel-models-to-complex-survey-data-in-r).
So, my question is now if a can use participants id as clusters in svydesign
library(survey)
w_data_long_cluster <- svydesign(ids = ~id, data = data_long, weights = data_long$weight)
summary(svyglm(factor(outcome1) ~ group*time, w_data_long_cluster, family="quasibinomial"))
Estimate Std. Error t value Pr(>|t|)
(Intercept) 1.875e+01 1.000e+00 18.746 0.0339 *
groupWoman -1.903e+01 1.536e+00 -12.394 0.0513 .
timePre 5.443e-09 5.443e-09 1.000 0.5000
groupWoman:timePre 2.877e-01 1.143e+00 0.252 0.8431
and still interpret groupWoman:timePre as differences in the average rate of change/improvement in the outcome over time between sex groups, as if I was using mixed models with participants as random effects.
Thank you once again!
A linear model with svyglm does not give the same parameter estimates as lme4::lmer. It does estimate the same parameters as lme4::lmer if the model is correctly specified, though.
Generalised linear models with svyglm or svy_vglm don't estimate the same parameters as lme4::glmer, as you note. However, they do estimate perfectly good regression parameters and if you aren't specifically interested in the variance components or in estimating the realised random effects (BLUPs) I would recommend just using svy_glm.
Another option if you have non-survey software for random effects versions of the models is to use that. If you scale the weights to sum to the sample size and if all the clustering in the design is modelled by random effects in the model, you will get at least a reasonable approximation to valid inference. That's what I've seen recommended for Bayesian survey modelling, for example.

What post-hoc test should be used for a glmer model with a continious and a categorical predictor variable?

I'm a bit of a newbie with stats and R, so need a bit of direction to find a suitable post-hoc test for my glmer model.
The model has a binary dependent variable (absent/present) and the predictor variables are interactive terms between a continuous variable(eg temp) and a categorical variable (species, n=3). Only interactive terms, rather than the continuous factor in isolation, produce significant results when an anova is run on the model. Species by itself has a large effect because one species is much rarer than the others. I'm trying to tease apart how the presence of these species varies across pH and between species.
I've tried lsmeans test with Tukey, and Firth's Bias-Reduced Logistic Regression, emmeans. I ran the effects function on the interactive terms, so had a rough expectation of what a post hoc could show, but the results logistf (firth's) have produced I was not expecting. Emmeans and tukey both gave the same results and ignored the continuous variable I assume because it's not a factor.
When I run firth's regression it produces chi-squared and p values that are either infinity for chi values or the p values astronomically small, even though what I saw through effects suggested no significant difference. I can't tell with the interactive term if there truly is an effect of the environmental variable or if the significant effect is because of the difference in species. Based on what I have seen of the logistf function, I didn't think it would produce a chi-square score. Is this an issue in coding or is it because of my data?
If I wasn't clear enough about something please let me know and if anyone has any suggestions or advice, they would be massively appreciated. Thanks!
The model and test code I used are below:
###glmer model
Large<-glmer(Abs.Pres~ Species:Q.Depth+Species:Conductivity+Species:Temp+Species:pH+Species:DO.P+(1|QID),
nAGQ=0,
family=binomial,
data=Stacked_Pref)
anova(Large)
Output:Analysis of Variance Table
npar Sum Sq Mean Sq F value
Species:Q.Depth 3 234.904 78.301 78.3014
Species:Conductivity 3 32.991 10.997 10.9970
Species:Temp 3 39.001 13.000 13.0004
Species:pH 3 25.369 8.456 8.4562
Species:DO.P 3 34.930 11.643 11.6434
###Firths
Lp<-logistf(Abs.Pres~Species:pH, data=Stacked_Pref, contrasts.arg=list(pH="contr.treatment", Species="contr.sum"))
> Lp
logistf(formula = Abs.Pres ~ Species:pH, data = Stacked_Pref,
contrasts.arg = list(pH = "contr.treatment", Species = "contr.sum"))
Model fitted by Penalized ML
Confidence intervals and p-values by Profile Likelihood
coef se(coef) lower 0.95 upper 0.95 Chisq p
(Intercept) 1.9711411 0.57309880 0.8552342 3.1015114 12.09107 5.066380e-04
SpeciesGoby:pH -0.3393185 0.07146049 -0.4804047 -0.2003108 23.31954 1.371993e-06
SpeciesMosquito:pH -0.3001385 0.07127771 -0.4408186 -0.1614419 18.24981 1.937453e-05
SpeciesRFBE:pH -0.4771393 0.07232469 -0.6200179 -0.3365343 45.73750 1.352096e-11
Likelihood ratio test=267.0212 on 3 df, p=0, n=3945

Test of second differences for average marginal effects in logistic regression

I have a question similar to the one here: Testing the difference between marginal effects calculated across factors. I used the same code to generate average marginal effects for two groups. The difference is that I am running a logistic rather than linear regression model. My average marginal effects are on the probability scale, so emmeans will not provide the correct contrast. Does anyone have any suggestions for how to test whether there is a significant difference in the average marginal effects between group 1 and group 2?
Thank you so much,
Ilana
It is a bit unclear what the issue really is, but I'll try. I'm supposing your logistic regression model was fitted using, say, glm:
mod <- glm(cbind(heads, tails) ~ treat, data = mydata, family = binomial())
If you then do
emm <- emmeans(mod, "treat")
emm ### marginal means
pairs(emm) ### differences
Your results will be presented on the logit scale.
If you want them on the probability scale, you can do
summary(emm, type = "response")
summary(pairs(emm), type = "response")
However, the latter will back-transform the differences of logits, thereby producing odds ratios.
If you actually want differences of probabilities rather than ratios of odds, use regrid(), which will construct a new grid of values after back-transforming (and hence it will forget the log transformation):
pairs(regrid(emm))
It seems possible that two or more factors are present and you want contrasts of contrasts on the probability scale. In that case, extend this idea by calling regrid() on the table of EMMs to put everything on the probability scale, then follow the analogous procedure used in the linked article.

How to test significance of polynomial (linear) trends among groups with unequal variances?

I am testing for a linear trend among several groups, however, my data has violated the assumption of equal variance among groups (tested by Levene's homogeneity of variance).
In SPSS, along with the significance of linear trend assuming equal variance, there is automatic output for significance where equal variances are not assumed. What 'test' or 'adjustment' is being done? Can I do this in R, and how?
Image of SPSS output: (https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcSZRs3EM3wJz5raHhav-LLBTmTyfLJO0z4xHDEzI-3uI15BoBQ5)
I'm struggling to find what exactly SPSS is doing, but it could be some sort of welch correction?
# TEST homogeneity of variance
leveneTest(ICECAP_A ~ SFMental_f, data = SCI)
p < 0.001 so we reject null of homogeneity of variance.
# Use built-in contr.poly() function: Tell R to get a polynomial contrast matrix for 5 levels/groups
contrasts(SCI$SFMental_f) <- contr.poly(n=5)
# call an ANOVA
anova.SFMental <- aov(ICECAP_A ~ SFMental_f, data = SCI)
# print output, show linear trend result
summary.aov(anova.SFMental, split=list (SFMental_f=list ("Linear"=1)))
Now I have the significance for linear trend. How do I get the significance if we do NOT assume equal variances?
It seems that SPSS does a correction using the [Welch-Satterthwaite Equation]1. Thanks to Andy Field for the tip. But there is no direct R alternative, so I constructed the contrasts in the usual way and ran a robust model with lmRob() instead.

regTermTest in R survey package does not appear to produce correct p-values

I am trying to use regTermTest to test for estimated with the survey package in R. This is a simulation study. I simulated 1,000 stratified random samples from a finite population. The correct model is given by y ~ x + Stratum, for some covariate x. The intercept varies by stratum. I fit y ~ x * Stratum using svyglm() and then ran regTermTest(myOutput, ~x:Stratum), to test for the presence of an interaction effect. I also the usual ANOVA using a linear model. The p-values from regTermTest ought to follow a uniform distribution, with 5% of them having values below 5%. Same for the ANOVA. The p-values for the regular linear model worked out, but regTermTest produced p-values that were way too small. The median p-value was 0.02, when it should have been 0.5.
Does anyone know what's going wrong here? And should I have posted this question to Cross-Validated?

Resources