Pairwise comparison after repeated measures ANOVA interpretation - r
I ran a one way ANOVA with repeated measures:
aov <- aov(score ~ group*time + Error(subject/time), data=valueAest)
However, the result is hard to interpret as to whether the group difference comes from before treatment, after treatment, or both?
Error: subject
Df Sum Sq Mean Sq F value Pr(>F)
group 1 12.01 12.013 4.424 0.0421 *
Residuals 38 103.17 2.715
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Error: subject:time
Df Sum Sq Mean Sq F value Pr(>F)
time 1 0.012 0.0125 0.029 0.866
group:time 1 0.013 0.0125 0.029 0.866
Residuals 38 16.475 0.4336
so I decided to run some post-hoc analysis. Looking at some answers from other people, I used the multcomp library:
lme_score = lme(score ~ group, data=valueAest, random = ~1|subject)
anova(lme_score)
require(multcomp)
summary(glht(lme_score, linfct=mcp(group = "Tukey")), test = adjusted(type = "bonferroni"))
but this gave me error of ‘ncol(linfct)’ is not equal to ‘length(coef(model))’.
I also tried:
Mixed_Fitted_Interaction<-emmeans(aov, ~group|time)
Mixed_Fitted_Interaction
# pairwise comparison
pairs(Mixed_Fitted_Interaction)
This gave me the following result:
time = recreation1:
contrast estimate SE df t.ratio p.value
map - VR -0.75 0.397 49.8 -1.890 0.0646
time = recreation2:
contrast estimate SE df t.ratio p.value
map - VR -0.80 0.397 49.8 -2.016 0.0492
This looks promising in my naive mind, can I draw the conclusion that the significance actually doesn't come from treatment, but from the population difference?
Related
Type 3 Anova and lm summary outputs different results, continuous variables
After running a lm() model I've found my coefficients in the output and the Anova (type III) outputs are not the same. I think lm() is supposed to use type III sum of squares. Just hoping to figure out why I'm getting different results? The p-values are the same. I've tried using the (options(contrasts=c("contr.sum", "contr.poly"))) in a couple of different ways for the Anova function but it doesn't seem to change the output, and I'm pretty sure the two Anovas are doing the same thing. I don't think I actually need to use contrasts for my data because all of my variables are continuous but I was just trying everything.
#First version of type III Anova:
ImmMarmod.III.aov <- car::Anova(IMMMAR_model.logRTL, type = 3)
ImmMarmod.III.aov
Anova Table (Type III tests)
Response: LOG10RTL
Sum Sq Df F value Pr(>F)
(Intercept) 2.36246 1 57.1843 8.789e-09 ***
fat 0.00208 1 0.0504 0.8237
hap 0.03801 1 0.9200 0.3442
bka 0.08196 1 1.9839 0.1681
age 0.00760 1 0.1841 0.6706
Residuals 1.40465 34
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
#second version of type III Anova:
options(contrasts = c("contr.treatment", "contr.poly"))
Anova(IMMMAR_model.logRTL, type=3) #type III
Anova Table (Type III tests)
#Response: LOG10RTL
# Sum Sq Df F value Pr(>F)
#(Intercept) 2.36246 1 57.1843 8.789e-09 ***
#fat 0.00208 1 0.0504 0.8237
#hap 0.03801 1 0.9200 0.3442
#bka 0.08196 1 1.9839 0.1681
#age 0.00760 1 0.1841 0.6706
#Residuals 1.40465 34
summary(IMMMAR_model.logRTL)
#Coefficients:
# Estimate Std. Error t value Pr(>|t|)
#(Intercept) 0.247246 0.032696 7.562 8.79e-09 ***
#fat 0.007855 0.034987 0.225 0.824
#hap 0.029878 0.031149 0.959 0.344
#bka 0.042890 0.030450 1.409 0.168
#age 0.014579 0.033982 0.429 0.671
I was expecting the results to be the same between the summary(lm) and Anova(model,type=3)
Any input would be greatly welcomed!
How to run 2x2 mixed anova with unequal sample size in R?
I am trying to run 2x2 mixed ANOVA with unequal sample size using R. The data was collected from 30 individuals with two different conditions (i.e., 2 levels of within factor), and they were allocated by k-means clustering analysis (i.e., # of group 1 = 11, # of group 2 = 19). Here is a sample of my code and the output: summary(aov(JH ~ Box+Group+Box:Group+Error(P/Box), data = d3)) Error: P Df Sum Sq Mean Sq F value Pr(>F) Group 1 0.027 0.02715 1.56 0.22 Box:Group 1 0.001 0.00078 0.04 0.83 Residuals 27 0.470 0.01741 Error: P:Box Df Sum Sq Mean Sq F value Pr(>F) Box 1 0.000000 1.00e-07 0.00 0.97 Group 1 0.000022 2.17e-05 0.24 0.63 Box:Group 1 0.000032 3.24e-05 0.35 0.56 Residuals 27 0.002488 9.21e-05 Unlike the output when I ran 2x2 mixed ANOVA with equal sample sizes using another dataset (the output attached below), I had another interaction effect under Error: P. summary(aov(HipLoadingK ~ Box*Sex+Error(P/Box), data = TW)) Error: P Df Sum Sq Mean Sq F value Pr(>F) Sex 1 0.02578 0.025779 8.038 0.00841 ** Residuals 28 0.08980 0.003207 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Error: P:Box Df Sum Sq Mean Sq F value Pr(>F) Box 1 0.003454 0.003454 4.271 0.0481 * Box:Sex 1 0.000066 0.000066 0.082 0.7765 Residuals 28 0.022645 0.000809 Is this result fine as long as I interpret appropriate results for each between/within main effects and interaction effect? Or should I edit my code to get the correct outputs? If so, please let me know what should be added or edited in my code.
When I change the dataset from a short format to a long format, I accidentally put the different order of group name for each participant. For instance, P01 was group A in X condition, but his or her group was B in Y condition. After I fixed the code, it worked well.
r - emmeans pairwise analysis for multilevel repeated measures ANCOVA
I'm building a repeated measures ANCOVA using a multi-level framework through the AOV package. I have one continuous response variable, two factor predictors, and 3 continuous covariates. My script for the model is below: ModelDV <- aov(DV ~ IV1 + IV2 + IV1*IV2 + CV1 + CV2 + CV3 + Error(PartID/(IV1 + IV2 + IV1:IV2)), data) A snippet of my data set shows how it is formatted: PartID DV IV1 IV2 CV1 CV2 CV3 1 56 CondA1 CondB1 Contunous values 2 45 CondA2 CondB2 - 3 32 CondA3 CondB1 - 4 21 CondA4 CondB2 - 1 10 CondA1 CondB1 - 2 19 CondA2 CondB2 - 3 35 CondA3 CondB1 - 4 45 CondA4 CondB2 - My condiitons are embedded in the error term of the participant ID since this is a fully within repeated measures model. I am attempting to conduct a pairwise analysis on these values. My output provides omnibus F-tests: Error: PartID Df Sum Sq Mean Sq F value Pr(>F) CV1 1 348 348 0.442 0.5308 CV2 1 9 9 0.011 0.9193 CV3 1 3989 3989 5.063 0.0654 . Residuals 6 4727 788 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Error: PartID:IV1 Df Sum Sq Mean Sq F value Pr(>F) IV1 1 6222 6222 17.41 0.0024 ** Residuals 9 3217 357 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Error: PartID:IV2 Df Sum Sq Mean Sq F value Pr(>F) IV2 2 6215 3107.7 16.18 9.51e-05 *** Residuals 18 3457 192.1 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 Error: PartID:IV1:IV2 Df Sum Sq Mean Sq F value Pr(>F) IV1:IV2 2 575.2 287.6 1.764 0.2 Residuals 18 2934.4 163.0 When calculating emmeans via: emm<-emmeans(Model, ~ IV1) pairs(emm) I get a sensible output. However, when using this for the covariates: emm<-emmeans(Model, ~ CV1) pairs(emm) I get the following output: contrast estimate SE df z.ratio p.value (nothing) nonEst NA NA NA NA Results are averaged over the levels of: IV1, IV2 What am I doing wrong here that a pairwise comparison is not working for the covariate?
Short answer is because you have made them covariates to control for them and not to consider them as part of the explanation for your model. You of course could do pairwise comparisons for the covariates outside the model but not inside the model framework. Longer blogpost using these tools I wrote here...
Post hoc for binary GLMM (lme4) and plot
So I'm an R novice attempting a GLMM and post hoc analysis... help! I've collected binary data on 9 damselflys under 6 light levels, 1=response to movement of optomotor drum, 0=no response. My data was imported into R with the headings 'Animal_ID, light_intensity, response'. Animal ID (1-9) repeated for each light intensity (3.36-0.61) (see below)
Using the following code (lme4 package), I've performed a GLMM and found a light level to have a significant effect on response:
d = data.frame(id = data[,1], var = data$Light_Intensity, Response = data$Response)
model <- glmer(Response~var+(1|id),family="binomial",data=d)
summary(model)
Returns
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) [glmerMod]
Family: binomial ( logit )
Formula: Response ~ var + (1 | Animal_ID)
Data: d
AIC BIC logLik deviance df.resid
66 72 -30 60 51
Scaled residuals:
Min 1Q Median 3Q Max
-3.7704 -0.6050 0.3276 0.5195 1.2463
Random effects:
Groups Name Variance Std.Dev.
Animal_ID (Intercept) 1.645 1.283
Number of obs: 54, groups: Animal_ID, 9
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.7406 1.0507 -1.657 0.0976 .
var 1.1114 0.4339 2.561 0.0104 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr)
var -0.846
Then running:
m1 <- update(model, ~.-var)
anova(model, m1, test = 'Chisq')
Returns
Data: d
Models:
m1: Response ~ (1 | Animal_ID)
model: Response ~ var + (1 | Animal_ID)
Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
m1 2 72.555 76.533 -34.278 68.555
model 3 66.017 71.983 -30.008 60.017 8.5388 1 0.003477 **
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
I've installed the multcomp and lsmeans packages in an attempt at performing a Tukey post hoc to see where the difference is, but have run into difficulties with both.
Running:
summary(glht(m1,linfct=mcp("Animal_ID"="Tukey")))
Returns:
"Error in mcp2matrix(model, linfct = linfct) :
Variable(s) ‘Animal_ID’ have been specified in ‘linfct’ but cannot be found in ‘model’! "
Running:
lsmeans(model,pairwise~Animal_ID,adjust="tukey")
Returns:
"Error in lsmeans.character.ref.grid(object = new("ref.grid", model.info = list( :
No variable named Animal_ID in the reference grid"
I'm aware that I'm probably being very stupid here, but any help would be very much appreciated. My confusion is snowballing.
Also, does anyone have any suggestions as to how I might best visualize my results (and how to do this)?
Thank you very much in advance!
UPDATE:
New code-
Light <- c("3.36","3.36","3.36","3.36","3.36","3.36","3.36","3.36","3.36","2.98","2.98","2.98","2.98","2.98","2.98","2.98","2.98","2.98","2.73","2.73","2.73","2.73","2.73","2.73","2.73","2.73","2.73","2.15","2.15","2.15","2.15","2.15","2.15","2.15","2.15","2.15","1.72","1.72","1.72","1.72","1.72","1.72","1.72","1.72","1.72","0.61","0.61","0.61","0.61","0.61","0.61","0.61","0.61","0.61")
Subject <- c("1","2","3","4","5","6","7","8","9","1","2","3","4","5","6","7","8","9","1","2","3","4","5","6","7","8","9","1","2","3","4","5","6","7","8","9","1","2","3","4","5","6","7","8","9","1","2","3","4","5","6","7","8","9")
Value <- c("1","0","1","0","1","1","1","0","1","1","0","1","1","1","1","1","1","1","0","1","1","1","1","1","1","0","1","0","0","1","1","1","1","1","1","1","0","0","0","1","0","0","1","0","1","0","0","0","1","1","0","1","0","0")
data <- data.frame(Light, Subject, Value)
library(lme4)
model <- glmer(Value~Light+(1|Subject),family="binomial",data=data)
summary(model)
Returns:
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) [
glmerMod]
Family: binomial ( logit )
Formula: Value ~ Light + (1 | Subject)
Data: data
AIC BIC logLik deviance df.resid
67.5 81.4 -26.7 53.5 47
Scaled residuals:
Min 1Q Median 3Q Max
-2.6564 -0.4884 0.2193 0.3836 1.2418
Random effects:
Groups Name Variance Std.Dev.
Subject (Intercept) 2.687 1.639
Number of obs: 54, groups: Subject, 9
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -1.070e+00 1.053e+00 -1.016 0.3096
Light1.72 -7.934e-06 1.227e+00 0.000 1.0000
Light2.15 2.931e+00 1.438e+00 2.038 0.0416 *
Light2.73 2.931e+00 1.438e+00 2.038 0.0416 *
Light2.98 4.049e+00 1.699e+00 2.383 0.0172 *
Light3.36 2.111e+00 1.308e+00 1.613 0.1067
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) Lg1.72 Lg2.15 Lg2.73 Lg2.98
Light1.72 -0.582
Light2.15 -0.595 0.426
Light2.73 -0.595 0.426 0.555
Light2.98 -0.534 0.361 0.523 0.523
Light3.36 -0.623 0.469 0.553 0.553 0.508
Then running:
m1 <- update(model, ~.-Light)
anova(model, m1, test= 'Chisq')
Returns:
Data: data
Models:
m1: Value ~ (1 | Subject)
model: Value ~ Light + (1 | Subject)
Df AIC BIC logLik deviance Chisq Chi Df Pr(>Chisq)
m1 2 72.555 76.533 -34.278 68.555
model 7 67.470 81.393 -26.735 53.470 15.086 5 0.01 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Finally, running:
library(lsmeans)
lsmeans(model,list(pairwise~Light),adjust="tukey")
Returns (it actually works now!):
$`lsmeans of Light`
Light lsmean SE df asymp.LCL asymp.UCL
0.61 -1.070208 1.053277 NA -3.1345922 0.9941771
1.72 -1.070216 1.053277 NA -3.1345997 0.9941687
2.15 1.860339 1.172361 NA -0.4374459 4.1581244
2.73 1.860332 1.172360 NA -0.4374511 4.1581149
2.98 2.978658 1.443987 NA 0.1484964 5.8088196
3.36 1.040537 1.050317 NA -1.0180467 3.0991215
Results are given on the logit (not the response) scale.
Confidence level used: 0.95
$`pairwise differences of contrast`
contrast estimate SE df z.ratio p.value
0.61 - 1.72 7.933829e-06 1.226607 NA 0.000 1.0000
0.61 - 2.15 -2.930547e+00 1.438239 NA -2.038 0.3209
0.61 - 2.73 -2.930539e+00 1.438237 NA -2.038 0.3209
0.61 - 2.98 -4.048866e+00 1.699175 NA -2.383 0.1622
0.61 - 3.36 -2.110745e+00 1.308395 NA -1.613 0.5897
1.72 - 2.15 -2.930555e+00 1.438239 NA -2.038 0.3209
1.72 - 2.73 -2.930547e+00 1.438238 NA -2.038 0.3209
1.72 - 2.98 -4.048874e+00 1.699175 NA -2.383 0.1622
1.72 - 3.36 -2.110753e+00 1.308395 NA -1.613 0.5897
2.15 - 2.73 7.347728e-06 1.357365 NA 0.000 1.0000
2.15 - 2.98 -1.118319e+00 1.548539 NA -0.722 0.9793
2.15 - 3.36 8.198019e-01 1.302947 NA 0.629 0.9889
2.73 - 2.98 -1.118326e+00 1.548538 NA -0.722 0.9793
2.73 - 3.36 8.197945e-01 1.302947 NA 0.629 0.9889
2.98 - 3.36 1.938121e+00 1.529202 NA 1.267 0.8029
Results are given on the log odds ratio (not the response) scale.
P value adjustment: tukey method for comparing a family of 6 estimates
Your model specifies Animal_ID as a random effect. The glht and lsmeans functions work only for fixed-effect comparisons.
r repeated measures output error (main effects shown twice, between subject shown as within)
I am having trouble discerning the error strata output from a repeated measures anova in R, as they appear to be doing something funky. I have a repeated measures anova where each participant gets a score for two different roles and a score for two different valences (Role and Valence are dichotomous categorical within subjects factors), and I am including them in a model that has Gender as a between subjects factor (also dichotomous categorical). My model is as follows: summary(aov(data = data, score ~ Role * Valence * Gender + Error(Subject_ID / (Role*Valence))) The output looks unusual: summary(aov(data = data, + score ~ Role * Valence * Gender + Error(Subject_ID / (Role*Valence)))) Error: Subject_ID Df Sum Sq Mean Sq Gender 1 0.06647 0.06647 Error: Valence Df Sum Sq Mean Sq Valence 1 6.774 6.774 Error: Subject_ID:Role Df Sum Sq Mean Sq Role 1 0.04595 0.04595 Error: Subject_ID:Valence Df Sum Sq Mean Sq Valence:Gender 1 0.06981 0.06981 Error: Subject_ID:Role:Valence Df Sum Sq Mean Sq Role:Valence 1 1.329 1.329 Error: Within Df Sum Sq Mean Sq F value Pr(>F) Role 1 0.00 0.0000 0.000 0.9986 Gender 1 0.08 0.0781 0.382 0.5371 Role:Valence 1 0.65 0.6457 3.159 0.0767 Role:Gender 1 0.04 0.0354 0.173 0.6777 Valence:Gender 1 0.04 0.0443 0.217 0.6420 Role:Valence:Gender 1 0.24 0.2447 1.197 0.2749 Residuals 252 51.50 0.2044 Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1 I don't understand why the F and P values only displayed for the within error strata, why my main effects are appearing twice (e.g., Role under within and Subject_ID:Role), or why my between subjects variable is showing up in the Within error strata. I'm not sure how to begin troubleshooting this, so any insights would be greatly appreciated.