I am trying to use the rma.uni function from the metafor package to estimate the impact of fishing gears on my abundance data. Following the method published in Sciberas et al. 2018 (DOI: 10.1111/faf.12283), I think I used correctly the function, however, I am not sure how to interpret the output.
In the function, c is the log response ratio and var_cis the associated variance. log2(t+1) represent times in days.
In my data, gear is a factor with three levels: CD, QSD and KSD.
As I am not familiar with models in general and especially this type of model, I read online documentation including this :
https://faculty.nps.edu/sebuttre/home/R/contrasts.html
Thus, I understood that only two levels from my factor gear need to be display in the output.
Below is the output I have when I run the rma.uni function. My questions are:
if gearCD is considered as a 'reference' in the model then it would mean that the effect of gearKSD is 0.14 more positive (I don't know how to word it) than gearCD and that on the opposite, gearQSD is 0.12 times more damaging ?
How should I interpret the fact that the pvalues for gearKSD and gearQSD are not significant ? Does it mean that their intercept is not significantly different from the one of gearCD ? If so, is the intercept of gearCD the same thing than intercpt?
Do you know how I could obtain one intercept value for each level of my factor gear ? I am aiming at distinguishing the intial impact of these three gears so it would be of interest to have one interpect per gear.
Similarly, if I had interaction terms with log2(t+1) (for example gearKSD:log2(t+1)) the interpreation would be silimar to how we interpret intercept ?
I am sorry I know these are a lot of questions.. Also, isn't it weird to have a R² of 100% and all other values at 0 (tau, tau² and I²)?
Thank you all very much for your help !
rma.uni(c,var_c,mods=~gear+log2(t+1),data=data_AB,method="REML")
Mixed-Effects Model (k = 15; tau^2 estimator: REML)
tau^2 (estimated amount of residual heterogeneity): 0 (SE = 0.0097)
tau (square root of estimated tau^2 value): 0
I^2 (residual heterogeneity / unaccounted variability): 0.00%
H^2 (unaccounted variability / sampling variability): 1.00
R^2 (amount of heterogeneity accounted for): 100.00%
Test for Residual Heterogeneity:
QE(df = 11) = 7.5027, p-val = 0.7570
Test of Moderators (coefficients 2:4):
QM(df = 3) = 31.7446, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
intrcpt -1.1145 0.1407 -7.9200 <.0001 -1.3904 -0.8387 ***
gearKSD 0.1488 0.1025 1.4517 0.1466 -0.0521 0.3497
gearQSD -0.1274 0.0916 -1.3919 0.1640 -0.3069 0.0520
log2(t + 1) 0.1007 0.0195 5.1626 <.0001 0.0625 0.1389 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Related
I have carried out a binomial GLMM to determine how latitude and native status (native/non-native) of a set of plant species affects herbivory damage. I am now trying to determine the statistical power of my model when I change the effect sizes. My model looks like this:
latglmm <- glmer(cbind(chewing,total.cells-chewing) ~ scale(latitude) * native.status + scale(sample.day.of.year) + (1|genus) + (1|species) + (1|catalogue.number), family=binomial, data=mna)
where cbind(chewing,total.cells-chewing) gives me a proportion (of leaves with herbivory damage), native.status is either "native" or "non-native" and catalogue.number acts as an observation-level random effect to deal with overdispersion. There are 10 genus, each with at least 1 native and 1 non-native species to make 26 species in total. The model summary is:
Generalized linear mixed model fit by maximum likelihood (Laplace Approximation) ['glmerMod']
Family: binomial ( logit )
Formula: cbind(chewing, total.cells - chewing) ~ scale(latitude) * native.status +
scale(sample.day.of.year) + (1 | genus) + (1 | species) + (1 | catalogue.number)
Data: mna
AIC BIC logLik deviance df.resid
3986.7 4023.3 -1985.4 3970.7 706
Scaled residuals:
Min 1Q Median 3Q Max
-1.3240 -0.4511 -0.0250 0.1992 1.0765
Random effects:
Groups Name Variance Std.Dev.
catalogue.number (Intercept) 1.26417 1.1244
species (Intercept) 0.08207 0.2865
genus.ID (Intercept) 0.33431 0.5782
Number of obs: 714, groups: catalogue.number, 713; species, 26; genus.ID, 10
Fixed effects:
Estimate Std. Error z value Pr(>|z|)
(Intercept) -2.61310 0.20849 -12.534 < 2e-16 ***
scale(latitude) -0.17283 0.06370 -2.713 0.00666 **
native.statusnon-native 0.11434 0.15554 0.735 0.46226
scale(sample.day.of.year) 0.28521 0.05224 5.460 4.77e-08 ***
scale(latitude):native.statusnon-native -0.02986 0.09916 -0.301 0.76327
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Correlation of Fixed Effects:
(Intr) scallt ntv.s- scaldy
scalelat 0.012
ntv.sttsnn- -0.304 -0.014
scaledoy 0.018 -0.085 -0.027
scllt:ntv.- -0.011 -0.634 0.006 -0.035
I should add that the actual model I have been using is a glmmTMB model as lme4 still had some overdispersion even with the observation-level random effect, but this is not compatible with simr so I am using lme4 (the results are very similar for both). I want to see what happens to the model power when I increase or decrease the effect sizes of latitude and native status but when I run fixef(latglmm1)["scale(latitude)"]<--1 and fixef(latglmm1)["native.statusnon-native"]<--1 and try this:
powerSim(latglmm, fcompare(~ scale(latitude) + native.status))
I get the following output:
Power for model comparison, (95% confidence interval):====================================================================|
100.0% (69.15, 100.0)
Test: Likelihood ratio
Comparison to ~scale(latitude) + native.status + [re]
Based on 10 simulations, (0 warnings, 0 errors)
alpha = 0.05, nrow = 1428
Time elapsed: 0 h 1 m 5 s
The output is the same (100% power) no matter what I change fixef() to. Based on other similar questions online I have ensured that my data has no NA values and according to my powerSim there are no warnings or errors to address. I am completely lost as to why this isn't working so any help would be greatly appreciated!
Alternatively, if anyone has any recommendations for other methods to carry out similar analysis I would love to hear them. What I really want is to get a p-value for each effect size I input but statistical power would be very valuable too.
Thank you!
I have been using the metafor package for some meta-analyses and would like to adjust for a single continuous covariate (mean age) using meta-regression. However, I require some clarification regarding the outputs and what they mean. Below I have shared the output for the base case analysis as well as the meta-regression (same studies in both, with the only difference being the addition of covariates for the meta-regression).
Base case output
Random-Effects Model (k = 36; tau^2 estimator: DL)
logLik deviance AIC BIC AICc
-18.8613 60.5927 41.7226 44.8896 42.0862
tau^2 (estimated amount of total heterogeneity): 0.0633 (SE = 0.0327)
tau (square root of estimated tau^2 value): 0.2515
I^2 (total heterogeneity / total variability): 51.46%
H^2 (total variability / sampling variability): 2.06
Test for Heterogeneity:
Q(df = 35) = 72.1031, p-val = 0.0002
Model Results:
estimate se zval pval ci.lb ci.ub
0.1266 0.0633 2.0014 0.0453 0.0026 0.2506 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Meta-regression (output)
Mixed-Effects Model (k = 36; tau^2 estimator: DL)
logLik deviance AIC BIC AICc
-18.7696 60.4092 43.5391 48.2897 44.2891
tau^2 (estimated amount of residual heterogeneity): 0.0677 (SE = 0.0346)
tau (square root of estimated tau^2 value): 0.2601
I^2 (residual heterogeneity / unaccounted variability): 52.84%
H^2 (unaccounted variability / sampling variability): 2.12
R^2 (amount of heterogeneity accounted for): 0.00%
Test for Residual Heterogeneity:
QE(df = 34) = 72.1024, p-val = 0.0001
Test of Moderators (coefficient(s) 2):
QM(df = 1) = 0.2456, p-val = 0.6202
Model Results:
estimate se zval pval ci.lb ci.ub
intrcpt -0.3741 1.0140 -0.3690 0.7122 -2.3616 1.6133
mods 0.0085 0.0172 0.4955 0.6202 -0.0252 0.0423
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
My questions are:
Why are we observing an R-squared of 0% in the meta-regression (is it simply because the covariate is not significant or do you suspect something is not correct)?
How can we interpret the outputs of the meta-regression? With back-transformation of the logHRs we suspect something like below, but would like to make sure that I am interpreting the ‘intrcpt’ and ‘mods’ values correctly.
I have assumed mods represents the pooled HR taking into account the adjustment for age.
I have assumed intrcpt represents the covariate effect (beta) – i.e. the amount that the logHR changes for a one unit increase in age. Also, I have back-transformed this output, which I am not sure is appropriate, or if I should present as is.
Let me first note that I haven't been able to reproduce this error on anything outside of my data set. However, here is the general idea. I have a data frame and I'm trying to build a simple logistic regression to understand the marginal effect of Amount on IsWon. Both models perform poorly, it's one predictor after all, but they produce two different coefficients
First is the glm output:
> summary(mod4)
Call:
glm(formula = as.factor(IsWon) ~ Amount, family = "binomial",
data = final_data_obj_samp)
Deviance Residuals:
Min 1Q Median 3Q Max
-1.2578 -1.2361 1.0993 1.1066 3.7307
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 0.18708622416 0.03142171761 5.9540 0.000000002616 ***
Amount -0.00000315465 0.00000035466 -8.8947 < 0.00000000000000022 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
(Dispersion parameter for binomial family taken to be 1)
Null deviance: 6928.69 on 4999 degrees of freedom
Residual deviance: 6790.87 on 4998 degrees of freedom
AIC: 6794.87
Number of Fisher Scoring iterations: 6
Notice that negative coefficient for Amount.
And now the lrm function from rms
Logistic Regression Model
lrm(formula = as.factor(IsWon) ~ Amount, data = final_data_obj_samp,
x = TRUE, y = TRUE)
Model Likelihood Discrimination Rank Discrim.
Ratio Test Indexes Indexes
Obs 5000 LR chi2 137.82 R2 0.036 C 0.633
0 2441 d.f. 1 g 0.300 Dxy 0.266
1 2559 Pr(> chi2) <0.0001 gr 1.350 gamma 0.288
max |deriv| 0.0007 gp 0.054 tau-a 0.133
Brier 0.242
Coef S.E. Wald Z Pr(>|Z|)
Intercept 0.1871 0.0314 5.95 <0.0001
Amount 0.0000 0.0000 -8.89 <0.0001
Both models do a poor job, but one estimates a positive coefficient and the other a negative coefficient. Sure, the values are negligible, but can someone help me understand this.
For what it's worth, here's what the plot of the lrm object looks like.
> plot(Predict(mod2, fun=plogis))
The plot shows the predicted probabilities of winning have a very negative relationship with Amount.
It seems like lrm is estimating the coefficient to the nearest ±0.0000 value. Since the coefficient value is well below that, it is simply rounding it to 0.0000. Hence it seems positive but may in fact not be.
You should not rely on the printed result from summary to check for coefficients. The summary table is controlled by print, hence will always subject to rounding problem. Have you tried mod4$coef (get coefficients of glm model mod4) and mod2$coef (get coefficients of lrm model mod2)? It is good idea to read the "values" section of ?glm and ?lrm.
I am very new to R and I want to know if random effects in the different areas where my research took place could be biasing my results, and providing a false positive effect on my conditions. My data is based on natural observations taking place in four different conditions, over 9 areas. It is a between subjects design- each row is an observation of a different subject.
Condition =factor (4 levels)
Area = random effect (9 levels)
GeneralDisplayingBehaviours = DV
ObsTime = How many minuets the observations took place for
This is what my model looks like
data2$Condition<-as.factor(data2$Condition)
data2$Area<-as.factor(data2$Area)
data2$Condition<-relevel(data2$Condition,ref="22")
data2$Area<-relevel(data2$Area,ref="1")
mod<-glmmadmb(GeneralDisplayingBehaviours~Condition+ObsTime+(1|Area), data=data2, family="nbinom", zeroInflation=TRUE)
This is the out put:
Call:
glmmadmb(formula = GeneralDisplayingBehaviours ~ Condition +
ObsTime + (1 | Area), data = data2, family = "nbinom", zeroInflation = TRUE)
AIC: 2990.1
Coefficients:
Estimate Std. Error z value Pr(>|z|)
(Intercept) 1.6233 0.5019 3.23 0.0012 **
Condition12 1.3291 0.1330 9.99 <2e-16 ***
Condition11 1.2965 0.1294 10.02 <2e-16 ***
Condition21 0.0715 0.1351 0.53 0.5966
ObsTime 0.0829 0.0341 2.43 0.0151 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Number of observations: total=360, Area=9
Random effect variance(s):
Group=Area
Variance StdDev
(Intercept) 8.901e-09 9.434e-05
Negative binomial dispersion parameter: 1.7376 (std. err.: 0.16112)
Zero-inflation: 0.16855 (std. err.: 0.02051 )
Log-likelihood: -1487.06
I would then go on to change the condition ref for each level, and I think I would have to do the same with area.
How would I interpret these results and report them?
What does this tell me about my random effect of area? Does it impact the DV?
Are the conditions significantly different in terms of the DV, and which conditions are significantly different for which?
For the last question I have a feeling I would need to do multiple comparisons so how would I do this in GLMMABMD?
Thank you
I run two meta-analysis and want to proof that the caluclated mean effect size (in fisher z) differs between both meta-analysis.
As i am quit new in R and not such a pro in statistics, could you provide the appropriate test and how to conduct it in R?
Here are my current results of the two meta analysis:
> results1GN
Random-Effects Model (k = 4; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.0921 (SE = 0.0752)
tau (square root of estimated tau^2 value): 0.3034
I^2 (total heterogeneity / total variability): 99.98%
H^2 (total variability / sampling variability): 5569.05
Test for Heterogeneity:
Q(df = 3) = 22183.0526, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
0.3663 0.1517 2.4139 0.0158 0.0689 0.6637 *
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> results1NN
Random-Effects Model (k = 72; tau^2 estimator: REML)
tau^2 (estimated amount of total heterogeneity): 0.0521 (SE = 0.0096)
tau (square root of estimated tau^2 value): 0.2282
I^2 (total heterogeneity / total variability): 95.98%
H^2 (total variability / sampling variability): 24.85
Test for Heterogeneity:
Q(df = 71) = 1418.1237, p-val < .0001
Model Results:
estimate se zval pval ci.lb ci.ub
0.2594 0.0282 9.2016 <.0001 0.2042 0.3147 ***
I elaborate on the previous response, which was a bit rude.
On first step, you may consider the confidence interval of you result.
You may consult the Wikipedia page on the topic for a quick summary:
http://en.wikipedia.org/wiki/Confidence_interval
Anyway, in your example you have two effect sizes with overlapping confidence interval:
results1GN = 0.36 (95%CI: 0.069 to 0.66)
results1NN = 0.26 (95%CI: 0.20 to 0.31).
So, the two results are not statistically different.
The package 'metafor' also includes a function (anova) to compare "nested models", and I quote:
"For two (nested) models of class "rma.uni", the function provides a full versus reduced model comparison in terms of model fit statistics and a likelihood ratio test".
Note that "When specifying two models for comparison, the function provides a likelihood ratio test comparing the two models. The models must be based on the same set of data and should be nested for the likelihood ratio test to make sense".
You may found more details and examples on the use of the function here:
http://www.inside-r.org/packages/cran/metafor/docs/anova.rma.uni
Also, consult the manual of the package and the the site of the package maintainer.
Hope this may be helpful.