Interpreting Interaction Coefficients within Multiple Linear Regression Model - r

I am struggling with the interpretation of the coefficients within interaction models.
I am looking at the outcome of an interaction model of 2 binary (dummy variables). I was just wondering how I interpret the:
- Intercept (is everything at 0)?
- The slope coefficients?
- The interaction coefficients?
In standard multiple linear regression, we talk about the change in y when we have a 1-unit change in x, holding everything else constnat. How do we interpret this in interactions? Especially since both my variables are dummy?
Hope this makes sense and thanks very much in advance.

How do we interpret this in interactions?
The meaning of the regression coefficients in models having interaction do not remain the same as in the case of simple linear regression without interaction simply because of the added interaction term/terms.
The regression coefficients no longer indicate the change in the mean response with a unit increase of the predictor variable, with the other predictor variable held constant at any given level. This interpretation is only valid after accounting for the dependence on the level of the other predictor variable.
Ex:
A SLRM with interaction terms:
E(Y) = B0 + B1X1 + B2X2 + B3X1X2
Interpretation:
It can be shown that the change in the mean response with a unit increase in X1 when X2 is held constant is:
B1 + B3X2
And, the change in the mean response with a unit increase in X2 when X1 is held
constant is:
B2 + B3X1
I was just wondering how I interpret the: - Intercept (is everything at 0)?
The intercept is the prediction from the regression model when all the predictors are at level zero.
The slope coefficients?
In case of no interaction coefficients.
E(Y) = B0 + B1X1 + B2X2
The coefficients B1, B2 indicate, respectively, how much higher (lower) the response functions for dummies X1, X2 are than the one for, both dummies zero.
Thus, B1 and B2 measure the differential effects of the dummy variables on the height of the response function i.e. E(Y).
You can inspect that only the slope changes:
When X1 = 1 and X2 = 0.
E(Y) = B0 + B1
and, when X1 = 0 and X2 + 1.
E(Y) = B0 + B2
The interaction coefficients?
By interaction coefficients, I understand the regression coefficients for model with interaction.
The model:
E(Y) = B0 + B1X1 + B2X2 + B3X1X2
When both X1 and X2 are 1, then the model becomes:
E(Y) = B0 + B1 + B2 + B3.
Which translates to an increase or decrease in the height of the response function.
You can create a more interesting example with a third continuous predictor and explore the interaction relationship of the continuous variable with the dummies, in which case the slope of the regression would also change instead of only the intercept. And, hence the interpretation that how much higher (lower) one response function is than the other for any given level of X1 and X2 would not be valid as the slope also would have changed and thus the effect of the dummy predictor also would have been more evident.
When interaction effects are present, the effect of the qualitative predictor (dummy variable) can be studied by comparing the regression functions within the scope of the model for the different classes of the dummy variable.
Reference: Kutner et. al. Applied Linear Statistical Models

Related

Interpretation of main effects when interaction is present in gam

Consider a GAM model with the following structure:
y~gam(s(x1, by=x2) + x2 + s(x3)) where x1 and x2 are continuous variables and x2 is categorical. If I want to know the effect of x1 (in terms of deviance explained), I remove x1 from the model and compare the deviance explained (following this thread), like this:
model1 <- y~gam(s(x1, by=x2) + x2 + s(x3))
model2 <- y~gam(x2 + s(x3))
## deviance explained by x1:
summary(model1)$dev.expl-summary(model2)$dev.expl
But what if I want to know the effect of x2? I am not interested in the effect of x2 on x1; I just want to know the effect of x2 by itself. Could I do this:
model3 <- y~gam(s(x1, by=x2) + s(x3))
## deviance explained by x2:
summary(model1)$dev.expl-summary(model3)$dev.expl
I know that for linear models, if a significant interaction is present, one cannot remove the main effects of the variables in that interaction, even if they are not significant. Does the same apply here, in that I cannot know the effect of x2 on y independently of its effect on x1?
Yes, the same apply here. Whenever there are any interactions involving a variable, you cannot make any affirmation over the effects of this variable.
However, notice that this type of effect you are retrieving from explained deviance, doesn't have the same interpretability as the usual in linear models, where you affirm that a modification of a single unit in x2 represents an increase of beta2 over the mean of y. In fact, they are two different effects. Hence, by removing, only the x2 parameter you can still say that you have an explained deviance increase that is interpretable. The only difference is that the interpretation is in terms of information loss, or uncertainty decrease, which is absolutely fine to do.

R - multiple linear regression, generating an extra variable with a specific regression coefficient

I want to run a simulated power analysis for a dataset I have. Let's assume the dataset has four variables (column names of the dataset):
Y - which is the dependent variable, is continuous and normally distributed.
X1 - an independent variable, is continuous and has a normal distribution.
X2 - an independent variable, is continous and is NOT normally distributed.
X3 - an independent variable, is continous and is NOT normally distributed.
Now, this data consists of 5000 rows, so there are 5000 entries.
I've run a linear regression using the following formula:
summary(lm( Y ~ X1 + X2 + X3)), and determined the regression coefficients of X1, X2, and X3 to be B1, B2, and B3 respectively.
I now have a fifth variable (x4) which I don't have access to but I believe is normally distributed. Now, the linear model can be updated using the following formula:
lm(Y ~ X1 + X2 + X3 + X4), with the regression coefficient of B4.
I don't know what B4 is, but I have various scenarious where B4 is between 0.2 - 0.5.
I want to run monte-carlo simulations to check what sample size is required to achieve 80% power at various B4. To do this, I need to generate a normally distributed variable that can simulate x4, and and has a regression coefficient of B4. Is there any way to generate this in R?

Linear mixed effects model with heteroscedastic structure for the errors

In a randomized clinical trial in the ophthalmology field, I would like to find the R implementation for a linear mixed effects model given by
log(y_ijk) = b0 + b1_j + b2_k + b3_jk + w_i + e_ijk
where y_ijk is an amount of residual antibiotics, b1_j is the effect of structure (j=1 if cornea and j=2 if aquous humor), b2_k are the effects of treatments (k=1,2,3), b3 are interaction effects, w_i are random effects and e_ijk are random errors. We have pairs of observations for each individual (the residual antibiotic is measured in both structures for each subject).
My main difficulty is that the model should incorporate a heteroscedastic structure for the errors, so that the variances are different for each treatment group. In this way,
e_ijk ~ Normal(0,sigma_k), k = 1,2,3
Thanks

Back transform mixed-effects model's regression coefficients for fixed-effects from log to original scale

I am running a mixed-effects model with the lme4 package. The model specifications are: log(dv) ~ 1 + A*B*C + (1+A*B|random1) + (1+A|random2), where A and B are within-group conditions and C is a between-group condition.
The first problem is that the coefficients for fixed effects are on the log scale and only the intercept makes sense when I do exp(coef) (see below).
The second problem is even if I do an exponentiation transform, how should I account for the random-effects structure? As I understand it, the random-effects structure affects the fixed-effects coefficients (I might be wrong here).
This is a sample output of my fixed-effects coefficients:
Estimate
(Intercept) 6.533079
A1 0.062171
A2 0.077409
B1 -0.184366
B2 -0.154115
C 0.152238
A1:B1 -0.015494
A2:B1 -0.017655
A1:B2 0.001674
A2:B2 -0.003641
A1:C 0.013021
A2:C 0.038995
B1:C 0.010087
B2:C 0.013721
A1:B1:C 0.016025
A2:B1:C 0.016453
A1:B2:C 0.012746
A2:B2:C 0.003113
Now, exp(6.533079) gives 687.5118, which makes sense in the original scale, but the rest of the numbers do not make sense once transformed.

Regression from error term to dependent variable (lavaan)

I want to test a structural equation model (SEM). There are 3 indicators, I1 to I3, that make up a latent construct LC. This construct should explain a dependent variable DV.
Now, assume that unique variance of the indicators will contribute additional explanation to the DV. Something like this:
IV1 ↖
IV2 ← LC → DV
IV3 ↙ ↑
↑ │
e3 ───────┘
In lavaan the error terms/residuals of IV3, e3, are usually not written:
model = '
# latent variables
LV =~ IV1 + IV2 + IV3
# regression
DV ~ LV
'
Further, the residual of I3 must be split into a compontent that contributes to explain DV, and one residual of the residual.
I do not want to explain DV directly by IV3, because its my goal to show how much unique explanation IV3 can contribute to DV. I want to maximize the path IV3 → LC → DV, and then put the residual into I3 → DV.
Question:
How do I put this down in a SEM?
Bonus question:
Does it make sense from a SEM persective that each of the IVs has such a path to DV?
Side note:
What I already did, was to compute this traditionally, using a series of computations. I:
Computed a pendant to LV, average of IV1 to IV3
Did 3 regressions IVx → LC
Did a multiple regression of the IVxs residuals to DV.
Removing the common variance seems to make one of the residuals superfluous, so the regression model cannot estimate each of the residuals, but skips the last one.
For your question:
How do I put this down in a SEM model? Is it possible at all?
The answer, I think, is yes--at least if I understand you correctly.
If what you want to do is predict an outcome using a latent variable and the unique variance of one of its indicators, this can be easily accomplished in lavaan. See example code below: the first example involves predicting an outcome from a latent variable alone, whereas the second example predicts the same outcome from the same latent variable as well as the unique variance of one of the indicators of that latent variable:
#Call lavaan and use HolzingerSwineford1939 data set
library(lavaan)
dat = HolzingerSwineford1939
#Model 1: x4 predicted by lv (visual)
model1 = '
visual =~ x1 + x2 + x3
x4 ~ visual
'
#Fit model 1 and get fit measures and r-squared estimates
fit1 <- cfa(model1, data = dat, std.lv = T)
summary(fit1, fit.measures = TRUE, rsquare=T)
#Model 2: x4 predicted by lv (visual) and residual of x3
model2 = '
visual =~ x1 + x2 + x3
x4 ~ visual + x3
'
#Fit model 2 and get fit measures and r-squared estimates
fit2 <- cfa(model2, data = dat, std.lv = T)
summary(fit2, fit.measures = TRUE,rsquare=T)
Notice that the R-squared for x4 (the hypothetical outcome) is much larger when predicted by both the latent variable onto which x3 loads, and x3's unique variance.
As for your second question:
Bonus question: Does that make sense? And even more: Does it make sense from a SEM view (theoretically is does) that each of the independet variables has such a path to DV?
It can make sense, in some cases, to specify such paths, but I would not do so in absentia of strong theory. For example, perhaps you think a variable is a weak, but theoretically important indicator of a greater latent variable--such as the experience of "awe" is for "positive affect". But perhaps your investigation isn't interested in the latent variable, per se--you are interested in the unique effects of awe for predicting something above and beyond its manifestation as a form of positive affect. You might therefore specify a regression pathway from the unique variance of awe to the outcome, in addition to the pathway from positive affect to the outcome.
But could/should you do this for each of your variables? Well, no, you couldn't. As you can see, this particular case only has one remaining degree of freedom, so the model is on the verge of being under-identified (and would be, if you specified the remaining two possible paths from the unique variances of x1 and x2 to the outcome of x4).
Moreover, I think many would be skeptical of your motivation for attempting to specify all these pathways. Modelling the pathway from the latent variable to the outcome allows you to speak to a broader process; what would you learn by modelling each and every pathway from unique variance to outcome? Sure, you might be able to say, "Well the remaining "stuff" in this variable predicts x4!"...but what could you say about the nature of that "stuff"--it's just isolated manifest variance. Instead, I think you would be on stronger theoretical ground to consider additional common factors that might underly the remaining variance of your variables (e.g., method factors); this would add more conceptual specificity to your analyses.

Resources