Linear model with equality constraints across two factor variables - r

Question

This question is related to https://stats.stackexchange.com/questions/3143/linear-model-with-constraints, but a slightly different scenario.
I have a simple 2-factor linear model with continuous outcome Y. factor1 has ~350 categorical values, and factor2 has the same ~350 categories. I want to constrain the coefficient on each level to sum to zero across the two factors.
(The reason for this is that each level of factor1 and factor2 enters either positively or negatively in any training example, but never appears twice in the same example.)
Here is an example dataset illustrating the situation, where there are four levels of each factor:
Y factor1 factor2
1 -1.2470416 A B
2 4.3368592 C D
3 1.0005147 D A
4 -2.8309146 A C
5 1.7501315 B D
6 -0.8372193 B A
7 3.3542627 C A
8 4.3319422 D C
9 1.4937895 D B
10 2.0951559 A D
11 -2.6610207 C D
12 -4.9917367 D B
13 2.2424169 D A
14 1.0205409 C A
15 -3.4584576 C B
The statistical model I want to estimate is:
$$ y_{(i,j)} = \alpha_i-\beta_j+\varepsilon_{(i,j)} $$
where $(i,j)$ is an outcome that depends on the pair. factor1 marks $i$ and factor2 marks $j$. If group A shows up in factor2, the parameter on A should be equal to the negative of if it showed up in factor1. Thus, I would like to set $\alpha$ equal to $\beta$ for all $i$ and $j$.
I can estimate a (nonsensical) version of this model in lm() fairly easily as follows:
Y <- c( -1.2470416, 4.3368592 , 1.0005147 , -2.8309146 , 1.7501315 , -0.8372193 , 3.3542627 , 4.3319422 , 1.4937895 , 2.0951559 , -2.6610 207 , -4.9917367 , 2.2424169 , 1.0205409 , -3.4584576 )
factor1 <- c( "A" , "C" , "D" , "A" , "B" , "B" , "C" , "D" , "D" , "A" , "C" , "D" , "D" , "C" , "C")
factor2 <- c( "B", "D", "A", "C", "D", "A", "A", "C", "B", "D", "D", "B", "A", "A", "B")
DF <- data.frame(Y,factor1,factor2)
lm(Y~factor1+factor2,data=DF)
and I get the following output:
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.5363 2.5856 0.207 0.841
factor1B -0.4579 3.1121 -0.147 0.887
factor1C 0.4047 2.4925 0.162 0.875
factor1D 1.8737 2.4098 0.778 0.459
factor2B -3.6252 2.2050 -1.644 0.139
factor2C -0.7226 2.8903 -0.250 0.809
factor2D 0.7561 2.2094 0.342 0.741
Note that, theoretically, factor1C should equal -factor2C as dictated by my model. This is not the case in the simple lm() output because I didn't impose any constraints.
So what I would like to do is to estimate
Y ~ factor1 + factor2 [subject to factor1+factor2=0 for each level of factor1, factor2]
In plain English, this would be something like
model2 <- lm(Y~factor1-factor2, data=DF)
But this of course is not how R interprets that expression (because putting a minus sign in a model statement tells R to exclude that variable from the model).
I've read up on contrasts, but I don't think there is a way to do this. I've also read up on glmc, but didn't see a straightforward way of incorporating it for factors that have this many levels. Also, it's not clear to me that generating a new factor3 = factor1-factor2 is a well-defined operation for this specific scenario. Finally, I tried running model3 <- lm(Y+factor2 ~ factor1, data=DF) but received an error.
My sense is that I would need to create a constraint matrix by looping through the levels of each variable. I'm sufficiently new to R that I'm not sure exactly how this is done. Any help would be appreciated.
Note that it is quite easy to do this in Stata, as follows:
input ID y factor1 factor2
1 -1.2470416 1 2
2 4.3368592 3 4
3 1.0005147 4 1
4 -2.8309146 1 3
5 1.7501315 2 4
6 -0.8372193 2 1
7 3.3542627 3 1
8 4.3319422 4 3
9 1.4937895 4 2
10 2.0951559 1 4
11 -2.6610207 3 4
12 -4.9917367 4 2
13 2.2424169 4 1
14 1.0205409 3 1
15 -3.4584576 3 2
end
constraint 1 2.factor1 = -2.factor2
constraint 2 3.factor1 = -3.factor2
constraint 3 4.factor1 = -4.factor2
cnsreg y i.factor1 i.factor2, constraints(1/3)
which gives the following output:
Constrained linear regression Number of obs = 15
F( 3, 11) = 0.73
Prob > F = 0.5554
Root MSE = 2.9875
( 1) 2.factor1 + 2.factor2 = 0
( 2) 3.factor1 + 3.factor2 = 0
( 3) 4.factor1 + 4.factor2 = 0
------------------------------------------------------------------------------
y | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
factor1 |
B | 2.104393 1.439085 1.46 0.172 -1.063011 5.271798
C | .5222649 1.377463 0.38 0.712 -2.509511 3.55404
D | .6589209 1.266188 0.52 0.613 -2.127941 3.445783
|
factor2 |
B | -2.104393 1.439085 -1.46 0.172 -5.271798 1.063011
C | -.5222649 1.377463 -0.38 0.712 -3.55404 2.509511
D | -.6589209 1.266188 -0.52 0.613 -3.445783 2.127941
|
_cons | .5054862 .829675 0.61 0.555 -1.320616 2.331589
------------------------------------------------------------------------------
How does one do the above in R?

Answer 1

As noted in the most popular (but not accepted) answer in https://stats.stackexchange.com/questions/3143/linear-model-with-constraints, this problem is easily solved by creating a new variable which is the difference in the "one-hot" encoded factors.
In Stata, one can do this as follows:
* one-hot encode each of the factors
qui tab factor1, gen(f1dum)
qui tab factor2, gen(f2dum)
* generate difference in one-hot vectors
forv x=1/4{
gen fdiffdum`x' = f1dum`x'-f2dum`x'
}
* regress y on differenced one-hot vectors
reg y fdiffdum2 fdiffdum3 fdiffdum4
Which gives the following output:
Source | SS df MS Number of obs = 15
-------------+---------------------------------- F(3, 11) = 0.73
Model | 19.5429062 3 6.51430205 Prob > F = 0.5554
Residual | 98.1766922 11 8.92515383 R-squared = 0.1660
-------------+---------------------------------- Adj R-squared = -0.0614
Total | 117.719598 14 8.40854274 Root MSE = 2.9875
------------------------------------------------------------------------------
y | Coef. Std. Err. t P>|t| [95% Conf. Interval]
-------------+----------------------------------------------------------------
fdiffdum2 | 2.104393 1.439085 1.46 0.172 -1.063011 5.271798
fdiffdum3 | .5222648 1.377463 0.38 0.712 -2.509511 3.55404
fdiffdum4 | .6589209 1.266188 0.52 0.613 -2.127941 3.445783
_cons | .5054862 .829675 0.61 0.555 -1.320616 2.331589
------------------------------------------------------------------------------
In R, one can do this as follows:
factor1mat <- model.matrix(~factor1, DF)
factor2mat <- model.matrix(~factor2, DF)
factordiffmat <- factor1mat - factor2mat
summary(lm(Y~factordiffmat, data=DF))
Coefficients: (1 not defined because of singularities)
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.5055 0.8297 0.609 0.555
factordiffmat(Intercept) NA NA NA NA
factordiffmatfactor1B 2.1044 1.4391 1.462 0.172
factordiffmatfactor1C 0.5223 1.3775 0.379 0.712
factordiffmatfactor1D 0.6589 1.2662 0.520 0.613