trying to use exact=TRUE feature in R glmnet - r

Question

I am trying to use exact=TRUE feature in glmnet. But I am getting an error message.
> fit = glmnet(as.matrix(((x_values))), (as.matrix(y_values)),penalty=variable.list$penalty)
> coef.exact = coef(fit, s = 0.03, exact = TRUE)
Error: used coef.glmnet() or predict.glmnet() with `exact=TRUE` so must in addition supply original argument(s) x and y and penalty.factor in order to safely rerun glmnet
How can I supply penalty.factor to coef.exact?
Options tried:-
> coef.exact = coef(as.matrix(((x_values))), (as.matrix(y_values)),penalty=variable.list$penalty, s = 0.03, exact = TRUE)
Error: $ operator is invalid for atomic vectors
>
> coef.exact = coef((as.matrix(((x_values))), (as.matrix(y_values)),penalty=variable.list$penalty), s = 0.03, exact = TRUE)
Error: unexpected ',' in "coef.exact = coef((as.matrix(((x_values))),"
>
> coef.exact = coef((as.matrix(((x_values))) (as.matrix(y_values)) penalty=variable.list$penalty), s = 0.03, exact = TRUE)
Error: unexpected symbol in "coef.exact = coef((as.matrix(((x_values))) (as.matrix(y_values)) penalty"
>
> coef.exact = coef(fit(as.matrix(((x_values))), (as.matrix(y_values)),penalty=variable.list$penalty), s = 0.03, exact = TRUE)
Error in fit(as.matrix(((x_values))), (as.matrix(y_values)), penalty = variable.list$penalty) :
could not find function "fit"
>
> coef.exact = coef(glmnet(as.matrix(((x_values))), (as.matrix(y_values)),penalty=variable.list$penalty), s = 0.03, exact = TRUE)
Error: used coef.glmnet() or predict.glmnet() with `exact=TRUE` so must in addition supply original argument(s) x and y and penalty.factor in order to safely rerun glmnet
>

Answer 1

Here is an example using mtcars as sample data. Note it's always advisable to provide a minimal & reproducible code example including sample data when posting on SO.
# Fit mpg ~ wt + disp
x <- as.matrix(mtcars[c("wt", "disp")]);
y <- mtcars[, "mpg"];
fit <- glmnet(x, y, penalty = 0.1);
# s is our regularisation parameter, and since we want exact results
# for s=0.035, we need to refit the model using the full data (x,y)
coef.exact <- coef(fit, s = 0.035, exact = TRUE, x = x, y = y, penalty.factor = 0.1);
coef.exact;
#3 x 1 sparse Matrix of class "dgCMatrix"
# 1
#(Intercept) 34.40289989
#wt -3.00225110
#disp -0.02016836
The reason why you explicitly need to provide x and y again is given in ?coef.glmnet (also see #FelipeAlvarenga post).
So in your case, the following should work:
fit = glmnet(x = as.matrix(x_values), y = y_values, penalty=variable.list$penalty)
coef.exact = coef(
fit,
s = 0.03,
exact = TRUE,
x = as.matrix(x_values),
y = y_values,
penalty.factor = variable.list$penalty)
Some comments
Perhaps the confusion arises from the difference between the model's overall regularisaton parameter (s or lambda) and the penalty.factors that you can apply to every coefficient. The latter allows for differential regularisation of individual parameters, whereas s controls the effect of overall L1/L2 regularisation.

Answer 2

In coef the parameter s corresponds to the penalty parameter. In the help files:
s Value(s) of the penalty parameter lambda at which predictions are
required. Default is the entire sequence used to create the model.
[...]
With exact=TRUE, these different values of s are merged (and sorted)
with object$lambda, and the model is refit before predictions are
made. In this case, it is required to supply the original data x= and
y= as additional named arguments to predict() or coef(). The workhorse
predict.glmnet() needs to update the model, and so needs the data used
to create it. The same is true of weights, offset, penalty.factor,
lower.limits, upper.limits if these were used in the original call.
Failure to do so will result in an error.
Therefore, to use exact = T you must assign your original penalties, x, y and any other parameter you inputted in your original model