Question regarding LASSO confidence intervals using selectiveinference package in R - r

Question

I want to get the confidence intervals for LASSO regression. For this, I used the selective inference package in R.
The fixedLassoInf function in this package provides the confidence intervals for lasso regression for a given value of lambda. Also, we can pass the coefficient vector obtained from glmnet package to this function.
The coefficients for LASSO logistic regression for a given lambda using glmnet package is as follows:
require(ISLR)
require(glmnet)
require(selectiveInference)
y1 <- Default$default
x1 <- model.matrix(default ~ student + balance + income + student*income, Default)[, -1]
lasso.mod1 <- glmnet(x1,y1, alpha = 1, lambda = 0.0003274549,family='binomial')
lasso.mod$beta
> lasso.mod1$beta
4 x 1 sparse Matrix of class "dgCMatrix"
s0
studentYes -6.131640e-01
balance 5.635401e-03
income 2.429232e-06
studentYes:income .
Then I used the fixedLassoInf function in selective inference package in R, to get the confidence intervals:
y1 <- Default$default
beta = coef(lasso.mod1, x=x1, y=y1, s=lambda/1000, exact=T)
y1= ifelse(y1=="NO",0,1)
out = fixedLassoInf(x1,(y1),beta,lambda,family="binomial",alpha=0.05)
out
However, I am getting following Warning messages:
**
Warning messages:
1: In fixedLogitLassoInf(x, y, beta, lambda, alpha = alpha, type = "partial", :
Solution beta does not satisfy the KKT conditions (to within specified tolerances)
2: In fixedLogitLassoInf(x, y, beta, lambda, alpha = alpha, type = "partial", :
Solution beta does not satisfy the KKT conditions (to within specified tolerances). You might try rerunning glmnet with a lower setting of the 'thresh' parameter, for a more accurate convergence.
3: glm.fit: algorithm did not converge
**
Also as the output I am getting something not correct,
Call:
fixedLassoInf(x = x1, y = (y1), beta = beta, lambda = lambda,
family = "binomial", alpha = 0.05)
Testing results at lambda = 0.000, with alpha = 0.050
Var Coef Z-score P-value LowConfPt UpConfPt LowTailArea UpTailArea
1 1142.801 1884.776 1 -Inf -60.633 0 0
2 0.386 1664.734 0 0.023 Inf 0 0
3 0.029 3318.110 0 0.001 Inf 0 0
4 -0.029 -1029.985 1 -Inf -0.003 0 0
Note: coefficients shown are partial regression coefficients
Based on the warning message, there is a problem with the Karush Kuhn Tucker (KKT) condition.
Can anyone help me to figure this out?
Thank you.

Answer 1

One of my university teachers always said
Fitting is an art, not a technique.
What I mean: Do expect that you need manual work for parameter guessing and multiple iterations of fitting. You might even question the method of fitting itself, but let's not go that path.
Anyhow, R will not do the magic of finding the correct model (now: number of parameters for LASSO) for you. From the output you show, you seem to have 4 variables, of which 3 are close to zero, therefore I suggest to start with...
Bounding the maximal number of variables in the model, i.e. dfmax=2 seems a good start
Limiting the maximum number of variables ever to be nonzero, e.g. pmax=2
The documentation of glment further details on other options.