glmnet refuses to predict - r

Question

I have a glm model that works. Since I'd like to add (ridge) regularization I thought I'd switch to glmnet. For some reason I cannot get glmnet to work. It seems to always predict the first class, never the second, which results in low accuracy and kappa = 0.
Below is some code to reproduce the problem. What am I doing wrong?
The test data it generates looks like this:
Since the data cannot be linearly separated two polynomial terms A^2 and B^2 are added.
A glm model predicts the data correctly (with accuracy = 1 and kappa = 1). Here is its prediction boundary:
While a glmnet model always has kappa = 0, no matter what lambda it tries:
lambda Accuracy Kappa Accuracy SD Kappa SD
0 0.746 0 0.0295 0
1e-04 0.746 0 0.0295 0
0.01 0.746 0 0.0295 0
0.1 0.746 0 0.0295 0
1 0.746 0 0.0295 0
10 0.746 0 0.0295 0
Code to reproduce the problem:
library(caret)
# generate test data
set.seed(42)
n <- 500; m <- 100
data <- data.frame(A=runif(n, 98, 102), B=runif(n, 98, 102), Type="foo")
data <- subset(data, sqrt((A-100)^2 + (B-100)^2) > 1.5)
data <- rbind(data, data.frame(A=rnorm(m, 100, 0.25), B=rnorm(m, 100, 0.25), Type="bar"))
# add a few polynomial features to match ellipses
polymap <- function(data) cbind(data, A2=data$A^2, B2=data$B^2)
data <- polymap(data)
plot(x=data$A, y=data$B, pch=21, bg=data$Type, xlab="A", ylab="B")
# train a binomial glm model
model.glm <- train(Type ~ ., data=data, method="glm", family="binomial",
preProcess=c("center", "scale"))
# train a binomial glmnet model with ridge regularization (alpha = 0)
model.glmnet <- train(Type ~ ., data=data, method="glmnet", family="binomial",
preProcess=c("center", "scale"),
tuneGrid=expand.grid(alpha=0, lambda=c(0, 0.0001, 0.01, 0.1, 1, 10)))
print(model.glm) # <- Accuracy = 1, Kappa = 1 - good!
print(model.glmnet) # <- Accuracy = low, Kappa = 0 - bad!
Calling glmnet directly (without caret) results in the same problem:
x <- as.matrix(subset(data, select=-c(Type)))
y <- data$Type
model.glmnet2 <- cv.glmnet(x=x, y=y, family="binomial", type.measure="class")
preds <- predict(model.glmnet2, x, type="class", s="lambda.min")
# all predictions are class 1...
EDIT: Plot of the scaled data and the decision boundary found by glm:
Model: -37 + 6317*A + 6059*B - 6316*A2 - 6059*B2

Answer 1

You should center and scale data prior to making polynomial versions of the predictor. Numerically, things work better that way:
set.seed(42)
n <- 500; m <- 100
data <- data.frame(A=runif(n, 98, 102), B=runif(n, 98, 102), Type="foo")
data <- subset(data, sqrt((A-100)^2 + (B-100)^2) > 1.5)
data <- rbind(data, data.frame(A=rnorm(m, 100, 0.25), B=rnorm(m, 100, 0.25), Type="bar"))
data2 <- data
data2$A <- scale(data2$A, scale = TRUE)
data2$B <- scale(data2$B, scale = TRUE)
data2$A2 <- data2$A^2
data2$B2 <- data2$B^2
# train a binomial glm model
model.glm2 <- train(Type ~ ., data=data2, method="glm")
# train a binomial glmnet model with ridge regularization (alpha = 0)
model.glmnet2 <- train(Type ~ ., data=data2, method="glmnet",
tuneGrid=expand.grid(alpha=0,
lambda=c(0, 0.0001, 0.01, 0.1, 1, 10)))
From these:
> getTrainPerf(model.glm2)
TrainAccuracy TrainKappa method
1 1 1 glm
> getTrainPerf(model.glmnet2)
TrainAccuracy TrainKappa method
1 1 1 glmnet
Max