Subscript out of bounds error while computing accuracy at different cutoffs - r

Question

I have constructed a logistic regression and now I want to calculate the predictive accuracy for various cutoff values ranging from 0 to 1. This is the for loop I have been using. But I am getting
subscript out of bounds
Here predtrain contains 300 predicted output probabilities each ranging from 0 to 1. Each value is compared to cutoff eff. Finally a table/confusion matrix has to be generated comparing the original values (train$CAN) with f1. Some thing like this:
tab
# pred2
# 0 1
# 0 1 93
# 1 0 206
code I have written is this:
predtrain <- predict(logreg1, newdata = train, type = 'response')
eff<-seq(0,1,by = 0.05)
for (i in 1:length(eff) {
f1 <- ifelse(predtrain > eff[i], 1, 0)
t1 <- table(train$CAN, f1)
effy <- (t1[1,1]+t1[2,2])/(t1[1,1]+t1[1,2]+t1[2,2]+t1[2,1])
eff[[i]] <-effy
}

Answer 1

The reason you're getting subscript out of bounds errors is that you're trying to create confusion matrices with cutoffs like 0 and 1 -- this will create a confusion matrix with a single column (all predictions are either positive or negative), causing code like t1[2,2] to cause your error.
In reality all you're trying to do is to compute the predictive accuracy at different cutoffs, which can be accomplished without creating tables at all with something like:
cutoffs <- seq(0, 1, by=0.05)
eff <- sapply(cutoffs, function(cutoff) {
sum((predtrain > cutoff) == train$CAN) / length(predtrain)
})
To see this in action, let's consider a small example model:
set.seed(144)
x <- runif(100)
train <- data.frame(x, CAN=as.numeric(runif(100)+x >= 1))
logreg1 <- glm(CAN~x, data=train, family="binomial")
predtrain <- predict(logreg1, newdata = train, type = 'response')
Now we can get the predictive accuracy at each cutoff:
eff <- sapply(cutoffs, function(cutoff) {
sum((predtrain > cutoff) == train$CAN) / length(predtrain)
})
plot(cutoffs, eff)
You could alternately use a package like the ROCR package to grab metrics. For instance, here is how you could grab the sensitivity at each cutoff:
library(ROCR)
pred <- prediction(predtrain, train$CAN)
perf <- performance(pred, "sens")
eff <- sapply(cutoffs, function(cutoff) max(perf#y.values[[1]][perf#x.values[[1]] >= cutoff]))
plot(cutoffs, eff)

Answer 2

But to calculate something like specificity and sensitivity doesn't it become more difficult? I have written using two for loops, I know it is not very effective but I do get the table from which I can calculate performance variables. Can this method be improved?
enter code here
z <- seq(0,1,by = 0.05)
t1 <- vector(mode = "list", length = length(z))
for(i in 1:length(z)) {
predtrain <- predict(logreg1, newdata = train, type = 'response')
for(j in 1:length(predtrain)){
predtrain[j] <- ifelse(predtrain[j]>z[i], 1, 0)
}
t1[[i]] <- table(train$CAN, predtrain)
} t1