Bug in R e1071 Naive Bayes? - r

Question

I have no experience in the R community, so please point me somewhere else if this is not the appropriate forum...
Long story short, I'm afraid that e1071::naiveBayes favors giving labels by alphabetical order.
In an earlier question here I had noticed some strange behavior with numerical predictors in the e1071 implementation of naive Bayes. While I got a more reasonable answer, some probabilities seemed biased upwards.
Can anyone shed any light on why this simulation would end up like this? I can only imagine that it is a bug at this point...
library(e1071)
# get a data frame with numObs rows, and numDistinctLabels possible labels
# each label is randomly drawn from letters a-z
# each label has its own distribution of a numeric variable
# this is normal(i*100, 10), i in 1:numDistinctLabels
# so, if labels are t, m, and q, t is normal(100, 10), m is normal(200, 10), etc
# the idea is that all labels should be predicted just as often
# but it seems that "a" will be predicted most, "b" second, etc
doExperiment = function(numObs, numDistinctLabels){
possibleLabels = sample(letters, numDistinctLabels, replace=F)
someFrame = data.frame(
x=rep(NA, numObs),
label=rep(NA, numObs)
)
numObsPerLabel = numObs / numDistinctLabels
for(i in 1:length(possibleLabels)){
label = possibleLabels[i]
whichAreNA = which(is.na(someFrame$label))
whichToSet = sample(whichAreNA, numObsPerLabel, replace=F)
someFrame[whichToSet, "label"] = label
someFrame[whichToSet, "x"] = rnorm(numObsPerLabel, 100*i, 10)
}
someFrame = as.data.frame(unclass(someFrame))
fit = e1071::naiveBayes(label ~ x, someFrame)
# The threshold argument doesn't seem to change the matter...
someFrame$predictions = predict(fit, someFrame, threshold=0)
someFrame
}
# given a labeled frame, return the label that was predicted most
getMostFrequentPrediction = function(labeledFrame){
names(which.max(sort(table(labeledFrame$prediction))))
}
# run the experiment a few thousand times
mostPredictedClasses = sapply(1:2000, function(x) getMostFrequentPrediction(doExperiment(100, 5)))
# make a bar chart of the most frequently predicted labels
plot(table(mostPredictedClasses))
This gives a plot like:
Giving every label the same normal distribution (i.e. mean 100, stdev 10) gives:
Regarding confusion in comment:
This is maybe getting away from Stack Overflow territory here, but anyways...
While I would expect classification to be less clumpy, the effect of the standard deviations does a lot to flatten out the pdfs, and you can observe if you do this enough that one or two actually tend to dominate (red and black in this case).
Too bad we can't exploit the knowledge that the standard deviation is the same for all of them.
If you add just a little noise to the mean it becomes much more evenly distributed, even though there's still some misclassification.

Answer 1

The problem is not naiveBayes, it's your getMostFrequentPrediction function. You are returning only one value even when there are ties for first. Since you are using table(), the counts are being implicitly sorted alphabetically in the table. So when you grab the first max value, it will also be the "smallest" alphabetically speaking. So if you dun this a bunch of times:
getMostFrequentPrediction(data.frame(predictions=sample(rep(letters[1:3], 5))))
you will always get "a" even though the letters "a" "b" and "c" all appear 5 times.
If you want to randomly choose one of the most frequently predicted categories, here's another possible implementation
getMostFrequentPrediction = function(labeledFrame){
tt<-table(labeledFrame$predictions)
names(sample(tt[tt==max(tt)], 1))
}
This gives