h2o deeplearning error when specifying nfolds for cross validation - r

Question

has this issue been resolved by now? I encounter the same error message.
Usecase: I am doing binary classification using h2o's deeplearning() function. Below, I provide randomly generated data the same size as my actual usecase. System specs:
# R version 3.3.2 (2016-10-31)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# Running under: Windows >= 8 x64 (build 9200)
# h2o version h2o_3.20.0.2
I am currently learning how to use h2o, so I have played with that function quite a bit. Everything runs smoothly until I specify parameters for cross validation.
The problem occurs when specifying the nfolds parameter for cross-validation. Interestingly, I can specify low values for nfolds and everything goes fine. For my use case, even nfolds > 3 produced an error message (see below). I provide an example below, here I was able to specify nfolds < 7 (not really consistent... sometimes just up to nfolds = 3). Above those values, the REST API give the above mentioned error: object not found for argument: key.
# R version 3.3.2 (2016-10-31)
# Platform: x86_64-w64-mingw32/x64 (64-bit)
# Running under: Windows >= 8 x64 (build 9200)
# h2o version h2o_3.20.0.2
#does not matter whether run on debian or windows, does not matter how many threads are used
#error occurs with options for cross validation, otherwise works fine
#no error occurs with specifying a low nfold number(in my actual use case, maximum of 3 folds possible without running into that error message)
require(h2o)
h2o.init(nthreads = -1)
x = matrix(rnorm(900*96, mean=10, sd=2), nrow=900, ncol=96)
y = sample(size=900, x=c(0,1), replace=T)
sampleData = cbind(x, y)
sampleData = data.frame(sampleData)
sampleData[,97] = as.factor(sampleData[,97])
m = h2o.deeplearning(x = 1:96, y = 97,
training_frame = as.h2o(sampleData), reproducible = T,
activation = "Tanh", hidden = c(64,16), epochs = 10000, verbose=T,
nfolds = 4, balance_classes = TRUE, #Cross-validation
standardize = TRUE, variable_importances = TRUE, seed=123,
stopping_rounds=2, stopping_metric="misclassification", stopping_tolerance=0.01, #early stopping
)
performance = h2o.performance(model = m)
print(performance)
######### gives error message
# ERROR: Unexpected HTTP Status code: 404 Not Found (url = http://localhost:xxxxx/3/Models/DeepLearning_model_R_1535036938222_489)
#
# water.exceptions.H2OKeyNotFoundArgumentException
# [1] "water.exceptions.H2OKeyNotFoundArgumentException: Object 'DeepLearning_model_R_1535036938222_489' not found for argument: key"
I cannot understand why it does work only for low values of nfolds. Any suggestions? What am I missing here? I've searched most remotely related threads on Google Groups and also here on stackoverflow, but without success. If this is to do with a changed API for h2o 3.x as suggested above (though that post was 18 months ago...) I would highly appreciate some documentary on how to correctly specify the syntax to do CV with h2o.deeplearning(). Thanks in advance!

Answer 1

This is a bug caused by setting the verbose parameter to True, the workaround is to leave the verbose parameter as the default which is FALSE. I've created a jira ticket to track the issue here