I have established a deep learning model with the h2o package of the R software. I gained a model with good presence and I wanna to save it. However, I tried all kinds of methods but failed. The code "save()" and "save.image()" are provided in the base package of R software. I used the "save()" function to conserve my model. But when I want to use the built model to run new data, it is said that the "model" object is not found in the function. I am really confused about this problem for a few days. If you have any good ideas, just tell me. Thanks for your reading~
load("F:/R/Rstudy/myfile") ##download the saved file
library(h2o)
h2o.init()
Te <- read.csv("F:/Rdata/Test.csv") ## import testing data
Te <- as.h2o(Te)
Te[,2] <- as.factor(Te[,2])
perf <- h2o.performance(model, Te) ## test model
ERROR: Unexpected HTTP Status code: 404 Not Found (url = http://localhost:54321/3/ModelMetrics/models/DeepLearning_model_R_1533035975237_1/frames/RTMP_sid_8185_2)
ERROR MESSAGE:
Object 'DeepLearning_model_R_1533035975237_1' not found in function: predict for argument: model
You can use the below to save and retrieve the model.
build the model
model <- h2o.deeplearning(params)
save the model
model_path <- h2o.saveModel(object=model, path=getwd(), force=TRUE)
print(model_path)
/tmp/mymodel/DeepLearning_model_R_1441838096933
load the model
saved_model <- h2o.loadModel(model_path)
Reference - http://docs.h2o.ai/h2o/latest-stable/h2o-docs/save-and-load-model.html
Hope this helps,
ND
Related
I would like to implement a simple text classification using BERT model on R side. On my local side, I have no progress because of this issue. Therefore, I moved to kaggle platform. Also, I have started to replicate this kaggle kernel. However, I could not succeed to call transformer$TFBertModel because this line throws me the below error. As you might guess that there are few tutorials BERT on R side, so I am also appreciated for any simple BERT text classification tutorial based on R (over kaggle).
ERROR
Error in py_get_attr_impl(x, name, silent): RuntimeError: Failed to import transformer...
as:
library(tidyverse)
library(reticulate)
library(LiblineaR)
library(tidymodels)
library(rsample)
reticulate::py_install('transformers', pip = TRUE)
transformer = reticulate::import('transformers')
tf = reticulate::import('tensorflow')
builtins <- import_builtins()
tokenizer <- transformer$AutoTokenizer$from_pretrained('bert-base-uncased')
BERT = transformer$TFBertModel$from_pretrained("bert-base-uncased")
print("completed...")
I'm using h2o package and trying to create a learner using the below given code
install.packages("h2o")
library("h2o")
h2o.learner <- makeLearner("regr.h2o.deeplearning",predict.type = "response")
But I'm getting this error
> h2o.learner <- makeLearner("regr.h2o.deeplearning",predict.type = "response")
Error: could not find function "makeLearner"
Note: Few months back I used this code without any problem.
Any idea what could be possible thing for this error?
The correct code for this is simply
library(mlr)
h2o.learner = makeLearner("regr.h2o.deeplearning")
The makeLearner() is not part of H2O. It appears to be part of the mlr package. It also seems that mlr does have h2o support, so it might be as simple as adding a library(mlr) to the top of your script? (Making sure that the mlr package has been installed, already, of course.)
I am using MXnet for training a CNN (in R) and I can train the model without any error with the following code:
model <- mx.model.FeedForward.create(symbol=network,
X=train.iter,
ctx=mx.gpu(0),
num.round=20,
array.batch.size=batch.size,
learning.rate=0.1,
momentum=0.1,
eval.metric=mx.metric.accuracy,
wd=0.001,
batch.end.callback=mx.callback.log.speedometer(batch.size, frequency = 100)
)
But as this process is time-consuming, I run it on a server during the night and I want to save the model for the purpose of using it after finishing the training.
I used:
save(list = ls(), file="mymodel.RData")
and
mx.model.save("mymodel", 10)
But none of them can save the model! for example when I load the "mymodel.RData", I can not predict the labels for the test set!
Another example is when I load the "mymodel.RData" and try to plot it with the following code:
graph.viz(model$symbol$as.json())
I get the following error:
Error in model$symbol$as.json() : external pointer is not valid
Can anybody give me a solution for saving and then loading this model for future use?
Thanks
You can save the model by
model <- mx.model.FeedForward.create(symbol=network,
X=train.iter,
ctx=mx.gpu(0),
num.round=20,
array.batch.size=batch.size,
learning.rate=0.1,
momentum=0.1,
eval.metric=mx.metric.accuracy,
wd=0.001,
epoch.end.callback=mx.callback.save.checkpoint("model_prefix")
batch.end.callback=mx.callback.log.speedometer(batch.size, frequency = 100)
)
A mxnet model is an R list, but its first component is not an R object but a C++ pointer and can't be saved and reloaded as an R object. Therefore, the model needs to be serialized to behave as an actual R object. The serialized object is also a list, but its first object is a text string containing model information.
To save a model:
modelR <- mx.serialize(model)
save(modelR, file="~/model1.RData")
To retrieve it and use it again:
load("~/model1.RData", verbose=TRUE)
model <- mx.unserialize(modelR)
The best practice for saving a snapshot of your training progress is to use save_snapshot (http://mxnet.io/api/python/module.html#mxnet.module.Module.save_checkpoint) as part of the callback after every epoch training. In R the equivalent command is probably mx.callback.save.checkpoint, but I'm not using R and not sure about it usage.
Using these snapshots can also allow you to take advantage of the low cost option of using AWS Spot market (https://aws.amazon.com/ec2/spot/pricing/ ), which for example now offers and instance with 16 K80 GPUs for $3.8/hour compare to the on-demand price of $14.4. Such 80%-90% discount is common in the spot market and can optimize the speed and cost of your training, as long as you use these snapshots correctly.
I created a model using Apache OpenNLP's command line tool to recognize named entities. The below code created the model using the file sentences4OpenNLP.txt as a training set.
opennlp TokenNameFinderTrainer -type maxent -model C:\Users\Documents\en-ner-org.bin -lang en -data C:\Users\Documents\apache-opennlp-1.6.0\sentences4OpenNLP.txt -encoding UTF-8
I tested the model from the command line by passing it sentences to tag, and the model seemed to be working well. However, I am unable to successfully use the model from R. I am using the below lines in attempts to create an organization annotating function. Using the same code to load a model downloaded from OpenNLP works fine.
modelNER <- "C:/Users/Documents/en-ner-org.bin"
oa <- openNLP::Maxent_Entity_Annotator(language = "en",
kind = "organization",
probs = TRUE,
model = modelNER)
When the above code is run I get an error saying:
Could not instantiate the opennlp.tools.namefind.TokenNameFinderFactory. The initialization throw an exception.
opennlp.tools.util.ext.ExtensionNotLoadedException: Unable to find implementation for opennlp.tools.util.BaseToolFactory, the class or service opennlp.tools.namefind.TokenNameFinderFactory could not be located!
at opennlp.tools.util.ext.ExtensionLoader.instantiateExtension(ExtensionLoader.java:97)
at opennlp.tools.util.BaseToolFactory.create(BaseToolFactory.java:106)
at opennlp.tools.util.model.BaseModel.initializeFactory(BaseModel.java:254)
Error in .jnew("opennlp.tools.namefind.TokenNameFinderModel", .jcast(.jnew("java.io.FileInputStream", :
java.lang.IllegalArgumentException: opennlp.tools.util.InvalidFormatException: Could not instantiate the opennlp.tools.namefind.TokenNameFinderFactory. The initialization throw an exception.
at opennlp.tools.util.model.BaseModel.loadModel(BaseModel.java:237)
at opennlp.tools.util.model.BaseModel.<init>(BaseModel.java:181)
at opennlp.tools.namefind.TokenNameFinderModel.<init>(TokenNameFinderModel.java:110)
Any advice on how to fix the error would be a big help. Thanks in advance.
Resolved the error. The R function openNLP::Maxent_Entity_Annotator was not compatible with the named entity recognition (NER) model being produced by OpenNLP 1.6.0. Building the NER model using OpenNLP 1.5.3 resulted in openNLP::Maxent_Entity_Annotator running without error.
I am trying to use predict function in R using a model saved earlier. The model was created and saved using the following code:
lrModel1 <- glm(response ~ .,data = modelData,family = binomial,model = TRUE)
save(lrModel1,file = "lrModel100.rda")
When I load the model for later use as follows and try to use the predict function on it as follows:
bar <- load("lrModel100.rda")
predicted <- predict(bar,validationData,type = "response")
I get the following error:
Error in UseMethod("predict") :
no applicable method for 'predict' applied to an object of class "character"
Is there a way to get the model object name from the saved RDA file and use it for prediction?
Thank you.
Ravi
As #droopy told you the model's name doesn't change if you save and load. You can use get to use the model:
predicted <- predict(get(bar),validationData,type = "response")
If you have saved the model earlier it may throw this error.
Reload the library(glmnet) and make sure that number of variables in X & Y are same.
I have same problem before.
I used caret to build model, and save the model as a rds file.(saveRDS)
When I readRDS my file, and use this model to predict, I encountered this problem.
After I use "library(caret)", my problem is solved.
So I think if you save your model, and re-open your model to predict, you have to reload the package you used for building model.