If you're thinking such a noob while reading the title - yes I am.
I've googled but didn't find a single guide that allowed me to view how a pre-trained torch neural network is designed/coded. I have already downloaded the pre-trained network (file format .t7) and I have torch installed. Can anyone help me view how it is coded (what size filters used, parameters used etc.)?
May be it's not on google because it's not possible? Will be happy to answer any additional questions you have or if anything isn't clear.
Thank You.
I think it is not possible to get the underlying code. But you can get a summary of the model which includes the layers and the main parameters just by using print.
model = SumModel(vocab_size=vocab_size, hiddem_dim=hidden_dim, batch_size=batch_size)
# saving model
torch.save(model, 'test_model.save')
# print summary of original
print(' - original model summary:')
print(model)
print()
# load saved model
loaded_model = torch.load('test_model.save')
# print summary of loaded model
print(' - loaded model summary:')
print(loaded_model)
This will output a summary which looks like this.
- original model summary:
SumModel(
(word_embedding): Embedding(530734, 128)
(encoder): LSTM(128, 128, batch_first=True)
(decoder): LSTM(128, 128, batch_first=True)
(output_layer): Linear(in_features=128, out_features=530734, bias=True)
)
- loaded model summary:
SumModel(
(word_embedding): Embedding(530734, 128)
(encoder): LSTM(128, 128, batch_first=True)
(decoder): LSTM(128, 128, batch_first=True)
(output_layer): Linear(in_features=128, out_features=530734, bias=True)
)
Tested with Pytorch 0.4.0
As you can see both outputs for the original and the loaded model are consistent.
I hope this helps.
Related
I'm triying to run the code of section 5.4.2 of the F. Chollet's book "Deep Learning with R", on Visualizing convnet filters. The code is as follows:
library(keras)
model <- application_vgg16(
weights = 'imagenet',
include_top = FALSE
)
layer_name <- 'block3_conv1'
filter_index <- 1
layer_output <- get_layer(model, layer_name)$output
loss <- k_mean(layer_output[,,,filter_index])
grads <- k_gradients(loss, model$input)[[1]]
but throws Error in py_call_impl(): ! RuntimeError: tf.gradients is not supported when eager execution is enabled. Use tf.GradientTape instead.
According to this github issue this can easily be resolved adding two lines of code at the beginning of the script:
library(tensorflow)
tf$compat$v1$disable_eager_execution()
and effectively, the tf$executing_eagerly() call transitions from TRUE to FALSE. Now, k_gradients() throws another error:
Error in `py_call_impl()`:
! TypeError: You are passing KerasTensor(type_spec=TensorSpec(shape=(), dtype=tf.float32, name=None), name='tf.math.reduce_mean/Mean:0', description="created by layer 'tf.math.reduce_mean'"), an intermediate Keras symbolic input/output, to a TF API that does not allow registering custom dispatchers, such as `tf.cond`, `tf.function`, gradient tapes, or `tf.map_fn`. Keras Functional model construction only supports TF API calls that *do* support dispatching, such as `tf.math.add` or `tf.reshape`. Other APIs cannot be called directly on symbolic Kerasinputs/outputs. You can work around this limitation by putting the operation in a custom Keras layer `call` and calling that layer on this symbolic input/output.
How to overcome this issue and visualize convnet filters?
I am new to Machine Learning and it is the first time that I am using python's gensim in order to extract topics from text.
I successfully trained a model (for 100 topics) and then I had the idea to use that model in an HTTP API that I created using python flask. The endpoint gives as back terms for a given text.
Btw model is loaded when I initialize the API.
After trying this out on production, memory (on a small VM ~ 1GB Ram) exhausted and finally I got an error:
tags = tags + lda.topic_words(topic_index, num_of_keywords_for_topic, model, words)
File "/var/app/tagbee/lda.py", line 64, in topic_words
x2 = model.get_topic_terms(topicid=topic_index, topn=number_of_keywords)
File "/usr/local/lib/python3.6/dist-packages/gensim/models/ldamodel.py", line 1224, in get_topic_terms
topic = self.get_topics()[topicid]
File "/usr/local/lib/python3.6/dist-packages/gensim/models/ldamodel.py", line 1204, in get_topics
topics = self.state.get_lambda()
File "/usr/local/lib/python3.6/dist-packages/gensim/models/ldamodel.py", line 269, in get_lambda
return self.eta + self.sstats
MemoryError: Unable to allocate 96.6 MiB for an array with shape (100, 253252) and data type float32
So I have some questions:
Can a gensim LDA model be used that way, mean in an HTTP API?
If yes, what is the trick to make it happen? If it needs at least 90MB of memory per request, how does it scale?
Is there any alternative approach?
Thank you in advance!
Your question seems to be related to LDA or gensim only accidentally. The main point seems to be how to and maintain (and reuse) an object in memory across a number of Flask requests.
Inspired by Flask documentation and answers from this question:
Flask - Store values in memory between requests I propose the following approach:
from flask import g # global context of all queries
def get_lda_model():
if 'lda' not in g:
g.lda = # read a model file here
return g.lda
#app.route('/example_request_path', methods =['POST'])
def my_request():
lda = get_lda_model()
# use lda model here....
Once the LDA model is loaded you can reuse it very quickly in a number of requests without reloading it into memory. As long as your model is not going to be changed across requests it does not matter if this approach is thread-safe.
I have established a deep learning model with the h2o package of the R software. I gained a model with good presence and I wanna to save it. However, I tried all kinds of methods but failed. The code "save()" and "save.image()" are provided in the base package of R software. I used the "save()" function to conserve my model. But when I want to use the built model to run new data, it is said that the "model" object is not found in the function. I am really confused about this problem for a few days. If you have any good ideas, just tell me. Thanks for your reading~
load("F:/R/Rstudy/myfile") ##download the saved file
library(h2o)
h2o.init()
Te <- read.csv("F:/Rdata/Test.csv") ## import testing data
Te <- as.h2o(Te)
Te[,2] <- as.factor(Te[,2])
perf <- h2o.performance(model, Te) ## test model
ERROR: Unexpected HTTP Status code: 404 Not Found (url = http://localhost:54321/3/ModelMetrics/models/DeepLearning_model_R_1533035975237_1/frames/RTMP_sid_8185_2)
ERROR MESSAGE:
Object 'DeepLearning_model_R_1533035975237_1' not found in function: predict for argument: model
You can use the below to save and retrieve the model.
build the model
model <- h2o.deeplearning(params)
save the model
model_path <- h2o.saveModel(object=model, path=getwd(), force=TRUE)
print(model_path)
/tmp/mymodel/DeepLearning_model_R_1441838096933
load the model
saved_model <- h2o.loadModel(model_path)
Reference - http://docs.h2o.ai/h2o/latest-stable/h2o-docs/save-and-load-model.html
Hope this helps,
ND
I am using MXnet for training a CNN (in R) and I can train the model without any error with the following code:
model <- mx.model.FeedForward.create(symbol=network,
X=train.iter,
ctx=mx.gpu(0),
num.round=20,
array.batch.size=batch.size,
learning.rate=0.1,
momentum=0.1,
eval.metric=mx.metric.accuracy,
wd=0.001,
batch.end.callback=mx.callback.log.speedometer(batch.size, frequency = 100)
)
But as this process is time-consuming, I run it on a server during the night and I want to save the model for the purpose of using it after finishing the training.
I used:
save(list = ls(), file="mymodel.RData")
and
mx.model.save("mymodel", 10)
But none of them can save the model! for example when I load the "mymodel.RData", I can not predict the labels for the test set!
Another example is when I load the "mymodel.RData" and try to plot it with the following code:
graph.viz(model$symbol$as.json())
I get the following error:
Error in model$symbol$as.json() : external pointer is not valid
Can anybody give me a solution for saving and then loading this model for future use?
Thanks
You can save the model by
model <- mx.model.FeedForward.create(symbol=network,
X=train.iter,
ctx=mx.gpu(0),
num.round=20,
array.batch.size=batch.size,
learning.rate=0.1,
momentum=0.1,
eval.metric=mx.metric.accuracy,
wd=0.001,
epoch.end.callback=mx.callback.save.checkpoint("model_prefix")
batch.end.callback=mx.callback.log.speedometer(batch.size, frequency = 100)
)
A mxnet model is an R list, but its first component is not an R object but a C++ pointer and can't be saved and reloaded as an R object. Therefore, the model needs to be serialized to behave as an actual R object. The serialized object is also a list, but its first object is a text string containing model information.
To save a model:
modelR <- mx.serialize(model)
save(modelR, file="~/model1.RData")
To retrieve it and use it again:
load("~/model1.RData", verbose=TRUE)
model <- mx.unserialize(modelR)
The best practice for saving a snapshot of your training progress is to use save_snapshot (http://mxnet.io/api/python/module.html#mxnet.module.Module.save_checkpoint) as part of the callback after every epoch training. In R the equivalent command is probably mx.callback.save.checkpoint, but I'm not using R and not sure about it usage.
Using these snapshots can also allow you to take advantage of the low cost option of using AWS Spot market (https://aws.amazon.com/ec2/spot/pricing/ ), which for example now offers and instance with 16 K80 GPUs for $3.8/hour compare to the on-demand price of $14.4. Such 80%-90% discount is common in the spot market and can optimize the speed and cost of your training, as long as you use these snapshots correctly.
I have a large dataset, and I'm trying to use R's package xgboost to perform a regression on it.
The function xgboost says that the argument data can be a local data file, from which I understand it should be the name of the file to be used. There are however no further specifications about it, so my question is how exactly should be this file.
I've tried
random=matrix(rnorm(15),5,3)
colnames(random)=c("first","second","label")
write.csv(random,"random.csv")
bst <- xgboost(data = "random.csv",
nthread = 7,
nround = 3,
objective="reg:linear",
verbose=FALSE)
but that returns
6x0 matrix with 0 entries is loaded from random.csv
Error in xgb.iter.update(bst$handle, dtrain, i - 1, obj) :
NumCol:need column access
Many thanks!
The xgboost local data file input does not support csv. Quoting from this link
Currently XGBoost supports local data files in the libsvm format. - See more at: http://blog.nycdatascience.com/uncategorized/xgboost-introduction/#sthash.bmlHst0T.dpuf
See this Cross Validated Question/Answer for more information on what the libsvm format is.
Hope this helps.
Sadly I am not yet allowed to comment, and this is a bit naive for a proper answer but just to be sure:
- Have you made sure that R is in the right folder? Try getwd() to check which working directory you are located and setwd() to change it.
I will delete my answer later since I will that it is incomplete,
greetings,
Daniel