I have the following Keras LSTM model using functional API:
model = Sequential()
model.add(Lambda(lambda x: x,input_shape=(timestep,n_feature)))
output = model.output
output = LSTM(8)(output)
output = Dense(2)(output)
inputTensor = model.input
myModel = Model([inputTensor], output)
myModel.compile(loss='mean_squared_error', optimizer='adam')
myModel.fit([trainX], trainY, epochs=100, batch_size=1, verbose=2, validation_split = 0.1)
The model works fine but I think there are redundant syntax in my architecture. For example, the Lambda layer is only used to define the input_shape, maybe it can be removed? Can the above code be simplified/cleaned (I want to keep using functional API)? Thanks!
You can write your model using functional API as follows-
x=Input(shape=(timestep,n_feature))
model=LSTM(8)(x)
model=Dense(2)(model)
myModel=Model(x,model)
Related
I have used RFE (Recursive Feature Elimination) method to find the best model out of 39 variables. I used the following codes:
set.seed(10)
ctrl <- rfeControl(functions = caretFuncs, method = "repeatedcv",repeats = 5,number= 5,allowParallel = TRUE)
RF_39 <- rfe(X, Y,sizes = c(1:39),method ='rf',rfeControl = ctrl,tuneGrid = data.frame(mtry=6))
The best model is made using 36 variables.
If I want to see those 36 variables I can use
predictors(RF_39) or RF_39$optVariable functions.
However, I need to see the variables used for making the other models rather than the best model.
For example, what are the variables used for making model number 12?
How can I see the variables of a specific model made by RFE method?
Thanks for your help.
I am trying to use MLflow in R. According to https://www.mlflow.org/docs/latest/models.html#r-function-crate, the crate flavor needs to be used for the model. My model uses the Random Forest function implemented in the ranger package:
model <- ranger::ranger(formula = model_formula,
data = trainset,
importance = "impurity",
probability=T,
num.trees = 500,
mtry = 10)
The model itself works and I can do the prediction on a testset:
test_prediction <- predict(model, testset)
As a next step, I try to bring the model in the crate flavor. I follow here the approach shown in https://docs.databricks.com/_static/notebooks/mlflow/mlflow-quick-start-r.html.
predictor <- crate(function(x) predict(model,.x))
This results however in an error, when I apply the "predictor" on the testset
predictor(testset)
Error in predict(model, .x) : could not find function "predict"
Does anyone know how to solve this issue? To I have to transfer the prediction function differently in the crate function? Any help is highly appreciated ;-)
In my experience, that Databricks quickstart guide is wrong.
According to the Carrier documentation, you need to use explicit namespaces when calling non-base functions inside of crate. Since predict is actually part of the stats package, you'd need to specify stats::predict. Also, since your crate function depends on the global object named model, you'd need to pass that as an argument to the crate function as well.
Your code would end up looking something like this (I can't test it on your exact use case, since I don't have your data, but this works for me on MLflow in Databricks):
model <- ranger::ranger(formula = model_formula,
data = trainset,
importance = "impurity",
probability=T,
num.trees = 500,
mtry = 10)
predictor <- crate(function(x) {
stats::predict(model,x)
}, model = model)
predictor(testset)
I'm trying to build a regression model with R using lightGBM,
and i'm getting a bit confused with some functions and when/how to use them.
First one is what i've written in the title, what's the difference between lgb.train() and lightgbm()?
The description in the documentation(https://cran.r-project.org/web/packages/lightgbm/lightgbm.pdf) says that lgb.train is 'Logic to train with LightGBM' and lightgbm is 'Simple interface for training a LightGBM model', while both their outcome value is lgb.Booster, a trained model.
One difference I've found is that lgb.train() does not work with valids = , while lightgbm() does.
Second one is about a function lgb.cv(), regarding a cross validation in lightGBM. How do you apply the output of lgb.cv() to a model?
As I understood from the documentation i've linked above, it seems like the output of both lgb.cv and lgb.train is a model.
Is it correct to use it like the example below?
lgbcv <- lgb.cv(params,
lgbtrain,
nrounds = 1000,
nfold = 5,
early_stopping_rounds = 100,
learning_rate = 1.0)
lgbcv <- lightgbm(params,
lgbtrain,
nrounds = 1000,
early_stopping_rounds = 100,
learning_rate = 1.0)
Thank you in advance!
what's the difference between lgb.train() and lightgbm()?
These functions both train a LightGBM model, they're just slightly different interfaces. The biggest difference is in how training data are prepared. LightGBM training requires a special LightGBM-specific representation of the training data, called a Dataset. To use lgb.train(), you have to construct one of these beforehand with lgb.Dataset(). lightgbm(), on the other hand, can accept a data frame, data.table, or matrix and will create the Dataset object for you.
Choose whichever method you feel has a more friendly interface...both will produce a single trained LightGBM model (class "lgb.Booster").
that lgb.train() does not work with valids = , while lightgbm() does.
This is not correct. Both functions accept the keyword argument valids. Run ?lgb.train and ?lightgbm for documentation on those methods.
How do you apply the output of lgb.cv() to a model?
I'm not sure what you mean, but you can find an example of how to use lgb.cv() in the docs that show up when you run ?lgb.cv.
library(lightgbm)
data(agaricus.train, package = "lightgbm")
train <- agaricus.train
dtrain <- lgb.Dataset(train$data, label = train$label)
params <- list(objective = "regression", metric = "l2")
model <- lgb.cv(
params = params
, data = dtrain
, nrounds = 5L
, nfold = 3L
, min_data = 1L
, learning_rate = 1.0
)
This returns an object of class "lgb.CVBooster". That object has multiple "lgb.Booster" objects in it (the trained models that lightgbm() or lgb.train() produce).
You can extract any one of these from model$boosters. However, in practice I don't recommend using the models from lgb.cv() directly. The goal of cross-validation is to get an estimate of the generalization error for a model. So you can use lgb.cv() to figure out the expected error for a given dataset + set of parameters (by looking at model$record_evals and model$best_score).
Let's say I built a nested model like this:
from keras.models import Sequential, Model
from keras.layers.core import Input, Dense
model_1 = Sequential()
model_1.add(Dense(...))
model_1.add(Dense(...))
input_2 = Input(...)
output_2 = Dense(...)(input_2)
model_2 = Model(inputs=input_2, outputs=output_2)
model = Sequential()
model.add(model_1)
model.add(model_2)
How can I transform this recursively into a "flat" model, that does not contain any Model or Sequential layers.
Since model_1 and model_2 might have been trained in advance the parameters should be conserved during the transformation.
I had a similar problem, and I got a working solution, but this doesn't seem very elegant.
The basic idea is to iterate through the layers of the sub-models and add them to the overall model one by one rather than adding the entire sub-models.
model = Sequential()
for layer1 in model1.layers:
model.add(layer1)
for layer2 in model2.layers:
model.add(layer2)
If the model already includes nested models, it is possible to iterate over them via:
model_flat = Sequential()
for layer_nested in model.get_layer('nested_model').layers:
model_flat.add(layer_nested)
I am using keras package in R to train a deep learning model. My data set is highly imbalanced. Therefore, I want to set class_weight argument in the fit function. Here is the fit function and its arguments that I used for my model
history <- model %>% fit(
trainData, trainClass,
epochs = 5, batch_size = 1000,
class_weight = ????,
validation_split = 0.2
)
In python I can set class_weight as follow:
class_weight={0:1, 1:30}
But I am not sure how to do it in R. In the help menu of R it describes class_weight as follow:
Optional named list mapping indices (integers) to a weight (float) to
apply to the model's loss for the samples from this class during
training. This can be useful to tell the model to "pay more attention"
to samples from an under-represented class.
Any idea or suggestions?
Class_weight needs to be a list, so
history <- model %>% fit(
trainData, trainClass,
epochs = 5, batch_size = 1000,
class_weight = list("0"=1,"1"=30),
validation_split = 0.2
)
seems to work. Keras internally uses a function called as_class_weights to change the list to a python-dictionary (see https://rdrr.io/cran/keras/src/R/model.R).
class_weight <- dict(list('0'=1,'1'=10))
class_weight
>>> {0: 1.0, 1: 10.0}
Looks just like the python dictionary that you mentioned above.
I found a generic solution in Python solution, so I converted into R:
counter=funModeling::freq(Y_data_aux_tr, plot=F) %>% select(var, frequency)
majority=max(counter$frequency)
counter$weight=ceil(majority/counter$frequency)
l_weights=setNames(as.list(counter$weight), counter$var)
Using it:
fit(..., class_weight = l_weights)
An advice if you are using fit_generator: since the weights are based on frequency, having a different number of training-validation samples may bias the validation results. They should be equally-sized.