I'm using Keras with tensorflow backend in R.
I want to create a model that can handle an arbitrary input sequence length.
When I try to define the following model:
model <- keras_model_sequential()
layer_lstm(model, 128, input_shape = c(NULL, 5))
I get the following error:
ValueError: Input 0 is incompatible with layer lstm_3: expected ndim=3, found ndim=2
I guess that it runs into difficulties since the batchsize is already variable. So I could do the following:
model <- keras_model_sequential()
layer_lstm(model, 128, input_shape = c(20, NULL, 5))
This runs without any error. Does this indeed signify a fixed batchsize of 20 a variable sequence length and an input lenght of 5? Or is this just wishfull thinking?
Related
I am trying to train a sequential classifier, with 1 input neuron, 3 output neurons. The data is in data frames X and Y, but how must I feed this data into fit function in keras library? In other words, what should be the variable type of train_x and train_y (for example, is it data frame, matrix, list, etc)?
[...]
predictor <- keras_model_sequential() %>%
layer_dense(units = 8, activation = "relu", input_shape = c(1)) %>%
layer_dense(units = 8, activation = "relu") %>%
layer_dense(units = 3, activation = "softmax")
[...]
train_x <- X
train_y <- Y
history <- predictor %>% fit(
train_x,
train_y,
epochs = 20,
verbose = 2
)
Edit:
If I can use dataframe, then how should I set input_shape?
The variable type for fit should be of vector, matrix, or array.
As per the documentation, it states below,
x -
Vector, matrix, or array of training data (or list if the model has multiple inputs). If all inputs in the model are named, you can also pass a list mapping input names to data. x can be NULL (default) if feeding from framework-native tensors (e.g. TensorFlow data tensors).
y - Vector, matrix, or array of target (label) data (or list if the
model has multiple outputs). If all outputs in the model are named,
you can also pass a list mapping output names to data. y can be NULL
(default) if feeding from framework-native tensors (e.g. TensorFlow
data tensors).
The model needs to know what input shape it should expect. For this reason, the first layer in a sequential model (and only the first, because following layers can do automatic shape inference) needs to receive information about its input shape. Ex: You can pass a batch_size argument to a layer. If you pass both batch_size=32 and input_shape=c(6, 8) to a layer, it will then expect every batch of inputs to have the batch shape (32, 6, 8).
Hope this answers your question. Happy Learning.
I am using R programming language and using Keras API to build a functional 1D CNN.
I have a matrix of my dataset of the following shape rows*features (6000*1024).
The input layer is set using the following code:
input_layer = layer_input(shape = 1024, batch_shape = c(nrow(train_matrix),1024), dtype = 'float64')
and then I am building a 1d conv layer as follows:
conv1 = input_layer %>% layer_conv_1d(filters = 32, kernel_size = 50, strides = 10, input_shape = 1024, batch_input_shape = list(NULL, 1024) ,dtype = 'float64', activation = 'relu' )
But I get the following error:
Error in py_call_impl(callable, dots$args, dots$keywords) :
ValueError: Input 0 is incompatible with layer conv1d: expected ndim=3, found ndim=2
I believe it is due to the fact that 1D cnn layer expects the input in the following form
Input shape: 3D tensor with shape: (batch_size, steps, input_dim)
I understand that I have to reshape my data as (NULL, nrow(train_matrix), 1; as this has been suggested in various answer for the same issue arising for keras when used in Python.
If I am right,
what values should I provide to input layer
how should i reshape my training data?
does that mean I have to reshape the test data as well?
also if my understanding is wrong what should be done otherwise ?
The input layer was producing a 2d tensor, whereas 1d convolutional layer expects a 3d tensor as input. A good explanation of as to why keras expects it and how should you reshape your 2d input to 3d can be found in this answer.
I used Keras R API function k_reshape( 2d_tensor, (list_of_new_dims)) --> k_reshape(input_layer, list(nrow(train_matrix), num_of_feature_vectors, 1).
nrow(train_matrix) - the total number of rows in my matrix (no of samples)
num_of_feature_vectors - total number of columns in matrix (total number of features)
1 - i want a 3d tensor with only 1 element at z axis, therefore z axis is initialised to 1
I am trying to tune the hyperparameters of my fully connected deep learning model using flags and tuning_run in R using the keras package. Where do I find the actual flag value used in each run?
I have tried looking for the hyperparameter values used in both the generated result data frame and the runs/ folder. While all the accuracy values, loss function and other meta details about the runs are there, the hyperparameters for which those results are generated are not included (I followed this example given here: https://tensorflow.rstudio.com/tools/tfruns/articles/tuning.html). I am calling my tuning_run as given below
runs <- tuning_run("test.R", flags = list(dropout1=c(0.5,0.4,0.3),dropout2=c(0.3,0.2),dense_units=c(128,256)),sample=0.3)
and my model consumes the flags like
model <- keras_model_sequential()
model %>%
layer_dense(units = 256, activation = 'relu', input_shape = c(784)) %>%
layer_dropout(rate = FLAGS$dropout_1) %>%
layer_dense(units = FLAGS$dense_units, activation = 'relu') %>%
layer_dropout(rate = FLAGS$dropout_2) %>%
layer_dense(units = 10, activation = 'softmax')
When I run it, and later look for the value of the flags for which a certain validation accuracy is generated for (the runs dataframe) This is what I observe
Data frame: 2 x 25
run_dir eval_loss eval_acc metric_loss metric_acc
1 runs/2019-03-29T00-14-10Z 0.1315 0.9794 0.0075 0.9977
2 runs/2019-03-29T00-10-37Z 0.1326 0.9816 0.0096 0.9973
metric_val_loss metric_val_acc
1 0.1475 0.9794
2 0.1443 0.9794
# ... with 18 more columns:
# samples, validation_samples, batch_size, epochs, epochs_completed,
# metrics, model, loss_function, optimizer, learning_rate, script, start,
# end, completed, output, source_code, context, type
I am wondering where to find the flag values used in each iteration. Or am I doing something wrong? Any help would be appreciated. Thanks!
I found out what the problem was. The flags need to be defined in the target script too for keras to report it. And that was why it wasn't showing the flags in the resulting frame.
Once I added these lines to the test.R it worked fine
FLAGS <- flags(
flag_numeric('dropout_1', 0.04, 'First dropout'),
flag_numeric('dropout_2', 0.3, 'Second dropout'),
flag_integer('dense_units', 128, 'Units in dense layer')
)
The same problem and the solution is discussed here: https://github.com/rstudio/tfruns/issues/24
I am trying to learn LSTM with keras in R. I am not being able to fully understand the conventions used in keras.
I have dataset that looks like below, with the first 3 columns considered as input and the last one as output.
Based on this, I am trying to build a stateless LSTM as follows:
model %>%
layer_lstm(units = 1024, input_shape = c(1, 3), return_sequences = T ) %>%
layer_lstm(units = 1024, return_sequences = F) %>%
# using linear activation on last layer, as output is needed in real number
layer_dense(units = 1, activation = "linear")
model %>% compile(loss = 'mse', optimizer = 'rmsprop')
The model looks like below
Layer (type) Output Shape Param #
=====================================================
lstm_1 (LSTM) (None, 1, 1024) 4210688
_____________________________________________________
lstm_2 (LSTM) (None, 1024) 8392704
_____________________________________________________
dense_3 (Dense) (None, 1) 1025
=====================================================
Total params: 12,604,417
Trainable params: 12,604,417
Non-trainable params: 0
_____________________________________________________
I am trying to train the model as follows:
history <- model %>% fit(dt[,1:3], dt[,4], epochs=50, shuffle=F)
However, i am getting the following error when I try to execute the code.
Error in py_call_impl(callable, dots$args, dots$keywords) :
ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (3653, 3)
Not sure what I am missing here.
Update: After looking around in internet, it seems that I need to reshape the dataset into a 3 dimensional (batchsize, timestep, #features) array. However, I am not using any batch, thus not sure how to reshape my data.
Update on 29.01.2018: This is what worked for me. I used input_shape = c(1, 3) in my first LSTM layer, as I have 3 features and I am not using any batch. Thus, I also ended up reshaping my data using the following function:
reshapeDt <- function(data){ # data is the original train matrix (training dataset)
rows <- nrow(data)
cols <- ncol(data)-1
dt <- array(dim=c(rows, 1, cols))
for(i in 1:rows){
dt[i,1,] <- data[i,1:cols]
}
dt
}
This means that the call to fit looks like below:
model %>% fit(reshapeDt(dt), dt[,4], epochs=50, shuffle=F)
This means that dim(reshapeDt(dt)) returns number_of_rows_in_dt 1 3.
Input shapes for LSTM layers should be (batch, time_steps, features).
You must organize your data to have this shape.
It seems that you have only one sequence, with 6 time steps, and 3 features. So, input_shape=(6,3). You can actually use (None,3) for sequences with variable length.
Your input array dt should have shape (1,length,3).
I am using keras package in R to train a deep learning model. My data set is highly imbalanced. Therefore, I want to set class_weight argument in the fit function. Here is the fit function and its arguments that I used for my model
history <- model %>% fit(
trainData, trainClass,
epochs = 5, batch_size = 1000,
class_weight = ????,
validation_split = 0.2
)
In python I can set class_weight as follow:
class_weight={0:1, 1:30}
But I am not sure how to do it in R. In the help menu of R it describes class_weight as follow:
Optional named list mapping indices (integers) to a weight (float) to
apply to the model's loss for the samples from this class during
training. This can be useful to tell the model to "pay more attention"
to samples from an under-represented class.
Any idea or suggestions?
Class_weight needs to be a list, so
history <- model %>% fit(
trainData, trainClass,
epochs = 5, batch_size = 1000,
class_weight = list("0"=1,"1"=30),
validation_split = 0.2
)
seems to work. Keras internally uses a function called as_class_weights to change the list to a python-dictionary (see https://rdrr.io/cran/keras/src/R/model.R).
class_weight <- dict(list('0'=1,'1'=10))
class_weight
>>> {0: 1.0, 1: 10.0}
Looks just like the python dictionary that you mentioned above.
I found a generic solution in Python solution, so I converted into R:
counter=funModeling::freq(Y_data_aux_tr, plot=F) %>% select(var, frequency)
majority=max(counter$frequency)
counter$weight=ceil(majority/counter$frequency)
l_weights=setNames(as.list(counter$weight), counter$var)
Using it:
fit(..., class_weight = l_weights)
An advice if you are using fit_generator: since the weights are based on frequency, having a different number of training-validation samples may bias the validation results. They should be equally-sized.