I get an error when i try to run this code. i followed a guide on youtube for building a neural network. Everything works except when i try to run this code to fit the model.
history <- modnn %>% fit(
train_X_matrix, train_Y, epochs = 50, batch_size = 600,
validation_data = list(validation_X_matrix,validation_Y))
```
the error i get when i try to run the code above, all the names you see are the names of the columns. So the features of the model:
[error in visual studio](https://i.stack.imgur.com/ZaPXw.png)
some extra info about the variables i use. Here i created a matrix of the input variables. They did this in the guide. I tried train_x_data as input then it gave the same error but immediately so not after 1 epoch
```
# dependent and independent variables in 1 dataframe
train_X_data <- data.frame(train_X,train_y)
validation_X_data <- data.frame(validation_X,validation_y)
train_X_matrix <- model.matrix(average_daily_rate ~. -1 , data = train_X_data)
train_Y <- train_X_data$average_daily_rate
validation_X_matrix <- model.matrix(average_daily_rate ~. -1, data = validation_X_data)
validation_Y <- validation_X_data$average_daily_rate
```
The model i use, it is just a simple single layer model for testing.
# 1) single layer model structure
# step 1 make architecture powerful enough
modnn <- keras_model_sequential() %>%
layer_dense(units = 500, activation = "relu",
input_shape = ncol(train_X)) %>%
layer_dense(units = 1)
summary(modnn)
modnn %>% compile(loss = "mse",
optimizer = optimizer_rmsprop(),
metrics = list("mean_absolute_error"))
The error occurs after running the first epoch. I tought it was because the model could not read the names of the columns, but i tried a lot of things and nothing seemed to work.
Does anyone have an idea on how to fix this?
Related
I am having trouble extracting the output of an intermediate layer in a Keras NN model. Although there is a ton of resource on this most of them are geared towards for python users. I have tried working towards the tutorial posted here and here. Essentially both of the tutorial says to recreate the original model and then somehow extract the intermediate layers by simply calling it. The issue I am having is model$input returns nothing in my code and I have no idea where layer_name is in a model. After playing around with this method I tried creating a new model to the layer I am interested in and put in the trained weights of the original model in order to get the intermediate layer output. However, I found myself at a lack of resource for R users on how to place weights into a new model. I would appreciate if anyone would help with walking through a concrete example on how to do either method or a better one. Here is my original model:
network <- keras_model_sequential()
network %>%
layer_dropout(rate = 0.1) %>%
layer_dense(units = 64, activation = "relu", input_shape = length(training_data) ) %>%
layer_dense(units = 8, activation = "sigmoid" ) %>%
layer_dense(units = 64, activation = "relu" ) %>%
layer_dense(units = length(training_data), activation = "softmax")
I'm working on testing a model before I let it rip on a full dataset. My data is RGB images in an array, so, my training dataset currently has the dimensions
> dim(ff_train)
[1] 10 500 500 3
So, 10 images, each 500x500 with 3 color layers (RGB).
My test data is the same
> dim(ff_test)
[1] 10 500 500 3
I've setup my model like so:
model <- keras_model_sequential() %>%
layer_dense(units = 16, activation = "relu",
input_shape = c(10)) %>%
layer_dense(units = 16, activation = "relu") %>%
layer_dense(units = 1, activation = "sigmoid")
model %>% compile(
optimizer = "rmsprop",
loss = "binary_crossentropy",
metrics = c("accuracy")
)
history <- model %>% fit(
x = ff_train,
y = ff_train_labels$fraction_yes,
epochs = 20,
validation_data = list(ff_test, ff_test_labels$fraction_yes))
where input shape is 10 as I have 10 images. I also have 10 labels for each which are numbers in a numeric vector between 0 and 1 (fraction of an event occurring in a sample) - both are of length 10.
However, when I run the model, I get the error
Error in py_call_impl(callable, dots$args, dots$keywords) :
ValueError: in user code:
which, after following googling around led me to https://github.com/rstudio/keras/issues/1063 stating that the problem is a mismatch in my dimensions or structure between train and test which.... seems incorrect?
What am I missing here? Where is the dimensional mismatch?
your input is image, so you need to use Convolution layer as first layer, after which you need flatten layer, this will be before Dense layer.
without flatten layer your output is 4 dimension, but you need 2 dimension output.
your output is dimension is 10,500,500,1
but you need 10,10
your final dense layer should have 10 neurons.
I am trying to tune the hyperparameters of my fully connected deep learning model using flags and tuning_run in R using the keras package. Where do I find the actual flag value used in each run?
I have tried looking for the hyperparameter values used in both the generated result data frame and the runs/ folder. While all the accuracy values, loss function and other meta details about the runs are there, the hyperparameters for which those results are generated are not included (I followed this example given here: https://tensorflow.rstudio.com/tools/tfruns/articles/tuning.html). I am calling my tuning_run as given below
runs <- tuning_run("test.R", flags = list(dropout1=c(0.5,0.4,0.3),dropout2=c(0.3,0.2),dense_units=c(128,256)),sample=0.3)
and my model consumes the flags like
model <- keras_model_sequential()
model %>%
layer_dense(units = 256, activation = 'relu', input_shape = c(784)) %>%
layer_dropout(rate = FLAGS$dropout_1) %>%
layer_dense(units = FLAGS$dense_units, activation = 'relu') %>%
layer_dropout(rate = FLAGS$dropout_2) %>%
layer_dense(units = 10, activation = 'softmax')
When I run it, and later look for the value of the flags for which a certain validation accuracy is generated for (the runs dataframe) This is what I observe
Data frame: 2 x 25
run_dir eval_loss eval_acc metric_loss metric_acc
1 runs/2019-03-29T00-14-10Z 0.1315 0.9794 0.0075 0.9977
2 runs/2019-03-29T00-10-37Z 0.1326 0.9816 0.0096 0.9973
metric_val_loss metric_val_acc
1 0.1475 0.9794
2 0.1443 0.9794
# ... with 18 more columns:
# samples, validation_samples, batch_size, epochs, epochs_completed,
# metrics, model, loss_function, optimizer, learning_rate, script, start,
# end, completed, output, source_code, context, type
I am wondering where to find the flag values used in each iteration. Or am I doing something wrong? Any help would be appreciated. Thanks!
I found out what the problem was. The flags need to be defined in the target script too for keras to report it. And that was why it wasn't showing the flags in the resulting frame.
Once I added these lines to the test.R it worked fine
FLAGS <- flags(
flag_numeric('dropout_1', 0.04, 'First dropout'),
flag_numeric('dropout_2', 0.3, 'Second dropout'),
flag_integer('dense_units', 128, 'Units in dense layer')
)
The same problem and the solution is discussed here: https://github.com/rstudio/tfruns/issues/24
I am trying to learn LSTM with keras in R. I am not being able to fully understand the conventions used in keras.
I have dataset that looks like below, with the first 3 columns considered as input and the last one as output.
Based on this, I am trying to build a stateless LSTM as follows:
model %>%
layer_lstm(units = 1024, input_shape = c(1, 3), return_sequences = T ) %>%
layer_lstm(units = 1024, return_sequences = F) %>%
# using linear activation on last layer, as output is needed in real number
layer_dense(units = 1, activation = "linear")
model %>% compile(loss = 'mse', optimizer = 'rmsprop')
The model looks like below
Layer (type) Output Shape Param #
=====================================================
lstm_1 (LSTM) (None, 1, 1024) 4210688
_____________________________________________________
lstm_2 (LSTM) (None, 1024) 8392704
_____________________________________________________
dense_3 (Dense) (None, 1) 1025
=====================================================
Total params: 12,604,417
Trainable params: 12,604,417
Non-trainable params: 0
_____________________________________________________
I am trying to train the model as follows:
history <- model %>% fit(dt[,1:3], dt[,4], epochs=50, shuffle=F)
However, i am getting the following error when I try to execute the code.
Error in py_call_impl(callable, dots$args, dots$keywords) :
ValueError: Error when checking input: expected lstm_1_input to have 3 dimensions, but got array with shape (3653, 3)
Not sure what I am missing here.
Update: After looking around in internet, it seems that I need to reshape the dataset into a 3 dimensional (batchsize, timestep, #features) array. However, I am not using any batch, thus not sure how to reshape my data.
Update on 29.01.2018: This is what worked for me. I used input_shape = c(1, 3) in my first LSTM layer, as I have 3 features and I am not using any batch. Thus, I also ended up reshaping my data using the following function:
reshapeDt <- function(data){ # data is the original train matrix (training dataset)
rows <- nrow(data)
cols <- ncol(data)-1
dt <- array(dim=c(rows, 1, cols))
for(i in 1:rows){
dt[i,1,] <- data[i,1:cols]
}
dt
}
This means that the call to fit looks like below:
model %>% fit(reshapeDt(dt), dt[,4], epochs=50, shuffle=F)
This means that dim(reshapeDt(dt)) returns number_of_rows_in_dt 1 3.
Input shapes for LSTM layers should be (batch, time_steps, features).
You must organize your data to have this shape.
It seems that you have only one sequence, with 6 time steps, and 3 features. So, input_shape=(6,3). You can actually use (None,3) for sequences with variable length.
Your input array dt should have shape (1,length,3).
I am using keras package in R to train a deep learning model. My data set is highly imbalanced. Therefore, I want to set class_weight argument in the fit function. Here is the fit function and its arguments that I used for my model
history <- model %>% fit(
trainData, trainClass,
epochs = 5, batch_size = 1000,
class_weight = ????,
validation_split = 0.2
)
In python I can set class_weight as follow:
class_weight={0:1, 1:30}
But I am not sure how to do it in R. In the help menu of R it describes class_weight as follow:
Optional named list mapping indices (integers) to a weight (float) to
apply to the model's loss for the samples from this class during
training. This can be useful to tell the model to "pay more attention"
to samples from an under-represented class.
Any idea or suggestions?
Class_weight needs to be a list, so
history <- model %>% fit(
trainData, trainClass,
epochs = 5, batch_size = 1000,
class_weight = list("0"=1,"1"=30),
validation_split = 0.2
)
seems to work. Keras internally uses a function called as_class_weights to change the list to a python-dictionary (see https://rdrr.io/cran/keras/src/R/model.R).
class_weight <- dict(list('0'=1,'1'=10))
class_weight
>>> {0: 1.0, 1: 10.0}
Looks just like the python dictionary that you mentioned above.
I found a generic solution in Python solution, so I converted into R:
counter=funModeling::freq(Y_data_aux_tr, plot=F) %>% select(var, frequency)
majority=max(counter$frequency)
counter$weight=ceil(majority/counter$frequency)
l_weights=setNames(as.list(counter$weight), counter$var)
Using it:
fit(..., class_weight = l_weights)
An advice if you are using fit_generator: since the weights are based on frequency, having a different number of training-validation samples may bias the validation results. They should be equally-sized.