How to get the labels of clusters after using autoencoder in R - r

Question

I'm a beginner in machine learning. I'm doing clustering using autoencoder in R (h2o package). For now, I've done the following codes:
`mydata = h2o.importFile(path = mfile)
NN_model = h2o.deeplearning(
x = 2:9,
training_frame = mydata,
hidden = c(2),
epochs = 100,
activation = "Tanh",
autoencoder = TRUE
)
train_supervised_features = h2o.deepfeatures(NN_model, mydata, layer=1)`
For my data, there are not too many columns (as you can see in the codes, only 8 variables now), but lots of rows.
Then I plot the 2 nodes in "train_supervised_features" obtained above. And I got the plot like this
It is clear that there are 8 clusters of my data (right?)
My question is, how can I extract the labels from the autoencoder results? I want to add the labels to original data, and plot in colors using these labels, something like this:

Answer 1

Since the autoencoder doesn't have any idea about "clusters", you would have to call h2o.kmeans() on the 2D dataset first, to get the cluster assignments. Then you can score the dataset using that k-means model, using h2o.predict(model, 2Ddata), and you'll get a cluster label for each row.

Answer 2

Basically, AE can be a good DNN for reinstructing the input, however, you can use the latent layer for clustering.
The important steps:
1- For initializing the cluster assignment you may apply simple k-means for your data and label them;
2- Train your AE to learn features;
3- Try to extract the output of the layer before the latent layer which includes the trained space of your data;
4- Cluster the data with a k-means.
I hope it helps,
Here is an example I have provided in Keras:
library(keras)
library(caret)
library(tidyverse)
c(c(xtrain, ytrain), c(xtest, ytest)) %<-% dataset_mnist()
xtrain = xtrain/255
xtest = xtest/255
input_size = dim(xtrain)[2]*dim(xtrain)[3]
latent_size = 10
print(input_size)
x_train = array_reshape(xtrain, dim=c(dim(xtrain)[1], input_size))
x_test = array_reshape(xtest, dim=c(dim(xtest)[1], input_size))
x <- rbind( x_test, x_train )/255.0
# Encoder
encoder_input = layer_input(shape = input_size)
encoder_output = encoder_input %>%
layer_dense(units=256, activation = "relu") %>%
layer_activation_leaky_relu() %>%
layer_dense(units=latent_size) %>%
layer_activation_leaky_relu()
encoderoder = keras_model(encoder_input, encoder_output)
summary(encoderoder)
# Decoder
decoder_input = layer_input(shape = latent_size)
decoder_output = decoder_input %>%
layer_dense(units=256, activation = "relu") %>%
layer_activation_leaky_relu() %>%
layer_dense(units = input_size, activation = "relu") %>%
layer_activation_leaky_relu()
decoderoder = keras_model(decoder_input, decoder_output)
summary(decoderoder)
# Autoencoder
autoencoderoder_input = layer_input(shape = input_size)
autoencoderoder_output = autoencoderoder_input %>%
encoderoder() %>%
decoderoder()
autoencoderoder = keras_model(autoencoderoder_input, autoencoderoder_output)
summary(autoencoderoder)
autoencoderoder %>% compile(optimizer="rmsprop", loss="binary_crossentropy")
autoencoderoder %>% fit(x_train,x_train, epochs=20, batch_size=256)
encoderoded_imgs = encoderoder %>% predict(x_test)
decoderoded_imgs = decoderoder %>% predict(encoderoded_imgs)
# Images plot
pred_images = array_reshape(decoderoded_imgs, dim=c(dim(decoderoded_imgs)[1], 28, 28))
n = 10
op = par(mfrow=c(12,2), mar=c(1,0,0,0))
for (i in 1:n)
{
plot(as.raster(pred_images[i,,]))
plot(as.raster(xtest[i,,]))
}
# Saving trained Net
autoencoderoder_weights <- autoencoderoder %>%
keras::get_weights()
keras::save_model_weights_hdf5(object = autoencoderoder,filepath = '..../autoencoderoder_weights.hdf5',overwrite = TRUE)
encoderoder_model <- keras_model(inputs = encoder_input, outputs = encoderoder$output)
encoderoder_model %>% keras::load_model_weights_hdf5(filepath = "..../autoencoderoder_weights.hdf5",skip_mismatch = TRUE,by_name = TRUE)
encoderoder_model %>% compile(
loss='mean_squared_error',
optimizer='adam',
metrics = c('accuracy')
)
embeded_points <-
encoderoder_model %>%
keras::predict_on_batch(x = x_train)
summary(encoderoder_model)
# Getting layer
layer_name<-"dense_1"
intermediate_layer_model <- keras_model(inputs = encoderoder_model$input, outputs = get_layer(encoderoder_model, layer_name)$output)
intermediate_output <- predict(intermediate_layer_model, x)
# Clustering latent space
km <- stats::kmeans( intermediate_output, centers = 10L, nstart = 20L )
labPrediction <- km$cluster
plot(labPrediction)
# The End
labels are available in "labPrediction" file
For the reference:
https://www.datatechnotes.com/2020/02/how-to-build-simple-autoencoder-with-keras-in-r.html