I want to combine more types of activations in output layer in Keras interface for R. Also, I want to use different loss functions for different outputs. Lets say I want to have first two neurons linear with MSE loss, second 2 neurons sigmoid with BCE loss and last output will be relu with MAE loss. By now I have this and it is not working:
model <- keras_model_sequential()
model %>% layer_dense(units=120, activation="selu",
input_shape=dim(X)[2]) # this is hidden layer, this works fine
model %>% layer_dense(units=120, activation=as.list(c(rep("linear",2),
rep("sigmoid",2), "relu"))) # output layer which is not working
model %>% compile(loss=as.list(c(rep("mean_squared_error",2),
rep("binary_crossentropy",2), "mean_absolute_error")), # problem here ?
optimizer=optimizer_adam(lr=0.001) ,metrics = "mae")
and after this I fit the model with model %>% fit(...) .
Error is the following:
Error in py_call_impl(callable, dots$args, dots$keywords) :
ValueError: When passing a list as loss, it should have one entry per model outputs.
The model has 1 outputs, but you passed loss=['mean_squared_error', 'mean_squared_error', ...
Any help is appreciated.
EDIT : only rewrited code so that is better readable.
I think that if you want to have multiple outputs you need to use the functional (that is, not the sequential) API - see some examples here: https://keras.rstudio.com/articles/functional_api.html
Related
My question is closely related to this previous one: Simulation-based hypothesis testing on spatial point pattern hyperframes using "envelope" function in spatstat
I have obtained an mppm object by fitting a model on several independent datasets using the mppmfunction from the R package spatstat. How can I study its envelope to compare it to my observations ?
I fitted my model as such:
data <- listof(NMJ1,NMJ2,NMJ3)
data <- hyperframe(X=1:3, Points=data)
model <- mppm(Points ~marks*sqrt(x^2+y^2), data)
where NMJ1, NMJ2, and NMJ3 are marked ppp and are independent realizations of the same experiment.
However, the envelope function does not accept inputs of type mppm:
> envelope(model, Kcross.inhom, nsim=10)
Error in UseMethod("envelope") :
no applicable method for 'envelope' applied to an object of class "c('mppm', 'list')"
The answer provided to the previously mentioned question indicates how to plot global envelopes for each pattern, and to use the product rule for multiple testing. However, my fitted model implies that my 3 ppp objects are statistically equivalent, and are independent realizations of the same experiment (ie no different covariates between them). I would thus like to obtain one single plot comparing my fitted model to my 3 datasets. The following code:
gamma= 1 - 0.95^(1/3)
nsims=round(1/gamma-1)
sims <- simulate(model, nsim=2*nsims)
SIMS <- list()
for(i in 1:nrow(sims)) SIMS[[i]] <- as.solist(sims[i,,drop=TRUE])
Hplus <- cbind(data, hyperframe(Sims=SIMS))
EE1 <- with(Hplus, envelope(Points, Kcross.inhom, nsim=nsims, simulate=Sims))
pool(EE1[1],EE1[2],EE1[3])
leads to the following error:
Error in pool.envelope(`1` = list(r = c(0, 0.78125, 1.5625, 2.34375, 3.125, :
Arguments 2 and 3 do not belong to the class “envelope”
Wrong type of subset index. Use
pool(EE1[[1]], EE1[[2]], EE1[[3]])
or just
pool(EE1)
These would have given an error message that the envelope commands should have been called with savefuns=TRUE. So you just need to change that step as well.
However, statistically this procedure makes little sense. You have already fitted a model, which allows for rigorous statistical inference using anova.mppm and other tools. Instead of this, you are generating simulated data from the fitted model and performing a Monte Carlo test, with all the fraught issues of multiple testing and low power. There are additional problems with this approach - for example, even if the model is "the same" for each row of the hyperframe, the patterns are not statistically equivalent unless the windows of the point patterns are identical, and so on.
I have a database of famous people's voices and I'm currently trying to build a model which predicts whether a voice sounds male or female using the randomForest function. This works fine when I delete the celebrity's name from the dataframe before building the model, however I want to be able to find out which voices the model gets wrong, so I've tried creating the same model without deleting the voice name first. This is the code:
library(randomForest)
# Build a random forest model
rf_model <- randomForest(Gender ~ -VoiceName, data = dataset)
# Compute the accuracy of the random forest on a validation set
validation$pred <- predict(rf_model, validation)
mean(validation$pred == validation$Gender)
However, when I run this, I see the following error:
Error in reformulate(attributes(Terms)$term.labels) :
'termlabels' must be a character vector of length at least one
This makes no sense to me, because each of these voice samples have a VoiceName which returns TRUE when I run is.character(dataset$VoiceName). Does anybody have any tips on what's going wrong here? Thank you.
You need to set a "dot" between ~ and -VoiceName. This means
rf_model <- randomForest(Gender ~. -VoiceName, data = dataset)
I am new to machine learning tools and have installed Keras in R. While considering simpler models, I want to use neural network now for more special purposes. Generally, the neural network should be a function Phi: R^d -> R, where the input is d-dimensional.
Given are d-dimensional data for n time points such that the neural network calculates for each time an 1-dimensional target value. Thereof, I have M samples, i.e. the input is somewhat (samples,times,input_dimension)=(M,n,d), on which the neural network is separately applied. The output should be of the form (samples,times)=(M,n), so that for each time, the predicted value of the neural network is compared with the desired target - and this for every sample. Just for information, the range of the numbers are around d=5, n=1000, M=100.
Based on this, one would suggest to run a "usual" neural network on M*n samples with d-dimensional input and 1-dimensional target. However, the problem is that the loss function depends on the previous evaluations of the neural networks in each time step, i.e. the loss is of the form
l(y_pred,y_target) = sum_{i=1}^n (y^i_pred-y^i_target+f_i(...))^2
where y^i_pred and y^i_target are the predicted and target values of the ith time step, respectively, and f_i is an additional function (that depends on the second derivative of the neural network, but that is another story, and on the previous losses).
So far I have the following code in order to illustrate my problem:
input <- array(data1,dim=c(M,n,d))
target <- array(data2,dim=c(M,n))
myloss <- function(f,y_true,y_pred) {
K <- backend()
return(K$sum((y_pred-y_true+f)^2))
}
library(keras)
NN <- keras_model_sequential()
NN %>%
layer_dense(units=20,activation='relu',input_shape=c(n,d)) %>%
layer_dense(units=20,activation='relu') %>%
layer_dense(units=1,activation='relu')
summary(NN)
NN %>% compile(
loss function(y_true,y_pred) myloss(f,y_true,y_pred),
optimizer = "adam",
metrics = "acc"
)
history <- NN %>% fit(
input, target,
epochs = 30, batch_size = 20,
validation_split = 0.1
)
I get various error messages (concerning dimension of target and custom loss function), therefore my question: Is it even possible to incorporate my problem into a Keras model? Or should I use convolutional neural networks? I looked also at recurrent networks, but in my case, only the loss is dependent on the "previous values". Perhaps somebody can give an advice, I would appreciate your help.
I am new to both Bayes and JAGS, so please forgive my ignorance.
I have been sent an R script (by a colleague) which is written using JAGS code.
The author of this code has defined the set of coda samples as below:
codaSamples = coda.samples( jagsModel , variable.names=parameters ,
n.iter=nPerChain , thin=thinSteps )
I wish to obtain the following, and have had limited success:
Gelman diagnostics: I have used "show(gelman.diag(codaSamples))" which is fine for a single simulation. However, how do I output to a file each of the gelman diagnostics, per parameter, for every simulation of interest? Of more interest, is it possible just to record the proportion of simulations where the Rhat value is >1.1?
Density plot: I have used "show(densplot(codaSamples))". However, this produces each plot on a separate plot (I have 96 parameters in the model). Is there an equivalence to "autocorr.plot", which places several plots per page?
Quantiles: I used "show (summary(codaSamples))", but although this gave the mean, SD, and specific centiles for each parameter (which is what I wanted), it also gave the MCMC matrix. Is there anyway in which to just specify the basic statistical properties for each parameter?
Posterior distribution: Is there a way to calculate the centile at which a given value (say zero), of each parameter lies below/above? Then to summarise across all simulations?
Thank you in advance for any help that you can offer.
record gelman output with something like:
write.csv(gelman.diag(codaSamples)$psrf, file = "gelman.csv")
to plot density of mcmc: mcmc object has a specific S3 class mcmc.list so class(codaSamples) should return mcmc.list. There is a plot S3 method for this class of object with arguments: trace and density.
plot(codaSamples, trace = FALSE, density = TRUE)
That should correspond to your expectations.
summary(codaSamples) gives statistic of posteriors for the parameters. Not sure I understand the question, but codaSamples is a list of length: the number of chains, in columns: your estimeted parameters, and in rows: each iterations of the chain. So you can calculate every statistics you want with this object or sub-object (quantiles, ...).
For example the quantile for the first parameter :
quantile(codaSamples[[1]][,1])
[[1]] because I take the first chain and [,1] for the first parameters.
To look at the structure of an object you need to use str(). With this function you can select all sub-object you want.
I've got the following code:
breast.svr=ksvm(Diagnosis~.,data=breast.train,kernel="rbfdot",C=4)
pred.svr=predict(breast.svr,newdata=breast.test)
tabel <- table(breast.test[,1],pred.svr)/nrow(breast.test)
tabel[1,2] + tabel[2,1]
The result is:
Support Vector Machine object of class "ksvm"
SV type: C-svc (classification)
parameter : cost C = 4
Gaussian Radial Basis kernel function.
Hyperparameter : sigma = 0.149121426557224
Number of Support Vectors : 99
Objective Function Value : -143.4679
Training error : 0.028947
I know that I can extract a lot of information from this model on the following manner:
coef(breast.svr)
But I don't know what to do with it? How can I interpret this? How can I make from this a model like: f(x) = ...? More specific, how can I say which predictor variables that are important?
Kernel SVM is by it's very nature not very intepretable. Each kernel uses many predictor variables so its hard to say which predictor variable is important. If you care about predictability, try to use linear regression or other interpretable models.