This question already has answers here:
Using neuralnet with caret train and adjusting the parameters
(2 answers)
Closed 8 years ago.
So I've read a paper that had used neural networks to model out a dataset which is similar to a dataset I'm currently using. I have 160 descriptor variables that I want to model out for 160 cases (regression modelling). The paper I read used the following parameters:-
'For each split, a model was developed for each of the 10 individual train-test folds. A three layer back-propagation net with 33 input neurons and 16 hidden neurons was used with online weight updates, 0.25 learning rate, and 0.9 momentum. For each fold, learning was conducted from a total of 50 different random initial weight starting points and the network was allowed to iterate through learning epochs until the mean absolute error (MAE) for the validation set reached a minimum. '
Now they used a specialist software called Emergent in order to do this, which is a very specialised neuronal network model software. However, as I've done previous models before in R, I have to keep to it. So I'm using the caret train function in order to do 10 cross fold validation, 10 times with the neuralnet package. I did the following:-
cadets.nn <- train(RT..seconds.~., data = cadet, method = "neuralnet", algorithm = 'backprop', learningrate = 0.25, hidden = 3, trControl = ctrl, linout = TRUE)
I did this to try and tune the parameters as closely to the ones used in the paper, however I get the following error message:-
layer1 layer2 layer3 RMSE Rsquared RMSESD RsquaredSD
1 1 0 0 NaN NaN NA NA
2 3 0 0 NaN NaN NA NA
3 5 0 0 NaN NaN NA NA
Error in train.default(x, y, weights = w, ...) :
final tuning parameters could not be determined
In addition: There were 50 or more warnings (use warnings() to see the first 50)
Do you know what I'm doing wrong? It works when I do nnet, but I can't tune the parameters for that to make it similar to the ones used in the paper I'm trying to mimic.
This is what I get in the warnings() fifty times:-
1: In eval(expr, envir, enclos) :
model fit failed for Fold01.Rep01: layer1=1, layer2=0, layer3=0 Error in neuralnet(form, data = data, hidden = nodes, ...) :
formal argument "hidden" matched by multiple actual arguments
2: In data.frame(..., check.names = FALSE) :
row names were found from a short variable and have been discarded
3: In eval(expr, envir, enclos) :
model fit failed for Fold01.Rep01: layer1=3, layer2=0, layer3=0 Error in neuralnet(form, data = data, hidden = nodes, ...) :
formal argument "hidden" matched by multiple actual arguments
4: In data.frame(..., check.names = FALSE) :
row names were found from a short variable and have been discarded
5: In eval(expr, envir, enclos) :
model fit failed for Fold01.Rep01: layer1=5, layer2=0, layer3=0 Error in neuralnet(form, data = data, hidden = nodes, ...) :
formal argument "hidden" matched by multiple actual arguments
Thanks!
From the error message, the 'hidden' parameter is not being properly matched. Looking at the documentation, there are only three training parameters for method = "neuralnet", layer1, layer2, layer3. Take a look at the link and use a different a method where you can specify the desired parameters.
Related
I have a large dataset (~100k observations) of presence/absence data that I am trying to fit a Hierarchical GAM with individual effects that have a Shared penalty (e.g. 'S' in Pedersen et al. 2019). The data consists of temp as numeric, region (5 groups) as a factor.
Here is a simple version of the model that I am trying to fit.
modS1 <- gam(occurrence ~ s(temp, region), family = binomial,
data = df, method = "REML")
modS2 <- gam(occurrence ~ s(temp, region, k= c(10,4), family = binomial,
data = df, method = "REML")
In the first case I received the following error:
Which I assumed it because k was set too high for region given there are only 5 different regions in the data set.
Error in smooth.construct.tp.smooth.spec(object, dk$data, dk$knots) :
NA/NaN/Inf in foreign function call (arg 1)
In addition: Warning messages:
1: In mean.default(xx) : argument is not numeric or logical: returning NA
2: In Ops.factor(xx, shift[i]) : ‘-’ not meaningful for factors
In the second case I attempt to lower k for region and receive this error:
Error in if (k < M + 1) { : the condition has length > 1
In addition: Warning messages:
1: In mean.default(xx) : argument is not numeric or logical: returning NA
2: In Ops.factor(xx, shift[i]) : ‘-’ not meaningful for factors
I can fit Models G and GI and I from Pedersen et al. 2019 with no issues. It is models GS and S where I run into issues.
If anyone has any insights I would really appreciate it!
The bs = "fs" argument in the code you're using as a guide is important. If we start at the ?s help page and click on the link to the ?smooth.terms help page, we see:
Factor smooth interactions
bs="fs" Smooth factor interactions are often produced using by variables (see gam.models), but a special smoother class (see factor.smooth.interaction) is available for the case in which a smooth is required at each of a large number of factor levels (for example a smooth for each patient in a study), and each smooth should have the same smoothing parameter. The "fs" smoothers are set up to be efficient when used with gamm, and have penalties on each null space component (i.e. they are fully ‘random effects’).
You need to use a smoothing basis appropriate for factors.
Notably, if you take your source code and remove the bs = "fs" argument and attempt to run gam(log(uptake) ∼ s(log(conc), Plant_uo, k=5, m=2), data=CO2, method="REML"), it will produce the same error that you got.
Click here to access the train and test data I used. I m new to SVM. I was trying the svm package in R to train my data which consists of 40 attributes and 39 labels. All attributes are of double type(most of them are 0's or 1's becuase I performed dummy encoding on the categorical attriubutes ) , the class label was of different strings which i later converted to a factor and its now of Integer type.
model=svm(Category~.,data=train1,scale=FALSE)
p1=predict(model,test1,"prob")
This was the result i got once i trained the model using SVM.
Call:
svm(formula = Category ~ ., data = train1, scale = FALSE)
Parameters:
SVM-Type: C-classification
SVM-Kernel: radial
cost: 1
gamma: 0.02564103
Number of Support Vectors: 2230
I used the predict function
Error in predict.svm(model, test1, "prob") :
NAs in foreign function call (arg 1)
In addition: Warning message:
In predict.svm(model, test1, "prob") : NAs introduced by coercion
I'm not understanding why this error is appearing, I checked all attributes of my training data none of them have NA's in them. Please help me with this.
Thanks
I'm assuming you are using the package e1071 (you don't specify which package are you using, and as far as I know there is no package called svm).
The error message is confusing, but the problem is that you are passing "prob" as the 3rd argument, while the function expects a boolean. Try it like this:
require(e1071)
model=svm(Category~.,data=train1, scale=FALSE, probability=TRUE)
p1=predict(model,test1, probability = TRUE)
head(attr(p1, "probabilities"))
This is a sample of the output I get.
WARRANTS OTHER OFFENSES LARCENY/THEFT VEHICLE THEFT VANDALISM NON-CRIMINAL ROBBERY ASSAULT WEAPON LAWS BURGLARY
1 0.04809877 0.1749634 0.2649921 0.02899535 0.03548131 0.1276913 0.02498949 0.08322866 0.01097913 0.03800846
SUSPICIOUS OCC DRUNKENNESS FORGERY/COUNTERFEITING DRUG/NARCOTIC STOLEN PROPERTY SECONDARY CODES TRESPASS MISSING PERSON
1 0.03255891 0.003790755 0.006249521 0.01944938 0.004843043 0.01305858 0.009727582 0.01840337
FRAUD KIDNAPPING RUNAWAY DRIVING UNDER THE INFLUENCE SEX OFFENSES FORCIBLE PROSTITUTION DISORDERLY CONDUCT ARSON
1 0.01884472 0.006089563 0.001378799 0.003289503 0.01071418 0.004562048 0.003107619 0.002124643
FAMILY OFFENSES LIQUOR LAWS BRIBERY EMBEZZLEMENT SUICIDE
1 0.0004787845 0.001669914 0.0007471968 0.0007465053 0.0007374036
Hope it helps.
I am doing a simulation study for sample size calculation for multilevel modeling. I am using R for both simulation and estimation . As posted in this post , confint function was not working and that was due to model mis-specification . As, I have one independent variable (X) at individual level and one independent variable (Z) at group level and since the simulation study assumes a zero correlation between the intercept deviations and X effect deviations across grouping factors , the model need to be specified as :
fit <- lmer(Y~X+Z+X:Z+(X||group),data=sim_data)
Now the confint.merMod function works . But another problem arises . I am checking the effect of sample size for various combination of parameters . But for small group size , I found warning message more than 50 (though it produces a result also) . Some of those are :
Warning messages:
1: In optwrap(optimizer, par = thopt, fn = mkdevfun(rho, ... :
convergence code 3 from bobyqa: bobyqa -- a trust region step failed to reduce q
4: In zeta(shiftpar, start = opt[seqpar1][-w]) :
slightly lower deviances (diff=-1.42109e-14) detected
5: In nextpar(mat, cc, i, delta, lowcut, upcut) :
Last two rows have identical or NA .zeta values: using minstep
7: In zetafun(np, ns) : slightly lower deviances (diff=-1.42109e-14) detected
8: In FUN(X[[i]], ...) : non-monotonic profile
24: In nextpar(mat, cc, i, delta, lowcut, upcut) :
unexpected decrease in profile: using minstep
25: In profile.merMod(object, which = parm, signames = oldNames, ... :
non-monotonic profile for (Intercept)
But there are no warning message , if I increase the number of group size .
In this case , should I worry of the warnings ?
I want to estimate the coefficients for an AR process based on weekly data where the lags occur at t-1, t-52, and t-53. I will naturally lose a year of data to do this.
I currently tried:
lags <- rep(0,54)
lags[1]<- NA
lags[52] <- NA
lags[53] <- NA
testResults <- arima(data,order=c(53,0,0),fixed=lags)
Basically I tried using an ARIMA and shutting off the MA/differencing. I used 0's for the terms I wanted to exclude (plus intercept, and NAs for the terms I wanted.
I get the following error:
Error in optim(init[mask], armafn, method = optim.method, hessian =TRUE, :
non-finite finite-difference value [1]
In addition: Warning message:
In arima(data, order = c(53, 0, 0), fixed = lags) :
some AR parameters were fixed: setting transform.pars = FALSE
I'm hoping there is an easier method or potential solution to this error. I want to avoid creating columns with the lagged variables and simply running a regression. Thanks!
So I've read a paper that had used neural networks to model out a dataset which is similar to a dataset I'm currently using. I have 160 descriptor variables that I want to model out for 160 cases (regression modelling). The paper I read used the following parameters:-
'For each split, a model was developed for each of the 10 individual train-test folds. A three layer back-propagation net with 33 input neurons and 16 hidden neurons was used with online weight updates, 0.25 learning rate, and 0.9 momentum. For each fold, learning was conducted from a total of 50 different random initial weight starting points and the network was allowed to iterate through learning epochs until the mean absolute error (MAE) for the validation set reached a minimum. '
Now they used a specialist software called Emergent in order to do this, which is a very specialised neuronal network model software. However, as I've done previous models before in R, I have to keep to it. So I'm using the caret train function in order to do 10 cross fold validation, 10 times with the neuralnet package. I did the following:-
cadets.nn <- train(RT..seconds.~., data = cadet, method = "neuralnet", algorithm = 'backprop', learningrate = 0.25, hidden = 3, trControl = ctrl, linout = TRUE)
I did this to try and tune the parameters as closely to the ones used in the paper, however I get the following error message:-
layer1 layer2 layer3 RMSE Rsquared RMSESD RsquaredSD
1 1 0 0 NaN NaN NA NA
2 3 0 0 NaN NaN NA NA
3 5 0 0 NaN NaN NA NA
Error in train.default(x, y, weights = w, ...) :
final tuning parameters could not be determined
In addition: There were 50 or more warnings (use warnings() to see the first 50)
Do you know what I'm doing wrong? It works when I do nnet, but I can't tune the parameters for that to make it similar to the ones used in the paper I'm trying to mimic.
This is what I get in the warnings() fifty times:-
1: In eval(expr, envir, enclos) :
model fit failed for Fold01.Rep01: layer1=1, layer2=0, layer3=0 Error in neuralnet(form, data = data, hidden = nodes, ...) :
formal argument "hidden" matched by multiple actual arguments
2: In data.frame(..., check.names = FALSE) :
row names were found from a short variable and have been discarded
3: In eval(expr, envir, enclos) :
model fit failed for Fold01.Rep01: layer1=3, layer2=0, layer3=0 Error in neuralnet(form, data = data, hidden = nodes, ...) :
formal argument "hidden" matched by multiple actual arguments
4: In data.frame(..., check.names = FALSE) :
row names were found from a short variable and have been discarded
5: In eval(expr, envir, enclos) :
model fit failed for Fold01.Rep01: layer1=5, layer2=0, layer3=0 Error in neuralnet(form, data = data, hidden = nodes, ...) :
formal argument "hidden" matched by multiple actual arguments
Thanks!
train sets hidden for you (based on the values given by layer-layer3. You are trying to specify that argument twice, hence:
formal argument "hidden" matched by multiple actual arguments
HTH,
Max
I think for beginners it's not obvious at all that the layer specification cannot be passed directly into the train function.
One must read the documentation very carefully to understand the following passage for ...:
Errors will occur if values for tuning parameters are passed here.
So first, you must realize that the hidden parameter of the neuralnet::neuralnet is defined as a tuning parameter and therefore may not be passed directly to the train function (by ...). You find the tuning parameter definitions by:
getModelInfo("neuralnet")$neuralnet$parameters
parameter class label
1 layer1 numeric #Hidden Units in Layer 1
2 layer2 numeric #Hidden Units in Layer 2
3 layer3 numeric #Hidden Units in Layer 3
Instead, you must pass the hidden layer definition by the tuneGrid parameter - not obvious at all because that is normally reserved for tuning the parameters, not passing them.
So you can define the hidden layers as follows:
tune.grid.neuralnet <- expand.grid(
layer1 = 10,
layer2 = 10,
layer3 = 10
)
and then pass that to the caret::train function call as:
model.neuralnet.caret <- caret::train(
formula.nps,
data = training.set,
method = "neuralnet",
linear.output = TRUE,
tuneGrid = tune.grid.neuralnet, # cannot pass parameter hidden directly!!
metric = "RMSE",
trControl = trainControl(method = "none", seeds = seed)