I am looking for an extension to the r-functions DEoptim() and SCEoptim() from
the packages DEoptim and SCEoptim, respectively.
The extension could be in an altered package or perhaps a guide how to calculate/program a Variance-Covariance Matrix from parameters results of global optimisers.
On a sidenote: I am a little surprised not to find the "standard" methods of e.g. optim() in the earlier mentioned optimisers.
you can also use the subplex package/function. Which gives you the hessian in the output list.
Related
I try to optimize the averaged prediction of two logistic regressions in a classification task using a superlearner.
My measure of interest is classif.auc
The mlr3 help file tells me (?mlr_learners_avg)
Predictions are averaged using weights (in order of appearance in the
data) which are optimized using nonlinear optimization from the
package "nloptr" for a measure provided in measure (defaults to
classif.acc for LearnerClassifAvg and regr.mse for LearnerRegrAvg).
Learned weights can be obtained from $model. Using non-linear
optimization is implemented in the SuperLearner R package. For a more
detailed analysis the reader is referred to LeDell (2015).
I have two questions regarding this information:
When I look at the source code I think LearnerClassifAvg$new() defaults to "classif.ce", is that true?
I think I could set it to classif.auc with param_set$values <- list(measure="classif.auc",optimizer="nloptr",log_level="warn")
The help file refers to the SuperLearner package and LeDell 2015. As I understand it correctly, the proposed "AUC-Maximizing Ensembles through Metalearning" solution from the paper above is, however, not impelemented in mlr3? Or do I miss something? Could this solution be applied in mlr3? In the mlr3 book I found a paragraph regarding calling an external optimization function, would that be possible for SuperLearner?
As far as I understand it, LeDell2015 proposes and evaluate a general strategy that optimizes AUC as a black-box function by learning optimal weights. They do not really propose a best strategy or any concrete defaults so I looked into the defaults of the SuperLearner package's AUC optimization strategy.
Assuming I understood the paper correctly:
The LearnerClassifAvg basically implements what is proposed in LeDell2015 namely, it optimizes the weights for any metric using non-linear optimization. LeDell2015 focus on the special case of optimizing AUC. As you rightly pointed out, by setting the measure to "classif.auc" you get a meta-learner that optimizes AUC. The default with respect to which optimization routine is used deviates between mlr3pipelines and the SuperLearner package, where we use NLOPT_LN_COBYLA and SuperLearner ... uses the Nelder-Mead method via the optim function to minimize rank loss (from the documentation).
So in order to get exactly the same behaviour, you would need to implement a Nelder-Mead bbotk::Optimizer similar to here that simply wraps stats::optim with method Nelder-Mead and carefully compare settings and stopping criteria. I am fairly confident that NLOPT_LN_COBYLA delivers somewhat comparable results, LeDell2015 has a comparison of the different optimizers for further reference.
Thanks for spotting the error in the documentation. I agree, that the description is a little unclear and I will try to improve this!
I'm writing my own software package for a certain type of mixed models in R and would like an easy and efficient way to form design matrices. The fixed part ("X") is not a problem, I use model.matrix, or sparse.model.matrix from the Matrix package if I want it sparse.
I'm looking for an equivalent function, preferably in a well tested package, to create the random effects design matrix ("Z").
For example, if I fit an lmer / glmer model using the lme4 package, I can get the desired matrix by calling getME(fitted_object, "Zt"), but I don't want to fit a model just to get the "Zt" or Z matrix. Ideally, I would like to call (e.g) model.matrix.random(formula) and get it directly, for any valid formula.
In response to a discussion in the comments, ideas about how such a function could be programmed are also welcome, but I was hoping there would be a well tested implementation already. I tried looking through the source code for lme4 to see how they do it but wasn't able to find the relevant code.
I am working on estimating a non-linear function. I am using the optim and Rsolonp packages to do so and I am able to obtain the standard errors of my estimated parameters by using the Hessian that these functions calculate. As my problem seems to be a quite non-smooth one I found out that I can use the psoptim package. Nevertheless, I cannot figure out how I can obtain a Hessian using this package. Any help would be much appreciated.
I have done it in Excel but need to run a proper simulation in R.
I need to minimize function F(x) (x is a vector) while having constraints that sum(x)=1, all values in x are [0,1] and another function G(x) > G_0.
I have tried it with optim and constrOptim. None of them give you this option.
The problem you are referring to is (presumably) a non-linear optimization with non-linear constraints. This is one of the most general optimization problems.
The package I have used for these purposes is called nloptr: see here. From my experience, it is both versatile and fast. You can specify both equality and inequality constaints by setting eval_g_eq and eval_g_ineq, correspondingly. If the jacobians are known explicitly (can be derived analytically), specify them for faster convergence; otherwise, a numerical approximation is used.
Use this list as a general reference to optimization problems.
Write the set of equations using the Lagrange multiplier, then solve using the R command nlm.
You can do this in the OpenMx Package (currently host at the site listed below. Aiming for 2.0 relase on cran this year)
It is a general purpose package mostly used for Structural Equation Modelling, but handling nonlinear constraints.
FOr your case, make an mxModel() with your algebras expressed in mxAlgebras() and the constraints in mxConstraints()
When you mxRun() the model, the algebras will be solved within the constraints, if possible.
http://openmx.psyc.virginia.edu/
Does Matlab have an equivalent to nlminb in R?
I realize that lsqcurvefit is available in Matlab, but I specifically want a function that uses a derivative-based method, ideally exactly the same one as nlminb uses.
nlminb is described in this Stats.StackExhange.com answer.
I do not want to use the 'trust-region-refelective' method emplyed by lsqcurvefit for constrained problems.
Matlab's fmincon uses Quasi-Newton methods with constraints if the appropriate 'Algorithm' option is specified. Apparently R's nlminb is based on the L-BFGS-B code. Using the 'interior-point' algorithm this method of approximating the Hessian can be specified:
options = optimoptions('fmincon','Algorithm','interior-point','Hessian','lbfgs');
Unless you're running out of memory, the value of using 'lbfgs' over the default 'bfgs' is questionable. Try them all.