This set of exercises has the student use a QP solver to solve an SVM in R. The suggested solver is the quadprog package. The quadratic problem is given as:
From the remark about the linear SVM, $K=XX'$, $K$ is a singular matrix usually, at most rank $p$ where $X$ is $n\times p$. But the solver quadprog requires a positive definite matrix, not just PSD, in the place of $K$, as mentioned many places (and verified). Any ideas what the instructor had in mind?
I think the workaround would be to add a small number (such as 1e-7) to the diagonal elements of the matrix which is supposed to be positive definite. I am not certain about the math behind it, but the sources below, as well as my experience, suggest that this solution works.
source: https://stats.stackexchange.com/questions/179900/optimizing-a-support-vector-machine-with-quadratic-programming
source: https://teazrq.github.io/stat542/hw/HW6.pdf
Related
I am currently trying to reconstruct some of the function of R's kappa condition number estimation function, which estimates the condition number of a matrix X by:
Working out the QR decomposition of X.
Calling to LAPACK's dtrcon or LINPACK's dtrco (depending on what the underlying dependencies on the users system are), and calculating the condition number of R, the upper triangular matrix which should have the same condition number as X (see here).
I have been trying to understand what the LAPACK and LINPACK algorithms do as it may be extremely useful for my own coding.
I have managed to find the algorithm that LINPACK uses, which was described here, but have had no luck finding the origin of LAPACK's algorithm. The comments in R's kappa function suggest that these are using different algorithms (see here) but I am unsure...
Long story short, my question is:
Does anyone know if LAPACK's dtrcon and LINPACK's dtrco are using the same algorithm and if not, what algorithm is LAPACK's dtrcon using?
Thank you in advance!
I have a mathematical optimization which I wish to solve in R consider this system/problem:
How Can I solve this problem in R?
In this model Budget, p_l for all l and mu_target are fixed constants while muis a given m-dimensional vector and R is a given n by m matrix.
I have looked into constrOptim and lp but I don't have the imagination to implement the constraints
Those functions require that I have a "constraint" matrix but my problem is that I simply don't know how to design that constraint matrix. There are not many examples with decision variables on both sides of the equations.
Have a look on the nloptr package. It has quite extensive documentation with examples. Lots of algorithms to choose from, depending what problem you are trying to resolve.
NLoptr link
Recently, I have constructed a stats model with the negative log-likelihood to be minimized. There are nine parameters to be estimate (in fact I wanna add two more further). Several optimization method in R have been used,including optim,GenSA, DEoptim,Solnp. Then I got a minimum satisfied.
In the next procedure to compute t-value, it is necessary to compute se:
sqrt(diag(solve(hessian)))
However, error occurs due to hessian matrix is not positive semi-definite that negative numbers exist in the main diagonal elements. I have tried optimHess or numericHessian to compute different hessian (the hessians are different) but failed all the same. The work suspends.
This question I think is common in multiple parametric statistics. I ask for help that how should I do in this situation.
There is a paper by Jeff Gill and Gary King discussing this issue. It may help. Essentially, even if theoretically the Hessian should be definite positive at the minimum, because of numerical issues it may not. The paper discusses methods to deal with such matrices.
I have done it in Excel but need to run a proper simulation in R.
I need to minimize function F(x) (x is a vector) while having constraints that sum(x)=1, all values in x are [0,1] and another function G(x) > G_0.
I have tried it with optim and constrOptim. None of them give you this option.
The problem you are referring to is (presumably) a non-linear optimization with non-linear constraints. This is one of the most general optimization problems.
The package I have used for these purposes is called nloptr: see here. From my experience, it is both versatile and fast. You can specify both equality and inequality constaints by setting eval_g_eq and eval_g_ineq, correspondingly. If the jacobians are known explicitly (can be derived analytically), specify them for faster convergence; otherwise, a numerical approximation is used.
Use this list as a general reference to optimization problems.
Write the set of equations using the Lagrange multiplier, then solve using the R command nlm.
You can do this in the OpenMx Package (currently host at the site listed below. Aiming for 2.0 relase on cran this year)
It is a general purpose package mostly used for Structural Equation Modelling, but handling nonlinear constraints.
FOr your case, make an mxModel() with your algebras expressed in mxAlgebras() and the constraints in mxConstraints()
When you mxRun() the model, the algebras will be solved within the constraints, if possible.
http://openmx.psyc.virginia.edu/
I am trying to do some computations using Laplace transforms in R. I used the
continued fractions approach to compute Laplace transform of a birth-death
process as described in Abate 1999. But I cannot find a simple numerical routine to compute the inverse Laplace transform (evaluated at 0 in my case). Does anyone have ideas on how to do this in R?
Computing inverse Laplace transforms numerically is tricky. I remember seeing some relatively recent results on the ACM. Googling around a bit, I found some
Python code implementing one of these algorithms. Maybe you can adapt it to your purposes.