I have a problem that I have expressed as the minimization of a convex quadratic program with linear constraints. The problem is that I want to disallow any point that is strictly interior (i.e. I only find the answer useful if it is on a vertex of the feasible region.
I'd like to do this without modifying the objective function. I have already considered several modifications that would make this a non-issue, but they all have the unfortunate result of making the program non-convex.
By my estimation my only option for an efficient solution would be a solver that uses a penalty method to approach a solution from the outside of the feasible region. Does anyone know a decent solver for this?
My current objective function is a sum of parabolic cylinders.
Can you just find the vertices of the feasible region and then take the one which minimizes the objective function? This should just involve a bit of linear algebra and then a limited number of evaluations of the objective function.
Related
I work with Julia, but I think the question is more general. Suppose that one wants to find the spectrum of a very large (sparse) unitary matrix U numerically. As is reported in many entries, diagonalizing by brute force using eigs ends without eigenvalue convergence.
The trick would be then to work with simpler expressions, i.e. with
U_Re = real(U + U')*0.5
U_Im = real((U - U')*-0.5im)
My question is, is there a way to obtain a uniform sampling in finding the eigenvalues? That is, I would like to obtain, say 10e3 eigenvalues for U_Re and U_Im in the interval [-1,1].
I am not entirely sure how uniform sampling of the eigenvalues would work, but I think you are looking for ARPACK. ARPACK would use matrix-vector products to find your eigenvalues, so I am not entirely sure if the Real/Im decomposition is required in this case (hard to say without knowing a lot about the U).
Also, you might want to look at FEAST algorithm, which would benefit a lot from the given search contour.
I am not aware of the existing linking of Julia to those libraries, but I don't think it is a problem since Julia can call C functions.
Here, I gave some brief ideas, and Computational Science might be a better place to find the right crowd. However, a lot more details about U, its sparsity, size, and what does "uniform sampling of eigenvalues in the interval" means would be required.
So, in this question I'd be grateful for hints and further information if I am correct or no.
To calculate the position upon range-measurments to fixed anchors (like GPS) you need to solve the trilateration problem, for example: non-linear least squares, geometrical algorithms or the particle filter, which also is able to solve the trilateration-problem as such.
Due to noise/errors the result might be a jagged line -> you can use the Kalman-Filter to smooth it. So far: Particle - calculation, Kalman - smoothing. Now:
Is it possible to use a Kalman-Filter NOT to smoothen an already existing result, BUT to solve the trilateration as such?
Regarding the particle filter: How to use the particle filter NOT to solve trilateration, BUT to smoothen an already existing result (e.g. calculated with NLLS)?
Best and thank you for any hints, papers, videos, solutions etc.!
The Kalman filter is an optimal solver for linear Gaussian problems. It is often used to solve the trilateration problem (Question 1). To use it in this problem the Jacobian (partial derivative of the range measurement with respect to the position) is linearized at the current position estimate. That process, linearization of the Jacobian, defines the Kalman filter as an Extended Kalman Filter, or EKF in the literature. That works well for GPS because the range to the transmitter is so great that the error in the Jacobian estimate due to position error is small enough to be negligible if the Kalman filter is crudely initialized, for example within 100 km. It breaks down when the 'fixed anchors' are closer to the user. The closer the anchor, the more quickly the line-of-sight vector to the anchor is changing with the position estimate. In these cases Unscented Kalman Filters (UKF) or Particle Filters (PF) are sometimes used instead of an EKF.
The best introduction to the KF and EKF in my view is Applied Optimal Estimation by Gelb. That book has been in print since 1974, and there is a reason why. A discussion of the breakdown of the EKF when the anchor is close can be found in the paper "The Scaled Unscented Transformation" by Julier, which can be found here.
For question 2, the answer is yes, certainly a PF could be used to smooth a solution that is created, for example, by replacing the range measurements with an epoch-by-epoch result from a least-squares solver for the position. I would not recommend the approach. The power of the PF, and the reason we pay the price of computing everything for each particle, is that it handles the non-linearities. To 'pre-linearize' the problem before handing it to the PF defeats its purpose.
I'm having a hard time understanding why it would be useful to use the Taylor series for a function in order to gain an approximation of a function, instead of just using the function itself when programming. If I can tell my computer to compute e^(.1) and it will give me an exact value, why would I take an approximation instead?
Taylor series are generally not used to approximate functions. Usually, some form of minimax polynomial is used.
Taylor series converge slowly (it takes many terms to get the accuracy desired) and are inefficient (they are more accurate near the point around which they are centered and less accurate away from it). The largest use of Taylor series is likely in mathematics classes and papers, where they are useful for examining the properties of functions and for learning about calculus.
To approximate functions, minimax polynomials are often used. A minimax polynomial has the minimum possible maximum error for a particular situation (interval over which a function is to be approximated, degree available for the polynomial). There is usually no analytical solution to finding a minimax polynomial. They are found numerically, using the Remez algorithm. Minimax polynomials can be tailored to suit particular needs, such as minimizing relative error or absolute error, approximating a function over a particular interval, and so on. Minimax polynomials need fewer terms than Taylor series to get acceptable results, and they “spread” the error over the interval instead of being better in the center and worse at the ends.
When you call the exp function to compute ex, you are likely using a minimax polynomial, because somebody has done the work for you and constructed a library routine that evaluates the polynomial. For the most part, the only arithmetic computer processors can do is addition, subtraction, multiplication, and division. So other functions have to be constructed from those operations. The first three give you polynomials, and polynomials are sufficient to approximate many functions, such as sine, cosine, logarithm, and exponentiation (with some additional operations of moving things into and out of the exponent field of floating-point values). Division adds rational functions, which is useful for functions like arctangent.
For two reasons. First and foremost - most processors do not have hardware implementations of complex operations like exponentials, logarithms, etc... In such cases the programming language may provide a library function for computing those - in other words, someone used a taylor series or other approximation for you.
Second, you may have a function that not even the language supports.
I recently wanted to use lookup tables with interpolation to get an angle and then compute the sin() and cos() of that angle. Trouble is that it's a DSP with no floating point and no trigonometric functions so those two functions are really slow (software implementation). Instead I put sin(x) in the table instead of x and then used the taylor series for y=sqrt(1-x*x) to compute the cos(x) from that. This taylor series is accurate over the range I needed with only 5 terms (denominators are all powers of two!) and can be implemented in fixed point using plain C and generates code that is faster than any other approach I could think of.
I have an arbitrary curve (defined by a set of points) and I would like to generate a polynomial that fits that curve to an arbitrary precision. What is the best way to tackle this problem, or is there already a library or online service that performs this task?
Thanks!
If your "arbitrary curve" is described by a set of points (x_i,y_i) where each x_i is unique, and if you mean by "fits" the calculation of the best least-squares polynomial approximation of degree N, you can simply obtain the coefficients b of the polynomial using
b = polyfit(X,Y,N)
where X is the vector of x_i values, Y is the vector of Y_i values. In this way you can increase N until you obtain the accuracy you require. Of course you can achieve zero approximation error by calculating the interpolating polynomial. However, data fitting often requires some thought beforehand - you need to give thought to what you want the approximation to achieve. There are a variety of mathematical ways of assessing approximation error (by using different norms), the choice of which will depend on your requirements of the resulting approximation. There are also many potential pitfalls (such as overfitting) that you may come across and blindly attempting to fit curves may result in an approximation that is theoritically sound but utterly useless to you in practical terms. I would suggest doing a little research on approximation theory if the above method does not meet your requirements, as has been suggested in the comments on your question.
Or rather, what is the definition of a combinatorial algorithm and a linear algorithm, resp.?
To make it clear because obviously the first responders misunderstood the question: I am not looking for a definition of an algorithm running in linear time vs non-linear time. A linear algorithm is somehow related to linear programming, which is a technique for finding or approximating solutions to linear optimization problems.
Since NP-hard problems are so hard, there is a whole field trying to find approximate solutions. The traveling salesman problem for instance has several approximate solutions which run in polynomial time and produce a solution which is within a given bound of the best solution.
Some of these approximating algorithms are called a linear algorithm, others a combinatorial algorithm; and the latter seems to be preferred (Why?). These are the two concepts I would like to understand.
The issue is one of problem formulation.
Just as you said Traveling Salesperson Problem (TSP) is NP-hard precisely because it has a discrete problem formulation (the salesperson either visits a city or not at a particular time). This discrete formulation makes the problem, and it's algorithm, combinatorial. (Note that not all combinatorial problems are NP-hard; consider sorting algorithms.)
However, the Linear-Programming (LP) relaxation of TSP results in a linear algorithm. This is because the problem has been reformulated such that the salesperson visits a city a certain proportion of the time. The main reason for using an LP relaxation is because the relaxed version can be solved in polynomial time. However, the solution to the LP relaxation is not necessarily a solution to the original problem.
A linear algorithm tends to work with just one set of data - 'Take all the numbers in set a, double them, and put the result in set b'. The number of operations is equal to the count of items in set a
A combinatorial one works on combinations of sets - 'For each number in set a, work out the sum of that number and each number in set b and print to screen'. The number of operations is the product of the size of set a and the size of set b.
Combinatorial algorithms "explode" as their input grows. Linear algorithms grows proportional to their input, while combinatorial algorithms grows proportional to an exponent (or worse) or their input: enumerating all possible paths through a graph, for example.