Algorithm for Solving a Linear Combination? [closed] - math

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I have run into the following problem that I need to solve in a project that I'm working on:
Given some number of vectors v_i (in the math sense), and a target vector H, compute a linear combination of the vectors v_i that most closely matches the target vector H, with the constraint that the coefficients must be in [0, 1].
I do not know much about what kind of algorithms / math should be used to approach such a problem. Any prods in the right general direction would be much appreciated!

It's a constrained least square problem. Basically you want to solve the optimization problem:
argmin ||Ax-H||
x
s.t. 0<=x_j<=1
where x=(x_1, ..., x_j, ..., x_n) consists the coefficients you are seeking, and a column of A corresponds to a vector v_i.

Assuming that you want to solve in the least squares sense, then you have a quadratic programming problem. For example, say that your set of vectors is
x1 = 1 2 3]' x2 = [3 2 1]'
and your target vector is
H = [1 -1 1]'
Then you can create the matrix whose columns are your vectors:
A = [1 3;
2 2;
3 1]
and the thing you are trying to minimize is
norm(A*x - H) = (A*x - H)' * (A*x - H) = x' * (A'*A) * x - (2*H'*A) * x + const
If you define
B = A' * A
C = -2 * H' * A
then you have a problem that can be solved optimally my Matlab's quadprog function
quadprog(B,C,[],[],[],[],0,1)
ans =
0.16667
0.16667
so the optimal solution in this case is
1/6 * x1 + 1/6 * x2 = [2/3, 2/3, 2/3]

This is a combinatorial optimization problem. This kind of problems are NP-hard. But I guess for the binary one, there should be polynomial algorithms that can solve, or there may be some relaxation to get an approximate solution. Some googling on "integer programming" may help.

Related

What is "h" in numerical differentiation? [closed]

Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 6 years ago.
Improve this question
I would like to know what h from the numerical differentiation formulas is and how I can calculate it when I have a function.
I am speaking about this formulas:
f'(x0) = (f(x0 + h) - f(x0)) / h
f'(x0) = (f(x0) - f(x0 - h)) / h
f'(x0) = (f(x0 + h) - f(x0 - h)) / 2*h
I would really appreciate any kind of help!
In such formulae h is usually a "very small number", similar to epsilon in Calculus.
For example, the derivative of f at a is defined as:
Note how h is defined as approaching 0.
When programming, e.g. doing numerical gradient computation, it usually works to set h to something very small - many programming environments have an "epsilon" quantity; lacking that, you can just use a very small floating-point number.
Using the usual 8 byte floats, sensible values for h are 1e-8 for the first and second formula and 1e-5 for the third central difference quotient. This is valid for medium values of x, for larger x one would have to include the scale of x in some way.
In general, for a kth order difference quotient with error order p, the balance between floating point noise and numerical error is reached for h about pow(2e-16, 1.0/(p+k)).

Reformulating a quadratic program suitable for R

My problem is one which should be quite common in statistical inference:
min{(P - k)'S(P - k)} subject to k >= 0
So my choice variable is k, a 3x1 vector. The 3x1 vector P and 3x3 matrix S are known. Is it possible to reformulate this problem so I can use R's solve.QP quadratic programming solver? This solver requires the problem to be in the form
min{-d'b + 0.5 b' D b} subject to A'b >= b_0.
So here the choice vector is is b. Is there a way I can make my problem fit into solve.QP? Thanks so much for any help.

Determine vector equation from parametric equations [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I'm really struggling with a problem at the moment. The problem is to find the equation for the plane that passes through the point (1,1,1) and is parallel to x - 3y - 2z - 4 = 0.
I have determined the direction vector as (1, -3, -2) and then I calculated the parametric equations as these:
x(t) = t + 1
y(t) = 1 - 3t
z(t) = 1 - 2t
But now I can't figure out how to determine the equation of the plane from those equations. Any help would be greatly appreciated! Thanks in advance.
This is a math question, not a programming question. You should ask it on math.stackexchange.com.
But since it's so simple, I'll just tell you how to solve it here.
You started with the plane equation x - 3y - 2z - 4 = 0. Any plane with the equation x - 3y - 2z + C = 0 (for any real number C) is parallel to your original plane.
So if you want the equation of a parallel plane passing through (1,1,1), just plug (1,1,1) into to the equation with C and then solve for C.
1 - 3*1 - 2*1 - C = 0
C = 3 + 2 - 1 = 4
So the equation of the parallel plane is x - 3y - 2z + 4 = 0.

Finding Multiplier Matrix [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 10 years ago.
Improve this question
I am trying to find unknown matrix multiply matrix with knowing matrix
A*c=b
where b is defined vector, A is defined matrix 8x8, c is unknown vector.
I know, I can not divide matrix but what is the solution for this situation ??
This is basically a system of simultaneous linear equations. You can solve it using Gaussian elimination.
As for matrix "division", what you really have in mind is an inverse matrix, i.e. a matrix A-1 such that
AA-1=A-1A=I
where I is the identity matrix. If A is invertible then A*c=b is equivalent to c=A-1b.
The answer by Adam is certainly correct, but you should know that calculating the inverse of the matrix might not be the best solution.
Another to look into is LU decomposition and forward-back substitution. It will be more computationally stable that full Gaussian elimination and calculating the inverse.
You solve the problem in steps like this:
Decompose A = LU; now you'll have LUc = b. L is lower triangular; U is upper triangular.
Let y = Uc; solve Ly = b for y.
Now that you have y, solve for the c vector you want: y = Uc.

Solving polynomial equation system possibly having infinite solutions?

I have to solve polynomial equation system which gives error as it has infinite solutions and i just require few solutions(any 2 or 3) so how can i get them? , Can i specify condition on solution like solutions whose values range between 1 to 10 so that i can get few value.
Equations are actually long complicated but infinite solutions are due to "sin(0)" at root.
You can try to add additional equations to the system, like x1 = 0, x2 = 0 etc., to restrict the number of possible solutions.
Which definition of a solution are you meaning here: That a given function has a value of zero for certain inputs or that a given system of multiple equations overlap in multiple points? The latter could be described as 2 planes intersecting on a line but this isn't necessarily what people may think of when they picture solving a polynomial equation system.
For example: x^2 =4 has only 2 solutions, but x^2=y^2 may have infinitely many solutions as x=y and x=-y are both lines that define where that equality would hold, yet both can be considered polynomial equations to my mind.
I presume you have read through things like SOLUTION OF EQUATIONS USING MATLAB, MATLAB Programming/Symbolic Toolbox, and Solving non linear equations, right? Those may have some ideas for how to use Matlab to do that.
In Mathematica you could use FindInstance to find one or more solutions to your equations. Here's how to get 2 solutions of a particular set of equations:
In[2]:= FindInstance[
x^2 + y^2 + z^2 == -1 && z^2 == 2 x - 5 y, {x, y, z}, 2]
Out[2]= {{x -> -(46/5) - (6 I)/5,
y -> 1/10 (25 - Sqrt[-5955 - 1968 I]),
z -> -Sqrt[1/10 ((-309 - 24 I) + 5 Sqrt[-5955 - 1968 I])]}, {x ->
11/5 - (43 I)/5, y -> 1/10 (25 - Sqrt[6997 + 5504 I]),
z -> Sqrt[(1/5 - I/10) ((2 - 85 I) + (2 + I) Sqrt[6997 + 5504 I])]}}
You can also give inequalities like 1 < var < 10 to FindInstance or to Reduce to further restrict possible solutions, as you suggested.
Mathematica's FindRoot function will give you the closest solution to a given value, so you can use FindRoot a few times with various inputs.
Any other mathematical program should have something similar, it just happens that I'm most familiar with Mathematica at the moment.
If you solve a system of the form f(x)=0 numerically, you can use FMINCON to add constraints. For example, you can specify that the solution should be between 1 and 10.
In a similar manner to Jonas, if you solve for f(x) = 0 numerically in Matlab, you can use fsolve. Should the polynomial have many potential outputs, you may well be able to iterate towards these from different initial points.
Beware local minima in your solution space though, they can be a serious problem for iterative solutions as they guide your algorithm to an answer that is not actually correct.
If the system is big or there are many solutions (isolated or high dimension components) you can use packages like HOM4PS2. If the system is (extremely) small you can solve it symbolically by finding the so called Grobner's basis, which gives you a equivalent (but different) set of polynomials whose solutions are almost obvious. Both Maple and Mathematica 7 can do this.
Mathematica provides quite a few features to help you. For example:
Plot3D[{0, x^2 - y^2}, {x, -1, 1}, {y, -1, 1},
PlotStyle -> {Red, Green}]
a = ToRules#Reduce[x^2 - y^2 == 0, {x, y}];
Plot[Evaluate#({x, y} /. {a}), {x, -1, 1}]

Resources