Since lpSolve does not allow to use != for the constraint directions, what is an alternative way to get the same result?
I would like to maximize x1 + x2 with constraints: x1 <= 5 and
x2 != 5
and keep using lpSolve R package.
I've tried using a combination of > < in order to replicate the same behaviour of !=, however I do not obtain the result I expected.
f.obj<-c(1,1)
f.con<-matrix(c(1,0,0,1),nrow=2,ncol=2,byrow=TRUE)
f.dir<-c("<=","!=")
f.rhs<-c(5,5)
lp("max",f.obj,f.con,f.dir,f.rhs)$solution
Since lpSolve does not support !=, I get the error message:
Error in lp("max",f.obj,f.con,f.dir,f.rhs): Unknown constraint direction found
EDIT
I would like to maximize x1 + x2 with constraints: x1 <= 5 and
x2 < 10 and x2 != 9.
So the solution would be 5 and 8.
You can't do that, even in theory, since the resulting constraint set is not closed. It is like trying to minimize x^2 over the set x > 0. For any proposed solution x0 in that set the solution x0/2 is better so there is no optimum.
I would just use x <= 5 as your constraint and if the constraint is not active (i.e. it turns out that x < 5) then you have found the solution; otherwise, there is no solution. If there is no solution you can try x <= 5 - eps for an arbitrarily chosen eps.
ADDED:
If what you intended was that the variables x1 and x2 are integer then
x < 10 and x != 9
is equivalent to
x <= 8
Note that lp has the all.int argument which defaults to FALSE.
ADDED 2:
If you just want to find multiple feasible solutions then if opt is the value of the objective from the first solution rerun adding the constraint (assuming a maximization problem):
objective <= opt - eps
where eps is an arbitrary small constant.
Also note that if the vectors x1 and x2 are two optimal solutions to an LP then since the constraint set is necessarily convex any convex combination of those solutions is also feasible and because the objective is linear all of those convex combinations must also be optimal so if there is more than one optimum then there are an infinite number of such optimal solutions so you can't simply enumerate them.
ADDED 3.
The feasible set of a linear program form a simplex (i.e. a polytope) and at least one vertex must be at the optimal value if such optimal value exists. If there are more than one vertex with the same optimal value then the points on the line connecting them are all optimal values as well. Although there are an infinite number of optimal values in that case there are only a finite number of vertices so you could enumerate them using the vertexenum package. Then evaluate the objective at each one. If there is one vertex whose objective value is greater than all others then that is the optimum. If there are multiple then we know that those plus all convex combinations of those are optimal. This might work if your problem is not too large.
Related
I have a data frame which contains decimal numbers in 5 columns. Try to think it as an excel file. For every column I find the sum. I provide an example below:
The problem is that I want to keep the optimal lines/observations that they will:
Have individual sum as much closer to 0 - for the col1, col2 and col3
AND simultaneously the col4 and col5 to sum (again individually) the closer to (let's say 3).
The example has created with a random sample, so it does not contain negative numbers, but I provided it as an example.
I think that problems of this case are solved via a Solver in R. I would like to find a code that solves the above problem.
I used before the package lpSolve with the function lp to solve linear programming problems with constraints. It will be easy to set up as you don't appear to have many constraints. However, because it is a single objective function, you need to define the objective function probably as the minimum difference of the sum of your 2 branches (read: Cols 1 to 3, and Cols 4 to 5).
Potentially there would be other methods but this one should be rather easy.
Hopefully this will help.
Regards
I think there are two obvious ways of doing this.
I'll write down the mathematical models that make more formal my interpretation of your problem.
Quadratic Formulation
The first is a least squares approach. Let
x(i) = 1 if row i is selected
0 otherwise
Then we can write:
min sum(j, w(j)*d(j)^2)
d(j) = sum(i, a(i,j)*x(i)) - t(j)
d(j) : free variable (can be substituted out if wanted)
where
t(j) : target sum for column j (0 and 3 in your example)
w(j) : weight for column j (choose 1 if there is no good reason to use something else)
a(i,j) : your data matrix (or data frame)
This is called a MIQP model (Mixed Integer Quadratic Programming). There are MIQP solvers available for R.
Linear Formulation
Instead of least squares we can choose to use least absolute deviations. A high-level model can look like:
min sum(j, w(j)*|d(j)|)
d(j) = sum(i, a(i,j)*x(i)) - t(j)
d(j) : free variable (can be substituted out if wanted)
To make this a proper MIP (Mixed Integer Programming) model we need to make everything linear. One possible formulation is:
min sum(j, w(j)*y(j))
d(j) = sum(i, a(i,j)*x(i)) - t(j)
-y(j) <= d(j) <= y(j) (we can write this as two inequalities)
d(j) : free variable
y(j) >= 0 (positive variable)
There are many MIP solvers available for use with R.
I have a Mixed Integer Programming (MIP) problem, currently modelled in Python's PuLP library. My issue is however very generic, syntax doesn't play a role here.
I want to add a constraint to my model that works like this:
if b=1 then x=y
The variable b is a binary variable taking values 0 or 1. x and y are variables that represent the current stock level. x as a continuous variable, y as an integer variable.
I know constraints can only be modelled in the following format:
a*x+c <= y # a, c are constants, x, y variables
I hope there is some workaround how I can model the above described if b then x equals y constraint.
Here are my approaches so far:
b*y <= x
y >= x*b # works in theory, but multiplication of 2 variables is not allowed
For 2 binary variables x and y the following is true:
M*y > x # represents: if x then y (M is a sufficient large constant)
I guess the solution involves a large M constant, maybe even further helper variables.
A little background: I want to model an inventory problem, with continuous stock levels. However, order decisions should only be possible in integer numbers. I therefore need the stock level to be modelled with float numbers. At the point of order (b==1) however in integer.
I hope someone can help here, even if this is rather theoretic than directly coding related. Hints to further resources that might help are also highly appreciated.
b=1 => x=y
can be modeled as:
y-M(1-b) <= x <= y+M(1-b)
I was wondering if there is a nice way (preferably using JuMP) to get all optimal solutions of a linear program (in case there are multiple optimal solutions).
An example
minimize the statistical distance (Kolmogorov distance) between two probabilities distributions.
min sum_{i=1}^{4} |P[i] - Q[i]| over free variable Q
P = [0.25,0.25,0.25,0.25]
sum_i P[i] = 1
Q[1] + Q[4] = 1
sum_i Q[i] = 1 -> Q[2],Q[3] = 0
Note we can phrase the optimization as a linear program, the objective becomes
min S >= sum_i S[i]
S[i] >= P[i]-Q[i]
S[i] >= Q[i]-P[i]
There is no unique solution to this problem, instead the subspace of optimal solution is spanned by
Q1 = [0.75,0,0,0.25]
Q2 = [0.25,0,0,0.75]
Both have the minimal distance of 0.5,
any convex combination of the these two solution is optimal.
I was wondering if there is a nice way to find all these optimal extreme points (points that span the optimal subspace)?
Why am I interested in this; the points that gives the maximal Bhattacharyya coefficient (concave function), lies somewhere in the middle of the optimal subspace of the statical distance.
So far I`ve tried to find optimal P,Q pairs (refering to example I gave) by making the algorithm favor miniziming the distance between P[i],Q[i], by adding a weight of 1.001 to this term in the sum. It seems to work to some extend, although I can hardly know for sure.
There is an interesting way to enumerate all possible optimal LP solutions (or rather all optimal LP bases) using a standard MIP solver. Basically the algorithm is:
step 1. solve LP/MIP
step 2. if infeasible or if objective starts to deteriorate: stop
step 3. add cuts (constraints) to the model to forbid current optimal solution
step 4. goto step 1
For an example see here.
LP solvers are not designed to enumerate all optimal solutions. Once you know the optimal objective value, you can define the polyhedron containing all optimal solutions and then use a vertex enumeration algorithm to collect the possibly very large set of extreme points of this polyhedron. All optimal solutions are convex combinations of these extreme points. From Julia, you could use the wrapper for cdd.
I don't know about julia, but there is a tool called PPL that you can use to determine all the vertices of the solution polyedron after you solved the linear program.
See my answer here to a similar question:
Find all alternative basic solutions using existing linear-programming tool.
Given a set of variables, x's. I want to find the values of coefficients for this equation:
y = a_1*x_1 +... +a_n*x_n + c
where a_1,a_2,...,a_n are all unknowns. Thinking this in perspective of data frame, I want to create this value of y for every rows in the data.
My question is: for y, a_1...a_n and c are all unknown, is there a way for me to find a set of solutions a_1,...,a_n under the condition that corr(y,x_1), corr(y,x_2) .... corr(y,x_n) are all greater than 0.7. For simplicity take correlation here as Pearson correlation. I know there would no be unique solution. But how can I construct a set of solutions for a_1,...,a_n to fulfill this condition?
Spent a day to search the idea but could not get any information out of it. Any programming language to tackle this problem is welcomed or at least some reference for this.
No, it is not possible in general. It may be possible in some special cases.
Given x₁, x₂, ... you want to find y = a₁x₁ + a₂x₂ + ... + c so that all the correlations between y and the x's are greater than some target R. Since the correlation is
Corr(y, xi) = Cov(y, xi) / Sqrt[ Var(y) * Var(xi) ]
your constraint is
Cov(y, xi) / Sqrt[ Var(y) * Var(xi) ] > R
which can be rearranged to
Cov(y, xi)² > R² * Var(y) * Var(xi)
and this needs to be true for all i.
Consider the simple case where there are only two columns x₁ and x₂, and further assume that they both have mean zero (so you can ignore the constant c) and variance 1, and that they are uncorrelated. In that case y = a₁x₁ + a₂x₂ and the covariances and variances are
Cov(y, x₁) = a₁
Cov(y, x₂) = a₂
Var(x₁) = 1
Var(x₂) = 1
Var(y) = (a₁)² + (a₂)²
so you need to simultaneously satisfy
(a₁)² > R² * ((a₁)² + (a₂)²)
(a₂)² > R² * ((a₁)² + (a₂)²)
Adding these inequalities together, you get
(a₁)² + (a₂)² > 2 * R² * ((a₁)² + (a₂)²)
which means that in order to satisfy both of the inequalities, you must have R < Sqrt(1/2) (by cancelling common factors on both sides of the inequality). So the very best you could do in this simple case is to choose a₁ = a₂ (the exact value doesn't matter as long as they are equal) and both of the correlations Corr(y,a₁) and Corr(y,a₂) will be equal to 0.707. You cannot achieve correlations higher than this between y and all of the x's simultaneously in this case.
For the more general case with n columns (each of which has mean zero, variance 1 and zero correlation between columns) you cannot simultaneously achieve correlations greater than 1 / sqrt(n) (as pointed out in the comments by #kazemakase).
In general, the more independent variables there are, the lower the correlation you will be able to achieve between y and the x's. Also (although I haven't mentioned it above) the correlations between the x's matter. If they are in general positively correlated, you will be able to achieve a higher target correlation between y and the x's. If they are in general uncorrelated or negatively correlated, you will only be able to achieve low correlations between y and the x's.
I am not expert in this field so read with extreme prejudice!
I am a bit confused by your y
Your y is a single constant and you want to have the correlation between it and all the x_i values be > 0.7 ? I am no math/statistics expert but my feelings for this are that this is achievable only if the correlation between x_i,x_j upholds the same condition. in that case you can simply do the average of x_i like this:
y=(x_1+x_2+x_3+...+x_n)/n
so the a_i=1.0/n and c=0.0 But still the question is:
What meaning has a correlation between 2 numbers only?
More reasonable would be if y is a function dependent on x
for example like this:
y(x) = a_1*(x-x_1)+... +a_n*(x-x_n) + c
or any other equation (hard to make any without knowing where it came from and for what purpose). Then you can compute the correlation between two sets
X = { x_1 , x_2 ,..., x_n }
Y = { y(x_1),y(x_2),...y(x_n) }
In that case I would give try approximation search for the c,a_i constants to maximize correlation between X,Y, but the results complexity for the whole thing would be insane. So instead I would tweak just one constant. at the time
set some safe c,a_1,a_2,... constants
tweak a_1
so compute correlation for (a_1-delta) and (a_1+delta) and then choose the direction which is in favor of correlation. then keep going in that direction until the correlation coefficient start to drop.
Then you can recursively to this again with smaller delta. Btw this is exactly what my approx class does from the link above.
loop #2 through all the a_i
loop this whole few times to enhance precision
May be you could compute the c after each run to minimize the distance between X,Y sets.
Objective function to be maximized : pos%*%mu where pos is the weights row vector and mu is the column vector of mean returns of d stocks
Constraints: 1) ones%*%pos = 1 where ones is a row vector of 1's of size 1*d (d is the number of stocks)
2) pos%*%cov%*%t(pos) = rb^2 # where cov is the covariance matrix of size d*d and rb is risk budget which is the free parameter whose values will be changed to draw the efficient frontier
I want to write a code for this optimization problem in R but I can't think of any function or library for help.
PS: solve.QP in library quadprog has been used to minimize covariance subject to a target return . Can this function be also used to maximize return subject to a risk budget ? How should I specify the Dmat matrix and dvec vector for this problem ?
EDIT :
library(quadprog)
mu <- matrix(c(0.01,0.02,0.03),3,1)
cov # predefined covariance matrix of size 3*3
pos <- matrix(c(1/3,1/3,1/3),1,3) # random weights vector
edr <- pos%*%mu # expected daily return on portfolio
m1 <- matrix(1,1,3) # constraint no.1 ( sum of weights = 1 )
m2 <- pos%*%cov # constraint no.2
Amat <- rbind(m1,m2)
bvec <- matrix(c(1,0.1),2,1)
solve.QP(Dmat= ,dvec= ,Amat=Amat,bvec=bvec,meq=2)
How should I specify Dmat and dvec ? I want to optimize over pos
Also, I think I have not specified constraint no.2 correctly. It should make the variance of portfolio equal to the risk budget.
(Disclaimer: There may be a better way to do this in R. I am by no means an expert in anything related to R, and I'm making a few assumptions about how R is doing things, notably that you're using an interior-point method. Also, there is likely an R package for what you're trying to do, but I don't know what it is or how to use it.)
Minimising risk subject to a target return is a linearly-constrained problem with a quadratic objective, looking like this:
min x^T Q x
subject to sum x_i = 1
sum ret_i x_i >= target
(and x >= 0 if you want to be long-only).
Maximising return subject to a risk budget is quadratically-constrained, however; it looks like this:
max ret^T x
subject to sum x_i = 1
x^T Q x <= riskbudget
(and maybe x >= 0).
Convex quadratic terms in the objective impose less of a computational cost in an interior-point method compared to introducing a convex quadratic constraint. With a quadratic objective term, the Q matrix just shows up in the augmented system. With a convex quadratic constraint, you need to optimise over a more complicated cone containing a second-order cone factor and you need to be careful about how you solve the linear systems that arise.
I would suggest you use the risk-minimisation formulation repeatedly, doing a binary search on the target parameter until you've found a portfolio approximately maximising return subject to your risk budget. I am suggesting this approach because it is likely sufficient for your needs.
If you really want to solve your problem directly, I would suggest using an interface Todd, Toh, and Tutuncu's SDPT3. This really is overkill; SDPT3 permits you to formulate and solve symmetric cone programs of your choosing. I would also note that portfolio optimisation problems are particularly special cases of symmetric cone programs; other approaches exist that are reportedly very successful. Unfortunately, I'm not studied up on them.