In JuMP, Julia v1.3.1,
using JuMP, GLPK
function example_basic(n = 4)
model = Model(GLPK.Optimizer)
#variable(model, x1, Bin)
#variable(model, x2, Bin)
#variable(model, C <= 1)
#objective(model, Max, C)
#constraint(model, x1 + x2 <= C)
# if verbose
# print(model)
# end
JuMP.optimize!(model)
obj_value = JuMP.objective_value(model)
# num_results = 1
num_results = result_count(model)
#show num_results
end
Returns 1. I just don't understand the programming behind this result. Because there are clearly 2 optimal solutions for this tiny linear problem:
(x1,x2)=(0,1)=(1,0)
How come JuMP returns 1?
My best guess is that GLPK doesn't support multiple solutions for Integer Programming because the documentation says:
Some solvers support returning multiple solutions. You can check how many solutions are available to query using result_count.
Then I tried another solver: Cbc, but the result was the same. If my issue is that both solvers don't support multiple solutions, how could I have the list of JuMP solvers that do?
Traditionally MIP solvers just return one solution. Most (or even all) free solvers follow this approach and just return one solution. Some commercial solvers have something called a solution pool to hold multiple solutions.
There is a way to enumerate optimal or second best MIP solutions by adding a cut to the problem and resolve. See How to ask for second best solution to a MIP using JuMP.
Related
JuMP provides a special syntax for creating indicator constraints.So, Which one is better, linearizing the indicator constraints and then write a code or using this feature?
In order to constrain the constraint x + y <= 2 to hold when z binary variable a is one:
#variable(model, x)
#variable(model, y)
#variable(model, z, Bin)
#constraint(model, z => {x + y <= 2})
Actually my question is which one is faster and more efficient, To linearize ourselves or to use code?
The answer is problem- and solver-dependent. You should try both approaches and time them to find out which is more efficient for your problem.
Some solvers (e.g., Gurobi) have special support for indicators, in which case it's probably faster to use the indicators directly. If you're using a solver that doesn't have special support for indicators, we convert the indicator constraint to a SOS-I constraint (https://jump.dev/MathOptInterface.jl/stable/submodules/Bridges/reference/#MathOptInterface.Bridges.Constraint.IndicatorSOS1Bridge).
The quality of a big-M type linearization will depend on using domain knowledge to select a good big-M. JuMP doesn't do big-M reformulations automatically.
I have multiple ODE problems that I am solving. where I need the solutions (u) and derivative of the solutions (du). For smaller ODEs it is practical for me to do the following
using DifferentialEquations
function SB(du,u,p,t)
du[1]=#. u[2]
du[2]=#. ((-0.5*u[2]^2)*(3-u[2]/(p[4]))+(1+(1-3*p[7])*u[2]/p[4])*((p[6]-p[5])/p[2]+2*p[1]/(p[2]*p[9]))*(p[9]/u[1])^(3*p[7])-2*p[1]/(p[2]*u[1])-4*p[3]*u[2]/(p[2]*u[1])-(1+u[2]/p[4])*(p[6]-p[5]+p[10]*sin(2*pi*p[8]*t))/p[2]-p[10]*u[1]*cos(2*pi*p[8]*t)*2*pi*p[8]/(p[2]*p[4]))/((1-u[2]/p[4])*u[1]+4*p[3]/(p[2]*p[4]))
end
R0=2e-6
ps=250e3
f=2e6
u0=([R0 0])
tspan=(0,100/f)
p=[0.0725, 998, 1e-3,1481, 0, 1.01e5,7/5,f, R0, ps]
prob = ODEProblem(SB,u0,tspan,p)
#time u = solve(prob,Tsit5(),alg_hints=[:stiff],saveat=0.01/f,reltol=1e-8,abstol=1e-8)
t=u.t
u2=#. ((-0.5*u[2,:]^2)*(3-u[2,:]/(p[4]))+(1+(1-3*p[7])*u[2,:]/p[4])*((p[6]-p[5])/p[2]+2*p[1]/(p[2]*p[9]))*(p[9]/u[1,:])^(3*p[7])-2*p[1]/(p[2]*u[1,:])-4*p[3]*u[2,:]/(p[2]*u[1,:])-(1+u[2,:]/p[4])*(p[6]-p[5]+p[10]*sin(2*pi*p[8]*t))/p[2]-p[10]*u[1,:]*cos(2*pi*p[8]*t)*2*pi*p[8]/(p[2]*p[4]))/((1-u[2,:]/p[4])*u[1,:]+4*p[3]/(p[2]*p[4]))
where u2 is bascially du[2] in the SB function. This quickly becomes impractical as the size of my ODEs grow (>500 coupled ODEs with >500X500 matrices). Is there way to ask DifferentialEquations.jl package (or any other way) to export du[i]s as it is solving the ODEs? I learned that DiffEqSensitivity.jl package is able to provide du/dps to check the sensitivity of the model to p. is there something similar to extract du/dts?
I would use two different components together. First, as you get to really large ODEs, you'll want to only save specific pieces of the solution, or reduced pieces. For this, the SavingCallback is very helpful.
http://diffeq.sciml.ai/latest/features/callback_library#SavingCallback-1
For example, the following solves an ODE and only saves the trace and the norm of the solution at each step:
using DiffEqCallbacks, OrdinaryDiffEq, LinearAlgebra
prob = ODEProblem((du,u,p,t) -> du .= u, rand(4,4), (0.0,1.0))
saved_values = SavedValues(Float64, Tuple{Float64,Float64})
cb = SavingCallback((u,t,integrator)->(tr(u),norm(u)), saved_values)
sol = solve(prob, Tsit5(), callback=cb)
Now you can use that to save what you need. The second piece is to use the integrator to get the derivatives. You can see that get_du! can be used to extract the current (already computed) derivative:
http://diffeq.sciml.ai/latest/basics/integrator#Misc-1
Additionally, you can make use of the interpolation on the integrator. integrator(t,Val{1}) will give the first derivative of the solution at the current t.
#ChrisRackauckas
I do need every time step that I am defining the solver to solve.
get_du!(out,integrator) gives me an array where all the points have the same value.
am I making a
mistake somewhere?
prob = ODEProblem(SB,u0,tspan,p)
Rdot=zeros(50001,2)
u = init(prob,SSPRK22(),dt=1e-9,reltol=1e-8,abstol=1e-8)
solve!(u)
get_du!(Rdot,u)
U=u.sol
basically derivative of the second output (du[2]) has to be equal to u2 defined in my previous post.`
I use extensively the julia's linear equation solver res = X\b. I have to use it millions of times in my program because of parameter variation. This was working ok because I was using small dimensions (up to 30). Now that I want to analyse bigger systems, up to 1000, the linear solver is no longer efficient.
I think there can be a work around. However I must say that sometimes my X matrix is dense, and sometimes is sparse, so I need something that works fine for both cases.
The b vector is a vector with all zeroes, except for one entry which is always 1 (actually it is always the last entry). Moreover, I don't need all the res vector, just the first entry of it.
If your problem is of the form (A - µI)x = b, where µ is a variable parameter and A, b are fixed, you might work with diagonalization.
Let A = PDP° where P° denotes the inverse of P. Then (PDP° - µI)x = b can be transformed to
(D - µI)P°x = P°b,
P°x = P°b / (D - µI),
x = P(P°b / (D - µI)).
(the / operation denotes the division of the respective vector elements by the scalars Dr - µ.)
After you have diagonalized A, computing a solution for any µ reduces to two matrix/vector products, or a single one if you can also precompute P°b.
Numerical instability will show up in the vicinity of the Eigenvalues of A.
Usually when people talk about speeding up linear solvers res = X \ b, it’s for multiple bs. But since your b isn’t changing, and you just keep changing X, none of those tricks apply.
The only way to speed this up, from a mathematical perspective, seems to be to ensure that Julia is picking the fastest solver for X \ b, i.e., if you know X is positive-definite, use Cholesky, etc. Matlab’s flowcharts for how it picks the solver to use for X \ b, for dense and sparse X, are available—most likely Julia implements something close to these flowcharts too, but again, maybe you can find some way to simplify or shortcut it.
All programming-related speedups (multiple threads—while each individual solver is probably already multi-threaded, it may be worth running multiple solvers in parallel when each solver uses fewer threads than cores; #simd if you’re willing to dive into the solvers themselves; OpenCL/CUDA libraries; etc.) then can be applied.
Best approach for efficiency would be to use: JuliaMath/IterativeSolvers.jl. For A * x = b problems, I would recommend x = lsmr(A, b).
Second best alternatives would be to give a bit more information to the compiler: instead of x = inv(A'A) * A' * b, do x = inv(cholfact(A'A)) A' * b if Cholesky decomposition works for you. Otherwise, you could try U, S, Vt = svd(A) and x = Vt' * diagm(sqrt.(S)) * U' * b.
Unsure if x = pinv(A) * b is optimized, but might be slightly more efficient than x = A \ b.
I was wondering if there is a nice way (preferably using JuMP) to get all optimal solutions of a linear program (in case there are multiple optimal solutions).
An example
minimize the statistical distance (Kolmogorov distance) between two probabilities distributions.
min sum_{i=1}^{4} |P[i] - Q[i]| over free variable Q
P = [0.25,0.25,0.25,0.25]
sum_i P[i] = 1
Q[1] + Q[4] = 1
sum_i Q[i] = 1 -> Q[2],Q[3] = 0
Note we can phrase the optimization as a linear program, the objective becomes
min S >= sum_i S[i]
S[i] >= P[i]-Q[i]
S[i] >= Q[i]-P[i]
There is no unique solution to this problem, instead the subspace of optimal solution is spanned by
Q1 = [0.75,0,0,0.25]
Q2 = [0.25,0,0,0.75]
Both have the minimal distance of 0.5,
any convex combination of the these two solution is optimal.
I was wondering if there is a nice way to find all these optimal extreme points (points that span the optimal subspace)?
Why am I interested in this; the points that gives the maximal Bhattacharyya coefficient (concave function), lies somewhere in the middle of the optimal subspace of the statical distance.
So far I`ve tried to find optimal P,Q pairs (refering to example I gave) by making the algorithm favor miniziming the distance between P[i],Q[i], by adding a weight of 1.001 to this term in the sum. It seems to work to some extend, although I can hardly know for sure.
There is an interesting way to enumerate all possible optimal LP solutions (or rather all optimal LP bases) using a standard MIP solver. Basically the algorithm is:
step 1. solve LP/MIP
step 2. if infeasible or if objective starts to deteriorate: stop
step 3. add cuts (constraints) to the model to forbid current optimal solution
step 4. goto step 1
For an example see here.
LP solvers are not designed to enumerate all optimal solutions. Once you know the optimal objective value, you can define the polyhedron containing all optimal solutions and then use a vertex enumeration algorithm to collect the possibly very large set of extreme points of this polyhedron. All optimal solutions are convex combinations of these extreme points. From Julia, you could use the wrapper for cdd.
I don't know about julia, but there is a tool called PPL that you can use to determine all the vertices of the solution polyedron after you solved the linear program.
See my answer here to a similar question:
Find all alternative basic solutions using existing linear-programming tool.
If I have a general function,f(z,a), z and a are both real, and the function f takes on real values for all z except in some interval (z1,z2), where it becomes complex. How do I determine z1 and z2 (which will be in terms of a) using Mathematica (or is this possible)? What are the limitations?
For a test example, consider the function f[z_,a_]=Sqrt[(z-a)(z-2a)]. For real z and a, this takes on real values except in the interval (a,2a), where it becomes imaginary. How do I find this interval in Mathematica?
In general, I'd like to know how one would go about finding it mathematically for a general case. For a function with just two variables like this, it'd probably be straightforward to do a contour plot of the Riemann surface and observe the branch cuts. But what if it is a multivariate function? Is there a general approach that one can take?
What you have appears to be a Riemann surface parametrized by 'a'. Consider the algebraic (or analytic) relation g(a,z)=0 that would be spawned from this branch of a parametrized Riemann surface. In this case it is simply g^2 - (z - a)*(z - 2*a) == 0. More generally it might be obtained using Groebnerbasis, as below (no guarantee this will always work without some amount of user intervention).
grelation = First[GroebnerBasis[g - Sqrt[(z - a)*(z - 2*a)], {x, a, g}]]
Out[472]= 2 a^2 - g^2 - 3 a z + z^2
A necessary condition for the branch points, as functions of the parameter 'a', is that the zero set for 'g' not give a (single valued) function in a neighborhood of such points. This in turn means that the partial derivative of this relation with respect to g vanishes (this is from the implicit function theorem of multivariable calculus). So we find where grelation and its derivative both vanish, and solve for 'z' as a function of 'a'.
Solve[Eliminate[{grelation == 0, D[grelation, g] == 0}, g], z]
Out[481]= {{z -> a}, {z -> 2 a}}
Daniel Lichtblau
Wolfram Research
For polynomial systems (and some class of others), Reduce can do the job.
E.g.
In[1]:= Reduce[Element[{a, z}, Reals]
&& !Element[Sqrt[(z - a) (z - 2 a)], Reals], z]
Out[1]= (a < 0 && 2a < z < a) || (a > 0 && a < z < 2a)
This type of approach also works (often giving very complicated solutions for functions with many branch cuts) for other combinations of elementary functions I checked.
To find the branch cuts (as opposed to the simple class of branch points you're interested in) in general, I don't know of a good approach. The best place to find the detailed conventions that Mathematica uses is at the functions.wolfram site.
I do remember reading a good paper on this a while back... I'll try to find it....
That's right! The easiest approach I've seen for branch cut analysis uses the unwinding number. There's a paper "Reasoning about the elementary functions of complex analysis" about this the the journal "Artificial Intelligence and Symbolic Computation". It and similar papers can be found at one of the authors homepage: http://www.apmaths.uwo.ca/~djeffrey/offprints.html.
For general functions you cannot make Mathematica calculate it.
Even for polynomials, finding an exact answer takes time.
I believe Mathematica uses some sort of quantifier elimination when it uses Reduce,
which takes time.
Without any restrictions on your functions (are they polynomials, continuous, smooth?)
one can easily construct functions which Mathematica cannot simplify further:
f[x_,y_] := Abs[Zeta[y+0.5+x*I]]*I
If this function is real for arbitrary x and any -0.5 < y < 0 or 0<y<0.5,
then you will have found a counterexample to the Riemann zeta conjecture,
and I'm sure Mathematica cannot give a correct answer.