I create this simple code as a demonstration:
using GLPK
using JuMP
m = Model(GLPK.Optimizer)
#variable(m, y[i=1:100], Bin)
#objective(m, Min, sum(y))
#constraint(m, [j=5:50], sum([y[i] for i in j:j+10]) >= 5)
optimize!(m)
Please note this integer program doesn't represent anything, it's only for example. The previous code doesn't output anything while I remembered using Gurobi or even GLPK and Julia JuMP used to output datas about where it is in the current solving process. How many nodes already treated, how long the algorithm has been running, current best bound and so on. Note that it is not related to the size of my integer program as it doesn't output anything too on a bigger program I run with more constraints and variables.
I also tried:
julia> get_optimizer_attribute(m, MOI.Silent())
false
Which is coherent with the following not changing anything:
julia> unset_silent(m)
false
Am I missing something?
I am running Julia 1.5.2, JuMP v0.21.5 and GLPK v0.14.4.
Set the logging level:
julia> set_optimizer_attribute(m, "msg_lev", GLPK.GLP_MSG_ALL)
3
julia> optimize!(m)
GLPK Simplex Optimizer, v4.64
46 rows, 100 columns, 506 non-zeros
67: obj = 2.500000000e+001 inf = 2.000e+001 (5)
74: obj = 2.600000000e+001 inf = 0.000e+000 (0)
* 77: obj = 2.500000000e+001 inf = 0.000e+000 (0)
OPTIMAL LP SOLUTION FOUND
GLPK Integer Optimizer, v4.64
46 rows, 100 columns, 506 non-zeros
100 integer variables, all of which are binary
Integer optimization begins...
+ 77: mip = not found yet >= -inf (1; 0)
+ 77: >>>>> 2.500000000e+001 >= 2.500000000e+001 0.0% (1; 0)
+ 77: mip = 2.500000000e+001 >= tree is empty 0.0% (0; 1)
INTEGER OPTIMAL SOLUTION FOUND
Related
Here is my code in Julia platform and I like to speed it up. Is there anyway that I can make this faster? It takes 0.5 seconds for a dataset of 50k*50k. I was expecting Julia to be a lot faster than this or I am not sure if I am doing a silly implementation.
ar = [[1,2,3,4,5], [2,3,4,5,6,7,8], [4,7,8,9], [9,10], [2,3,4,5]]
SV = rand(10,5)
function h_score_0(ar ,SV)
m = length(ar)
SC = Array{Float64,2}(undef, size(SV, 2), m)
for iter = 1:m
nodes = ar[iter]
for jj = 1:size(SV, 2)
mx = maximum(SV[nodes, jj])
mn = minimum(SV[nodes, jj])
term1 = (mx - mn)^2;
SC[jj, iter] = (term1);
end
end
return score = sum(SC, dims = 1)
end
You have some unnecessary allocations in your code:
mx = maximum(SV[nodes, jj])
mn = minimum(SV[nodes, jj])
Slices allocate, so each line makes a copy of the data here, you're actually copying the data twice, once on each line. You can either make sure to copy only once, or even better: use view, so there is no copy at all (note that view is much faster on Julia v1.5, in case you are using an older version).
SC = Array{Float64,2}(undef, size(SV, 2), m)
And no reason to create a matrix here, and sum over it afterwards, just accumulate while you are iterating:
score[i] += (mx - mn)^2
Here's a function that is >5x as fast on my laptop for the input data you specified:
function h_score_1(ar, SV)
score = zeros(eltype(SV), length(ar))
#inbounds for i in eachindex(ar)
nodes = ar[i]
for j in axes(SV, 2)
SVview = view(SV, nodes, j)
mx = maximum(SVview)
mn = minimum(SVview)
score[i] += (mx - mn)^2
end
end
return score
end
This function outputs a one-dimensional vector instead of a 1xN matrix in your original function.
In principle, this could be even faster if we replace
mx = maximum(SVview)
mn = minimum(SVview)
with
(mn, mx) = extrema(SVview)
which only traverses the vector once, instead of twice. Unfortunately, there is a performance issue with extrema, so it is currently not as fast as separate maximum/minimum calls: https://github.com/JuliaLang/julia/issues/31442
Finally, for absolutely getting the best performance at the cost of brevity, we can avoid creating a view at all and turn the calls to maximum and minimum into a single explicit loop traversal:
function h_score_2(ar, SV)
score = zeros(eltype(SV), length(ar))
#inbounds for i in eachindex(ar)
nodes = ar[i]
for j in axes(SV, 2)
mx, mn = -Inf, +Inf
for node in nodes
x = SV[node, j]
mx = ifelse(x > mx, x, mx)
mn = ifelse(x < mn, x, mn)
end
score[i] += (mx - mn)^2
end
end
return score
end
This also avoids the performance issue that extrema suffers, and looks up the SV element once per node. Although this version is annoying to write, it's substantially faster, even on Julia 1.5 where views are free. Here are some benchmark timings with your test data:
julia> using BenchmarkTools
julia> #btime h_score_0($ar, $SV)
2.344 μs (52 allocations: 6.19 KiB)
1×5 Matrix{Float64}:
1.95458 2.94592 2.79438 0.709745 1.85877
julia> #btime h_score_1($ar, $SV)
392.035 ns (1 allocation: 128 bytes)
5-element Vector{Float64}:
1.9545848011260765
2.9459235098820167
2.794383144368953
0.7097448590904598
1.8587691646610984
julia> #btime h_score_2($ar, $SV)
118.243 ns (1 allocation: 128 bytes)
5-element Vector{Float64}:
1.9545848011260765
2.9459235098820167
2.794383144368953
0.7097448590904598
1.8587691646610984
So explicitly writing out the innermost loop is worth it here, reducing time by another 3x or so. It's annoying that the Julia compiler isn't yet able to generate code this efficient, but it does get smarter with every version. On the other hand, the explicit loop version will be fast forever, so if this code is really performance critical, it's probably worth writing it out like this.
Julia absolute newcomer here with a question for you.
I'm teaching myself some Julia by porting some of my Mathematica and Python code (mostly scientific computations in physics etc.), and seeing what's what.
Until now things have been pretty smooth. And fast. Until now.
Now, I'm simulating an elementary lock-in amplifier, which, in essence, takes a - possibly very complicated - time-dependent signal, Uin(t), and produces an output, Uout(t), phase-locked at some reference frequency fref (that is, it highlights the component of Uin(t), which has a certain phase relationship with a reference sine wave). Little does the description matter, what matters is that it basically does that by calculating the integral (I'm actually omitting the phase here for clarity):
So, I set out and tested this in Mathematica and Julia:
I define a mockup Uin(t), pass some parameter values, and then build an array of Uout(t), at time t = 0, for a range of fref.
Julia: I used the QuadGK package for numerical integration.
T = 0.1
f = 100.
Uin(t) = sin(2pi * f * t) + sin(2pi * 2f *t)
Uout(t, fref)::Float64 = quadgk(s -> Uin(s) * sin(2pi * fref * s), t-T, t, rtol=1E-3)[1]/T
frng = 80.:1.:120.
print(#time broadcast(Uout, 0., frng))
Mathematica
T = 0.1;
f = 100.;
Uin[t_] := Sin[2 π f t] + Sin[2 π 2 f t]
Uout[t_, fref_] := NIntegrate[Sin[2 π fref s] Uin[s], {s, t - T, t}]/T
frng = Table[i, {i, 80, 120, 1}];
Timing[Table[Uout[0, fr], {fr, frng}]]
Results:
Julia timed the operation anywhere between 45 and 55 seconds, on an i7-5xxx laptop on battery power, which is a lot, while Mathematica did it in ~2 seconds. The difference is abysmal and, honestly, hard to believe. I know Mathematica has some pretty sweet and refined algorithms in its kernel, but Julia is Julia. So, question is: what gives?
P.S.: setting f and T as const does reduce Julia's time to ~8-10 seconds, but f and T cannot be consts in the actual program. Other than that, is there something obvious I'm missing?
EDIT Feb 2, 2020:
The slowing down seem to be due to the algorithm trying to hunt down precision when the value is near-zero, e.g. see below: for fref = 95 the calculation takes 1 whole second(!), while for adjacent frequency values it computes instantly (returned result is a tuple of (res, error)). Seems the quadgk function stalls at very small values):
0.000124 seconds (320 allocations: 7.047 KiB)
fref = 94.0 (-0.08637214864144352, 9.21712218998258e-6)
1.016830 seconds (6.67 M allocations: 139.071 MiB, 14.35% gc time)
fref = 95.0 (-6.088184966010742e-16, 1.046186419361636e-16)
0.000124 seconds (280 allocations: 6.297 KiB)
fref = 96.0 (0.1254003757465191, 0.00010132083518769636)
Notes: this is regardless of what tolerance I ask to be produced. Also, Mathematica generally hits machine precision tolerances by default, while somewhat slowing down at near-zeros, and numpy/scipy just fly through the whole thing, but produce less precise results than Mathematica (at default settings; didn't look much into this).
Your problem is related to the choice of error tolerance. Relative error of 1e-3 doesn't sound so bad, but it actually is when the integral is close to zero. In particular, this happens when fref = 80.0 (and 85, 90, 95, not 100, 105, etc.):
julia> Uout(0.0, 80.0, f, T)
1.2104987553880609e-16
To quote from the docstring of quadgk:
(Note that it is useful to specify a positive atol in cases where
norm(I) may be zero.)
Let's try to set an absolute tolerance, for example 1e-6, and compare. First the code (using the code from #ARamirez):
Uin(t, f) = sin(2π * f * t) + sin(4π * f * t)
function Uout(t, fref, f , T)
quadgk(s -> Uin(s, f) * sin(2π * fref * s), t-T, t, rtol=1e-3)[1]/T
end
function Uout_new(t, fref, f , T) # with atol
quadgk(s -> Uin(s, f) * sin(2π * fref * s), t-T, t, rtol=1e-3, atol=1e-6)[1]/T
end
Then the benchmarking (use BenchmarkTools for this)
using BenchmarkTools
T = 0.1
f = 100.0
freqs = 80.0:1.0:120.0
#btime Uout.(0.0, $freqs, $f, $T);
6.302 s (53344283 allocations: 1.09 GiB)
#btime Uout_new.(0.0, $freqs, $f, $T);
1.270 ms (11725 allocations: 262.08 KiB)
OK, that's 5000 times faster. Is that ok?
The first problem I see with your code is that it is type unstable. This is caused because you are using global variables (see Performance Tip number one at Julia Performance Tips) :
The compiler cannot know the types of f and T, which you are using inside your functions, therefore it cannot do an efficient compilation. This is also why when you mark them as const, the performance improves: now the compiler has the guarantee that they will not change their type, so it can efficiently compile your two functions.
How to see that your code is unstable
If you run your first function with the macro #code_warntype like this:
#code_warntype Uin(0.1,f)
You will see an output like this:
julia> #code_warntype Uin(0.1)
Variables
#self#::Core.Compiler.Const(Uin, false)
t::Float64
Body::Any
1 ─ %1 = (2.0 * Main.pi)::Core.Compiler.Const(6.283185307179586, false)
│ %2 = (%1 * Main.f * t)::Any
│ %3 = Main.sin(%2)::Any
│ %4 = (2.0 * Main.pi)::Core.Compiler.Const(6.283185307179586, false)
│ %5 = (2.0 * Main.f)::Any
│ %6 = (%4 * %5 * t)::Any
│ %7 = Main.sin(%6)::Any
│ %8 = (%3 + %7)::Any
└── return %8
All those Anys tell you that the compile doesn't know the type of the output at any step.
How to fix
You can redefine your functions to take in f and T as variables:
Uin(t,f) = sin(2.0pi * f * t) + sin(2.0pi * 2.0f *t)
Uout(t, fref,f,T)::Float64 = quadgk(s -> Uin(s,f) * sin(2pi * fref * s), t-T, t, rtol=1E-3)[1]/T
With these redefinitions, your code runs much faster. If you try to check them with #code_warntype you will see that now the compiler correctly infers the type of everything.
For further performance improvements, you can check out the Julia Performance Tips
In particular, the generally adviced method to measure performance instead of using #time is #btime from the package BenchmarkTools. It is so because when running #time you are measuring also compilation times (another option is to run #time two times - the second measure will be correct since all functions had a chance to compile).
There are various things you can do to speed it up further. Chaning the order of the integration did help a bit, using Float32 instead of Float64 made a small improvement and using #fastmath made a further small improvement. One can also make use of SLEEFPirates.sin_fast
using QuadGK, ChangePrecision
#changeprecision Float32 begin
T = 0.1
f = 100.
#inline #fastmath Uin(t,f) = sin(2pi * f * t) + sin(2pi * 2f *t)
#fastmath Uout(t, fref,f,T) = first(quadgk(s -> Uin(s,f) * sin(2pi * fref * s), t-T, t, rtol=1e-2, order=10))/T
frng = 80.:1.:120.
#time Uout.(0., frng, f, T)
end
The objective function is f(x,y)=sqrt(x^2+2*y^2-xy), subject to 10 > x > 0, 10 > y > 0, x > y. I am going to find the x and y which maximize objective function. I am required to use Nonlinear models in MathProgBase.jl packages. The tutorial from https://mathprogbasejl.readthedocs.io/en/latest/nlp.html is difficult for me to follow since I am a beginner. I really appreciate your help!
It seems that JuMP doesn't support strictly greater/lower constraints (as result that most solver engines doesn't either).
A modellisation in Julia of your problem would be:
using JuMP, Ipopt
m = Model(with_optimizer(Ipopt.Optimizer, print_level=0))
#variable(m, 0 <= x <= 10, start=1)
#variable(m, 0 <= y <= 10, start=1)
#constraint(m, c1, y <= x )
#NLobjective(m, Max, sqrt(x^2+2*y^2-x*y))
optimize!(m)
status = termination_status(m)
if (status == MOI.OPTIMAL || status == MOI.LOCALLY_SOLVED || status == MOI.TIME_LIMIT) && has_values(m)
if (status == MOI.OPTIMAL)
println("** Problem solved correctly **")
else
println("** Problem returned a (possibly suboptimal) solution **")
end
println("- Objective value : ", objective_value(m))
println("- Optimal solutions:")
println("x: $(value.(x))")
println("y: $(value.(y))")
else
println("The model was not solved correctly.")
println(status)
end
(see https://lobianco.org/antonello/personal/blog/2017/0203_jump_for_gams_users for an explanation of the various steps)
That results of the script is:
** Problem returned a (possibly suboptimal) solution **
- Objective value : 14.14213575988668
- Optimal solutions:
x: 10.0
y: 10.0
I was curious how quick and accurate, algorithm from Rosseta code ( https://rosettacode.org/wiki/Ackermann_function ) for (4,2) parameters, could be. But got StackOverflowError.
julia> using Memoize
#memoize ack3(m, n) =
m == 0 ? n + 1 :
n == 0 ? ack3(m-1, 1) :
ack3(m-1, ack3(m, n-1))
# WARNING! Next line has to calculate and print number with 19729 digits!
julia> ack3(4,2) # -> StackOverflowError
# has to be -> 2003529930406846464979072351560255750447825475569751419265016973710894059556311
# ...
# 4717124577965048175856395072895337539755822087777506072339445587895905719156733
EDIT:
Oscar Smith is right that trying ack3(4,2) is unrealistic. This is version translated from Rosseta's C++:
module Ackermann
function ackermann(m::UInt, n::UInt)
function ack(m::UInt, n::BigInt)
if m == 0
return n + 1
elseif m == 1
return n + 2
elseif m == 2
return 3 + 2 * n;
elseif m == 3
return 5 + 8 * (BigInt(2) ^ n - 1)
else
if n == 0
return ack(m - 1, BigInt(1))
else
return ack(m - 1, ack(m, n - 1))
end
end
end
return ack(m, BigInt(n))
end
end
julia> import Ackermann;Ackermann.ackermann(UInt(1),UInt(1));#time(a4_2 = Ackermann.ackermann(UInt(4),UInt(2)));t = "$a4_2"; println("len = $(length(t)) first_digits=$(t[1:20]) last digits=$(t[end-20:end])")
0.000041 seconds (57 allocations: 33.344 KiB)
len = 19729 first_digits=20035299304068464649 last digits=445587895905719156733
Julia itself does not have an internal limit to the stack size, but your operating system does. The exact limits here (and how to change them) will be system dependent. On my Mac (and I assume other POSIX-y systems), I can check and change the stack size of programs that get called by my shell with ulimit:
$ ulimit -s
8192
$ julia -q
julia> f(x) = x > 0 ? f(x-1) : 0 # a simpler recursive function
f (generic function with 1 method)
julia> f(523918)
0
julia> f(523919)
ERROR: StackOverflowError:
Stacktrace:
[1] f(::Int64) at ./REPL[1]:1 (repeats 80000 times)
$ ulimit -s 16384
$ julia -q
julia> f(x) = x > 0 ? f(x-1) : 0
f (generic function with 1 method)
julia> f(1048206)
0
julia> f(1048207)
ERROR: StackOverflowError:
Stacktrace:
[1] f(::Int64) at ./REPL[1]:1 (repeats 80000 times)
I believe the exact number of recursive calls that will fit on your stack will depend upon both your system and the complexity of the function itself (that is, how much each recursive call needs to store on the stack). This is the bare minimum. I have no idea how big you'd need to make the stack limit in order to compute that Ackermann function.
Note that I doubled the stack size and it more than doubled the number of recursive calls — this is because of a constant overhead:
julia> log2(523918)
18.998981503278365
julia> 2^19 - 523918
370
julia> log2(1048206)
19.99949084151746
julia> 2^20 - 1048206
370
Just fyi, even if you change the max recursion depth, you won't get the right answer as Julia uses 64 bit integers, so integer overflow with make stuff not work. To get the right answer, you will have to use big ints to have any hope. The next problem is that you probably don't want to memoize, as almost all of the computations are not repeated, and you will be computing the function more than 10^19729 different inputs, which you really do not want to store.
This is the code:
s=5
k=3
mod= RobustModel(solver=GurobiSolver())
#defUnc(mod,ξ[i=1:k,j=1:s])
adaptive(mod,ext[1:k], policy=Affine, depends_on=ξ[1:k])
#defVar(mod, obj)
#setObjective(mod, Max, obj)
The error is:
UndefVarError: Affine not defined.
Why does this problem happen?
You seem to be missing an # in front of adaptive.
Here is an example of your operations using more recent JuMP macro syntax:
julia> using JuMP, JuMPeR, Gurobi
julia> s = 5
5
julia> k = 3
3
julia> mod = RobustModel(solver=GurobiSolver())
Feasibility problem with:
* 0 linear constraints
* 0 variables
Solver is Gurobi
julia> #uncertain(mod,ξ[i=1:k,j=1:s])
3x5 Array{JuMPeR.Uncertain,2}:
ξ[1,1] ξ[1,2] ξ[1,3] ξ[1,4] ξ[1,5]
ξ[2,1] ξ[2,2] ξ[2,3] ξ[2,4] ξ[2,5]
ξ[3,1] ξ[3,2] ξ[3,3] ξ[3,4] ξ[3,5]
julia> #adaptive(mod,ext[1:k], policy=Affine, depends_on=ξ[1:k])
3-element Array{JuMPeR.Adaptive,1}:
JuMPeR.Adaptive(Feasibility problem with:
* 0 linear constraints
* 0 variables
Solver is Gurobi,1)
JuMPeR.Adaptive(Feasibility problem with:
* 0 linear constraints
* 0 variables
Solver is Gurobi,2)
JuMPeR.Adaptive(Feasibility problem with:
* 0 linear constraints
* 0 variables
Solver is Gurobi,3)
julia> #variable(mod, obj)
obj
julia> #objective(mod, Max, obj)
obj