I can do single variable numeric integration in Julia using quadgk. Some simple examples:
julia> f(x) = cos(x)
f (generic function with 1 method)
julia> quadgk(f, 0, pi)
(8.326672684688674e-17,0.0)
julia> quadgk(f, 0, pi/2)
(1.0,1.1102230246251565e-16)
julia> g(x) = cos(x)^2
g (generic function with 1 method)
julia> quadgk(g, 0, pi/2)
(0.7853981633974483,0.0)
julia> pi/4
0.7853981633974483
The documentation for quadgk doesn't seem to imply an support for multidimensional integration, and sure enough I get an error if I attempt to misuse it for a 2D integral:
julia> quadgk( h, 0, pi/2, 0, pi/2)
ERROR: `h` has no method matching h(::Float64)
The documentation does suggest there are some external packages for integration, but doesn't name them. I'm guessing that one such package can do two dimensional integrals. What's the best such package for this task?
I think you'll want to check out the Cubature package:
https://github.com/stevengj/Cubature.jl
Arguably, quadgk should simply be removed from the standard library because it's limited and just misleads people into not looking for a package to do integration.
In addition to Cubature.jl, there is another Julia package that allows you to compute multidimensional numerical integrals: Cuba.jl (https://github.com/giordano/Cuba.jl). You can install it using package manager:
Pkg.add("Cuba")
The complete documentation of the package is available at https://cubajl.readthedocs.org (also in PDF version)
Disclaimer: I'm the author of the package.
Cuba.jl is simply a Julia wrapper around Cuba Library, by Thomas Hahn, and provides four independent algorithms to calculate integrals: Vegas, Suave, Divonne, Cuhre.
The integral of cos(x) in the domain [0, 1] can be computed with one of the following commands:
Vegas((x,f)->f[1]=cos(x[1]), 1, 1)
Suave((x,f)->f[1]=cos(x[1]), 1, 1)
Divonne((x,f)->f[1]=cos(x[1]), 1, 1)
Cuhre((x,f)->f[1]=cos(x[1]), 1, 1)
As a more advanced example, the integral
where Ω = [0, 1]³ and
can be computed with the following Julia script:
using Cuba
function integrand(x, f)
f[1] = sin(x[1])*cos(x[2])*exp(x[3])
f[2] = exp(-(x[1]^2 + x[2]^2 + x[3]^2))
f[3] = 1/(1 - x[1]*x[2]*x[3])
end
result = Cuhre(integrand, 3, 3, epsabs=1e-12, epsrel=1e-10)
answer = [(e-1)*(1-cos(1))*sin(1), (sqrt(pi)*erf(1)/2)^3, zeta(3)]
for i = 1:3
println("Component $i")
println(" Result of Cuba: ", result[1][i], " ± ", result[2][i])
println(" Exact result: ", answer[i])
println(" Actual error: ", abs(result[1][i] - answer[i]))
end
which gives the following output
Component 1
Result of Cuba: 0.6646696797813739 ± 1.0050367631018485e-13
Exact result: 0.6646696797813771
Actual error: 3.219646771412954e-15
Component 2
Result of Cuba: 0.4165383858806454 ± 2.932866749838454e-11
Exact result: 0.41653838588663805
Actual error: 5.9926508200192075e-12
Component 3
Result of Cuba: 1.2020569031649702 ± 1.1958522385908214e-10
Exact result: 1.2020569031595951
Actual error: 5.375033751420233e-12
You can try the HCubature.jl package:
using HCubature
# Integrating cos(x) between 1.0 and 2.0
hcubature(x -> cos(x[1]), [1.0], [2.0])
# Integrating cos(x1)sin(x2) with domains of [1.0,2.0] for x1 and [1.1,3.0] for x2
hcubature(x -> cos(x[1]) * sin(x[2]), [1.0, 1.1], [2.0, 3.0])
Related
I am a newbie to Julia and Flux with some experience in Tensorflow Keras and python. I tried to use the Flux.withgradient command to write a user-defined training function with more flexibility. Here is the training part of my code:
loss, grad = Flux.withgradient(modelDQN.evalParameters) do
qEval = modelDQN.evalModel(evalInput)
Flux.mse(qEval, qTarget)
end
Flux.update!(modelDQN.optimizer, modelDQN.evalParameters, grad)
This code works just fine. But if I put the command qEval = modelDQN.evalModel(evalInput) outside the do end loop, as follows:
qEval = modelDQN.evalModel(evalInput)
loss, grad = Flux.withgradient(modelDQN.evalParameters) do
Flux.mse(qEval, qTarget)
end
Flux.update!(modelDQN.optimizer, modelDQN.evalParameters, grad)
The model parameters will not be updated. As far as I know, the do end loop works as an anonymous function that takes 0 arguments. Then why do we need the command qEval = modelDQN.evalModel(evalInput) inside the loop to get the model updated?
The short answer is that anything to be differentiated has to happen inside the (anonymous) function which you pass to gradient (or withgradient), because this is very much not a standard function call -- Zygote (Flux's auto-differentiation library) traces its execution to compute the derivative, and can't transform what it can't see.
Longer, this is Zygote's "implicit" mode, which relies on global references to arrays. The simplest use is something like this:
julia> using Zygote
julia> x = [2.0, 3.0];
julia> g = gradient(() -> sum(x .^ 2), Params([x]))
Grads(...)
julia> g[x] # lookup by objectid(x)
2-element Vector{Float64}:
4.0
6.0
If you move some of that calculation outside, then you make a new array y with a new objectid. Julia has no memory of where this came from, it is completely unrelated to x. They are ordinary arrays, not a special tracked type.
So if you refer to y in the gradient, Zygote cannot infer how this depends on x:
julia> y = x .^ 2 # calculate this outside of gradient
2-element Vector{Float64}:
4.0
9.0
julia> g2 = gradient(() -> sum(y), Params([x]))
Grads(...)
julia> g2[x] === nothing # represents zero
true
Zygote doesn't have to be used in this way. It also has an "explicit" mode which does not rely on global references. This is perhaps less confusing:
julia> gradient(x1 -> sum(x1 .^ 2), x) # x1 is a local variable
([4.0, 6.0],)
julia> gradient(x1 -> sum(y), x) # sum(y) is obviously indep. x1
(nothing,)
julia> gradient((x1, y1) -> sum(y1), x, y)
(nothing, Fill(1.0, 2))
Flux is in the process of changing to use this second form. On v0.13.9 or later, something like this ought to work:
opt_state = Flux.setup(modelDQN.optimizer, modelDQN) # do this once
loss, grads = Flux.withgradient(modelDQN.model) do m
qEval = m(evalInput) # local variable m
Flux.mse(qEval, qTarget)
end
Flux.update!(opt_state, modelDQN.model, grads[1])
so I wrote a minimum example to show what I'm trying to do. Basically I want to solve a optimization problem with multiple variables. When I try to do this in JuMP I was having issues with my function obj not being able to take a forwardDiff object.
I looked here: and it seemed to do with the function signature :Restricting function signatures while using ForwardDiff in Julia . I did this in my obj function, and for insurance did it in my sub-function as well, but I still get the error
LoadError: MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{JuMP.var"#110#112"{typeof(my_fun)},Float64},Float64,2})
Closest candidates are:
Float64(::Real, ::RoundingMode) where T<:AbstractFloat at rounding.jl:200
Float64(::T) where T<:Number at boot.jl:715
Float64(::Int8) at float.jl:60
This still does not work. I feel like I have the bulk of the code correct, just some weird of type thing going on that I have to clear up so autodifferentiate works...
Any suggestions?
using JuMP
using Ipopt
using LinearAlgebra
function obj(x::Array{<:Real,1})
println(x)
x1 = x[1]
x2 = x[2]
eye= Matrix{Float64}(I, 4, 4)
obj_val = tr(eye-kron(mat_fun(x1),mat_fun(x2)))
println(obj_val)
return obj_val
end
function mat_fun(var::T) where {T<:Real}
eye= Matrix{Float64}(I, 2, 2)
eye[2,2]=var
return eye
end
m = Model(Ipopt.Optimizer)
my_fun(x...) = obj(collect(x))
#variable(m, 0<=x[1:2]<=2.0*pi)
register(m, :my_fun, 2, my_fun; autodiff = true)
#NLobjective(m, Min, my_fun(x...))
optimize!(m)
# retrieve the objective value, corresponding x values and the status
println(JuMP.value.(x))
println(JuMP.objective_value(m))
println(JuMP.termination_status(m))
Use instead
function obj(x::Vector{T}) where {T}
println(x)
x1 = x[1]
x2 = x[2]
eye= Matrix{T}(I, 4, 4)
obj_val = tr(eye-kron(mat_fun(x1),mat_fun(x2)))
println(obj_val)
return obj_val
end
function mat_fun(var::T) where {T}
eye= Matrix{T}(I, 2, 2)
eye[2,2]=var
return eye
end
Essentially, anywhere you see Float64, replace it by the type in the incoming argument.
I found the problem:
in my mat_fun the type of the return had to be "Real" in order for it to propgate through. Before it was Float64, which was not consistent with the fact I guess all types have to be Real with the autodifferentiate. Even though a Float64 is clearly Real, it looks like the inheritence isn't perserved i.e you have to make sure everything that is returned and inputed are type Real.
using JuMP
using Ipopt
using LinearAlgebra
function obj(x::AbstractVector{T}) where {T<:Real}
println(x)
x1 = x[1]
x2 = x[2]
eye= Matrix{Float64}(I, 4, 4)
obj_val = tr(eye-kron(mat_fun(x1),mat_fun(x2)))
#println(obj_val)
return obj_val
end
function mat_fun(var::T) where {T<:Real}
eye= zeros(Real,(2,2))
eye[2,2]=var
return eye
end
m = Model(Ipopt.Optimizer)
my_fun(x...) = obj(collect(x))
#variable(m, 0<=x[1:2]<=2.0*pi)
register(m, :my_fun, 2, my_fun; autodiff = true)
#NLobjective(m, Min, my_fun(x...))
optimize!(m)
# retrieve the objective value, corresponding x values and the status
println(JuMP.value.(x))
println(JuMP.objective_value(m))
println(JuMP.termination_status(m))
I have some problems with JuMP. When I run it, it says:
MethodError: no method matching (::Interpolations.Extrapolation{Float64, 1, ScaledInterpolation{Float64, 1, Interpolations.BSplineInterpolation{Float64, 1, Vector{Float64}, BSpline{Linear{Throw{OnGrid}}}, Tuple{Base.OneTo{Int64}}}, BSpline{Linear{Throw{OnGrid}}}, Tuple{StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}}}}, BSpline{Linear{Throw{OnGrid}}}, Throw{Nothing}})(::AffExpr)
Use square brackets [] for indexing an Array.
thanks!
using JuMP
import Ipopt
β = 0.88
Nb = 1000
δ = 1.5
wage = 1
rate = 1
grid_b = range(0, 5, length = 1000)
w = 5 * (grid_b).^2
w_func = LinearInterpolation(grid_b, w)
choice1 = Model(Ipopt.Optimizer)
#variable(choice1, x >= 0)
#NLobjective(choice1, Max, x^δ/(1-δ) + β * (w_func.((grid_b[3]*(1+rate)+wage-x) * 3)))
optimize!(choice1)
If I try to run your code, I get
ERROR: UndefVarError: LinearInterpolation not defined
which package is that from? Also, what version of JuMP are you using?
You can’t use arbitrary functions in JuMP. You need to use a user-defined function:
https://jump.dev/JuMP.jl/stable/manual/nlp/#User-defined-Functions
But this will only work if it’s possible to automatically differentiate the function. I don’t know if that works with Interpolations.jl.
p.s. Please provide a link when posting the same question in multiple places: https://discourse.julialang.org/t/methoderror-no-method-matching-in-jump/66543/2
I'm new to Julia and JuMP, a library I'm going to use.
Trying to define the following constraint, after having defined the variables, I receive an error:
for r = 1:11
for d = 1:7
for s = 1:12
#constraint(model, mod(ris_day_ora[r,d,s],0.5)==0)
end
end
end
Here the error:
ERROR: LoadError: MethodError: no method matching mod(::VariableRef, ::Float64)
Could you please help me?
Thanks a lot in advance!
You cannot have a mod in a JuMP constraint.
You need to reformulate the model and there are many ways to do that.
In your case the easiest thing would be to declare ris_day_ora as Int and then divide it everywhere by 2.
#variable(model, ris_day_ora[1:11, 1:7, 1:12] >=0, Int)
And now everywhere in the code use ris_day_ora[r,d,s]/2.0 instead of ris_day_ora[r,d,s].
Edit:
if your variable ris_day_ora takes three values 0, 0.5, 1 you just model it as:
#variable(model, 0 <= ris_day_ora[1:11, 1:7, 1:12] <= 2, Int)
And in each place in model use it as 0.5 * ris_day_ora[r,d,s]
Edit 2
Perhaps you are looking for a more general solution. Consider x that can only be either 0.1, 0.3, 0.7 this could be written as:
#variable(model, x)
#variable(model, helper[1:3], Bin)
#contraint(model, x == 0.1*helper[1] + 0.3*helper[2] + 0.7*helper[3])
#contraint(model, sum(helper) == 1)
I would like to minimize a linear programming system with linear constraints "equalities".
The system summarized in the following code "Python 3"
>>> obj_func = [1,1,1]
>>> const = [[[1, 0, 0], [0, 1, 0], [0, 0, 1], [1, 1, 1]]]
>>> constraints= np.reshape(const, (-1, 3))
>>> constraints
array([[1, 0, 0],
[0, 1, 0],
[0, 0, 1],
[1, 1, 1]])
>>> rhs = [0.4498162176582741, 0.4498162176582741, 0.10036756468345168, 1.0]
Using scipy.optimization.linprg:
>>> res = linprog(obj_func, constraints, rhs, method="interior-point", options={"disp":True})
>>> res
con: array([], dtype=float64)
fun: 1.4722956444515663e-09
message: 'Optimization terminated successfully.'
nit: 4
slack: array([0.44981622, 0.44981622, 0.10036756, 1. ])
status: 0
success: True
x: array([4.34463075e-10, 4.34463075e-10, 6.03369494e-10])
The same system summarized in R and minimized using lpSolve:
> obj.func = c(1,1,1)
> constraints = matrix(c(1,0,0,0,1,0,0,0,1,1,1,1), nrow= 4, byrow = TRUE)
> rhs = c(0.4498162+0i, 0.4498162+0i, 0.1003676+0i, 1.0000000+0i)
> f.dir = c("=","=","=","=")
>
> res = lp("min",obj.func,constraints,f.dir,rhs,compute.sens=FALSE)
> res
Success: the objective function is 1
As detailed above, the results are not close to each other although it is the same system so I did the same work for other systems but the results are also far.
My question: I know it is not necessary that every LP has a unique solution but I think they should produce close values ! In my case, I tried to minimize many systems using both solvers but the results are too far. For example,
First system: linprog gave 1.4722956444515663e-09 while lpSolve gave 1
Another system: linprog gave 1.65952852061376e-11 while lpSolve gave 0.8996324
Another system: linprog gave 3.05146726445553e-12 while lpSolve gave 0.8175745
You are solving different models.
res = linprog(obj_func, constraints, rhs, method="interior-point", options={"disp":True})
means
res = linprog(obj_func, A_ub=constraints, b_ub=rhs, method="interior-point", options={"disp":True})
effecting in constraints:
x0 <= 0.4498162176582741
...
instead of
x0 == 0.4498162176582741
So linprog is using inequalities only while lpsolve is using equalities only (without me checking if f.dir = c("=","=","=","=") is doing what i think it's doing; but the result shows this more or less).
The linprog-result:
x: array([4.34463075e-10, 4.34463075e-10, 6.03369494e-10])
is a typical zero-vector output of an interior-point method (only approximates integral solutions)! In contrast to commercial solvers like Gurobi, there is no crossover step.
Be careful when reading the docs (which contain this information).