I'm following the quickstart guide on JuMP.
My julia version is 0.7, the code is this:
using JuMP
m = Model()
l = zeros(10)
u = ones(10)
##variable(m, x)
##variable(m, 0 <= x[1:10] <= 1)
#variable(m, l<=x[1:10]<=u)
The first two variable macros (commented out) work fine, but the last one produces an error.
MethodError: no method matching constructvariable!(::Model, ::getfield(JuMP, Symbol("#_error#107")){Tuple{Symbol,Expr}}, ::Array{Float64,1}, ::Array{Float64,1}, ::Symbol, ::String, ::Float64)
Closest candidates are:
constructvariable!(::Model, ::Function, !Matched::Number, !Matched::Number, ::Symbol, ::AbstractString, ::Number; extra_kwargs...) at /home/lhk/.julia/packages/JuMP/Xvn0n/src/macros.jl:968
constructvariable!(::Model, ::Function, !Matched::Number, !Matched::Number, ::Symbol, !Matched::Number, !Matched::Array{T,1} where T, !Matched::Array{Float64,1}, !Matched::AbstractString, !Matched::Number; extra_kwargs...) at /home/lhk/.julia/packages/JuMP/Xvn0n/src/macros.jl:961
Stacktrace:
[1] top-level scope at /home/lhk/.julia/packages/JuMP/Xvn0n/src/macros.jl:1259
[2] top-level scope at In[18]:7
How can I have different bounds for each entry in a vector valued variable ?
Argh, this is actually really easy:
l = zeros(10)
u = ones(10)
#variable(m, l[idx] <= x[idx = 1:10] <= u[idx])
Related
with Julia 1.5.3, I wanted to pass a list or parameters to the distributed workers.
I first tried in a non distributed way :
using Distributed
#everywhere begin
using SharedArrays
solve(a,b,c) = return (1,2,3)
d_rates = LinRange(0.01, 0.33, 5)
m_rates = LinRange(0.01, 0.25, 5)
population_size = 10^3
max_iterations_perloop = 10^3
nb_repeats = 2
nb_params = length(d_rates)*length(m_rates)*nb_repeats
para = enumerate(Base.product(d_rates, m_rates, population_size, max_iterations_perloop, 1:nb_repeats))
results = SharedArray{Tuple{Int, Int, Int}}(nb_params)
end
for (y , x) in para
results[y] = solve(x[1], x[2], x[3])
end
which worked fine. And then changed the final loop to:
#sync #distributed for (y , x) in para
results[y] = solve(x[1], x[2], x[3])
end
I then got an error (truncated):
ERROR: LoadError: TaskFailedException:
MethodError: no method matching firstindex(::Base.Iterators.Enumerate{Base.Iterators.ProductIterator{Tuple{LinRange{Float64},LinRange{Float64},Int64,Int64,UnitRange{Int64}}}})
Closest candidates are:
firstindex(::Cmd) at process.jl:638
firstindex(::Core.SimpleVector) at essentials.jl:599
firstindex(::Base64.Buffer) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Base64/src/buffer.jl:18
...
Stacktrace:
[1] (::Distributed.var"#159#161"{var"#271#272",Base.Iterators.Enumerate{Base.Iterators.ProductIterator{Tuple{LinRange{Float64},LinRange{Float64},Int64,Int64,UnitRange{Int64}}}}})() at ./task.jl:332
Stacktrace:
[1] sync_end(::Channel{Any}) at ./task.jl:314
[2] top-level scope at task.jl:333
[3] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1088
[4] include_string(::Module, ::String, ::String) at ./loading.jl:1096
[5] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./essentials.jl:710
[6] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:709
Is it possible to pass such a list, if so how?
I assume that all your workers are on a single server and that you have actually added some workers using the addprocs command. The first problem with your code is that you create the SharedArray on all workers. Rather than that the syntax of a SharedArray is the following:
help?> SharedArray
SharedArray{T}(dims::NTuple; init=false, pids=Int[])
SharedArray{T,N}(...)
Construct a SharedArray of a bits type T and size dims across the processes specified by pids - all of which have to be on the same host. (...)
This means that you create SharedArray only once from the master worker and you can specify the workers that are aware of it using the pids argument (if you do not specify pids all worker processes have the access).
Hence your code will look like this:
using Distributed, SharedArrays
addprocs(4)
#everywhere using SharedArrays
#everywhere solve(a,b,c) = return (1,2,3)
#(...) # your setup code without #everywhere
results = SharedArray{Tuple{Int, Int, Int}}(nb_params)
#sync #distributed for (y , x) in collect(para)
results[y] = solve(x[1], x[2], x[3])
end
Note that you will need collect because #distributed macro needs to know the size of the Vector and it does not work good with iterators.
I have the following code in which the JuMP constraint is throwing an error.
using JuMP
using MosekTools
K = 3
N = 2
penalties = [1.0, 3.9, 8.7]
function A_tau(r::Number, n::Number, tau::Float64)
fac = 1
for m in 1:r
fac *= (n - (m - 1))
end
if n >= r
return fac * tau ^ (n - r)
else
return 0.0
end
end
function A_tau_mat(tau::Float64)
mat = Array{Float64, 2}(undef, N+1, N+1)
for i in 1:N+1
for j in 1:N+1
mat[i, j] = A_tau(i, j, tau)
end
end
return mat
end
m = Model(optimizer_with_attributes(Mosek.Optimizer, "QUIET" => false, "INTPNT_CO_TOL_DFEAS" => 1e-7))
#variable(m, p[1:1:K,1:1:N+1])
#variable(m, A[1:1:K+1,1:1:K,1:1:N+1,1:1:N+1])
#constraint(m, -A_tau_mat(0.0) * p[1, :] == [0.0, 0.0, 0.0])
optimize!(m)
println("p value is ", value.(p[1, :]))
println(A_tau_mat(0.0))
The error happens when the #constraint line is added and there's no error without it. The error is as follows. The error shows no method matching CartesianIndex ::Int64.
ERROR: LoadError: MethodError: no method matching -(::CartesianIndex{1}, ::Int64)
Closest candidates are:
-(!Matched::Complex{Bool}, ::Real) at complex.jl:307
-(!Matched::Missing, ::Number) at missing.jl:115
-(!Matched::MutableArithmetics.Zero, ::Any) at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/rewrite.jl:55
...
Stacktrace:
[1] _add_mul_array(::Array{GenericAffExpr{Float64,VariableRef},1}, ::Array{Float64,2}, ::JuMP.Containers.DenseAxisArray{VariableRef,1,Tuple{StepRange{Int64,Int64}},Tuple{Dict{Int64,Int64}}}) at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/linear_algebra.jl:166
[2] mutable_operate! at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/linear_algebra.jl:196 [inlined]
[3] mutable_operate_to!(::Array{GenericAffExpr{Float64,VariableRef},1}, ::typeof(*), ::Array{Float64,2}, ::JuMP.Containers.DenseAxisArray{VariableRef,1,Tuple{StepRange{Int64,Int64}},Tuple{Dict{Int64,Int64}}}) at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/linear_algebra.jl:208
[4] operate at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/linear_algebra.jl:221 [inlined]
[5] operate at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/rewrite.jl:43 [inlined]
[6] operate_fallback! at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/interface.jl:275 [inlined]
[7] operate!(::typeof(MutableArithmetics.sub_mul), ::MutableArithmetics.Zero, ::Array{Float64,2}, ::JuMP.Containers.DenseAxisArray{VariableRef,1,Tuple{StepRange{Int64,Int64}},Tuple{Dict{Int64,Int64}}}) at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/rewrite.jl:70
[8] top-level scope at /Users/prikshetsharma/.julia/packages/MutableArithmetics/0tlz5/src/rewrite.jl:227
[9] top-level scope at /Users/prikshetsharma/.julia/packages/JuMP/qhoVb/src/macros.jl:440
[10] top-level scope at /Users/prikshetsharma/Documents/clotorch/src/clotorch/flight/trajectory.jl:72
[11] include(::Function, ::Module, ::String) at ./Base.jl:380
[12] include(::Module, ::String) at ./Base.jl:368
[13] exec_options(::Base.JLOptions) at ./client.jl:296
[14] _start() at ./client.jl:506
in expression starting at /Users/prikshetsharma/Documents/clotorch/src/clotorch/flight/trajectory.jl:72
How to fix this error and use the constraint like I want to? What's wrong with this constraint?
You can try this
#variable(m, p[1:K,1:N+1])
#variable(m, A[1:K+1,1:K,1:N+1,1:N+1])
#constraint(m, -A_tau_mat(0.0) * p[1, :] .== [0.0, 0.0, 0.0])
There are two problems. First, the p's type in your original is DenseAxisArray rather than the normal Array because a StepRange (1:1:3) rather than a UnitRange (1:3) is provided. Though having the same elements, their types are different and the matrix multiplication implementation for it is somewhat problematic. I think it is a bug that should be fixed on the JuMP side. The other change is the dot . before the ==, which indicates broadcasting.
I'm trying to write a ray tracer in julia but my main function returns StackOverFlowError. Below is my main function:
function trace(ray::Ray, surfaces::Array{Any, 1}, depth::Int64, maxDepth::Int64)
material, t = findIntersection(ray, surfaces, Inf)
if typeof(material) == Empty
Vec3(0,0,0)
end
if depth > maxDepth
Vec3(0,0,0)
end
if material.isLight == true
material.emittance
end
normal = material.normal
ρ = material.reflectance
BRDF = ρ/3.14159
R = randHemi(normal)
cosθ = dot(R,normal)
newRay = Ray(ray.s + t*ray.d, R)
In = trace(newRay, surfaces, depth+1, maxDepth)
2.0*3.14159*BRDF*In
end
And here is my intersection function:
function findIntersection(ray::Ray, surfaces::Array{Any, 1}, tmin::Float64)
hitSurface = Empty(Vec3(0,0,0), Vec3(0,0,0), Vec3(0,0,0), false)
for surface in surfaces
t = intersect(surface, ray)
if t < tmin
hitSurface = surface
tmin = t
end
end
return hitSurface, tmin
end
But I receive this error:
StackOverflowError:
Stacktrace:
[1] findIntersection(::Ray, ::Array{Any,1}, ::Float64) at .\In[31]:10
[2] trace(::Ray, ::Array{Any,1}, ::Int64, ::Int64) at .\In[33]:2
[3] trace(::Ray, ::Array{Any,1}, ::Int64, ::Int64) at .\In[33]:18 (repeats
14493 times)
[4] top-level scope at In[35]:1
What is the cause of error?
Your problem is that you missed a return.
What you meant to have was
if depth > maxDepth
return Vec3(0,0,0)
end
Without the return, that line just allocates a Vec3, and the code continues looping.
Stackoverflow error usually occurs when there are too many recursive calls. In a nutshell, in most programming languages there is a limit on how many recursive calls you can make.
In your example, the trace functions calls itself recursively and it can cause an stack overflow. You have a parameters maxDepth that can limit that. Setting it to a lower value will probably solve this particular issue.
I am having trouble with pmap() throwing a BoundsError when setting the values of array elements - my code works for 1 worker but not >1. I have written a minimum working example which roughly follows the real code flow:
Get source data
Define set of points over which to iterate
Initialise array points to be calculated
Calculate each array point
The main file:
#pmapdemo.jl
using Distributed
#addprocs(length(Sys.cpu_info())) # uncomment this line for error
#everywhere include(joinpath(#__DIR__, "pmapdemo2.jl"))
function main()
# Get source data
source = Dict{String, Any}("t"=>zeros(5),
"x"=>zeros(5,6),
"y"=>zeros(5,3),
"z"=>zeros(5,3))
# Define set of points over which to iterate
iterset = Dict{String, Any}("t"=>source["t"],
"x"=>source["x"],
"y"=>fill(2, size(source["t"])[1], 1),
"z"=>fill(2, size(source["t"])[1], 1))
data = Dict{String, Any}()
# Initialise array points to be calculated
MyMod.initialisearray!(data, iterset)
# Calculate each array point
MyMod.calcarray!(data, iterset, source)
#show data
end
main()
The functionality file:
#pmapdemo2.jl
module MyMod
using Distributed
#everywhere using SharedArrays
# Initialise data array
function initialisearray!(data, fieldset)
zerofield::SharedArray{Float64, 4} = zeros(size(fieldset["t"])[1],
size(fieldset["x"])[2],
size(fieldset["y"])[2],
size(fieldset["z"])[2])
data["field"] = deepcopy(zerofield)
end
# Calculate values of array elements according to values in source
function calcpoint!((data, source, a, b, c, d))
data["field"][a,b,c,d] = rand()
end
# Set values in array
function calcarray!(data, iterset, source)
for a in eachindex(iterset["t"])
# [additional functionality f(a) here]
b = eachindex(iterset["x"][a,:])
c = eachindex(iterset["y"][a,:])
d = eachindex(iterset["z"][a,:])
pmap(calcpoint!, Iterators.product(Iterators.repeated(data,1), Iterators.repeated(source,1), Iterators.repeated(a,1), b, c, d))
end
end
end
The error output:
ERROR: LoadError: On worker 2:
BoundsError: attempt to access 0×0×0×0 Array{Float64,4} at index [1]
setindex! at ./array.jl:767 [inlined]
setindex! at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/SharedArrays/src/SharedArrays.jl:500 [inlined]
_setindex! at ./abstractarray.jl:1043
setindex! at ./abstractarray.jl:1020
calcpoint! at /home/dave/pmapdemo2.jl:25
#112 at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Distributed/src/process_messages.jl:269
run_work_thunk at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Distributed/src/process_messages.jl:56
macro expansion at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Distributed/src/process_messages.jl:269 [inlined]
#111 at ./task.jl:259
Stacktrace:
[1] (::getfield(Base, Symbol("##696#698")))(::Task) at ./asyncmap.jl:178
[2] foreach(::getfield(Base, Symbol("##696#698")), ::Array{Any,1}) at ./abstractarray.jl:1866
[3] maptwice(::Function, ::Channel{Any}, ::Array{Any,1}, ::Base.Iterators.ProductIterator{Tuple{Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Int64}},Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}}) at ./asyncmap.jl:178
[4] #async_usemap#681 at ./asyncmap.jl:154 [inlined]
[5] #async_usemap at ./none:0 [inlined]
[6] #asyncmap#680 at ./asyncmap.jl:81 [inlined]
[7] #asyncmap at ./none:0 [inlined]
[8] #pmap#213(::Bool, ::Int64, ::Nothing, ::Array{Any,1}, ::Nothing, ::Function, ::Function, ::WorkerPool, ::Base.Iterators.ProductIterator{Tuple{Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Int64}},Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}}) at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Distributed/src/pmap.jl:126
[9] pmap(::Function, ::WorkerPool, ::Base.Iterators.ProductIterator{Tuple{Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Int64}},Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}}) at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Distributed/src/pmap.jl:101
[10] #pmap#223(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::Function, ::Function, ::Base.Iterators.ProductIterator{Tuple{Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Int64}},Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}}) at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Distributed/src/pmap.jl:156
[11] pmap(::Function, ::Base.Iterators.ProductIterator{Tuple{Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Dict{String,Any}}},Base.Iterators.Take{Base.Iterators.Repeated{Int64}},Base.OneTo{Int64},Base.OneTo{Int64},Base.OneTo{Int64}}}) at /build/julia/src/julia-1.1.1/usr/share/julia/stdlib/v1.1/Distributed/src/pmap.jl:156
[12] calcarray!(::Dict{String,Any}, ::Dict{String,Any}, ::Dict{String,Any}) at /home/dave/pmapdemo2.jl:20
[13] main() at /home/dave/pmapdemo.jl:19
[14] top-level scope at none:0
in expression starting at /home/dave/pmapdemo.jl:23
In pmapdemo2.jl, replacing data["field"][a,b,c,d] = rand() with #show a, b, c, d demonstrates that all workers are running and have full access to the variables being passed, however instead replacing it with #show data["field"] throws the same error. Surely the entire purpose of SharedArrays is to avoid this? Or am I misunderstanding how to use it with pmap?
This is a crosspost from the Julia discourse here.
pmap will do the work of passing the data to the processes, so you don't need to use SharedArrays. Typically, the function provided to pmap (and indeed map) will be a pure function (and therefore doesn't mutate any variable) which returns one element of an output array. That function is mapped across each element of the input array, and the pmap function will construct the output array for you. For example, in your case, the code may look a bit like this
calcpoint(source, (a,b,c,d)) = rand() # Or some function of source and the indices a,b,c,d
field["data"] = pmap(calcpoint, Iterators.repeated(source), Iterators.product(a,b,c,d))
Dear users of the language julia. I have a problem when using the optimize function of the Optim package. What is the error of the code below?
using Optim
using Distributions
rng = MersenneTwister(1234);
d = Weibull(1,1)
x = rand(d,1000)
function pdf_weibull(x, lambda, k)
k/lambda * (x/lambda).^(k-1) * exp((-x/lambda)^k)
end
function obj(x::Vector, lambda, k)
soma = 0
for i in x
soma = soma + log(pdf_weibull(i,lambda,k))
end
-soma
end
obj(x, pars) = obj(x, pars...)
optimize(vars -> obj(x, vars...), [1.0,1.0])
Output
julia> optimize(vars -> obj(x, vars...), [1.0,1.0])
ERROR: DomainError:
Exponentiation yielding a complex result requires a complex argument.
Replace x^y with (x+0im)^y, Complex(x)^y, or similar.
Stacktrace:
[1] nan_dom_err at ./math.jl:300 [inlined]
[2] ^ at ./math.jl:699 [inlined]
[3] (::##2#4)(::Float64, ::Float64, ::Float64) at ./<missing>:0
[4] pdf_weibull(::Float64, ::Float64, ::Float64) at ./REPL[6]:2
[5] obj(::Array{Float64,1}, ::Float64, ::Float64) at ./REPL[7]:4
[6] (::##5#6)(::Array{Float64,1}) at ./REPL[11]:1
[7] value(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/NLSolversBase/src/interface.jl:19
[8] initial_state(::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/solvers/zeroth_order/nelder_mead.jl:139
[9] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/optimize.jl:25
[10] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##5#6}, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/interface.jl:62
[11] #optimize#148(::Array{Any,1}, ::Function, ::Function, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/interface.jl:52
[12] optimize(::Function, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/interface.jl:52
[13] macro expansion at ./REPL.jl:97 [inlined]
[14] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
It is a simple problem to obtain the maximum likelihood estimates of the parameters that index the weibull distribution.
Best regards.
The reason of your problem is that your definition of pdf_weibull is incorrect. Here is a corrected definition:
function pdf_weibull(x, lambda, k)
k/lambda * (x/lambda)^(k-1) * exp(-(x/lambda)^k)
end
Note that I have moved - sign in exp part of the expression. If you change this all will work as expected.
Now - why does Julia complain with DomainError. The reason is that because of the error in your code you try to calculate the value of something like (-1.0)^0.5. In Julia ^ is implemented in type stable way. This means, in particular, that when it is passed Float64 as both arguments it guarantees to return Float64 or throw an error. Clearly (-1.0)^0.5 cannot be computed in real domain - that is why an error is thrown. If you passed (-1+0im)^0.5 then there would be no error as we are passing a complex number to ^ so the result can also be complex number in a type stable way.