I am trying to make one program of mine that uses Python Libraries NetworkX and Matplotlib imported using PyCall and it is returning a strange error that I am unable to understand.
The program is:
#everywhere function runTestGrid(;lenghtGrid=6,heightGrid=6,redundancyRange=0.1:0.1:0.1,failsRange=1:1:1,repetitionNumber=1,failType=1)
d1 = length(redundancyRange)
d2 = length(failsRange)
d3 = repetitionNumber
grid_data = SharedArray{Float64,3}((d1,d2,d3),init=0)
nNodes = lenghtGrid*heightGrid - 1
#sync #parallel for redundancyIndex in 1:1:d1
redundancyValue = redundancyRange[redundancyIndex]
gGrid = nx.grid_2d_graph(lenghtGrid,heightGrid,periodic=true)
tGrid = nx.minimum_spanning_tree(gGrid)
sGrid = System(gGrid,tGrid,redundancyValue)
for repetitionIndex in 1:1:d3
notServed = failureRecoveryRun(copySystem(sGrid),failsRange=failsRange,failType=failType,reconnectionMethod="single_route")
for nFailsIndex in 1:1:d2
grid_data[redundancyIndex,nFailsIndex,repetitionIndex] = (nNodes - notServed[nFailsIndex])/nNodes
end
end
end
writeDataFile("grid_data.txt",grid_data)
end
and I get the error message:
ERROR: LoadError: On worker 2:
UndefVarError: __anon__ not defined
deserialize_module at ./serialize.jl:754
handle_deserialize at ./serialize.jl:695
deserialize at ./serialize.jl:634
handle_deserialize at ./serialize.jl:703
deserialize at ./serialize.jl:634
deserialize_expr at ./serialize.jl:882
handle_deserialize at ./serialize.jl:693
deserialize at ./serialize.jl:634
deserialize_expr at ./serialize.jl:882
handle_deserialize at ./serialize.jl:693
deserialize at ./serialize.jl:634
deserialize_expr at ./serialize.jl:882
handle_deserialize at ./serialize.jl:693
deserialize_array at ./serialize.jl:871
handle_deserialize at ./serialize.jl:672
deserialize at ./serialize.jl:1075
handle_deserialize at ./serialize.jl:723
deserialize at ./serialize.jl:782
handle_deserialize at ./serialize.jl:723
deserialize_array at ./serialize.jl:871
handle_deserialize at ./serialize.jl:672
deserialize_typename at ./serialize.jl:938
deserialize at ./distributed/clusterserialize.jl:52
handle_deserialize at ./serialize.jl:723
deserialize at ./serialize.jl:634
deserialize_datatype at ./serialize.jl:967
handle_deserialize at ./serialize.jl:676
deserialize at ./serialize.jl:634
deserialize_datatype at ./serialize.jl:993
handle_deserialize at ./serialize.jl:674
deserialize at ./serialize.jl:634
handle_deserialize at ./serialize.jl:681
deserialize_msg at ./distributed/messages.jl:98
#99 at ./event.jl:73
Stacktrace:
[1] #remotecall_fetch#141(::Array{Any,1}, ::Function, ::Function, ::Base.Distributed.Worker, ::Base.Distributed.RRID, ::Vararg{Any,N} where N) at ./distributed/remotecall.jl:354
[2] remotecall_fetch(::Function, ::Base.Distributed.Worker, ::Base.Distributed.RRID, ::Vararg{Any,N} where N) at ./distributed/remotecall.jl:346
[3] #remotecall_fetch#144(::Array{Any,1}, ::Function, ::Function, ::Int64, ::Base.Distributed.RRID, ::Vararg{Any,N} where N) at ./distributed/remotecall.jl:367
[4] remotecall_fetch(::Function, ::Int64, ::Base.Distributed.RRID, ::Vararg{Any,N} where N) at ./distributed/remotecall.jl:367
[5] call_on_owner(::Function, ::Future, ::Int64, ::Vararg{Int64,N} where N) at ./distributed/remotecall.jl:440
[6] wait(::Future) at ./distributed/remotecall.jl:455
[7] sync_end() at ./task.jl:274
[8] macro expansion at ./task.jl:303 [inlined]
[9] #runTestGrid#55(::Int64, ::Int64, ::StepRangeLen{Float64,Base.TwicePrecision{Float64},Base.TwicePrecision{Float64}}, ::StepRange{Int64,Int64}, ::Int64, ::Int64, ::Function) at projeto.jl:24
[10] (::#kw##runTestGrid)(::Array{Any,1}, ::#runTestGrid) at ./<missing>:0
while loading projeto.jl, in expression starting on line 257
Being that the code on line 257 is simply the call of the function and that I do not define any __anon__ variable in my program.
Related
I'm trying to apply automatic differentiation (ForwardDiff) to a function that contains an instance of find_zero (Roots) and am encountering an error that seems to relate to find_zero not accepting the ForwardDiff.Dual type.
Here's a (contrived) minimal working example that illustrates the issue:
using Distributions
using Roots
using StatsFuns
using ForwardDiff
function test_fun(θ::AbstractVector{T}) where T
μ,σ,p = θ;
z_star = find_zero(z -> logistic(z) - p, 0.0)
return pdf(Normal(μ,σ),z_star)
end
test_fun([0.0,1.0,0.75])
ForwardDiff.gradient(test_fun,[0.0,1.0,0.75])
This results in the following error:
ERROR: MethodError: no method matching Float64(::ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3})
Closest candidates are:
Float64(::Real, ::RoundingMode) where T<:AbstractFloat at rounding.jl:200
Float64(::T) where T<:Number at boot.jl:716
Float64(::Irrational{:invsqrt2}) at irrationals.jl:189
...
Stacktrace:
[1] convert(::Type{Float64}, ::ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}) at ./number.jl:7
[2] setproperty!(::Roots.UnivariateZeroState{Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Symbol, ::ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}) at ./Base.jl:34
[3] update_state(::Roots.Secant, ::Roots.DerivativeFree{Roots.DerivativeFree{var"#5#6"{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}}}, ::Roots.UnivariateZeroState{Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Roots.UnivariateZeroOptions{Float64,Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}) at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/derivative_free.jl:163
[4] find_zero(::Roots.Secant, ::Roots.AlefeldPotraShi, ::Roots.DerivativeFree{Roots.DerivativeFree{var"#5#6"{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}}}, ::Roots.UnivariateZeroState{Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Roots.UnivariateZeroOptions{Float64,Float64,ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}, ::Roots.NullTracks) at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:868
[5] find_zero(::Roots.DerivativeFree{var"#5#6"{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3}}}, ::Float64, ::Roots.Secant, ::Roots.AlefeldPotraShi; tracks::Roots.NullTracks, verbose::Bool, p::Nothing, kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:689
[6] #find_zero#36 at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/derivative_free.jl:123 [inlined]
[7] find_zero at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/derivative_free.jl:120 [inlined]
[8] #find_zero#5 at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:707 [inlined]
[9] find_zero at /bbkinghome/asharris/.julia/packages/Roots/TZpjF/src/find_zero.jl:707 [inlined]
[10] test_fun at ./REPL[7856]:3 [inlined]
[11] vector_mode_dual_eval at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/apiutils.jl:37 [inlined]
[12] vector_mode_gradient(::typeof(test_fun), ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},1}}) at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:106
[13] gradient(::Function, ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},1}}, ::Val{true}) at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:19
[14] gradient(::Function, ::Array{Float64,1}, ::ForwardDiff.GradientConfig{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3,Array{ForwardDiff.Dual{ForwardDiff.Tag{typeof(test_fun),Float64},Float64,3},1}}) at /bbkinghome/asharris/.julia/packages/ForwardDiff/QOqCN/src/gradient.jl:17 (repeats 2 times)
[15] top-level scope at REPL[7858]:1
[16] run_repl(::REPL.AbstractREPL, ::Any) at /builddir/build/BUILD/julia/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:288
I have limited experience using the FowardDiff package and am probably misunderstanding how the Dual type works, so I would really appreciate if someone knows how to solve this issue. Thanks so much!
z_star = find_zero(z -> logistic(z) - p, 0.0)
You have a fixed initial condition which is non-dual. Make it dual.
z_star = find_zero(z -> logistic(z) - p, zero(eltype(θ))
I am getting this error when using the Interact.jl #manipulate macro in Juno:
UndefVarError: WebIOServer not defined
enter code here
setup_server() at webio.jl:76
show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::MIME{Symbol("application/prs.juno.plotpane+html")}, ::Widget{:manipulate,Any}) at webio.jl:68
show(::IOContext{Base.GenericIOBuffer{Array{UInt8,1}}}, ::String, ::Widget{:manipulate,Any}) at multimedia.jl:109
displayinplotpane(::Widget{:manipulate,Any}) at showdisplay.jl:51
displayandrender(::Widget{:manipulate,Any}) at showdisplay.jl:131
(::Atom.var"#208#213"{String})() at eval.jl:136
#invokelatest#1 at essentials.jl:712 [inlined]
invokelatest at essentials.jl:711 [inlined]
macro expansion at dynamic.jl:24 [inlined]
eval(::String, ::Int64, ::String, ::String, ::Bool) at eval.jl:113
invokelatest(::Any, ::Any, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{,Tuple{}}}) at essentials.jl:712
invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at essentials.jl:711
macro expansion at eval.jl:41 [inlined]
(::Atom.var"#188#189")() at task.jl:358
Piece of code (the function generatePlot is defined previously for generating Makie plots):
#manipulate for variable = Dict(string(vars[i])=>vars[i] for i=1:16),
labl = Dict(string(label[i])=>label[i] for i=1:length(label))
generatePlot(variable, labl)
end
Env:
[c601a237] Interact v0.10.3
[ee78f7c6] Makie v0.11.0
[85f8d34a] NCDatasets v0.10.2
[2913bbd2] StatsBase v0.33.0
[0f1e0344] WebIO v0.8.14
Any thoughts on what can be the problem?
Suppose I make my own custom vector type with it's own custom show method:
struct MyVector{T} <: AbstractVector{T}
v::Vector{T}
end
function Base.show(io::IO, v::MyVector{T}) where {T}
println(io, "My custom vector with eltype $T with elements")
for i in eachindex(v)
println(io, " ", v.v[i])
end
end
If I try making one of these objects at the REPL I get unexpected errors related to functions I never intended to call:
julia> MyVector([1, 2, 3])
Error showing value of type MyVector{Int64}:
ERROR: MethodError: no method matching size(::MyVector{Int64})
Closest candidates are:
size(::AbstractArray{T,N}, ::Any) where {T, N} at abstractarray.jl:38
size(::BitArray{1}) at bitarray.jl:77
size(::BitArray{1}, ::Integer) at bitarray.jl:81
...
Stacktrace:
[1] axes at ./abstractarray.jl:75 [inlined]
[2] summary(::IOContext{REPL.Terminals.TTYTerminal}, ::MyVector{Int64}) at ./show.jl:1877
[3] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::MyVector{Int64}) at ./arrayshow.jl:316
[4] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.3/REPL/src/REPL.jl:132
[5] display(::REPL.REPLDisplay, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.3/REPL/src/REPL.jl:136
[6] display(::Any) at ./multimedia.jl:323
...
Okay, whatever so I'll implement Base.size so it'll leave me alone:
julia> Base.size(v::MyVector) = size(v.v)
julia> MyVector([1, 2, 3])
3-element MyVector{Int64}:
Error showing value of type MyVector{Int64}:
ERROR: getindex not defined for MyVector{Int64}
Stacktrace:
[1] error(::String, ::Type) at ./error.jl:42
[2] error_if_canonical_getindex(::IndexCartesian, ::MyVector{Int64}, ::Int64) at ./abstractarray.jl:991
[3] _getindex at ./abstractarray.jl:980 [inlined]
[4] getindex at ./abstractarray.jl:981 [inlined]
[5] isassigned(::MyVector{Int64}, ::Int64, ::Int64) at ./abstractarray.jl:405
[6] alignment(::IOContext{REPL.Terminals.TTYTerminal}, ::MyVector{Int64}, ::UnitRange{Int64}, ::UnitRange{Int64}, ::Int64, ::Int64, ::Int64) at ./arrayshow.jl:67
[7] print_matrix(::IOContext{REPL.Terminals.TTYTerminal}, ::MyVector{Int64}, ::String, ::String, ::String, ::String, ::String, ::String, ::Int64, ::Int64) at ./arrayshow.jl:186
[8] print_matrix at ./arrayshow.jl:159 [inlined]
[9] print_array at ./arrayshow.jl:308 [inlined]
[10] show(::IOContext{REPL.Terminals.TTYTerminal}, ::MIME{Symbol("text/plain")}, ::MyVector{Int64}) at ./arrayshow.jl:345
[11] display(::REPL.REPLDisplay, ::MIME{Symbol("text/plain")}, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.3/REPL/src/REPL.jl:132
[12] display(::REPL.REPLDisplay, ::Any) at /Users/mason/julia/usr/share/julia/stdlib/v1.3/REPL/src/REPL.jl:136
[13] display(::Any) at ./multimedia.jl:323
...
Hmm, now it wants getindex
julia> Base.getindex(v::MyVector, args...) = getindex(v.v, args...)
julia> MyVector([1, 2, 3])
3-element MyVector{Int64}:
1
2
3
What? That wasn't the print formatting I told it to do! what's going on here?
The problem is that in julia, Base defines a method Base.show(io::IO ::MIME"text/plain", X::AbstractArray) which is actually more specific than the Base.show(io::IO, v::MyVector) for the purposes of display. This section of the julia manual describes the sort of custom printing that AbstractArray uses. So if we want to use our custom show method, we instead need to do
julia> function Base.show(io::IO, ::MIME"text/plain", v::MyVector{T}) where {T}
println(io, "My custom vector with eltype $T and elements")
for i in eachindex(v)
println(io, " ", v.v[i])
end
end
julia> MyVector([1, 2, 3])
My custom vector with eltype Int64 and elements
1
2
3
See also: https://discourse.julialang.org/t/extending-base-show-for-array-of-types/31289
I am getting the following Error for PyCall and am on Julia v 1.2.0
ERROR: LoadError: PyCall.PyError("\$(Expr(:escape, :(ccall(#= /home/omkar/.julia/packages/PyCall/ttONZ/src/pyfncall.jl:44 =# #pysym(:PyObject_Call), PyPtr, (PyPtr, PyPtr, PyPtr), o, pyargsptr, kw))))", PyCall.PyObject(Ptr{PyCall.PyObject_struct} #0x00007f9ea56a2520), PyCall.PyObject(Ptr{PyCall.PyObject_struct} #0x00007f9db65dcca8), PyCall.PyObject(Ptr{PyCall.PyObject_struct} #0x00007f9d004f9f08))
Stacktrace:
[1] pyerr_check at /home/omkar/.julia/packages/PyCall/ttONZ/src/exception.jl:60 [inlined]
[2] pyerr_check at /home/omkar/.julia/packages/PyCall/ttONZ/src/exception.jl:64 [inlined]
[3] macro expansion at /home/omkar/.julia/packages/PyCall/ttONZ/src/exception.jl:84 [inlined]
[4] __pycall!(::PyCall.PyObject, ::Ptr{PyCall.PyObject_struct}, ::PyCall.PyObject, ::Ptr{Nothing}) at /home/omkar/.julia/packages/PyCall/ttONZ/src/pyfncall.jl:44
[5] _pycall!(::PyCall.PyObject, ::PyCall.PyObject, ::Tuple{String,String,String,String,String,Float64,String,Float64}, ::Int64, ::Ptr{Nothing}) at /home/omkar/.julia/packages/PyCall/ttONZ/src/pyfncall.jl:29
[6] _pycall!(::PyCall.PyObject, ::PyCall.PyObject, ::Tuple{String,String,String,String,String,Float64,String,Float64}, ::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at /home/omkar/.julia/packages/PyCall/ttONZ/src/pyfncall.jl:11
[7] #call#111(::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}, ::PyCall.PyObject, ::String, ::Vararg{Any,N} where N) at /home/omkar/.julia/packages/PyCall/ttONZ/src/pyfncall.jl:89
[8] (::PyCall.PyObject)(::String, ::Vararg{Any,N} where N) at /home/omkar/.julia/packages/PyCall/ttONZ/src/pyfncall.jl:89
I have already tried update and below is the setup for the build file
ENV["PYTHON"] = " "
Pkg.build("PyCall")
using PyCall, Conda
pyexec = pyimport("sys").executable
Dear users of the language julia. I have a problem when using the optimize function of the Optim package. What is the error of the code below?
using Optim
using Distributions
rng = MersenneTwister(1234);
d = Weibull(1,1)
x = rand(d,1000)
function pdf_weibull(x, lambda, k)
k/lambda * (x/lambda).^(k-1) * exp((-x/lambda)^k)
end
function obj(x::Vector, lambda, k)
soma = 0
for i in x
soma = soma + log(pdf_weibull(i,lambda,k))
end
-soma
end
obj(x, pars) = obj(x, pars...)
optimize(vars -> obj(x, vars...), [1.0,1.0])
Output
julia> optimize(vars -> obj(x, vars...), [1.0,1.0])
ERROR: DomainError:
Exponentiation yielding a complex result requires a complex argument.
Replace x^y with (x+0im)^y, Complex(x)^y, or similar.
Stacktrace:
[1] nan_dom_err at ./math.jl:300 [inlined]
[2] ^ at ./math.jl:699 [inlined]
[3] (::##2#4)(::Float64, ::Float64, ::Float64) at ./<missing>:0
[4] pdf_weibull(::Float64, ::Float64, ::Float64) at ./REPL[6]:2
[5] obj(::Array{Float64,1}, ::Float64, ::Float64) at ./REPL[7]:4
[6] (::##5#6)(::Array{Float64,1}) at ./REPL[11]:1
[7] value(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/NLSolversBase/src/interface.jl:19
[8] initial_state(::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}, ::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/solvers/zeroth_order/nelder_mead.jl:139
[9] optimize(::NLSolversBase.NonDifferentiable{Float64,Array{Float64,1},Val{false}}, ::Array{Float64,1}, ::Optim.NelderMead{Optim.AffineSimplexer,Optim.AdaptiveParameters}, ::Optim.Options{Float64,Void}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/optimize.jl:25
[10] #optimize#151(::Array{Any,1}, ::Function, ::Tuple{##5#6}, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/interface.jl:62
[11] #optimize#148(::Array{Any,1}, ::Function, ::Function, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/interface.jl:52
[12] optimize(::Function, ::Array{Float64,1}) at /home/pedro/.julia/v0.6/Optim/src/multivariate/optimize/interface.jl:52
[13] macro expansion at ./REPL.jl:97 [inlined]
[14] (::Base.REPL.##1#2{Base.REPL.REPLBackend})() at ./event.jl:73
It is a simple problem to obtain the maximum likelihood estimates of the parameters that index the weibull distribution.
Best regards.
The reason of your problem is that your definition of pdf_weibull is incorrect. Here is a corrected definition:
function pdf_weibull(x, lambda, k)
k/lambda * (x/lambda)^(k-1) * exp(-(x/lambda)^k)
end
Note that I have moved - sign in exp part of the expression. If you change this all will work as expected.
Now - why does Julia complain with DomainError. The reason is that because of the error in your code you try to calculate the value of something like (-1.0)^0.5. In Julia ^ is implemented in type stable way. This means, in particular, that when it is passed Float64 as both arguments it guarantees to return Float64 or throw an error. Clearly (-1.0)^0.5 cannot be computed in real domain - that is why an error is thrown. If you passed (-1+0im)^0.5 then there would be no error as we are passing a complex number to ^ so the result can also be complex number in a type stable way.