ZeroMQ and Dash on linux cause "Address already in use" error\ - julia

Puzzled by this error message as nothing is running on the port for Dash (I run sudo netstat -nlp to make sure) and the ZMQ is working, on it’s own, perfectly as is the Dash functionality. Put them together, which doesn’t seem unreasonable, and they should work well. I don’t see how the bindings are causing the issue.
I’m using linux and I was told that ZMQ and Dash are using the same port. My problem is that I don’t know how to address this issue.
here’s the error
ave#deepthought:~/tontine_2022/just_messing$ julia 6_25_min_dash.jl
starting IN Socket 5555
Received request: END
dying
ERROR: LoadError: TaskFailedException
Stacktrace:
[1] wait
# ./task.jl:334 [inlined]
[2] hot_restart(func::Dash.var"#72#74"{Dash.DashApp, String, Int64}; check_interval::Float64, env_key::String, suppress_warn::Bool)
# Dash ~/.julia/packages/Dash/yscRy/src/utils/hot_restart.jl:23
[3] run_server(app::Dash.DashApp, host::String, port::Int64; debug::Bool, dev_tools_ui::Nothing, dev_tools_props_check::Nothing, dev_tools_serve_dev_bundles::Nothing, dev_tools_hot_reload::Nothing, dev_tools_hot_reload_interval::Nothing, dev_tools_hot_reload_watch_interval::Nothing, dev_tools_hot_reload_max_retry::Nothing, dev_tools_silence_routes_logging::Nothing, dev_tools_prune_errors::Nothing)
# Dash ~/.julia/packages/Dash/yscRy/src/server.jl:64
[4] build_dash()
# Main ~/tontine_2022/just_messing/6_25_min_dash.jl:46
[5] top-level scope
# ~/tontine_2022/just_messing/6_25_min_dash.jl:68
nested task error: LoadError: StateError("Address already in use")
Stacktrace:
[1] bind(socket::Socket, endpoint::String)
# ZMQ ~/.julia/packages/ZMQ/R3wSD/src/socket.jl:58
[2] top-level scope
# ~/tontine_2022/just_messing/6_25_min_dash.jl:10
[3] include(mod::Module, _path::String)
# Base ./Base.jl:418
[4] include(x::String)
# Main.##274 ~/.julia/packages/Dash/yscRy/src/utils/hot_restart.jl:21
[5] top-level scope
# ~/.julia/packages/Dash/yscRy/src/utils/hot_restart.jl:21
[6] eval
# ./boot.jl:373 [inlined]
[7] eval
# ./Base.jl:68 [inlined]
[8] (::Dash.var"#21#22"{String, Symbol})()
# Dash ./task.jl:423
in expression starting at /home/dave/tontine_2022/just_messing/6_25_min_dash.jl:10
in expression starting at /home/dave/tontine_2022/just_messing/6_25_min_dash.jl:68
dave#deepthought:~/tontine_2022/just_messing$
can someone look at my code to see what I am doing wrong please?
here’s the ZMQ pull code
using ZMQ
using Dash
using DataFrames
context = Context()
in_socket = Socket(context, PULL)
ZMQ.bind(in_socket, "tcp://*:5555")
println("starting IN Socket 5555")
dash_columns = ["sym","price","sdmove","hv20","hv10","hv5","iv","iv%ile","prc%ile","volume"]
df_dash_table = DataFrame([col => (col == "sym" ? String : Float64)[] for col in dash_columns ])
function build_dash()
app = dash()
app.layout = html_div() do
html_h1("tontine2"),
dash_datatable( id="table", columns=[Dict("name" =>i, "id" => i) for i in names(df_dash_table)],
data = Dict.(pairs.(eachrow(df_dash_table))),
editable=false,
filter_action="native",
sort_action="native",
sort_mode="multi",
row_selectable="multi",
row_deletable=false,
selected_rows=[],
## page_action="native",
## page_current= 0,
## page_size= 10,
)#end dash_datatable
end
run_server(app, "0.0.0.0", debug=true)
end
while true
message = String(ZMQ.recv(in_socket))
println("Received request: $message")
if message == "END"
println("dying")
break
end
end
build_dash()
and here’s the code to trigger the event
using ZMQ
context = Context()
stk_socket = Socket(context, PUSH)
ZMQ.connect(stk_socket, "tcp://localhost:5555")
ZMQ.send(stk_socket,"END")
ZMQ.close(stk_socket)
ZMQ.close(context)

This works here:
using ZMQ
using Dash
using Distributed
app = dash(external_stylesheets = ["https://codepen.io/chriddyp/pen/bWLwgP.css"])
app.layout = html_div() do
html_h1("Hello Dash"),
html_div("Dash.jl: Julia interface for Dash"),
dcc_graph(
id = "example-graph",
figure = (
data = [
(x = [1, 2, 3], y = [4, 1, 2], type = "bar", name = "SF"),
(x = [1, 2, 3], y = [2, 4, 5], type = "bar", name = "Montréal"),
],
layout = (title = "Dash Data Visualization",)
)
)
end
#spawn run_server(app, "0.0.0.0", 8080)
function testpush()
context = Context()
stk_socket = Socket(context, PUSH)
ZMQ.connect(stk_socket, "tcp://localhost:5555")
ZMQ.send(stk_socket,"END")
ZMQ.close(stk_socket)
ZMQ.close(context)
end
context = Context()
in_socket = Socket(context, PULL)
ZMQ.bind(in_socket, "tcp://*:5555")
sleep(1)
#spawn testpush()
sleep(5)
If you are running the program in an editor task, please note that the spawned process will now continue until you restart the editor. Perhaps a spawned process is clinging to the port?

Related

Jupyter with Julia results in unexpected type error: no method matching

I get an unexpected type error when running the following Julia code in Jupyter, where a seemingly straightforward import goes wrong:
include("./imp.jl")
include("./imp2.jl")
n = Main.Imp.Network([1,2])
Imp2.p2(n)
This results in the following error:
MethodError: no method matching p(::Main.Imp.Network)
Closest candidates are:
p(::Main.Imp2.Imp.Network) at /Users/cg/Dropbox/code/Julia/learning/imp.jl:11
The code is the below. How does this happen?
Imp.jl:
module Imp
export Network, p
mutable struct Network
a::Array{Any,1}
end
function p(network::Network)
network
end
end
Imp2.jl:
module Imp2
include("./imp.jl")
function p2(network)
Imp.p(network)
end
end
More error below:
Stacktrace:
[1] p2(network::Main.Imp.Network)
# Main.Imp2 ~/Dropbox/code/Julia/learning/imp2.jl:5
[2] top-level scope
# In[3]:4
[3] eval
# ./boot.jl:360 [inlined]
[4] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
# Base ./loading.jl:1116
You can either do:
module Imp2
using Main.Imp
function p2(network)
Imp.p(network)
end
end
OR (without sourcing imp.jl outside of module defintion)
module Imp2
include("./imp.jl")
using .Imp
function p2(network)
Imp.p(network)
end
end
In the second case your Julia code could look like:
julia> using Main.Imp2
julia> n = Imp2.Imp.Network([1,2])
Main.Imp2.Imp.Network(Any[1, 2])
julia> Imp2.p2(n)
Main.Imp2.Imp.Network(Any[1, 2])
Addtionally if you add export Imp to the Imp2 module, you could write Imp.Network([1,2]) instead of Imp2.Imp.Network([1,2]).

LoadError: UndefVarError: #defVar not defined

I have a piece of code about JuMP. When I run it ,it says that LoadError: UndefVarError: #defVar not defined. I have tried using global forward or backward but both fails.
See:
function T1(w_func,grid_b,β,u,z)
# objective for each grid point
for j in 1:cp.Nb
b = grid_b[j]
choice1 = Model(solver=GLPKSolverLP())
#defVar (choice1, a >= 0)
#setObjective(choice1, Max, u(a) + cp.β * (w_func.((b*(1+cp.r)+cp.w-a) .* cp.z[i])))
results1 = solve(choice1)
Tw1 = getObjectiveValue(choice1)
c_choice1 = getValue(x)
return Tw, σ
end
end
LoadError: UndefVarError: #defVar not defined
in expression starting at In[44]:37
Stacktrace:
[1] top-level scope
# :0
[2] eval
# ./boot.jl:360 [inlined]
[3] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
# Base ./loading.jl:1094
thanks
It seems that you're using an outdated code. Look at the fresh documentation and make sure you have installed the latest versions of libraries and Julia.
In short, #defVar and #setObjective were replaced by #variable and #objective correspondingly.
function T1(w_func,grid_b,β,u,z)
# objective for each grid point
for j in 1:cp.Nb
b = grid_b[j]
choice1 = Model(solver=GLPKSolverLP())
#variable(choice1, a >= 0)
#objective(choice1, Max, u(a) + cp.β * (w_func.((b*(1+cp.r)+cp.w-a) .* cp.z[i])))
results1 = solve(choice1)
Tw1 = getObjectiveValue(choice1)
c_choice1 = getValue(x)
return Tw, σ
end
end

How to pass a list of parameters to workers in Julia Distributed

with Julia 1.5.3, I wanted to pass a list or parameters to the distributed workers.
I first tried in a non distributed way :
using Distributed
#everywhere begin
using SharedArrays
solve(a,b,c) = return (1,2,3)
d_rates = LinRange(0.01, 0.33, 5)
m_rates = LinRange(0.01, 0.25, 5)
population_size = 10^3
max_iterations_perloop = 10^3
nb_repeats = 2
nb_params = length(d_rates)*length(m_rates)*nb_repeats
para = enumerate(Base.product(d_rates, m_rates, population_size, max_iterations_perloop, 1:nb_repeats))
results = SharedArray{Tuple{Int, Int, Int}}(nb_params)
end
for (y , x) in para
results[y] = solve(x[1], x[2], x[3])
end
which worked fine. And then changed the final loop to:
#sync #distributed for (y , x) in para
results[y] = solve(x[1], x[2], x[3])
end
I then got an error (truncated):
ERROR: LoadError: TaskFailedException:
MethodError: no method matching firstindex(::Base.Iterators.Enumerate{Base.Iterators.ProductIterator{Tuple{LinRange{Float64},LinRange{Float64},Int64,Int64,UnitRange{Int64}}}})
Closest candidates are:
firstindex(::Cmd) at process.jl:638
firstindex(::Core.SimpleVector) at essentials.jl:599
firstindex(::Base64.Buffer) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Base64/src/buffer.jl:18
...
Stacktrace:
[1] (::Distributed.var"#159#161"{var"#271#272",Base.Iterators.Enumerate{Base.Iterators.ProductIterator{Tuple{LinRange{Float64},LinRange{Float64},Int64,Int64,UnitRange{Int64}}}}})() at ./task.jl:332
Stacktrace:
[1] sync_end(::Channel{Any}) at ./task.jl:314
[2] top-level scope at task.jl:333
[3] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1088
[4] include_string(::Module, ::String, ::String) at ./loading.jl:1096
[5] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N; kwargs::Base.Iterators.Pairs{Union{},Union{},Tuple{},NamedTuple{(),Tuple{}}}) at ./essentials.jl:710
[6] invokelatest(::Any, ::Any, ::Vararg{Any,N} where N) at ./essentials.jl:709
Is it possible to pass such a list, if so how?
I assume that all your workers are on a single server and that you have actually added some workers using the addprocs command. The first problem with your code is that you create the SharedArray on all workers. Rather than that the syntax of a SharedArray is the following:
help?> SharedArray
SharedArray{T}(dims::NTuple; init=false, pids=Int[])
SharedArray{T,N}(...)
Construct a SharedArray of a bits type T and size dims across the processes specified by pids - all of which have to be on the same host. (...)
This means that you create SharedArray only once from the master worker and you can specify the workers that are aware of it using the pids argument (if you do not specify pids all worker processes have the access).
Hence your code will look like this:
using Distributed, SharedArrays
addprocs(4)
#everywhere using SharedArrays
#everywhere solve(a,b,c) = return (1,2,3)
#(...) # your setup code without #everywhere
results = SharedArray{Tuple{Int, Int, Int}}(nb_params)
#sync #distributed for (y , x) in collect(para)
results[y] = solve(x[1], x[2], x[3])
end
Note that you will need collect because #distributed macro needs to know the size of the Vector and it does not work good with iterators.

Cannot return value from a function: ERROR: LoadError: ArgumentError: `nothing` should not be printed; use `show`, `repr`, or

I've been solving the problem sets from Harvard's CS50 with Julia. This script is meant to be my solution of [plurality elections.]1
println("How many contenders do we have?")
const max_candidates = parse(Int, readline()) # a maximal number of candidates
# Let us define a composite type for the candidates in our elections
mutable struct Candidate
name::String
votes::Int64
end
function vote(name)
for i in 1:max_candidates
if candidates[i].name == name
candidates[i].votes = candidates[i].votes + 1
end
end
end
function print_winner()
max_votes = 0
for i in 1:max_candidates
if candidates[i].votes > max_votes
max_votes = candidates[i].votes
end
end
for i in 1:max_candidates
if candidates[i].votes == max_votes
candidates[i].name
end
end
end
candidates = Vector{Candidate}(undef, max_candidates)
for i in 1:max_candidates -1
println("Name of the candidate: ?")
name = readline()
votes = 0
candidates[i] = Candidate(name, votes)
println("Thank you, let us move to the next candidate.")
end
#The last candidate i registered outside of the loop because I do no want
#the line println("Thank you, let us move to the next candidate.") to be executed after them.
println("Name of the last candidate: ?")
name = readline()
votes = 0
candidates[max_candidates] = Candidate(name, votes)
println("How many voters do we have?")
voter_count = parse(Int, readline())
for i in 1:voter_count
println("Who are you voting for?")
name = readline()
vote(name)
end
winner = print_winner()
println(winner)
When I run this script I get the following error
ERROR: LoadError: ArgumentError: `nothing` should not be printed; use `show`, `repr`, or custom output instead.
Stacktrace:
[1] print(::Base.TTY, ::Nothing) at ./show.jl:566
[2] print(::Base.TTY, ::Nothing, ::Char) at ./strings/io.jl:42
[3] println(::Base.TTY, ::Nothing) at ./strings/io.jl:69
[4] println(::Nothing) at ./coreio.jl:4
[5] top-level scope at none:0
[6] include at ./boot.jl:317 [inlined]
[7] include_relative(::Module, ::String) at ./loading.jl:1044
[8] include(::Module, ::String) at ./sysimg.jl:29
[9] exec_options(::Base.JLOptions) at ./client.jl:266
[10] _start() at ./client.jl:425
in expression starting at /home/jerzy/C.../plurality.jl:65
The expression referred in the error message as "expression starting at /home/jerzy/C.../plurality.jl:65" is the last name of the script.
I do not understand what is this nothing?
Nevertheless, following the suggestions of the error message I modified the last line of my code, changing it from:
println(winner)
to
show(winner)
and got the following output:
nothing
I did a some research here and there, yet being a newbie, I don't understand why can't I return a value from my function print_winner. From what I read, return statements are not obligatory.
In the definition of print_winner, when I substitute
candidates[i].name
with
println(candidates[i].name)
and then when the last line is
winner = print_winner()
then I am able to finally get the name of the winner. But it is not the way I want it. I want to return a value and assign it to a variable, and then do something with this variable. I would be able to do this in PHP or Racket, why cannot I in Julia?
The function print_winner doesn't return anything, in which case the object nothing is actually returned. So winner gets the value nothing (from winner = print_winner()), and println(winner) is equivalent to println(nothing), which leads to the error.
I want to return a value and assign it to a variable
Then just do that: return a value from print_winner. Julia can't know what you would want this function to return, so you have to be explicit about it. By default, Julia returns the value of the function expression, which in this case is the result of the last expression, which is here a for loop. The expression-value of a for-loop is nothing in Julia.

Broadcasting with ArrayFire.jl

I'm trying to broadcast a simple conditional function on an AFArray type.
using ArrayFire
f = x -> x > .5 ? "foo" : "bar"
a = rand(Float32, 10, 10)
b = rand(AFArray{Float32}, 10, 10)
display( f.(a) ) # This one works...
display( f.(b) ) # ...this one does not
Here's the stacktrace :
ERROR: LoadError: TypeError: non-boolean (ArrayFire.AFArray{Bool,2}) used in boolean context
Stacktrace:
[1] (::##1#2)(::ArrayFire.AFArray{Float32,2}) at D:\LeMinaw\Dev\MandelFrac\test.jl:3
[2] broadcast_c(::Function, ::Type{ArrayFire.AFArray}, ::ArrayFire.AFArray{Float32,2}) at C:\Users\LeMinaw\.julia\v0.6\ArrayFire\src\array.jl:317
[3] broadcast(::Function, ::ArrayFire.AFArray{Float32,2}) at .\broadcast.jl:455
[4] include_from_node1(::String) at .\loading.jl:576
[5] include(::String) at .\sysimg.jl:14
while loading D:\LeMinaw\Dev\MandelFrac\test.jl, in expression starting on line 9
How can I handle this without copying the array from the GPU back to the host?

Resources