When I call consume(generator) I get this error. Is this a version problem?
function fib()
a = 0
produce(a)
b = 1
produce(b)
while true
a , b = b , a+b
produce(b)
end
end
generator = Task(fib)
consume(generator)
Here a way to do something similar using a Channel
global channel = Channel{Int}(1)
function fib()
# global here is not needed because we aren't modifying the channel handle, just the contents
# global channel
a = 0
put!(channel, a)
b = 1
put!(channel, b)
while true
a , b = b , a+b
put!(channel,b)
sleep(1)
end
end
#async while true; item = take!(channel); println("item: $item"); end
#async fib()
Note that #async will hide errors, so you may want to do a try catch with showerror(stderr, e, catch_backtrace()) if things are not running.
Each #async produces a Task handle.
Also the put! and take! will block when the channel if filled up. You may want to expand the channel size to handle a larger buffer.
Related
I finished writing the following program and began to do some cleanup after the debugging stage:
using BenchmarkTools
function main()
global solution = 0
global a = big"1"
global b = big"1"
global c = big"0"
global total = 0
while a < 100
while b < 100
c = a^b
s = string(c)
total = 0
for i in 1:length(s)
total = total + Int(s[i]) - 48
end
if total > solution
global solution = total
end
global b = b + 1
end
global b = 1
global a = a + 1
end
end
#elapsed begin
main()
end
#run #elapsed twice to ignore compilation overhead
t = #elapsed main()
print("Solution: ", solution)
t = t * 1000;
print("\n\nProgram completed in ", round.(t; sigdigits=5), " milliseconds.")
The runtime for my machine was around 150ms.
I decided to rearrange the globals to better match the typical layout of program, where globals are defined at the top:
using BenchmarkTools
global solution = 0
global a = big"1"
global b = big"1"
global c = big"0"
global total = 0
function main()
while a < 100
while b < 100
c = a^b
s = string(c)
total = 0
for i in 1:length(s)
total = total + Int(s[i]) - 48
end
if total > solution
global solution = total
end
global b = b + 1
end
global b = 1
global a = a + 1
end
end
#elapsed begin
main()
end
#run #elapsed twice to ignore compilation overhead
t = #elapsed main()
print("Solution: ", solution)
t = t * 1000;
print("\n\nProgram completed in ", round.(t; sigdigits=5), " milliseconds.")
Making that one change for where the globals were defined reduced the runtime on my machine to 0.0042ms.
Why is the runtime so drastically reduced?
Don't use globals.
Don't. Use. Globals. They are bad.
When you define your globals outside the main function, then the second time you run your function, a already equals 100, and main() bails out before doing anything at all.
Global variables are a bad idea, not just in Julia, but in programming in general. You can use them when defining proper constants, like π, and maybe some other specialized cases, but not for things like this.
Let me rewrite your function without globals:
function main_locals()
solution = 0
a = 1
while a < 100
b = 1
c = big(1)
while b < 100
c *= a
s = string(c)
total = sum(Int, s) - 48 * length(s)
solution = max(solution, total)
b += 1
end
a += 1
end
return solution
end
On my laptop this is >20x faster than your version with globals defined inside the function, that is, the version that actually works. The other one doesn't work as it should, so the comparison is not relevant.
Edit: I have even complicated this too much. The only thing you need to do is to remove all the globals from your first function, and return the solution, then it will work fine, and be almost as fast as the code I wrote:
function main_with_globals_removed()
solution = 0
a = big"1"
b = big"1"
c = big"0"
total = 0
while a < 100
while b < 100
c = a^b
s = string(c)
total = 0
for i in 1:length(s)
total = total + Int(s[i]) - 48
end
if total > solution
solution = total
end
b = b + 1
end
b = 1
a = a + 1
end
return solution # remember return!
end
Don't use globals.
In the first case, you are always assigning the globals and possibly changing types. Hence compiler needs to do extra work. I assume that the two programs generate different answers after the 2nd run because of the failure to reset globals…
Globals are discouraged in Julia for performance reasons because of potential type instability.
I am using Julia and I've designed a for loop that takes the outputs of a function in one loop and uses them as the input of that function in the next loop (and over and over). When I run this code, Julia flags an "undefined" error, however, if I run the code in debug mode, it executes perfectly. For example, the code looks like this:
function do_command(a,b,c,d)
a = a + 1
b = split(b, keepempty=false)[1]
c = split(b, keepempty=false)[1]
if a == 1000
d = true
else
d = false
end
return a, b, c, d
end
for ii in 1:length(x)
if ii == 1
a = 0
b = "string something"
c = ""
d = false
end
a,b,c,d = do_command(a,b,c,d)
if d == true
print(string(b))
break
end
end
What am I doing wrong here?
An issue with your code is that for introduces a new scope for each iteration of the loop. That is to say: variable a created within the loop body at iteration 1 is not the same as variable a created within the loop body at iteration 2.
In order to fix your problem, you should declare variables outside the loop, so that at each iteration, references to them from within the loop body would actually refer to the same variables from the enclosing scope.
I'd go with something like this:
function do_command(a,b,c,d)
a = a + 1
b = split(b, keepempty=false)[1]
c = split(b, keepempty=false)[1]
if a == 1000
d = true
else
d = false
end
return a, b, c, d
end
# Let's create a local scope: it's good practice to avoid global variables
let
# All these variables are declared in the scope introduced by `let`
a = 0
b = "string something"
c = ""
d = false
for ii in 1:10 #length(x)
# now these names refer to the variables declared in the enclosing scope
a,b,c,d = do_command(a,b,c,d)
if d == true
print(string(b))
break
end
end
end
I'd like to run heavy computations in Julia for a fixed duration, for example 10 seconds. I tried this:
timer = Timer(10.0)
while isopen(timer)
computation()
end
But this does not work, since the computations never let Julia's task scheduler take control. So I added yield() in the loop:
timer = Timer(10.0)
while isopen(timer)
yield()
computation()
end
But now there is significant overhead from calling yield(), especially when one call to computation() is short. I guess I could call yield() and isopen() only every 1000 iterations or so, but I would prefer a solution where I would not have to tweak the number of iterations every time I change the computations. Any ideas?
This pattern below uses threads and on my laptop has a latency of around 35ms for each 1,000,000 calls which is more than acceptable for any job.
Tested on Julia 1.5 release candidate:
function should_stop(timeout=10)
handle = Threads.Atomic{Bool}(false)
mytask = Threads.#spawn begin
sleep(timeout)
Threads.atomic_or!(handle, true)
end
handle
end
function do_some_job_with_timeout()
handle = should_stop(5)
res = BigInt() # save results to some object
mytask = Threads.#spawn begin
for i in 1:10_000_000
#TODO some complex computations here
res += 1 # mutate the result object
handle.value && break
end
end
wait(mytask) # wait for the job to complete
res
end
You can also used Distributed instead. The code below seems to have a much better latency - only about 1ms for each 1,000,000 timeout checks.
using Distributed
using SharedArrays
addprocs(1)
function get_termination_handle(timeout=5,workerid::Int=workers()[end])::SharedArray{Bool}
handle = SharedArray{Bool}([false])
proc = #spawnat workerid begin
sleep(timeout)
handle[1]=true
end
handle
end
function fun_within_timeout()
res = 0
h = get_termination_handle(0.1)
for i = 1:100_000_000
res += i % 2 == 0 ? 1 : 0
h[1] && break
end
res
end
I want to create a julia "server" that contains a cached value and serves that when requested and updates the value when an update arrives from other channels
My plan to do this is:
Use julia ZMQ (zeromq) which listens on a REP (reply) socket and delivers the value to any request coming to that REP socket.
Also, the program has a SUB (subscribe) socket that updates the value whenever the socket receives anything
The REP socket blocks using ZMQ.recv.
Maybe the SUB socket does as well, not sure
But basically, both parts need to run independently in a while loop, sharing some memory (variable)
So perhaps this needs to be done using SharedArrays, spawning processes
But I just can't figure out how to do this in code.
E.g. I can #spawn each such process, one has a REP, one has a SUB socket, but I don't know how to get their pid's to create a SharedArray
Can someone help please?
I am also open to different design solutions to solve the problem (basically data is being constantly updated from some source and other programs need to be able to get the most current copy of this data)
Thanks
Imran
EDIT:
I have kind of gotten a simple version to work as follows:
It has 2 independent REP/REQ sockets
The strange thing is that this sometimes works and sometimes after a few calls to readcache() and writecache(41) blocks in either the readcache or the writecache ... but I cannot reproduce, as it sometimes just works smoothly
Is this the correct way of solving this in julia?
using ZMQ
type CT
a::Int
b::String
end
ct = CT(1,"a")
readport = 5551
readproc = #spawn readcacheproc(ct,readport)
writeport = 5552
writeproc = #spawn writecacheproc(ct,writeport)
# test as follows
# readcache() # expect [1 a]
# writecache("test") # expect [4 test]
# readcache() # expect [4 test]
function readcache(port=readport)
ctx=Context()
s=Socket(ctx,REQ)
ZMQ.connect(s,"tcp://localhost:$port")
ZMQ.send(s,"")
println(bytestring(ZMQ.recv(s)))
ZMQ.close(s)
ZMQ.close(ctx)
end
function writecache(value,port=writeport)
ctx=Context()
s=Socket(ctx,REQ)
ZMQ.connect(s,"tcp://localhost:$port")
ZMQ.send(s,"$value")
println(bytestring(ZMQ.recv(s)))
ZMQ.close(s)
ZMQ.close(ctx)
end
function readcacheproc(cache,port=readport)
ctx=Context()
s=Socket(ctx,REP)
ZMQ.bind(s,"tcp://*:$port")
done = false
while !done
msg = bytestring(ZMQ.recv(s)) # actual msg is ignored
ZMQ.send(s,"$(cache.a) $(cache.b)")
end
ZMQ.close(s)
ZMQ.close(ctx)
end
function writecacheproc(cache,port=writeport)
ctx=Context()
s=Socket(ctx,REP)
ZMQ.bind(s,"tcp://*:$port")
done = false
while !done
msg = bytestring(ZMQ.recv(s))
cache.a = length(msg)
cache.b = msg
ZMQ.send(s,"new cache: $(cache.a) $(cache.b)")
end
ZMQ.close(s)
ZMQ.close(ctx)
end
The following seems to work, although I dont know whether this is the best way of solving it
using ZMQ
type CT
a::Int
b::String
end
ct = CT(1,"a")
readport = 5551
readproc = #spawn readcacheproc(ct,readport)
writeport = 5552
writeproc = #spawn writecacheproc(ct,writeport)
# test as follows
# readcache() # expect [1 a]
# writecache("test") # expect [4 test]
# readcache() # expect [4 test]
function readcache(port=readport)
ctx=Context()
s=Socket(ctx,REQ)
ZMQ.connect(s,"tcp://localhost:$port")
ZMQ.send(s,"")
println(bytestring(ZMQ.recv(s)))
ZMQ.close(s)
ZMQ.close(ctx)
end
function writecache(value,port=writeport)
ctx=Context()
s=Socket(ctx,REQ)
ZMQ.connect(s,"tcp://localhost:$port")
ZMQ.send(s,"$value")
println(bytestring(ZMQ.recv(s)))
ZMQ.close(s)
ZMQ.close(ctx)
end
function readcacheproc(cache,port=readport)
ctx=Context()
s=Socket(ctx,REP)
ZMQ.bind(s,"tcp://*:$port")
done = false
while !done
msg = bytestring(ZMQ.recv(s)) # actual msg is ignored
ZMQ.send(s,"$(cache.a) $(cache.b)")
end
ZMQ.close(s)
ZMQ.close(ctx)
end
function writecacheproc(cache,port=writeport)
ctx=Context()
s=Socket(ctx,REP)
ZMQ.bind(s,"tcp://*:$port")
done = false
while !done
msg = bytestring(ZMQ.recv(s))
cache.a = length(msg)
cache.b = msg
ZMQ.send(s,"new cache: $(cache.a) $(cache.b)")
end
ZMQ.close(s)
ZMQ.close(ctx)
end
I have a generic function in julia that the aim is to say if a member of a vector of
a given dimension is negative or not. After a few variations I have:
function any(vec)
dim = size(vec)
for i in 1:dim[2]
fflag = vec[1,i] < 0
println("Inside any, fflag = ", fflag)
if fflag == true
result = 0
println("blabla ", result)
break
else
result =1
println("blabla ", result)
continue
end
end
println("hey, what is result? ")
println(result)
return result
end
If I run a test I found the following result:
Inside any, fflag = false
blabla 1
Inside any, fflag = false
blabla 1
Inside any, fflag = false
blabla 1
hey, what is result?
result not defined
at In[7]:57
I don't know why the compiler says me that 'result' is not defined. I know the variable exist but why does not live outside the for loop?
The documentation on variable scoping clearly states that a for loop defines a new scope. This means result is going out of scope when execution leaves the for loop. Hence it is undefined when you call println(result)
Defining result in advance of the for loop should give the behaviour you are expecting:
function any(vec)
dim = size(vec)
result = -1
for i in 1:dim[2]
...
Or if you do not wish to assign a default value, and are sure the for loop will set its value, you can do:
function any(vec)
dim = size(vec)
local result
for i in 1:dim[2]
...
In the first example, if the for loop does not set a value, result will be -1.
In the the second example, not setting a value in the for loop will leave result undefined.