This is my idea
task=#task begin
while true
sleep(1)
foo()
if should_end
# ?
end
end
end
But there are some problems
are there any simple ways to contact other than using global should_end?
how to end the task from inside the expression?
While implementing something like this can be a good exercise, note that Timers are in the standard library and they are similar to the JS functions you may be used to because Julia and Node.js both use libuv internally. Whether you use Node.js setInterval or Timeout, or Julia Timer eventually uv_timer_start is called (although the creation and low-level management of timers is different in the respective runtimes).
To answer your 2nd question, you can just break in the if expression:
julia> should_end = false
false
julia> task = #task begin
while true
sleep(1)
if should_end
break
end
end
"Tasks can return a value"
end
Task (runnable) #0x00007fca1c9c3990
julia> schedule(task)
Task (runnable) #0x00007fca1c9c3990
julia> task
Task (runnable) #0x00007fca1c9c3990
julia> should_end = true;sleep(1)
julia> task
Task (done) #0x00007fca1c9c3990
julia> fetch(task)
"Tasks can return a value"
As for the 1st question, there is a lot of information in the Julia docs, e.g. the Asynchronous Programming section. As is described there, Channels can be used for communication between tasks (when suitable). Note that should_end doesn't have to be global. A task wraps a function and that function can capture variables in enclosing scopes (#task begin a = 1 end is really just Task(() -> begin a = 1 end).
Here is a short example using a Timer and a Channel to send data to a task:
function pinger(n)
ping() = parse(Float64, match(r"time=(.*) ms", read(`ping -c 1 8.8.8.8`, String))[1])
ch = Channel{Float64}(0)
pinger = Timer(0.0; interval=1.0) do timer
put!(ch, ping())
end
summer = #task begin
total = 0.0
count = 0
while count < n
total += take!(ch)
count += 1
end
total / count
end
schedule(summer)
fetch(summer)
end
julia> pinger(3)
19.5
Related
I'm new to elixir, I'm trying to find something similar to Python's ContextManager.
Problem:
I have a bunch of functions and I want to add latency metric around them.
Now we have:
def method_1 do
...
end
def method_2 do
...
end
... more methods
I'd like to have:
def method_1 do
start = System.monotonic_time()
...
end = System.monotonic_time()
emit_metric(end-start)
end
def method_2 do
start = System.monotonic_time()
...
end = System.monotonic_time()
emit_metric(end-start)
end
... more methods
Now code duplication is a problem
start = System.monotonic_time()
...
end = System.monotonic_time()
emit_metric(end-start)
So what is a better way to avoid code duplication in this case? I like the context manager idea in python. But now sure how I can achieve something similar in Elixir, thanks for the help in advance!
In Erlang/Elixir this is done through higher-order functions, take a look at BEAM telemetry. It is an Erlang and Elixir library/standard for collecting metrics and instrumenting your code - it is widely adopted by Pheonix, Ecto, cowboy and other libraries. Specifically, you'd be interested in :telemetry.span/3 function as it emits start time and duration measurements by default:
def some_function(args) do
:telemetry.span([:my_app, :my_function], %{metadata: "Some data"}, fn ->
result = do_some_work(args)
{result, %{more_metadata: "Some data here"}}
end)
end
def do_some_work(args) # actual work goes here
And then, in some other are of your code you listen to those events and log them/send them to APM:
:telemetry.attach_many(
"test-telemetry",
[[:my_app, :my_function, :start],
[:my_app, :my_function, :stop],
[:my_app, :my_function, :exception]],
fn event, measurements, metadata, config ->
# Handle the actual event.
end)
nil
)
I think the closest thing to python context manager would be to use higher order functions, i.e. functions taking a function as argument.
So you could have something like:
def measure(fun) do
start = System.monotonic_time()
result = fun.()
stop = System.monotonic_time()
emit_metric(stop - start)
result
end
And you could use it like:
measure(fn ->
do_stuff()
...
end)
Note: there are other similar instances where you would use a context manager in python that would be done in a similar way, on the top of my head: Django has a context manager for transactions but Ecto uses a higher order function for the same thing.
PS: to measure elapsed time, you probably want to use :timer.tc/1 though:
def measure(fun) do
{elapsed, result} = :timer.tc(fun)
emit_metric(elapsed)
result
end
There is actually a really nifty library called Decorator in which macros can be used to "wrap" your functions to do all sorts of things.
In your case, you could write a decorator module (thanks to #maciej-szlosarczyk for the telemetry example):
defmodule MyApp.Measurements do
use Decorator.Define, measure: 0
def measure(body, context) do
meta = Map.take(context, [:name, :module, :arity])
quote do
# Pass the metadata information about module/name/arity as metadata to be accessed later
:telemetry.span([:my_app, :measurements, :function_call], unquote(meta), fn ->
{unquote(body), %{}}
end)
end
end
end
You can set up a telemetry listener in your Application.start definition:
:telemetry.attach_many(
"my-app-measurements",
[[:my_app, :measurements, :function_call, :start],
[:my_app, :measurements, :function_call, :stop],
[:my_app, :measurements, :function_call, :exception]],
&MyApp.MeasurementHandler.handle_telemetry/4)
nil
)
Then in any module with a function call you'd like to measure, you can "decorate" the functions like so:
defmodule MyApp.Domain.DoCoolStuff do
use MyApp.Measurements
#decorate measure()
def awesome_function(a, b, c) do
# regular function logic
end
end
Although this example uses telemetry, you could just as easily print out the time difference within your decorator definition.
I am trying to understand the usage of #sync and #async macros in Julia. I am trying to get this MWE to work and the program does not terminate. Any help is appreciated.
function process_node(nodes, id)
#show id
sleep(1.0)
nodes[id] = true
return
end
function main()
nodes = Dict( i => false for i in 1:10 )
jobs = Channel{Int}(15)
for i in 1:10
put!(jobs, i)
end
#sync for id in jobs
#async process_node(nodes, id)
end
println("done")
end
main()
The program never gets to the line println("done"). I do not know why.
Thanks in advance.
There is nothing wrong about the use of #sync and #async in this example.
the loop for id in jobs never return because it blocks forever waiting endlessy for values no more inserted into the Channel.
Directly from the docs:
The returned Channel can be used as an iterable object in a for loop, in which case the loop variable takes on all the produced > values. The loop is terminated when the channel is closed.
One solution is to signal the end of the streams of values with a special Int value, for example -1 or if this is not possible with a nothing value, declaring jobs as Channel{Union{Int, Nothing}}.
function main()
nodes = Dict( i => false for i in 1:10 )
jobs = Channel{Int}(15)
for i in 1:10
put!(jobs, i)
end
put!(jobs, -1)
#sync for id in jobs
if id == -1
close(jobs)
else
#async process_node(nodes, id)
end
end
println("done")
end
While memoization of a function is a good idea, it could cause a program to crash because the program could potentially run out of memory.
Therefore it is NOT A SAFE OPTION to be used in a production program.
Instead I have developed caching with a fixed memory slots below with a soft limit and hard limit. When the cache slots is above the hard limit, it will have the least used slots deleted until the number of slots is reduced to the soft limit.
struct cacheType
softlimit::Int
hardlimit::Int
memory::Dict{Any,Any}
freq::Dict{Any,Int}
cacheType(soft::Int,hard::Int) = new(soft,hard,Dict(),Dict())
end
function tidycache!(c::cacheType)
memory_slots=length(c.memory)
if memory_slots > c.hardlimit
num_to_delete = memory_slots - c.softlimit
# Now sort the freq dictionary into array of key => AccessFrequency
# where the first few items have the lowest AccessFrequency
for item in sort(collect(c.freq),by = x -> x[2])[1:num_to_delete]
delete!(c.freq, item[1])
delete!(c.memory, item[1])
end
end
end
# Fibonacci function
function cachefib!(cache::cacheType,x)
if haskey(cache.memory,x)
# Increment the number of times this key has been accessed
cache.freq[x] += 1
return cache.memory[x]
else
# perform housekeeping and remove cache entries if over the hardlimit
tidycache!(cache)
if x < 3
cache.freq[x] = 1
return cache.memory[x] = 1
else
result = cachefib!(cache,x-2) + cachefib!(cache,x-1)
cache.freq[x] = 1
cache.memory[x] = result
return result
end
end
end
c = cacheType(3,4)
cachefib!(c,3)
cachefib!(c,4)
cachefib!(c,5)
cachefib!(c,6)
cachefib!(c,4)
println("c.memory is ",c.memory)
println("c.freq is ",c.freq)
I think this would be most useful in a production environment than just using memorization with no limits of memory consumption which could result in a program crashing.
In Python language, they have
#functools.lru_cache(maxsize=128, typed=False)
Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.
Is there an equivalent in Julia language?
There is LRUCache.jl, which provides an LRU type which basically acts like a Dict. Unfortunately, this doesn't seem to work with the Memoize.jl package, but you can use my answer to your other question:
using LRUCache
const fibmem = LRU{Int,Int}(3) # store only 3 values
function fib(n)
get!(fibmem, n) do
n < 3 ? 1 : fib(n-1) + fib(n-2)
end
end
I'm coming to Elixir from primarily a Javascript background. in JS, it's possible to write a higher order function "once" which returns a function that will invoke the passed in function only once, and returns the previous result on subsequent calls- the trick is manipulating variables that were captured via closure:
var once = (func) => {
var wasCalled = false, prevResult;
return (...args) => {
if (wasCalled) return prevResult;
wasCalled = true;
return prevResult = func(...args);
}
}
It seems to me that it's not possible to create this function in Elixir, due to its different variable rebinding behavior. Is there some other clever way to do it via pattern matching or recursion, or is it just not possible? Without macros that is, I'd imagine those might enable it. Thanks
Using the current process dictionary:
defmodule A do
def once(f) do
key = make_ref()
fn ->
case Process.get(key) do
{^key, val} -> val
nil ->
val = f.()
Process.put(key, {key, val})
val
end
end
end
end
Or if the function will be passed across processes, an ets table can be used:
# ... during application initialization
:ets.new(:cache, [:set, :public, :named_table])
defmodule A do
def once(f) do
key = make_ref()
fn ->
case :ets.lookup(:cache, key) do
[{^key, val}] -> val
[] ->
val = f.()
:ets.insert(:cache, {key, val})
val
end
end
end
end
Application.put_env / Application.get_env can also be used to hold global state, though usually is used for configuration settings.
It's not considered idiomatic in most cases, but you can do this with Agent:
defmodule A do
def once(fun) do
{:ok, agent} = Agent.start_link(fn -> nil end)
fn args ->
case Agent.get(agent, & &1) do
nil ->
result = apply(fun, args)
:ok = Agent.update(agent, fn _ -> {:ok, result} end)
result
{:ok, result} ->
result
end
end
end
end
Now if you run this:
once = A.once(fn sleep ->
:timer.sleep(sleep)
1 + 1
end)
IO.inspect once.([1000])
IO.inspect once.([1000])
IO.inspect once.([1000])
IO.inspect once.([1000])
You'll see that the first line is printed after 1 second, but the next 3 are printed instantly, because the result is fetched from the agent.
While both already given answers are perfectly valid, the most precise translation from your javascript is shown below:
defmodule M do
use GenServer
def start_link(_opts \\ []) do
GenServer.start_link(__MODULE__, nil, name: __MODULE__)
end
def init(_args) do
Process.sleep(1_000)
{:ok, 42}
end
def value() do
start_link()
GenServer.call(__MODULE__, :value)
end
def handle_call(:value, _from, state) do
{:reply, state, state}
end
end
(1..5) |> Enum.each(&IO.inspect(M.value(), label: to_string(&1)))
Use the same metric as in #Dogbert’s answer: the first value is printed with a delay, all subsequent are printed immediately.
This is an exact analog of your memoized function using GenServer stage. GenServer.start_link/3 returns one of the following:
{:ok, #PID<0.80.0>}
{:error, {:already_started, #PID<0.80.0>}}
That said, it is not restarted if it’s already started. I do not bother to check the returned value since we are all set in any case: if it’s the initial start, we call the heavy function, if we were already started, the vaklue is already at fingers in the state.
My code for display all days in this year.
I don't understand why if NewSec =< EndSec -> init:stop() end did not execute the first time in run_calendar?
I expect init:stop() could be executed first time but it is not.
What is wrong?
Code:
-module(cal).
-export([main/0]).
main() ->
StartSec = calendar:datetime_to_gregorian_seconds({{2009,1,1},{0,0,0}}),
EndSec = calendar:datetime_to_gregorian_seconds({{2009,12,31},{0,0,0}}),
run_calendar(StartSec,EndSec).
run_calendar(CurSec, EndSec) ->
{Date,_Time} = calendar:gregorian_seconds_to_datetime(CurSec),
io:format("~p~n", [Date]),
NewSec = CurSec + 60*60*24,
if NewSec =< EndSec -> init:stop() end,
run_calendar(NewSec, EndSec).
Result:
wk# erlc cal.erl
wk# erl -noshell -s cal main
{2009,1,1}
{2009,1,2}
{2009,1,3}
{2009,1,4}
{2009,1,5}
...
{2009,12,22}
{2009,12,23}
{2009,12,24}
{2009,12,25}
{2009,12,26}
{2009,12,27}
{2009,12,28}
{2009,12,29}
{2009,12,30}
{2009,12,31}
wk#
I believe that init:stop() is an asynchronous process that will attempt to shut down the runtime smoothly. According to the docs, "All applications are taken down smoothly, all code is unloaded, and all ports are closed before the system terminates."
It probably takes a while to actually stop, because you have an actively running process. If you change "init:stop()" to "exit(stop)", it will terminate immediately:
3> cal:main().
{2009,1,1}
** exception exit: stop
in function cal:run_calendar/2
Init:stop is asynchronous and it will take time to quit. An alternate way would be to wrap up the test in the call itself and use pattern matching to terminate the loop:
-module(cal).
-export([main/0]).
main() ->
StartSec = calendar:datetime_to_gregorian_seconds({{2009,1,1},{0,0,0}}),
EndSec = calendar:datetime_to_gregorian_seconds({{2009,12,31},{0,0,0}}),
run_calendar(false, StartSec, EndSec).
run_calendar(true, _StartSec, _EndSec) ->
finished;
run_calendar(false, CurSec, EndSec) ->
{Date,_Time} = calendar:gregorian_seconds_to_datetime(CurSec),
io:format("~p~n", [Date]),
NewSec = CurSec + 60*60*24,
run_calendar(NewSec =< EndSec, NewSec, EndSec).
(or something similar, hopefully you get the idea)
You have a mistake in your if statement
You said
if NewSec =< EndSec -> init:stop() end,
This is incorrect. You have to write something like:
if
A =< B -> do something ...;
true -> do something else
end
The if syntax is
if
Condition1 -> Actions1;
Condition2 -> Actions2;
...
end
One of these conditions must always be true.
Why is this?
Erlang is a functional language, not a statement language. In an functional
language every expression must have a value. if is an expression, so it must have a value.
The value of (if 2 > 1 -> 3 end) is 3 but what is the value of
(if 1 > 2 -> 3 end) - answer it has no value - but it must have a value
everything must have a value.
In a statement language everything is evaluated for its side effect -so this would
be a valid construction.
In Erlang you will generate an exception.
So your code generates an exception - which you don't trap so you don't see it and
init:stop() never gets called ...