F# events not working inside Async workflow - asynchronous

I want to do a Post-Fire-Reply to an agent. Basically the agent triggers an event then replies to the caller. However I either keep getting a timeout error or the events do not fire correctly. I tried doing Post-Fire, that stopped the timeout errors but the events do not fire.
let evt = new Event<int>()
let stream = evt.Publish
type Agent<'T> = MailboxProcessor<'T>
type Fire = Fire of int
let agent = Agent.Start(fun inbox ->
let rec loop() = async {
let! msg = inbox.Receive()
let (Fire i) = msg
evt.Trigger i }
loop())
let on i fn =
stream
|> Observable.filter (fun x -> x = i)
|> Observable.filter (fun x -> x <> 1)
|> Observable.subscribe (fun x -> fn x)
let rec collatz n =
printfn "%d" n
on n (fun i ->
if (i % 2 = 0) then collatz (i/2)
else collatz (3*n + 1)) |> ignore
agent.Post (Fire n) // this does not work
// evt.Trigger n // this does works
collatz 13
This is a simple experiment that repeatedly creates a function to find the next number in the Collatz series and then calls itself to return the value until it reaches 1.
What seems to happen is that the trigger only fires once. I tried experimenting with every combination of Async.RunSynchronously / Async.Start / StartChild / SynchronizationContext that I could think of but no progress. I found a blog similar to what I am doing but that didn't help me neither
EDIT
Thank you Fyodor Soikin for pointing out my oversight. The original problem still remains in that I wish to both fire events and reply with a result, but get a timeout.
let evt = new Event<int>()
let stream = evt.Publish
type Agent<'T> = MailboxProcessor<'T>
type Command =
| Fire of int
| Get of int * AsyncReplyChannel<int>
let agent = Agent.Start(fun inbox ->
let rec loop() = async {
let! msg = inbox.Receive()
match msg with
| Fire i -> evt.Trigger i
| Get (i,ch) ->
evt.Trigger i
ch.Reply(i)
return! loop() }
loop())
let on i fn =
stream
|> Observable.filter (fun x -> x = i)
|> Observable.filter (fun x -> x <> 1)
|> Observable.subscribe (fun x -> fn x)
let rec collatz n =
printfn "%d" n
on n (fun i ->
if (i % 2 = 0) then collatz (i/2)
else collatz (3*n + 1)) |> ignore
agent.PostAndReply (fun ch -> (Get (n, ch))) |> ignore // timeout
agent.PostAndAsyncReply (fun ch -> (Get (n, ch))) |> Async.Ignore |> Async.Start // works but I need the result
agent.PostAndAsyncReply (fun ch -> (Get (n, ch))) |> Async.RunSynchronously |> ignore // timeout
collatz 13

Your loop function doesn't loop. It receives the first message, triggers the event, and then just... exits. Never attempts to receive a second message.
You need to make that function work continuously: process the first message, then go right back to receive the next one, then go receive the next one, and so on. Like this:
let agent = Agent.Start(fun inbox ->
let rec loop() = async {
let! msg = inbox.Receive()
let (Fire i) = msg
evt.Trigger i
return! loop() }
loop())
Edit
Since you've reached your limit on questions, I will answer your edit here.
The reason you're getting timeouts in your second snippet is that you have a deadlock in your code. Let's trace the execution to see that.
THREAD 1: The agent is started.
THREAD 2: The first collatz call.
THREAD 2: The first collatz call posts a message to the agent.
THREAD 1: The agent receives the message.
THREAD 1: The agent triggers the event.
THREAD 1: As a result of the event, the second collatz call happens.
THREAD 1: The second collatz call posts a message to the agent.
THREAD 1: The second collatz call starts waiting for the agent to respond.
And this is where the execution ends. The agent cannot respond at this point (in fact, it cannot even receive the next message!), because its instruction pointer is still inside evt.Trigger. The evt.Trigger call hasn't yet returned, so the loop function hasn't yet recursed, so the inbox.Receive function hasn't yet been called, so the second message is still waiting in the agent's queue.
So you get yourself a classic deadlock: collatz is waiting for the agent to receive its message, but the agent is waiting for collatz to finish handling the event.
The simplest, dumbest solution to this would be to just trigger the event asynchronously:
async { evt.Trigger i } |> Async.Start
This will make sure that the event handler is executed not "right there", but asynchronously, possibly on a different thread. This will in turn allow the agent not to wait for the event to be processed before it can continue its own execution loop.
In general though, when dealing with multithreading and asynchrony, one should never call unknown code directly. The agent should never directly call evt.Trigger, or anything else that it doesn't control, because that code might be waiting on the agent itself (which is what happened in your case), thus introducing the deadlock.

Related

How can I create an F# async from a C# method with a callback?

Suppose I have some C# code that takes a callback:
void DoSomething(Action<string> callback);
Now, I want to use this in F#, but wrap it in an async. How would I go about this?
// Not real code
let doSomething = async {
let mutable result = null
new Action(fun x -> result <- x) |> Tasks.DoSomething
// Wait for result to be assigned
return result
}
For example, suppose DoSomething looks like this:
module Tasks
let DoSomething callback =
callback "Hello"
()
Then the output of the following should be "Hello":
let wrappedDoSomething = async {
// Call DoSomething somehow
}
[<EntryPoint>]
let main argv =
async {
let! resultOfDoSomething = wrappedDoSomething
Console.WriteLine resultOfDoSomething
return ()
} |> Async.RunSynchronously
0
The function Async.FromContinuations is, so to say, the "lowest level" of Async. All other async combinators can be expressed in terms of it.
It is the lowest level in the sense that it directly encodes the very nature of async computations - the knowledge of what to do in the three possible cases: (1) a successful completion of the previous computation step, (2) a crash of the previous computation step, and (3) cancellation from outside. These possible cases are expressed as the three function-typed arguments of the function that you pass to Async.FromContinuations. For example:
let returnFive =
Async.FromContinuations( fun (succ, err, cancl) ->
succ 5
)
async {
let! res = returnFive
printfn "%A" res // Prints "5"
}
|> Async.RunSynchronously
Here, my function fun (succ, err, cancl) -> succ 5 has decided that it has completed successfully, and calls the succ continuation to pass its computation result to the next step.
In your case, the function DoSomething expresses only one of the three cases - i.e. "what to do on successful completion". Once you're inside the callback, it means that whatever DoSomething was doing, has completed successfully. That's when you need to call the succ continuation:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( fun res -> succ res )
)
Of course, you can avoid a nested lambda-expression fun res -> succ res by passing succ directly into DoSomething as callback. Unfortunately, you'll have to explicitly specify which type of Action to use for wrapping it, which negates the advantage:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( System.Action<string> succ )
)
As an aside, note that this immediately uncovered a hole in the DoSomething's API: it ignores the error case. What happens if DoSomething fails to do whatever it was meant to do? There is no way you'd know about it, and the whole async workflow will just hang. Or, even worse: the process will exit immediately (depending on how the crash happens).
If you have any control over DoSomething, I suggest you address this issue.
You can try something like:
let doSomething callback = async {
Tasks.DoSomething(callback)
}
If your goal is to define the callback in the method you could do something like:
let doSomething () = async {
let callback = new Action<string>(fun result -> printfn "%A" result )
Tasks.DoSomething(callback)
}
If your goal is to have the result of the async method be used in the DoSomething callback you could do something like:
let doSomething =
Async.StartWithContinuations(
async {
return result
},
(fun result -> Tasks.DoSomething(result)),
(fun _ -> printfn "Deal with exception."),
(fun _ -> printfn "Deal with cancellation."))

How to ensure Async.StartChild is started before continuing?

I am trying to await an event with timeout. I am abstracting this behind a function startAwaitEventWithTimeout. Currently my code looks like this (including some debug output messages):
let startAwaitEventWithTimeout timeoutMs event =
async {
Console.WriteLine("Starting AwaitEvent in eventAwaiter")
let! eventWaiter = Async.StartChild(Async.AwaitEvent event, timeoutMs)
try
Console.WriteLine("Awaiting event in eventAwaiter")
let! res = eventWaiter
return Ok res
with :? TimeoutException ->
return Error ()
} |> Async.StartChild
Here's a test:
let testEvent = Event<string>()
[<EntryPoint>]
let run _ =
async {
Console.WriteLine("Starting event awaiter in main")
let! eventAwaiter = testEvent.Publish |> startAwaitEventWithTimeout 1000
Console.WriteLine("Triggering event")
testEvent.Trigger "foo"
Console.WriteLine("Awaiting event awaiter in main")
let! result = eventAwaiter
match result with
| Ok str -> Console.WriteLine("ok: " + str)
| Error () -> Console.WriteLine("TIMEOUT")
} |> Async.RunSynchronously
0
Unfortunately, even though everything is "awaited" as far as I can see, it seems the run function proceeds to triggering the event before Async.AwaitEvent has had a chance to subscribe to the event. In short, here is the output I get:
Starting event awaiter in main
Starting AwaitEvent in eventAwaiter
Triggering event
Awaiting event awaiter in main
Awaiting event in eventAwaiter
TIMEOUT
Here is what I would expect:
Starting event awaiter in main
Starting AwaitEvent in eventAwaiter
Awaiting event in eventAwaiter <-- this is moved up
Triggering event
Awaiting event awaiter in main
ok foo
I can work around the problem by adding e.g. do! Async.Sleep 100 between calling startAwaitEventWithTimeout and triggering the event, but of course this is less than ideal.
Have I done something incorrectly, and is there any way I can reliably ensure that AwaitEvent has been called before I trigger the event?
(Side note: I am doing this because we are calling remote processes over TCP, and all communication from the remote is done via events.)
Probably I am missing some requirement but your code can easily be refactored using continuations and the error fixed by itself.
let testEvent = Event<unit>()
let run _ =
let ts = new CancellationTokenSource(TimeSpan.FromSeconds(float 1))
let rc r = Console.WriteLine("ok")
let ec _ = Console.WriteLine("exception")
let cc _ = Console.WriteLine("cancelled")
Async.StartWithContinuations((Async.AwaitEvent testEvent.Publish), rc , ec, cc, ts.Token )
testEvent.Trigger()
run()
Edit: If you have a specific requirement to use async workflows, you can convert it by using TaskCompletionSource in TPL.
let registerListener timeout event=
let tcs = TaskCompletionSource()
let ts = new CancellationTokenSource(TimeSpan.FromSeconds(timeout))
let er _ = tcs.SetResult (Error())
Async.StartWithContinuations(Async.AwaitEvent event, tcs.SetResult << Ok , er , er , ts.Token)
Async.AwaitTask tcs.Task
let run _ =
let testEvent = Event<int>()
async {
let listener = registerListener (float 1) testEvent.Publish
testEvent.Trigger 2
let! ta = listener
match ta with
| Ok n -> printfn "ok: %d" n
| Error () -> printfn "error"
} |> Async.RunSynchronously
run()
Note that even though it is far easier to understand than spawning/awaiting multiple child computations, most of this code is still boilerplate and I am sure there must far easier solutions for setting a simple timeout value.
I do not think that you experience a race condition because you are consistently firing the event before the child computation is even started. Let's change the set-up - like you did for testing - to include a delay before firing.
open System
open System.Threading
let e = Event<_>()
let sleeper timeToFire = async{
do! Async.Sleep timeToFire
e.Trigger() }
let waiter = async{
do! Async.AwaitEvent e.Publish
return Ok() }
let foo timeToFire timeOut = async{
Async.Start(sleeper timeToFire)
let! child = Async.StartChild(waiter, timeOut)
try return! child
with :? TimeoutException -> return Error() }
foo 500 1000 |> Async.RunSynchronously
// val it : Result<unit,unit> = Ok null
foo 1000 500 |> Async.RunSynchronously
// val it : Result<unit,unit> = Error null
A race condition will now appear if the delay to firing is equal to the timeout.

F# handling Task cancellation

I am struggling to understand why some code is never executed.
Consider this extension method:
type WebSocketListener with
member x.AsyncAcceptWebSocket = async {
try
let! client = Async.AwaitTask <| x.AcceptWebSocketAsync Async.DefaultCancellationToken
if(not (isNull client)) then
return Some client
else
return None
with
| :? System.Threading.Tasks.TaskCanceledException ->
| :? AggregateException ->
return None
}
I know that AcceptSocketAsync throws a TaskCanceledException when the cancellation token is canceled. I have checked in a C# application. The idea is to return None.
However, that never happens. If I put a breakpoint in the last return None or even in the if expression it never stops there when the cancellation token has been cancelled. And I know it is awaiting in the Async.AwaitTask because if before cancelling, other client connects, it works and it stops in the breakpoints.
I am a little bit lost, why is the exception lost?
Cancellation uses a special path in F# asyncs - Async.AwaitTask will re-route execution of cancelled task to the cancellation continuation. If you want different behavior - you can always do this by manually:
type WebSocketListener with
member x.AsyncAcceptWebSocket = async {
let! ct = Async.CancellationToken
return! Async.FromContinuations(fun (s, e, c) ->
x.AcceptWebSocketAsync(ct).ContinueWith(fun (t: System.Threading.Tasks.Task<_>) ->
if t.IsFaulted then e t.Exception
elif t.IsCanceled then s None // take success path in case of cancellation
else
match t.Result with
| null -> s None
| x -> s (Some x)
)
|> ignore
)
}

Why would disposal of resources be delayed when using the "use" binding within an async computation expression?

I've got an agent which I set up to do some database work in the background. The implementation looks something like this:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
use conn = new System.Data.SqlClient.SqlConnection("...")
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
With this I discovered that if this was called several times in some amount of time I would start getting SqlConnection objects piling up and not being disposed, and eventually I would run out of connections in the connection pool (I don't have exact metrics on how many "several" is, but running an integration test suite twice in a row could always cause the connection pool to run dry).
If I change the use to a using then things are disposed properly and I don't have a problem:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
using (new System.Data.SqlClient.SqlConnection("...")) <| fun conn ->
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
It seems that the Using method of the AsyncBuilder is not properly calling its finally function for some reason, but it's not clear why. Does this have something to do with how I've written my recursive async expression, or is this some obscure bug? And does this suggest that utilizing use within other computation expressions could produce the same sort of behavior?
This is actually the expected behavior - although not entirely obvious!
The use construct disposes of the resource when the execution of the asynchronous workflow leaves the current scope. This is the same as the behavior of use outside of asynchronous workflows. The problem is that recursive call (outside of async) or recursive call using return! (inside async) does not mean that you are leaving the scope. So in this case, the resource is disposed of only after the recursive call returns.
To test this, I'll use a helper that prints when disposed:
let tester () =
{ new System.IDisposable with
member x.Dispose() = printfn "bye" }
The following function terminates the recursion after 10 iterations. This means that it keeps allocating the resources and disposes of all of them only after the entire workflow completes:
let rec loop(n) = async {
if n < 10 then
use t = tester()
do! Async.Sleep(1000)
return! loop(n+1) }
If you run this, it will run for 10 seconds and then print 10 times "bye" - this is because the allocated resources are still in scope during the recursive calls.
In your sample, the using function delimits the scope more explicitly. However, you can do the same using nested asynchronous workflow. The following only has the resource in scope when calling the Sleep method and so it disposes of it before the recursive call:
let rec loop(n) = async {
if n < 10 then
do! async {
use t = tester()
do! Async.Sleep(1000) }
return! loop(n+1) }
Similarly, when you use for loop or other constructs that restrict the scope, the resource is disposed immediately:
let rec loop(n) = async {
for i in 0 .. 10 do
use t = tester()
do! Async.Sleep(1000) }

Async.TryCancelled doesn't work with Async.RunSynchronously

I try to create an agent that updates UI based on user interaction. If user clicks on a button, the GUI should be refreshed. The preparation of model takes a long time, so it is desirable that if user clicks on other button, the preparation is cancelled and the new one is started.
What I have so far:
open System.Threading
type private RefreshMsg =
| RefreshMsg of AsyncReplyChannel<CancellationTokenSource>
type RefresherAgent() =
let mutable cancel : CancellationTokenSource = null
let doSomeModelComputation i =
async {
printfn "start %A" i
do! Async.Sleep(1000)
printfn "middle %A" i
do! Async.Sleep(1000)
printfn "end %A" i
}
let mbox =
MailboxProcessor.Start(fun mbx ->
let rec loop () = async {
let! msg = mbx.Receive()
match msg with
| RefreshMsg(chnl) ->
let cancelSrc = new CancellationTokenSource()
chnl.Reply(cancelSrc)
let update = async {
do! doSomeModelComputation 1
do! doSomeModelComputation 2
//do! updateUI // not important now
}
let cupdate = Async.TryCancelled(update, (fun c -> printfn "refresh cancelled"))
Async.RunSynchronously(cupdate, -1, cancelSrc.Token)
printfn "loop()"
return! loop()
}
loop ())
do
mbox.Error.Add(fun exn -> printfn "Error in refresher: %A" exn)
member x.Refresh() =
if cancel <> null then
// I don't handle whether the previous computation finished
// I just cancel it; might be improved
cancel.Cancel()
cancel.Dispose()
cancel <- mbox.PostAndReply(fun reply -> RefreshMsg(reply))
printfn "x.Refresh end"
//sample
let agent = RefresherAgent()
agent.Refresh()
System.Threading.Thread.Sleep(1500)
agent.Refresh()
I return a CancellationTokenSource for each request and store it in a mutable variable (the x.Refresh() is thread safe, it is called on UI thread).
If Refresh() is called for the first time, the cancellation source is returned. If Refresh() is called for the second time, I call Cancel which should abort the async task that I run through Async.RunSynchronously.
However, an exception is raised. The output from my sample is
x.Refresh end
start 1
middle 1
end 1
refresh cancelled
Error in refresher: System.OperationCanceledException: The operation was canceled.
at Microsoft.FSharp.Control.AsyncBuilderImpl.commit[a](Result`1 res)
Now as I think about this, it might make sense, because the thread on which the agent runs, was interrputed, right? But, how do I achieve the desired behaviour?
I need to cancel async workflow inside the agent, so that the agent can continue consuming new messages. Why do I use the mailbox processor? Cause it is guaranteed that only one thread is trying to create UI model, so I save resources.
Let's suppose I create UI model by downloading data from several web services, that's why I use async call. When user changes a combo and select other option, I want to stop querying the webservices (= cancel the async calls) with old value and want to create new model base od web services call with new value.
Any suggestion that I can use instead of my solution and will solve my problem, is also welcome.
I have difficulties in trying to understand what you want to achieve. But maybe this does not matter - the error just says that the workflow you are executing with RunSynchronously was canceled (RunSynchronously will throw the exception) - so you can wrap this call into a try-match block and just ignore the OC-Exception
a better option might be to refactor your cupdate and to the try-match inside of this - you can even bring the in TryCancelled into it if you catch the OC-Exceptions directly ;)
let update =
async {
try
do! doSomeModelComputation 1
do! doSomeModelComputation 2
with
| :? OperationCanceledException ->
printfn "refresh cancelled"
}
Async.RunSynchronously(update, -1, cancelSrc.Token)
But I still don't get the part why you want this Synchronously

Resources