I have stumbled upon a problem when calling a nested Async which happens to be null. An exception is raised but it can't be catched with any of the normal exception handling methods Async workflows provide.
The following is a simple test which reproduces the problem:
[<Test>]
let ``Nested async is null with try-with``() =
let g(): Async<unit> = Unchecked.defaultof<Async<unit>>
let f = async {
try
do! g()
with e ->
printf "%A" e
}
f |> Async.RunSynchronously |> ignore
which results in the follwing exception:
System.NullReferenceException : Object reference not set to an instance of an object.
at Microsoft.FSharp.Control.AsyncBuilderImpl.bindA#714.Invoke(AsyncParams`1 args)
at <StartupCode$FSharp-Core>.$Control.loop#413-40(Trampoline this, FSharpFunc`2 action)
at Microsoft.FSharp.Control.Trampoline.ExecuteAction(FSharpFunc`2 firstAction)
at Microsoft.FSharp.Control.TrampolineHolder.Protect(FSharpFunc`2 firstAction)
at Microsoft.FSharp.Control.AsyncBuilderImpl.startAsync(CancellationToken cancellationToken, FSharpFunc`2 cont, FSharpFunc`2 econt, FSharpFunc`2 ccont, FSharpAsync`1 p)
at Microsoft.FSharp.Control.CancellationTokenOps.starter#1121-1.Invoke(CancellationToken cancellationToken, FSharpFunc`2 cont, FSharpFunc`2 econt, FSharpFunc`2 ccont, FSharpAsync`1 p)
at Microsoft.FSharp.Control.CancellationTokenOps.RunSynchronously(CancellationToken token, FSharpAsync`1 computation, FSharpOption`1 timeout)
at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously(FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
at Prioinfo.Urkund.DocCheck3.Core2.Tests.AsyncTests.Nested async is null with try-with() in SystemTests.fs: line 345
I really think the exception should be caught in this case, or is this really the expected behavior? (I'm using Visual Studio 2010 Sp1 for the record)
Also, Async.Catch and Async.StartWithContinuations exhibits the same problem as demonstrated by these test cases:
[<Test>]
let ``Nested async is null with Async.Catch``() =
let g(): Async<unit> = Unchecked.defaultof<Async<unit>>
let f = async {
do! g()
}
f |> Async.Catch |> Async.RunSynchronously |> ignore
[<Test>]
let ``Nested async is null with StartWithContinuations``() =
let g(): Async<unit> = Unchecked.defaultof<Async<unit>>
let f = async {
do! g()
}
Async.StartWithContinuations(f
, fun _ -> ()
, fun e -> printfn "%A" e
, fun _ -> ())
It seems the exception is raised within the bind-method in the workflow builder and my guess is that as a result the normal error handling code is bypassed. It looks like a bug in the implementation of async workflows to me since I haven't found anything in the documentation or elsewhere which suggest that this is the intended behavior.
It is pretty easy to work around in most cases I think so it's not a huge problem for me at least but it is a bit unsettling since it means that you can't completely trust the async exception handling mechanism to be able to capture all exceptions.
Edit:
After giving it some thought I agree with kvb. Null asyncs should not really exist in normal code and could really only be produced if you do something you probably shouldn't (such as using Unchecked.defaultOf) or use reflection to produce the values (in my case it was a mocking framework involved). Thus it's not really a bug but more of an edge case.
I don't think it's a bug. As the name indicates Unchecked.defaultof<_> does not check that the values it produces are valid, and Async<unit> does not support null as a proper value (e.g. see the message if you try to use let x : Async<unit> = null). Async.Catch and the like are intended to catch exceptions thrown within asynchronous computations, not exceptions caused by sneaking behind the compiler's back and creating invalid asynchronous computations.
I fully agree with kvb - when you initialize a value using Unchecked.defaultOf, it means that the behaviour of using the value may be undefined, so this cannot be treated as bug. In practice, you don't have to worry about it, because you should never get null values of Async<'T> type.
To add some more details, the exception cannot be handled, because the translation looks as follows:
async.TryWith
( async.Bind ( Unchecked.defaultof<_>,
fun v -> async { printfn "continued" } ),
fun e -> printfn "%A" e)
The exception is thrown from the Bind method before the workflow returned by Bind is started (it happens after you call RunSynchronously, because the workflow is wrapped using Delay, but it happens outside of the workflow execution). If you want to handle this kinds of exceptions (arising from incorrectly constructed workflows), you can write a version of TryWith that runs the workflow and handles exceptions thrown outside of the execution:
let TryWith(work, handler) =
Async.FromContinuations(fun (cont, econt, ccont) ->
try
async { let! res = work in cont res }
|> Async.StartImmediate
with e ->
async { let! res = handler e in cont res }
|> Async.StartImmediate )
Then you can handle exceptions like this:
let g(): Async<unit> = Unchecked.defaultof<Async<unit>>
let f =
TryWith
( (async { do! g() }),
(fun e -> async { printfn "error %A" e }))
f |> Async.RunSynchronously
Related
Suppose I have some C# code that takes a callback:
void DoSomething(Action<string> callback);
Now, I want to use this in F#, but wrap it in an async. How would I go about this?
// Not real code
let doSomething = async {
let mutable result = null
new Action(fun x -> result <- x) |> Tasks.DoSomething
// Wait for result to be assigned
return result
}
For example, suppose DoSomething looks like this:
module Tasks
let DoSomething callback =
callback "Hello"
()
Then the output of the following should be "Hello":
let wrappedDoSomething = async {
// Call DoSomething somehow
}
[<EntryPoint>]
let main argv =
async {
let! resultOfDoSomething = wrappedDoSomething
Console.WriteLine resultOfDoSomething
return ()
} |> Async.RunSynchronously
0
The function Async.FromContinuations is, so to say, the "lowest level" of Async. All other async combinators can be expressed in terms of it.
It is the lowest level in the sense that it directly encodes the very nature of async computations - the knowledge of what to do in the three possible cases: (1) a successful completion of the previous computation step, (2) a crash of the previous computation step, and (3) cancellation from outside. These possible cases are expressed as the three function-typed arguments of the function that you pass to Async.FromContinuations. For example:
let returnFive =
Async.FromContinuations( fun (succ, err, cancl) ->
succ 5
)
async {
let! res = returnFive
printfn "%A" res // Prints "5"
}
|> Async.RunSynchronously
Here, my function fun (succ, err, cancl) -> succ 5 has decided that it has completed successfully, and calls the succ continuation to pass its computation result to the next step.
In your case, the function DoSomething expresses only one of the three cases - i.e. "what to do on successful completion". Once you're inside the callback, it means that whatever DoSomething was doing, has completed successfully. That's when you need to call the succ continuation:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( fun res -> succ res )
)
Of course, you can avoid a nested lambda-expression fun res -> succ res by passing succ directly into DoSomething as callback. Unfortunately, you'll have to explicitly specify which type of Action to use for wrapping it, which negates the advantage:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( System.Action<string> succ )
)
As an aside, note that this immediately uncovered a hole in the DoSomething's API: it ignores the error case. What happens if DoSomething fails to do whatever it was meant to do? There is no way you'd know about it, and the whole async workflow will just hang. Or, even worse: the process will exit immediately (depending on how the crash happens).
If you have any control over DoSomething, I suggest you address this issue.
You can try something like:
let doSomething callback = async {
Tasks.DoSomething(callback)
}
If your goal is to define the callback in the method you could do something like:
let doSomething () = async {
let callback = new Action<string>(fun result -> printfn "%A" result )
Tasks.DoSomething(callback)
}
If your goal is to have the result of the async method be used in the DoSomething callback you could do something like:
let doSomething =
Async.StartWithContinuations(
async {
return result
},
(fun result -> Tasks.DoSomething(result)),
(fun _ -> printfn "Deal with exception."),
(fun _ -> printfn "Deal with cancellation."))
I am struggling to understand why some code is never executed.
Consider this extension method:
type WebSocketListener with
member x.AsyncAcceptWebSocket = async {
try
let! client = Async.AwaitTask <| x.AcceptWebSocketAsync Async.DefaultCancellationToken
if(not (isNull client)) then
return Some client
else
return None
with
| :? System.Threading.Tasks.TaskCanceledException ->
| :? AggregateException ->
return None
}
I know that AcceptSocketAsync throws a TaskCanceledException when the cancellation token is canceled. I have checked in a C# application. The idea is to return None.
However, that never happens. If I put a breakpoint in the last return None or even in the if expression it never stops there when the cancellation token has been cancelled. And I know it is awaiting in the Async.AwaitTask because if before cancelling, other client connects, it works and it stops in the breakpoints.
I am a little bit lost, why is the exception lost?
Cancellation uses a special path in F# asyncs - Async.AwaitTask will re-route execution of cancelled task to the cancellation continuation. If you want different behavior - you can always do this by manually:
type WebSocketListener with
member x.AsyncAcceptWebSocket = async {
let! ct = Async.CancellationToken
return! Async.FromContinuations(fun (s, e, c) ->
x.AcceptWebSocketAsync(ct).ContinueWith(fun (t: System.Threading.Tasks.Task<_>) ->
if t.IsFaulted then e t.Exception
elif t.IsCanceled then s None // take success path in case of cancellation
else
match t.Result with
| null -> s None
| x -> s (Some x)
)
|> ignore
)
}
I've got an agent which I set up to do some database work in the background. The implementation looks something like this:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
use conn = new System.Data.SqlClient.SqlConnection("...")
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
With this I discovered that if this was called several times in some amount of time I would start getting SqlConnection objects piling up and not being disposed, and eventually I would run out of connections in the connection pool (I don't have exact metrics on how many "several" is, but running an integration test suite twice in a row could always cause the connection pool to run dry).
If I change the use to a using then things are disposed properly and I don't have a problem:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
using (new System.Data.SqlClient.SqlConnection("...")) <| fun conn ->
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
It seems that the Using method of the AsyncBuilder is not properly calling its finally function for some reason, but it's not clear why. Does this have something to do with how I've written my recursive async expression, or is this some obscure bug? And does this suggest that utilizing use within other computation expressions could produce the same sort of behavior?
This is actually the expected behavior - although not entirely obvious!
The use construct disposes of the resource when the execution of the asynchronous workflow leaves the current scope. This is the same as the behavior of use outside of asynchronous workflows. The problem is that recursive call (outside of async) or recursive call using return! (inside async) does not mean that you are leaving the scope. So in this case, the resource is disposed of only after the recursive call returns.
To test this, I'll use a helper that prints when disposed:
let tester () =
{ new System.IDisposable with
member x.Dispose() = printfn "bye" }
The following function terminates the recursion after 10 iterations. This means that it keeps allocating the resources and disposes of all of them only after the entire workflow completes:
let rec loop(n) = async {
if n < 10 then
use t = tester()
do! Async.Sleep(1000)
return! loop(n+1) }
If you run this, it will run for 10 seconds and then print 10 times "bye" - this is because the allocated resources are still in scope during the recursive calls.
In your sample, the using function delimits the scope more explicitly. However, you can do the same using nested asynchronous workflow. The following only has the resource in scope when calling the Sleep method and so it disposes of it before the recursive call:
let rec loop(n) = async {
if n < 10 then
do! async {
use t = tester()
do! Async.Sleep(1000) }
return! loop(n+1) }
Similarly, when you use for loop or other constructs that restrict the scope, the resource is disposed immediately:
let rec loop(n) = async {
for i in 0 .. 10 do
use t = tester()
do! Async.Sleep(1000) }
I've found this snippet:
http://fssnip.net/8o
But I'm working not only with retriable functions, but also with asynchronous such, and I was wondering how I make this type properly. I have a tiny piece of retryAsync monad that I'd like to use as a replacement for async computations, but that contains retry logic, and I'm wondering how I combine them?
type AsyncRetryBuilder(retries) =
member x.Return a = a // Enable 'return'
member x.ReturnFrom a = x.Run a
member x.Delay f = f // Gets wrapped body and returns it (as it is)
// so that the body is passed to 'Run'
member x.Bind expr f = async {
let! tmp = expr
return tmp
}
member x.Zero = failwith "Zero"
member x.Run (f : unit -> Async<_>) : _ =
let rec loop = function
| 0, Some(ex) -> raise ex
| n, _ ->
try
async { let! v = f()
return v }
with ex -> loop (n-1, Some(ex))
loop(retries, None)
let asyncRetry = AsyncRetryBuilder(4)
Consuming code is like this:
module Queue =
let desc (nm : NamespaceManager) name = asyncRetry {
let! exists = Async.FromBeginEnd(name, nm.BeginQueueExists, nm.EndQueueExists)
let beginCreate = nm.BeginCreateQueue : string * AsyncCallback * obj -> IAsyncResult
return! if exists then Async.FromBeginEnd(name, nm.BeginGetQueue, nm.EndGetQueue)
else Async.FromBeginEnd(name, beginCreate, nm.EndCreateQueue)
}
let recv (client : MessageReceiver) timeout =
let bRecv = client.BeginReceive : TimeSpan * AsyncCallback * obj -> IAsyncResult
asyncRetry {
let! res = Async.FromBeginEnd(timeout, bRecv, client.EndReceive)
return res }
Error is:
This expression was expected to have type Async<'a> but here has type 'b -> Async<'c>
Your Bind operation behaves like a normal Bind operation of async, so your code is mostly a re-implementation (or wrapper) over async. However, your Return does not have the right type (it should be 'T -> Async<'T>) and your Delay is also different than normal Delay of async. In general, you should start with Bind and Return - using Run is a bit tricky, because Run is used to wrap the entire foo { .. } block and so it does not give you the usual nice composability.
The F# specification and a free chapter 12 from Real-World Functional Programming both show the usual types that you should follow when implementing these operations, so I won't repeat that here.
The main issue with your approach is that you're trying to retry the computation only in Run, but the retry builder that you're referring to attempts to retry each individual operation called using let!. Your approach may be sufficient, but if that's the case, you can just implement a function that tries to run normal Async<'T> and retries:
let RetryRun count (work:Async<'T>) = async {
try
// Try to run the work
return! work
with e ->
// Retry if the count is larger than 0, otherwise fail
if count > 0 then return! RetryRun (count - 1) work
else return raise e }
If you actually want to implement a computation builder that will implicitly try to retry every single asynchronous operation, then you can write something like the following (it is just a sketch, but it should point you in the right direction):
// We're working with normal Async<'T> and
// attempt to retry it until it succeeds, so
// the computation has type Async<'T>
type RetryAsyncBuilder() =
member x.ReturnFrom(comp) = comp // Just return the computation
member x.Return(v) = async { return v } // Return value inside async
member x.Delay(f) = async { return! f() } // Wrap function inside async
member x.Bind(work, f) =
async {
try
// Try to call the input workflow
let! v = work
// If it succeeds, try to do the rest of the work
return! f v
with e ->
// In case of exception, call Bind to try again
return! x.Bind(work, f) }
I needed to reraise an exception that occurs while executing an async block, after logging the exception.
When I do the following the compiler thinks that I am not calling the reraise function from within the handler. What am I doing wrong?
let executeAsync context = async {
traceContext.Properties.Add("CorrelationId", context.CorrelationId)
try
do! runAsync context
return None
with
| e when isCriticalException(e) ->
logCriticalException e
reraise()
| e ->
logException e
return Some(e)
}
Rough! I think this is impossible, because reraise corresponds to a special IL instruction that grabs the exception from the top of the stack, but the way async expressions are compiled into a chain of continuations, I don't think the semantics hold!
For the same reason, the following won't compile either:
try
(null:string).ToString()
with e ->
(fun () -> reraise())()
In these situations, where I need to handle the exception outside of the actual with body, and would like to emulate reraise (that is, preserve the stack trace of the exception), I use this solution, so all together your code would look like:
let inline reraisePreserveStackTrace (e:Exception) =
let remoteStackTraceString = typeof<exn>.GetField("_remoteStackTraceString", BindingFlags.Instance ||| BindingFlags.NonPublic);
remoteStackTraceString.SetValue(e, e.StackTrace + Environment.NewLine);
raise e
let executeAsync context = async {
traceContext.Properties.Add("CorrelationId", context.CorrelationId)
try
do! runAsync context
return None
with
| e when isCriticalException(e) ->
logCriticalException e
reraisePreserveStackTrace e
| e ->
logException e
return Some(e)
}
Update: .NET 4.5 introduced ExceptionDispatchInfo which may allow a cleaner implementation of reraisePreserveStackTrace above.
I ran in to a similar problem, in a different context, but it boils down to this.
Exceptions cannot be thrown onto a different thread - calling reraise() would require an exception handler running in some sense 'above' the original async block in the code.
let runAsync context = async {return ()}
let isCriticalException e = true
let logCriticalException e = ()
let logException e = ()
let executeAsync context =
async {
do! runAsync context
return None
}
let run =
match executeAsync 5 |> Async.Catch |> Async.RunSynchronously with
|Choice1Of2(t) ->
printfn "%A" t
None
|Choice2Of2(exn) ->
match exn with
| e when isCriticalException(e) ->
logCriticalException e
raise (new System.Exception("See inner exception",e)) //stack trace will be lost at this point if the exn is not wrapped
| e ->
logException e
Some(e)
Note, we still can't use reraise, as we are now calling on a different thread, so we wrap the exception inside another one
As mentioned in comments and updates to answers, you can use ExceptionDispatchInfo as a workaround.
open System.Runtime.ExceptionServices
let inline reraiseAnywhere<'a> (e: exn) : 'a =
ExceptionDispatchInfo.Capture(e).Throw()
Unchecked.defaultof<'a>
Usage:
async {
try
do! someAsyncThing ()
return "Success"
with
| e ->
if errorIsOk e then return "Handled error"
else return (reraiseAnyWhere e)
}
Here is a GitHub Gist I made with unit tests to check its behavior.