I'm playing around with using SqlClient in F# and I'm having difficulty with using SqlDataReader.ReadAsync. I'm trying to do the F# equivalent of
while (await reader.ReadAsync) { ... }
What is the best way to do this in F#? Below is my full program. It works, but I'd like to know if there is a better way to do it.
open System
open System.Data.SqlClient
open System.Threading.Tasks
let connectionString = "Server=.;Integrated Security=SSPI"
module Async =
let AwaitVoidTask : (Task -> Async<unit>) =
Async.AwaitIAsyncResult >> Async.Ignore
// QUESTION: Is this idiomatic F#? Is there a more generally-used way of doing this?
let rec While (predicateFn : unit -> Async<bool>) (action : unit -> unit) : Async<unit> =
async {
let! b = predicateFn()
match b with
| true -> action(); do! While predicateFn action
| false -> ()
}
[<EntryPoint>]
let main argv =
let work = async {
// Open connection
use conn = new SqlConnection(connectionString)
do! conn.OpenAsync() |> Async.AwaitVoidTask
// Execute command
use cmd = conn.CreateCommand()
cmd.CommandText <- "select name from sys.databases"
let! reader = cmd.ExecuteReaderAsync() |> Async.AwaitTask
// Consume reader
// I want a convenient 'while' loop like this...
//while reader.ReadAsync() |> Async.AwaitTask do // Error: This expression was expected to have type bool but here has type Async<bool>
// reader.GetValue 0 |> string |> printfn "%s"
// Instead I used the 'Async.While' method that I defined above.
let ConsumeReader = Async.While (fun () -> reader.ReadAsync() |> Async.AwaitTask)
do! ConsumeReader (fun () -> reader.GetValue 0 |> string |> printfn "%s")
}
work |> Async.RunSynchronously
0 // return an integer exit code
There is one issue in your code which is that you're doing a recursive call using
do! While predicateFn action. This is a problem because it does not turn into a tail-call and so you could end up with memory leaks. The right way to do this is to use return! instead of do!.
Aside from that, your code works good. But you can actually extend the async computation builder to let you use ordinary while keyword. To do that, you need a slightly different version of While:
let rec While (predicateFn : unit -> Async<bool>) (action : Async<unit>) : Async<unit> =
async {
let! b = predicateFn()
if b then
do! action
return! While predicateFn action
}
type AsyncBuilder with
member x.While(cond, body) = Async.While cond body
Here, the body is also asynchronous and it is not a function. Then we add a While method to the computation builder (so we are adding another overload as an extension method). With this, you can actually write:
while Async.AwaitTask(reader.ReadAsync()) do // This is async!
do! Async.Sleep(1000) // The body is asynchronous too
reader.GetValue 0 |> string |> printfn "%s"
I'd probably do the same as you. If you can stomach refs though, you can shorten it to
let go = ref true
while !go do
let! more = reader.ReadAsync() |> Async.AwaitTask
go := more
reader.GetValue 0 |> string |> printfn "%s"
Related
I have some potentially very long running function, which may sometimes hang up. So, I thought that if I wrap it into an async workflow, then I should be able to cancel it. Here is an FSI example that does not work (but the same behavior happens with the compiled code):
open System.Threading
let mutable counter = 0
/// Emulates an external C# sync function that hung up.
/// Please, don't change it to some F# async stuff because
/// that won't fix that C# method.
let run() =
while true
do
printfn "counter = %A" counter
Thread.Sleep 1000
counter <- counter + 1
let onRunModel() =
let c = new CancellationTokenSource()
let m = async { do run() }
Async.Start (m, c.Token)
c
let tryCancel() =
printfn "Starting..."
let c = onRunModel()
printfn "Waiting..."
Thread.Sleep 5000
printfn "Cancelling..."
c.Cancel()
printfn "Waiting again..."
Thread.Sleep 5000
printfn "Completed."
#time
tryCancel()
#time
If you run it in FSI, then you will see something like that:
Starting...
Waiting...
counter = 0
counter = 1
counter = 2
counter = 3
counter = 4
Cancelling...
Waiting again...
counter = 5
counter = 6
counter = 7
counter = 8
counter = 9
Completed.
Real: 00:00:10.004, CPU: 00:00:00.062, GC gen0: 0, gen1: 0, gen2: 0
counter = 10
counter = 11
counter = 12
counter = 13
counter = 14
counter = 15
counter = 16
which means that it does not stop at all after c.Cancel() is called.
What am I doing wrong and how to make such thing work?
Here is some additional information:
When the code hangs up, it does it in some external sync C# library,
which I have no control of. So checking for cancellation token in
the code that I control is useless. That's why function run()
above was modeled that way.
I don't need any communication of completion and / or progress. It's already done via some messaging system and it is out of scope
of the question.
Basically I just need to kill background work as soon as I "decide" to do so.
You are handing off control to a code segment, which, albeit wrapped in an async block, has no means of checking for the cancellation. Were you to construct your loop directly wrapped in an async, or have it replaced by a recursive async loop, it will work as expected:
let run0 () = // does not cancel
let counter = ref 0
while true do
printfn "(0) counter = %A" !counter
Thread.Sleep 1000
incr counter
let m = async { run0 () }
let run1 () = // cancels
let counter = ref 0
async{
while true do
printfn "(1) counter = %A" !counter
Thread.Sleep 1000
incr counter }
let run2 = // cancels too
let rec aux counter = async {
printfn "(2) counter = %A" counter
Thread.Sleep 1000
return! aux (counter + 1) }
aux 0
printfn "Starting..."
let cts = new CancellationTokenSource()
Async.Start(m, cts.Token)
Async.Start(run1(), cts.Token)
Async.Start(run2, cts.Token)
printfn "Waiting..."
Thread.Sleep 5000
printfn "Cancelling..."
cts.Cancel()
printfn "Waiting again..."
Thread.Sleep 5000
printfn "Completed."
A word of caution though: Nested async calls in F# are automatically checked for cancellation, which is why do! Async.Sleep is preferable. If you are going down the recursive route, be sure to enable tail-recursion via return!. Further reading: Scott W.'s blog on Asynchronous programming, and Async in C# and F# Asynchronous gotchas in C# by Tomas Petricek.
This piece of code was developed to solve a situation where I couldn't get some calls to terminate/timeout. They would just hang. Maybe you can get some ideas that will help you solve your problem.
The interesting part for you would be only the two first functions. The rest is only to demonstrate how I'm using them.
module RobustTcp =
open System
open System.Text
open System.Net.Sockets
open Railway
let private asyncSleep (sleepTime: int) (error: 'a) = async {
do! Async.Sleep sleepTime
return Some error
}
let private asyncWithTimeout asy (timeout: int) (error: 'a) =
Async.Choice [ asy; asyncSleep timeout error ]
let private connectTcpClient (host: string) (port: int) (tcpClient: TcpClient) = async {
let asyncConnect = async {
do! tcpClient.ConnectAsync(host, port) |> Async.AwaitTask
return Some tcpClient.Connected }
match! asyncWithTimeout asyncConnect 1_000 false with
| Some isConnected -> return Ok isConnected
| None -> return Error "unexpected logic error in connectTcpClient"
}
let private writeTcpClient (outBytes: byte[]) (tcpClient: TcpClient) = async {
let asyncWrite = async {
let stream = tcpClient.GetStream()
do! stream.WriteAsync(outBytes, 0, outBytes.Length) |> Async.AwaitTask
do! stream.FlushAsync() |> Async.AwaitTask
return Some (Ok ()) }
match! asyncWithTimeout asyncWrite 10_000 (Error "timeout writing") with
| Some isWrite -> return isWrite
| None -> return Error "unexpected logic error in writeTcpClient"
}
let private readTcpClient (tcpClient: TcpClient) = async {
let asyncRead = async {
let inBytes: byte[] = Array.zeroCreate 1024
let stream = tcpClient.GetStream()
let! byteCount = stream.ReadAsync(inBytes, 0, inBytes.Length) |> Async.AwaitTask
let bytesToReturn = inBytes.[ 0 .. byteCount - 1 ]
return Some (Ok bytesToReturn) }
match! asyncWithTimeout asyncRead 2_000 (Error "timeout reading reply") with
| Some isRead ->
match isRead with
| Ok s -> return Ok s
| Error error -> return Error error
| None -> return Error "unexpected logic error in readTcpClient"
}
let sendReceiveBytes (host: string) (port: int) (bytesToSend: byte[]) = async {
try
use tcpClient = new TcpClient()
match! connectTcpClient host port tcpClient with
| Ok isConnected ->
match isConnected with
| true ->
match! writeTcpClient bytesToSend tcpClient with
| Ok () ->
let! gotData = readTcpClient tcpClient
match gotData with
| Ok result -> return Ok result
| Error error -> return Error error
| Error error -> return Error error
| false -> return Error "Not connected."
| Error error -> return Error error
with
| :? AggregateException as ex ->
(* TODO ? *)
return Error ex.Message
| ex ->
(*
printfn "Exception in getStatus : %s" ex.Message
*)
return Error ex.Message
}
let sendReceiveText (host: string) (port: int) (textToSend: string) (encoding: Encoding) =
encoding.GetBytes textToSend
|> sendReceiveBytes host port
|> Async.map (Result.map encoding.GetString)
I am attempting to create a list of strings which gets elements gradually inserted into asynchronously with the help of a mailbox processor. However I am not getting the desired output.
I have pretty much followed the code from https://fsharpforfunandprofit.com/posts/concurrency-actor-model/
however it does not seem to work as intended in my case. The code I have is as follows:
type TransactionQueue ={
queue : string list
} with
static member UpdateState (msg : string) (tq : TransactionQueue) =
{tq with queue = (msg :: tq.queue)}
static member Agent = MailboxProcessor.Start(fun inbox ->
let rec msgLoop (t : TransactionQueue) =
async{
let! msg = inbox.Receive()
let newT = TransactionQueue.UpdateState msg t
printfn "%A" newT
return! msgLoop newT
}
msgLoop {queue = []}
)
static member Add i = TransactionQueue.Agent.Post i
[<EntryPoint>]
let main argv =
// test in isolation
printfn "welcome to test"
let rec loop () =
let str = Console.ReadLine()
TransactionQueue.Add str
loop ()
loop ()
0
The result i keep getting is a list of the latest input only, the state is not kept. So if I enter "a" then "b" then "c" the queue will only have the value "c" instead of "a";"b";"c"
Any help or pointers would be most appreciated!
Just like C# Properties, your Agent is really a Property and thus behaves like a method with void parameter. That’s why you will get a new agent everytime Agent property is accessed.
In idiomatic F# there are two styles when implementing agents. If you don’t need to have many agent instances, just write a module and encapsule the agent-related stuff inside. Otherwise, OOP style should be used.
Code for style #1
module TransactionQueue =
type private Queue = Queue of string list
let private empty = Queue []
let private update item (Queue items) = Queue (item :: items)
let private agent = MailboxProcessor.Start <| fun inbox ->
let rec msgLoop queue = async {
let! msg = inbox.Receive ()
return! queue |> update msg |> msgLoop
}
msgLoop empty
let add item = agent.Post item
[<EntryPoint>]
let main argv =
// test in isolation
printfn "welcome to test"
let rec loop () =
let str = Console.ReadLine()
TransactionQueue.add str
loop ()
loop ()
Code for style #2
type Queue = Queue of string list with
static member Empty = Queue []
static member Update item (Queue items) =
Queue (item :: items)
type Agent () =
let agent = MailboxProcessor.Start <| fun inbox ->
let rec msgLoop queue = async {
let! msg = inbox.Receive ()
return! queue |> Queue.Update msg |> msgLoop
}
msgLoop Queue.Empty
member this.Add item = agent.Post item
[<EntryPoint>]
let main argv =
// test in isolation
printfn "welcome to test"
let agent = new Agent ()
let rec loop () =
let str = Console.ReadLine()
agent.Add str
loop ()
loop ()
Notice the use of Single-case union types for the Queue type.
Let's say I have a synchronous expensive operation:
let SomeExpensiveOp():string=
System.Console.WriteLine"about to begin expensive OP"
System.Threading.Thread.Sleep(TimeSpan.FromSeconds 2.0)
System.Console.WriteLine"finished expensive OP"
"foo"
That I wrap as an async job:
let SomeExpensiveOpAs():Async<string>=async {
return SomeExpensiveOp()}
Now I want to use this expensive operation to combine it with other two:
let SomeExpensiveOpSeq():seq<Async<string>>=
let op = SomeExpensiveOpAs()
seq {
for some in [Bar(); Baz()] do
yield async {
let! prefix = op
let! someAfterWaiting = some
return (String.Concat (prefix, someAfterWaiting))
}
}
The purpose of putting it into a seq<Async<'T>> is to be able to use Async.Parallel this way:
let DoSomething() =
let someAsyncOps = SomeExpensiveOpSeq() |> List.ofSeq
let newOp = SomeExpensiveOpAs()
let moreAsyncOps = (newOp::someAsyncOps)
let allStrings = Async.RunSynchronously(Async.Parallel moreAsyncOps)
for str in allStrings do
Console.WriteLine str
Console.WriteLine()
However, this makes SomeExpensiveOp to be executed three times. I would expect the second time to be executed an extra time because of the newOp call above, but I was expecting SomeExpensiveOpSeq to reuse the call to SomeExpensiveOp instead of calling it twice. How can I achieve SomeExpensiveOpSeq to only call SomeExpensiveOp once and reuse that for subsequent results?
The key observation here is that let! is invoking the async expression every time—nothing caches its result. Consider this example where we have expOp : Async<string> but we await it three times in an async expression:
let expOp = SomeExpensiveOpAs()
async {
let! a = expOp
let! b = expOp
let! c = expOp
return [a;b;c]
} |> Async.RunSynchronously
about to begin expensive OP
finished expensive OP
about to begin expensive OP
finished expensive OP
about to begin expensive OP
finished expensive OP
val it : string list = ["foo"; "foo"; "foo"]
You can see the async expensive op gets evaluated each time. If you only want to execute that expensive operation once, you could fully evaluate/await its result and use that instead of awaiting it multiple times:
let SomeExpensiveOpSeq():seq<Async<string>>=
let op = SomeExpensiveOpAs() |> Async.RunSynchronously
seq {
for some in [Bar(); Baz()] do
yield async {
let! someAfterWaiting = some
return (String.Concat (op, someAfterWaiting))
}
}
This will still result in the expensive op being executed twice in your code—once in SomeExpensiveOpSeq and another as a result of being prepended on to moreAsyncOps—but it could be refactored further to a single invocation. Basically, if all subsequent async ops depend on this expensive evaluation, why not evaluate it once/first and then use its value wherever necessary:
let SomeExpensiveOpSeq op : seq<Async<string>>=
seq {
for some in [Bar(); Baz()] do
yield async {
let! someAfterWaiting = some
return (String.Concat (op, someAfterWaiting))
}
}
let DoSomething() =
let newOp = SomeExpensiveOpAs() |> Async.RunSynchronously
let someAsyncOps = SomeExpensiveOpSeq newOp |> Async.Parallel |> Async.RunSynchronously
let allStrings = newOp::(List.ofArray someAsyncOps)
for str in allStrings do
Console.WriteLine str
Console.WriteLine()
> DoSomething();;
about to begin expensive OP
finished expensive OP
foo
foobar
foobaz
I tried to reduce this to the smallest possible repro, but it's still a bit long-ish, my apologies.
I have an F# project that references a C# project with code like the following.
public static class CSharpClass {
public static async Task AsyncMethod(CancellationToken cancellationToken) {
await Task.Delay(3000);
cancellationToken.ThrowIfCancellationRequested();
}
}
Here's the F# code.
type Message =
| Work of CancellationToken
| Quit of AsyncReplyChannel<unit>
let mkAgent() = MailboxProcessor.Start <| fun inbox ->
let rec loop() = async {
let! msg = inbox.TryReceive(250)
match msg with
| Some (Work cancellationToken) ->
let! result =
CSharpClass.AsyncMethod(cancellationToken)
|> Async.AwaitTask
|> Async.Catch
// THIS POINT IS NEVER REACHED AFTER CANCELLATION
match result with
| Choice1Of2 _ -> printfn "Success"
| Choice2Of2 exn -> printfn "Error: %A" exn
return! loop()
| Some (Quit replyChannel) -> replyChannel.Reply()
| None -> return! loop()
}
loop()
[<EntryPoint>]
let main argv =
let agent = mkAgent()
use cts = new CancellationTokenSource()
agent.Post(Work cts.Token)
printfn "Press any to cancel."
System.Console.Read() |> ignore
cts.Cancel()
printfn "Cancelled."
agent.PostAndReply Quit
printfn "Done."
System.Console.Read()
The issue is that, upon cancellation, control never returns to the async block. I'm not sure if it's hanging in AwaitTask or Catch. Intuition tells me it's blocking when trying to return to the previous sync context, but I'm not sure how to confirm this. I'm looking for ideas on how to troubleshoot this, or perhaps someone with a deeper understanding here can spot the issue.
POSSIBLE SOLUTION
let! result =
Async.FromContinuations(fun (cont, econt, _) ->
let ccont e = econt e
let work = CSharpClass.AsyncMethod(cancellationToken) |> Async.AwaitTask
Async.StartWithContinuations(work, cont, econt, ccont))
|> Async.Catch
What ultimately causes this behavior is that cancellations are special in F# Async. Cancellations effectively translate to a stop and teardown. As you can see in the source, cancellation in the Task makes it all the way out of the computation.
If you want the good old OperationCanceledException which you can handle as part of your computation, we can just make our own.
type Async =
static member AwaitTaskWithCancellations (task: Task<_>) =
Async.FromContinuations(fun (setResult, setException, setCancelation) ->
task.ContinueWith(fun (t:Task<_>) ->
match t.Status with
| TaskStatus.RanToCompletion -> setResult t.Result
| TaskStatus.Faulted -> setException t.Exception
| TaskStatus.Canceled -> setException <| OperationCanceledException()
| _ -> ()
) |> ignore
)
Cancellation is now just another exception - and exceptions, we can handle.
Here's the repro:
let tcs = TaskCompletionSource<unit>()
tcs.SetCanceled()
async {
try
let! result = tcs.Task |> Async.AwaitTaskWithCancellations
return result
with
| :? OperationCanceledException ->
printfn "cancelled"
| ex -> printfn "faulted %A" ex
()
} |> Async.RunSynchronously
let failing = async {
failwith "foo"
}
let test () =
try
Async.Start(failing)
with
| exn -> printf "caught"
This code doesn't catch the exception. How can I start an asynchronous workflow on a separate thread and catch the exception in the main program?
as an alternative you start the workflow as a task and use it's methods and properties instead. For example Task.Result will rethrow an exception again so this works, and is almost what you tried:
let test () =
try
Async.StartAsTask failing
|> fun t -> t.Result
with _ -> printfn "caught"
run
> test ();;
caught
val it : unit = ()
on a differnt thread
sorry - I just saw that you want it on a different thread - in this case you most likely want to use the internal approach RCH gave you - but you could use ContinueWith too (although a bit ugly):
open System.Threading.Tasks
let test () =
(Async.StartAsTask failing).ContinueWith(fun (t : Task<_>) -> try t.Result with _ -> printfn "caught")
run
> test ();;
caught
val it : Task = System.Threading.Tasks.Task {AsyncState = null;
CreationOptions = None;
Exception = null;
Id = 3;
IsCanceled = false;
IsCompleted = true;
IsFaulted = false;
Status = RanToCompletion;}
without Async.Catch
also you don't really need the Async.Catch:
let test () =
async {
try
do! failing
with _ -> printfn "caught"
} |> Async.Start
As there is no result awaited, there is no place where the exception could be caught. You need to wrap the computation. One possibility:
let failing = async {
failwith "foo"
}
let test () =
async {
let! res = failing |> Async.Catch
match res with
| Choice1Of2 _ -> printf "success"
| Choice2Of2 exn -> printfn "failed with %s" exn.Message
} |> Async.Start