I am confused about the
let decode buffer = Encoding.UTF8.GetString(buffer)
line, what is going on there ?
buffer is declared inside async, or is it a closure ?
https://github.com/sebfia/OffLog/blob/master/Shared/FeedStorage.fs#L22
let thaw<'T> path =
let decode buffer = Encoding.UTF8.GetString(buffer)
async {
try
match File.Exists(path) with
| false -> return None
| true ->
use fs = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.None)
let! buffer = fs.AsyncRead(fs.Length |> Convert.ToInt32)
let str = decode buffer
System.Diagnostics.Debug.WriteLine(str)
let bla = JsonConvert.DeserializeObject<'T>(str)
System.Diagnostics.Debug.WriteLine(typeof<'T>.GetType().FullName)
return bla |> Some
with | e -> System.Diagnostics.Debug.WriteLine(e.ToString()); return None
}
Related
I have an async block, within this block I call an async method from an external C# web service client library. This method call returns a data transfer object, or a custom exception of type ApiException. Initially, my function looked like this:
type Msg =
| LoginSuccess
| LoginError
| ApiError
let authUserAsync (client: Client) model =
async {
do! Async.SwitchToThreadPool()
let loginParams = new LoginParamsDto(Username = "some username", Password = "some password")
try
// AuthenticateAsync may throw a custom ApiException.
let! loggedInAuthor = client.AuthenticateAsync loginParams |> Async.AwaitTask
// Do stuff with loggedInAuthor DTO...
return LoginSuccess
with
| :? ApiException as ex ->
let msg =
match ex.StatusCode with
| 404 -> LoginError
| _ -> ApiError
return msg
}
But I found that the ApiException wasn't being caught. Further investigation revealed that the ApiException was in fact the inner exception. So I changed my code to this:
type Msg =
| LoginSuccess
| LoginError
| ApiError
let authUserAsync (client: Client) model =
async {
do! Async.SwitchToThreadPool()
let loginParams = new LoginParamsDto(Username = "some username", Password = "some password")
try
// AuthenticateAsync may throw a custom ApiException.
let! loggedInAuthor = client.AuthenticateAsync loginParams |> Async.AwaitTask
// Do stuff with loggedInAuthor DTO...
return LoginSuccess
with
| baseExn ->
let msg =
match baseExn.InnerException with
| :? ApiException as e ->
match e.StatusCode with
| 404 -> LoginError
| _ -> ApiError
| otherExn -> raise otherExn
return msg
}
Which seems to work. But being new to F# I'm wondering if there is there a more elegant or idiomatic way to catch an inner exception in this kind of situation?
I sometimes use an active pattern to catch the main or the inner exception:
let rec (|NestedApiException|_|) (e: exn) =
match e with
| null -> None
| :? ApiException as e -> Some e
| e -> (|NestedApiException|_|) e.InnerException
and then use it like this:
async {
try
...
with
| NestedApiException e ->
...
| otherExn ->
raise otherExn
}
The solution based on active patterns from Tarmil is very nice and I would use that if I wanted to catch the exception in multiple places across a file or project. However, if I wanted to do this in just one place, then I would probably not want to define a separate active pattern for it.
There is a somewhat nicer way of writing what you have using the when clause in the try ... with expression:
let authUserAsync (client: Client) model =
async {
try
// (...)
with baseExn when (baseExn.InnerException :? ApiException) ->
let e = baseExn :?> ApiException
match e.StatusCode with
| 404 -> return LoginError
| _ -> return ApiError }
This is somewhat repetitive because you have to do a type check using :? and then again a cast using :?>, but it is a bit nicer inline way of doing this.
As an alternative, if you use F# exceptions, you can use the full range of pattern matching available for product types. Error handling code reads a lot more succinctly.
exception TimeoutException
exception ApiException of message: string * inner: exn
try
action ()
with
| ApiException (_, TimeoutException) ->
printfn "Timed out"
| ApiException (_, (:? ApplicationException as e)) ->
printfn "%A" e
I have some potentially very long running function, which may sometimes hang up. So, I thought that if I wrap it into an async workflow, then I should be able to cancel it. Here is an FSI example that does not work (but the same behavior happens with the compiled code):
open System.Threading
let mutable counter = 0
/// Emulates an external C# sync function that hung up.
/// Please, don't change it to some F# async stuff because
/// that won't fix that C# method.
let run() =
while true
do
printfn "counter = %A" counter
Thread.Sleep 1000
counter <- counter + 1
let onRunModel() =
let c = new CancellationTokenSource()
let m = async { do run() }
Async.Start (m, c.Token)
c
let tryCancel() =
printfn "Starting..."
let c = onRunModel()
printfn "Waiting..."
Thread.Sleep 5000
printfn "Cancelling..."
c.Cancel()
printfn "Waiting again..."
Thread.Sleep 5000
printfn "Completed."
#time
tryCancel()
#time
If you run it in FSI, then you will see something like that:
Starting...
Waiting...
counter = 0
counter = 1
counter = 2
counter = 3
counter = 4
Cancelling...
Waiting again...
counter = 5
counter = 6
counter = 7
counter = 8
counter = 9
Completed.
Real: 00:00:10.004, CPU: 00:00:00.062, GC gen0: 0, gen1: 0, gen2: 0
counter = 10
counter = 11
counter = 12
counter = 13
counter = 14
counter = 15
counter = 16
which means that it does not stop at all after c.Cancel() is called.
What am I doing wrong and how to make such thing work?
Here is some additional information:
When the code hangs up, it does it in some external sync C# library,
which I have no control of. So checking for cancellation token in
the code that I control is useless. That's why function run()
above was modeled that way.
I don't need any communication of completion and / or progress. It's already done via some messaging system and it is out of scope
of the question.
Basically I just need to kill background work as soon as I "decide" to do so.
You are handing off control to a code segment, which, albeit wrapped in an async block, has no means of checking for the cancellation. Were you to construct your loop directly wrapped in an async, or have it replaced by a recursive async loop, it will work as expected:
let run0 () = // does not cancel
let counter = ref 0
while true do
printfn "(0) counter = %A" !counter
Thread.Sleep 1000
incr counter
let m = async { run0 () }
let run1 () = // cancels
let counter = ref 0
async{
while true do
printfn "(1) counter = %A" !counter
Thread.Sleep 1000
incr counter }
let run2 = // cancels too
let rec aux counter = async {
printfn "(2) counter = %A" counter
Thread.Sleep 1000
return! aux (counter + 1) }
aux 0
printfn "Starting..."
let cts = new CancellationTokenSource()
Async.Start(m, cts.Token)
Async.Start(run1(), cts.Token)
Async.Start(run2, cts.Token)
printfn "Waiting..."
Thread.Sleep 5000
printfn "Cancelling..."
cts.Cancel()
printfn "Waiting again..."
Thread.Sleep 5000
printfn "Completed."
A word of caution though: Nested async calls in F# are automatically checked for cancellation, which is why do! Async.Sleep is preferable. If you are going down the recursive route, be sure to enable tail-recursion via return!. Further reading: Scott W.'s blog on Asynchronous programming, and Async in C# and F# Asynchronous gotchas in C# by Tomas Petricek.
This piece of code was developed to solve a situation where I couldn't get some calls to terminate/timeout. They would just hang. Maybe you can get some ideas that will help you solve your problem.
The interesting part for you would be only the two first functions. The rest is only to demonstrate how I'm using them.
module RobustTcp =
open System
open System.Text
open System.Net.Sockets
open Railway
let private asyncSleep (sleepTime: int) (error: 'a) = async {
do! Async.Sleep sleepTime
return Some error
}
let private asyncWithTimeout asy (timeout: int) (error: 'a) =
Async.Choice [ asy; asyncSleep timeout error ]
let private connectTcpClient (host: string) (port: int) (tcpClient: TcpClient) = async {
let asyncConnect = async {
do! tcpClient.ConnectAsync(host, port) |> Async.AwaitTask
return Some tcpClient.Connected }
match! asyncWithTimeout asyncConnect 1_000 false with
| Some isConnected -> return Ok isConnected
| None -> return Error "unexpected logic error in connectTcpClient"
}
let private writeTcpClient (outBytes: byte[]) (tcpClient: TcpClient) = async {
let asyncWrite = async {
let stream = tcpClient.GetStream()
do! stream.WriteAsync(outBytes, 0, outBytes.Length) |> Async.AwaitTask
do! stream.FlushAsync() |> Async.AwaitTask
return Some (Ok ()) }
match! asyncWithTimeout asyncWrite 10_000 (Error "timeout writing") with
| Some isWrite -> return isWrite
| None -> return Error "unexpected logic error in writeTcpClient"
}
let private readTcpClient (tcpClient: TcpClient) = async {
let asyncRead = async {
let inBytes: byte[] = Array.zeroCreate 1024
let stream = tcpClient.GetStream()
let! byteCount = stream.ReadAsync(inBytes, 0, inBytes.Length) |> Async.AwaitTask
let bytesToReturn = inBytes.[ 0 .. byteCount - 1 ]
return Some (Ok bytesToReturn) }
match! asyncWithTimeout asyncRead 2_000 (Error "timeout reading reply") with
| Some isRead ->
match isRead with
| Ok s -> return Ok s
| Error error -> return Error error
| None -> return Error "unexpected logic error in readTcpClient"
}
let sendReceiveBytes (host: string) (port: int) (bytesToSend: byte[]) = async {
try
use tcpClient = new TcpClient()
match! connectTcpClient host port tcpClient with
| Ok isConnected ->
match isConnected with
| true ->
match! writeTcpClient bytesToSend tcpClient with
| Ok () ->
let! gotData = readTcpClient tcpClient
match gotData with
| Ok result -> return Ok result
| Error error -> return Error error
| Error error -> return Error error
| false -> return Error "Not connected."
| Error error -> return Error error
with
| :? AggregateException as ex ->
(* TODO ? *)
return Error ex.Message
| ex ->
(*
printfn "Exception in getStatus : %s" ex.Message
*)
return Error ex.Message
}
let sendReceiveText (host: string) (port: int) (textToSend: string) (encoding: Encoding) =
encoding.GetBytes textToSend
|> sendReceiveBytes host port
|> Async.map (Result.map encoding.GetString)
I am attempting to create a list of strings which gets elements gradually inserted into asynchronously with the help of a mailbox processor. However I am not getting the desired output.
I have pretty much followed the code from https://fsharpforfunandprofit.com/posts/concurrency-actor-model/
however it does not seem to work as intended in my case. The code I have is as follows:
type TransactionQueue ={
queue : string list
} with
static member UpdateState (msg : string) (tq : TransactionQueue) =
{tq with queue = (msg :: tq.queue)}
static member Agent = MailboxProcessor.Start(fun inbox ->
let rec msgLoop (t : TransactionQueue) =
async{
let! msg = inbox.Receive()
let newT = TransactionQueue.UpdateState msg t
printfn "%A" newT
return! msgLoop newT
}
msgLoop {queue = []}
)
static member Add i = TransactionQueue.Agent.Post i
[<EntryPoint>]
let main argv =
// test in isolation
printfn "welcome to test"
let rec loop () =
let str = Console.ReadLine()
TransactionQueue.Add str
loop ()
loop ()
0
The result i keep getting is a list of the latest input only, the state is not kept. So if I enter "a" then "b" then "c" the queue will only have the value "c" instead of "a";"b";"c"
Any help or pointers would be most appreciated!
Just like C# Properties, your Agent is really a Property and thus behaves like a method with void parameter. That’s why you will get a new agent everytime Agent property is accessed.
In idiomatic F# there are two styles when implementing agents. If you don’t need to have many agent instances, just write a module and encapsule the agent-related stuff inside. Otherwise, OOP style should be used.
Code for style #1
module TransactionQueue =
type private Queue = Queue of string list
let private empty = Queue []
let private update item (Queue items) = Queue (item :: items)
let private agent = MailboxProcessor.Start <| fun inbox ->
let rec msgLoop queue = async {
let! msg = inbox.Receive ()
return! queue |> update msg |> msgLoop
}
msgLoop empty
let add item = agent.Post item
[<EntryPoint>]
let main argv =
// test in isolation
printfn "welcome to test"
let rec loop () =
let str = Console.ReadLine()
TransactionQueue.add str
loop ()
loop ()
Code for style #2
type Queue = Queue of string list with
static member Empty = Queue []
static member Update item (Queue items) =
Queue (item :: items)
type Agent () =
let agent = MailboxProcessor.Start <| fun inbox ->
let rec msgLoop queue = async {
let! msg = inbox.Receive ()
return! queue |> Queue.Update msg |> msgLoop
}
msgLoop Queue.Empty
member this.Add item = agent.Post item
[<EntryPoint>]
let main argv =
// test in isolation
printfn "welcome to test"
let agent = new Agent ()
let rec loop () =
let str = Console.ReadLine()
agent.Add str
loop ()
loop ()
Notice the use of Single-case union types for the Queue type.
Edit: This turned out to be an F# bug which can be worked-around by using a custom option type instead of Fsharp's "Option".
In F#, I am trying to call a .net Task with Async.AwaitTask. The task is throwing an exception, and I can't seem to catch it with either try-catch or Async.Catch. And I know of no other way to catch exceptions. What is the solution? And what is the cause of the problem? Thanks for any explanation.
Here is some test code that shows my failure to catch the exception thrown by DownloadStringTaskAsync:
open System
open System.Net
[<EntryPoint>]
let main argv =
let test =
async{
let! exc = Async.Catch( async{
try
let w = new Net.WebClient();
let! str = Async.AwaitTask (w.DownloadStringTaskAsync "") // throws ArgumentException
return Some str
with
| _ ->
return None // not caught
}
)
match exc with
| Choice1Of2 r -> return r
| Choice2Of2 ext -> return None // not caught
}
let res = Async.RunSynchronously(test)
let str = Console.ReadLine();
0 // return an integer exit code
I need some way to catch the exception inside the "test" async function, preventing it to "bubble up".
I found this related question, but I can't seem to adapt the answer (which deals with Task, not with Task<'a>) to my needs.
Edit: Here is a screenshot showing the problem: https://i.gyazo.com/883f546c00255b210e53cd095b876cb0.png
"ArgumentException was unhandled by user code."
(Visual Studio 2013, .NET 4.5.1, Fsharp 3.1, Fsharp.core.dll 4.3.1.0.)
Could it be that the code is correct but some Fsharp or Visual Studio setting is preventing the exception from being caught?
TL;DR: The exception does get caught, it appears to be returning None out of async that's confusing here - the result is null, not None, which is screwing up pattern matching and generally being a bother. Use your own union type instead of Option to handle it.
Edit: Oh, and #takemyoxygen's comments about why you're seeing it straight away is correct. Debug->Exceptions, or just untick "Break when this exception type is user-unhandled" in the pop-up visible in your screenshot. It looks like VS is breaking on THROW, rather than actually on unhandled.
First, returning Some/None inside an Async.Catch makes it useless. Async.Catch returns Choice1Of2(val) unless there's an uncaught exception, so the code as is (if it worked) would return Choice1Of2(Some(string)) with no exception, or Choice1Of2(None) with exception. Either use try/with and handle the exception yourself, or only use Async.Catch.
Experiments:
Proof that we are catching: Add a debug output to the "with". The code DOES get there (add a breakpoint or look at debug output for proof). We get Choice1Of2(null), rather than Choice1Of2(None) in exc though. WEIRD. See image: http://i.imgur.com/e8Knx5a.png
open System
open System.Net
[<EntryPoint>]
let main argv =
let test =
async{
let! exc = Async.Catch( async{
try
let w = new Net.WebClient();
let! str = Async.AwaitTask (w.DownloadStringTaskAsync "") // throws ArgumentException
return Some str
with
| _ ->
System.Diagnostics.Debug.WriteLine "in with" // We get here.
return None // not caught
}
)
match exc with
| Choice1Of2 r -> return r
| Choice2Of2 ext -> return None // not caught
}
let res = Async.RunSynchronously(test)
let str = Console.ReadLine();
0 // return an integer exit code
Remove Async.Catch: Don't use Async.Catch (keep the debug output though). Again, we get to the debug output, so we're catching the exception, and as we're not wrapping in a Choice from Async.Catch, we get null instead of None. STILL WEIRD.
open System
open System.Net
[<EntryPoint>]
let main argv =
let test = async {
try
let w = new Net.WebClient();
let! str = Async.AwaitTask (w.DownloadStringTaskAsync "") // throws ArgumentException
return Some str
with
| _ ->
System.Diagnostics.Debug.WriteLine "in with"
return None }
let res = Async.RunSynchronously(test)
let str = Console.ReadLine();
0 // return an integer exit code
Only use Async.Catch: Don't use a try/with, just Async.Catch. This works a bit better. At the pattern match on exc, I have a Choice2Of2 containing the exception. However, when the None is returned out of the outermost async, it becomes null.
open System
open System.Net
[<EntryPoint>]
let main argv =
let test = async {
let! exc = Async.Catch(async {
let w = new Net.WebClient();
let! str = Async.AwaitTask (w.DownloadStringTaskAsync "") // throws ArgumentException
return str })
match exc with
| Choice1Of2 v -> return Some v
| Choice2Of2 ex -> return None
}
let res = Async.RunSynchronously(test)
let str = Console.ReadLine();
0 // return an integer exit code
Don't use Option: Interestingly, if you use a custom union type, it works perfectly:
open System
open System.Net
type UnionDemo =
| StringValue of string
| ExceptionValue of Exception
[<EntryPoint>]
let main argv =
let test = async {
let! exc = Async.Catch(async {
let w = new Net.WebClient();
let! str = Async.AwaitTask (w.DownloadStringTaskAsync "") // throws ArgumentException
return str })
match exc with
| Choice1Of2 v -> return StringValue v
| Choice2Of2 ex -> return ExceptionValue ex
}
let res = Async.RunSynchronously(test)
let str = Console.ReadLine();
0 // return an integer exit code
Using a try/with and no Async.Catch also works with a new Union:
open System
open System.Net
type UnionDemo =
| StringValue of string
| ExceptionValue of Exception
[<EntryPoint>]
let main argv =
let test = async {
try
let w = new Net.WebClient();
let! str = Async.AwaitTask (w.DownloadStringTaskAsync "") // throws ArgumentException
return StringValue str
with
| ex -> return ExceptionValue ex }
let res = Async.RunSynchronously(test)
let str = Console.ReadLine();
0 // return an integer exit code
It works even if the union is defined as the following:
type UnionDemo =
| StringValue of string
| ExceptionValue
The following gets null in moreTestRes.
[<EntryPoint>]
let main argv =
let moreTest = async {
return None
}
let moreTestRes = Async.RunSynchronously moreTest
0
Why this is the case I don't know (still happens in F# 4.0), but the exception is definitely being caught, but None->null is screwing it up.
I'm playing around with using SqlClient in F# and I'm having difficulty with using SqlDataReader.ReadAsync. I'm trying to do the F# equivalent of
while (await reader.ReadAsync) { ... }
What is the best way to do this in F#? Below is my full program. It works, but I'd like to know if there is a better way to do it.
open System
open System.Data.SqlClient
open System.Threading.Tasks
let connectionString = "Server=.;Integrated Security=SSPI"
module Async =
let AwaitVoidTask : (Task -> Async<unit>) =
Async.AwaitIAsyncResult >> Async.Ignore
// QUESTION: Is this idiomatic F#? Is there a more generally-used way of doing this?
let rec While (predicateFn : unit -> Async<bool>) (action : unit -> unit) : Async<unit> =
async {
let! b = predicateFn()
match b with
| true -> action(); do! While predicateFn action
| false -> ()
}
[<EntryPoint>]
let main argv =
let work = async {
// Open connection
use conn = new SqlConnection(connectionString)
do! conn.OpenAsync() |> Async.AwaitVoidTask
// Execute command
use cmd = conn.CreateCommand()
cmd.CommandText <- "select name from sys.databases"
let! reader = cmd.ExecuteReaderAsync() |> Async.AwaitTask
// Consume reader
// I want a convenient 'while' loop like this...
//while reader.ReadAsync() |> Async.AwaitTask do // Error: This expression was expected to have type bool but here has type Async<bool>
// reader.GetValue 0 |> string |> printfn "%s"
// Instead I used the 'Async.While' method that I defined above.
let ConsumeReader = Async.While (fun () -> reader.ReadAsync() |> Async.AwaitTask)
do! ConsumeReader (fun () -> reader.GetValue 0 |> string |> printfn "%s")
}
work |> Async.RunSynchronously
0 // return an integer exit code
There is one issue in your code which is that you're doing a recursive call using
do! While predicateFn action. This is a problem because it does not turn into a tail-call and so you could end up with memory leaks. The right way to do this is to use return! instead of do!.
Aside from that, your code works good. But you can actually extend the async computation builder to let you use ordinary while keyword. To do that, you need a slightly different version of While:
let rec While (predicateFn : unit -> Async<bool>) (action : Async<unit>) : Async<unit> =
async {
let! b = predicateFn()
if b then
do! action
return! While predicateFn action
}
type AsyncBuilder with
member x.While(cond, body) = Async.While cond body
Here, the body is also asynchronous and it is not a function. Then we add a While method to the computation builder (so we are adding another overload as an extension method). With this, you can actually write:
while Async.AwaitTask(reader.ReadAsync()) do // This is async!
do! Async.Sleep(1000) // The body is asynchronous too
reader.GetValue 0 |> string |> printfn "%s"
I'd probably do the same as you. If you can stomach refs though, you can shorten it to
let go = ref true
while !go do
let! more = reader.ReadAsync() |> Async.AwaitTask
go := more
reader.GetValue 0 |> string |> printfn "%s"