I get some list of data from a HTTP call. I then know what values to get for another HTTP call. I would like to have everything be asynchronous. But I need to use this data with Expecto's testCaseAsync : string -> Async<unit> -> Test. So, my goal is to get a signature like so Async<Item>[]
So, I would like to get a list of testCaseAsync.
So, I basically have something like this:
// Async<Async<Item>[]>
let getAsyncCalls =
async {
let! table = API.getTable ()
// Async<Item>[]
let items =
table.root
|> Array.map (fun x -> API.getItem x.id)
return item
}
If I run them in parallel I get:
// Async<Item[]>
let getAsyncCalls =
async {
let! table = API.getTable ()
// Item[]
let! items =
table.root
|> Array.map (fun x -> API.getItem x.id)
return item
}
So, that doesn't get me to Async<Item>[]. I'm not sure if this is possible. I would like to avoid Async.RunSynchronously for the API.getTable call since that can lead to deadlocks, right? It will most likely be called from a cached value (memoized) so I'm not sure that will make a difference.
I guess I'll keep working on it unless someone else is more clever than me :-) Thanks in advance!
In general, you cannot turn Async<Async<T>[]> into Async<T>[]. The problem is that to even get the length of the array, you need to perform some operation asynchronously, so there is no way to "lift" the array outside of the async. If you knew the length of the array in advance, then you can make this work.
The following function turns Async<'T[]> into Async<'T>[] provided that you give it the length of the array. As you figured out, the returned asyncs need to somehow share access to the one top-level async. The easiest way of doing this I can think of is to use a task. Adapting that for your use case should be easy:
let unwrapAsyncArray (asyncs:Async<'T[]>) len =
let task = asyncs |> Async.StartAsTask
Array.init len (fun i -> async {
let! res = Async.AwaitTask task
if res.Length <> len then failwith "Wrong length!"
return res.[i] }
)
Related
I have the following methods in F#:
let GetAllPatientNamesAsync() =
async {
let data = context.GetAllPatientNamesAsync()
return! Async.AwaitTask data
}
let getAllPatientNames = Async.RunSynchronously (FsNetwork.GetAllPatientNamesAsync())
This works the FIRST time:
{ m with names = getAllPatientNames}
But once getAllPatientNames has a value, calling it again does not execute the Async.RunSynchronously (FsNetwork.GetAllPatientNamesAsync()) -- rather getAllPatientNames returns its original value.
Since FsNetwork.GetAllPatientNamesAsync() is accessing the database, which is changing, after the first read getAllPatientNames is wrong.
Ofcourse, this works correctly no matter how many times I use it:
{ m with names = Async.RunSynchronously (FsNetwork.GetAllPatientNamesAsync()) }.
How should I write
let getAllPatientNames = Async.RunSynchronously (FsNetwork.GetAllPatientNamesAsync())
such that it ALWAYS executes the Async process? What am I doing wrong?
TIA
The problem is that
getAllPatientNames
is a a value and not a function. Make a it a function by changing it like below
let getAllPatientNames () = Async.RunSynchronously (FsNetwork.GetAllPatientNamesAsync())
F# has Async.Parallel with type signature seq<Async<'a>> -> Async<'a list>, which will take a seq of Async types, prep them to run in parallel, and output a single Async.
I was wondering if there is a similar Async.Sequential which has the same type signature, but runs each item in sequence one after the other? I feel like I must be missing something obvious?
I'd like to do something like the following:
async {
let! results =
myItems
|> List.map myAsynchronousOperation
|> Async.Sequential
... do something with results
}
I don't think there's one built in, but should be trivial to write your own:
let sequential s = async {
let results = ResizeArray()
for a in s do
let! result = a
results.Add(result)
return List.ofSeq results
}
Evaluating the items in a seq<Async<'a>> in-order and returning a list is effectively just a foldBack on the sequence where you want to evaluate the async before simply cons-ing it onto a list. While kvb's solution certainly works, you could also do it with a Seq.foldBack like so:
module Async =
let sequential s =
Seq.foldBack (fun cur acc -> async.Bind(cur, fun head ->
async.Bind(acc, fun tail -> async.Return(head :: tail)))) s (async.Return [])
With the release of F# 4.7 and FSharp.Core 4.7 Async.Sequential is now available "out-of-the-box".
Announcement: https://devblogs.microsoft.com/dotnet/announcing-f-4-7/
Source: https://github.com/dotnet/fsharp/blob/897afd3dad3bfe58a0495713e99a8094098c18c5/src/fsharp/FSharp.Core/async.fs#L1282
If one is willing to use the AsyncSeq library, then the following does what's needed in the given example:
async {
let! results =
myItems
|> AsyncSeq.ofSeq
|> AsyncSeq.mapAsync myAsynchronousOperation
|> AsyncSeq.toListAsync
... do something with results
}
Probably overkill if you're not already using AsyncSeq for other things though.
Suppose I have some C# code that takes a callback:
void DoSomething(Action<string> callback);
Now, I want to use this in F#, but wrap it in an async. How would I go about this?
// Not real code
let doSomething = async {
let mutable result = null
new Action(fun x -> result <- x) |> Tasks.DoSomething
// Wait for result to be assigned
return result
}
For example, suppose DoSomething looks like this:
module Tasks
let DoSomething callback =
callback "Hello"
()
Then the output of the following should be "Hello":
let wrappedDoSomething = async {
// Call DoSomething somehow
}
[<EntryPoint>]
let main argv =
async {
let! resultOfDoSomething = wrappedDoSomething
Console.WriteLine resultOfDoSomething
return ()
} |> Async.RunSynchronously
0
The function Async.FromContinuations is, so to say, the "lowest level" of Async. All other async combinators can be expressed in terms of it.
It is the lowest level in the sense that it directly encodes the very nature of async computations - the knowledge of what to do in the three possible cases: (1) a successful completion of the previous computation step, (2) a crash of the previous computation step, and (3) cancellation from outside. These possible cases are expressed as the three function-typed arguments of the function that you pass to Async.FromContinuations. For example:
let returnFive =
Async.FromContinuations( fun (succ, err, cancl) ->
succ 5
)
async {
let! res = returnFive
printfn "%A" res // Prints "5"
}
|> Async.RunSynchronously
Here, my function fun (succ, err, cancl) -> succ 5 has decided that it has completed successfully, and calls the succ continuation to pass its computation result to the next step.
In your case, the function DoSomething expresses only one of the three cases - i.e. "what to do on successful completion". Once you're inside the callback, it means that whatever DoSomething was doing, has completed successfully. That's when you need to call the succ continuation:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( fun res -> succ res )
)
Of course, you can avoid a nested lambda-expression fun res -> succ res by passing succ directly into DoSomething as callback. Unfortunately, you'll have to explicitly specify which type of Action to use for wrapping it, which negates the advantage:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( System.Action<string> succ )
)
As an aside, note that this immediately uncovered a hole in the DoSomething's API: it ignores the error case. What happens if DoSomething fails to do whatever it was meant to do? There is no way you'd know about it, and the whole async workflow will just hang. Or, even worse: the process will exit immediately (depending on how the crash happens).
If you have any control over DoSomething, I suggest you address this issue.
You can try something like:
let doSomething callback = async {
Tasks.DoSomething(callback)
}
If your goal is to define the callback in the method you could do something like:
let doSomething () = async {
let callback = new Action<string>(fun result -> printfn "%A" result )
Tasks.DoSomething(callback)
}
If your goal is to have the result of the async method be used in the DoSomething callback you could do something like:
let doSomething =
Async.StartWithContinuations(
async {
return result
},
(fun result -> Tasks.DoSomething(result)),
(fun _ -> printfn "Deal with exception."),
(fun _ -> printfn "Deal with cancellation."))
I'm trying to understand async workflows in F# but I found one part that I really don't understand.
The following code works fine:
let asynWorkflow = async{
let! result = Stream.TryOpenAsync(partition) |> Async.AwaitTask
return result
}
let stream = Async.RunSynchronously asynWorkflow
|> fun openResult -> if openResult.Found then openResult.Stream else Stream(partition)
I define a async workflow where TryOpenAsync returns a Task<StreamOpenResult> type. I convert it to Async<StreamOpenResult> with Async.AwaitTask. (Side quest: "Await"Task? It doesn't await it just convert it, does it? I think it has nothing to do with Task.Wait or the await keyword). I "await" it with let! and return it.
To start the workflow I use RunSynchronously which should start the workflow and return the result (bind it). On the result I check if the Stream is Found or not.
But now to my first question. Why do I have to wrap the TryOpenAsync call in another async computation and let! ("await") it?
E.g. the following code does not work:
let asynWorkflow = Stream.TryOpenAsync(partition) |> Async.AwaitTask
let stream = Async.RunSynchronously asynWorkflow
|> fun openResult -> if openResult.Found then openResult.Stream else Stream(partition)
I thought the AwaitTask makes it an Async<T> and RunSynchronously should start it. Then use the result. What do I miss?
My second question is why is there any "Async.Let!" function available? Maybe because it does not work or better why doesn't it work with the following code?
let ``let!`` task = async{
let! result = task |> Async.AwaitTask
return result
}
let stream = Async.RunSynchronously ( ``let!`` (Stream.TryOpenAsync(partition)) )
|> fun openResult -> if openResult.Found then openResult.Stream else Stream(partition)
I just insert the TryOpenAsync as a parameter but it does not work. By saying does not work I mean the whole FSI will hang. So it has something to do with my async/"await".
--- Update:
Result of working code in FSI:
>
Real: 00:00:00.051, CPU: 00:00:00.031, GC gen0: 0, gen1: 0, gen2: 0
val asynWorkflow : Async<StreamOpenResult>
val stream : Stream
Result of not working code in FSI:
>
And you cannot execute anything in the FSI anymore
--- Update 2
I'm using Streamstone. Here the C# example: https://github.com/yevhen/Streamstone/blob/master/Source/Example/Scenarios/S04_Write_to_stream.cs
and here the Stream.TryOpenAsync: https://github.com/yevhen/Streamstone/blob/master/Source/Streamstone/Stream.Api.cs#L192
I can't tell you why the second example doesn't work without knowing what Stream and partition are and how they work.
However, I want to take this opportunity to point out that the two examples are not strictly equivalent.
F# async is kind of like a "recipe" for what to do. When you write async { ... }, the resulting computation is just sitting there, not actually doing anything. It's more like declaring a function than like issuing a command. Only when you "start" it by calling something like Async.RunSynchronously or Async.Start does it actually run. A corollary is that you can start the same async workflow multiple times, and it's going to be a new workflow every time. Very similar to how IEnumerable works.
C# Task, on the other hand, is more like a "reference" to an async computation that is already running. The computation starts as soon as you call Stream.TryOpenAsync(partition), and it's impossible to obtain a Task instance before the task actually starts. You can await the resulting Task multiple times, but each await will not result in a fresh attempt to open a stream. Only the first await will actually wait for the task's completion, and every subsequent one will just return you the same remembered result.
In the async/reactive lingo, F# async is what you call "cold", while C# Task is referred to as "hot".
The second code block looks like it should work to me. It does run it if I provide dummy implementations for Stream and StreamOpenResult.
You should avoid using Async.RunSynchronously wherever possible because it defeats the purpose of async. Put all of this code within a larger async block and then you will have access to the StreamOpenResult:
async {
let! openResult = Stream.TryOpenAsync(partition) |> Async.AwaitTask
let stream = if openResult.Found then openResult.Stream else Stream(partition)
return () // now do something with the stream
}
You may need to put a Async.Start or Async.RunSynchronously at the very outer edge of your program to actually run it, but it's better if you have the async (or convert it to a Task) and pass it to some other code (e.g. a web framework) that can call it in a non-blocking manner.
Not that I want to answer your question with another question, but: why are you doing code like this anyway? That might help to understand it. Why not just:
let asyncWorkflow = async {
let! result = Stream.TryOpenAsync(partition) |> Async.AwaitTask
if result.Found then return openResult.Stream else return Stream(partition) }
There's little point in creating an async workflow only to immediately call RunSynchronously on it - it's similar to calling .Result on a Task - it just blocks the current thread until the workflow returns.
I've got an agent which I set up to do some database work in the background. The implementation looks something like this:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
use conn = new System.Data.SqlClient.SqlConnection("...")
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
With this I discovered that if this was called several times in some amount of time I would start getting SqlConnection objects piling up and not being disposed, and eventually I would run out of connections in the connection pool (I don't have exact metrics on how many "several" is, but running an integration test suite twice in a row could always cause the connection pool to run dry).
If I change the use to a using then things are disposed properly and I don't have a problem:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
using (new System.Data.SqlClient.SqlConnection("...")) <| fun conn ->
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
It seems that the Using method of the AsyncBuilder is not properly calling its finally function for some reason, but it's not clear why. Does this have something to do with how I've written my recursive async expression, or is this some obscure bug? And does this suggest that utilizing use within other computation expressions could produce the same sort of behavior?
This is actually the expected behavior - although not entirely obvious!
The use construct disposes of the resource when the execution of the asynchronous workflow leaves the current scope. This is the same as the behavior of use outside of asynchronous workflows. The problem is that recursive call (outside of async) or recursive call using return! (inside async) does not mean that you are leaving the scope. So in this case, the resource is disposed of only after the recursive call returns.
To test this, I'll use a helper that prints when disposed:
let tester () =
{ new System.IDisposable with
member x.Dispose() = printfn "bye" }
The following function terminates the recursion after 10 iterations. This means that it keeps allocating the resources and disposes of all of them only after the entire workflow completes:
let rec loop(n) = async {
if n < 10 then
use t = tester()
do! Async.Sleep(1000)
return! loop(n+1) }
If you run this, it will run for 10 seconds and then print 10 times "bye" - this is because the allocated resources are still in scope during the recursive calls.
In your sample, the using function delimits the scope more explicitly. However, you can do the same using nested asynchronous workflow. The following only has the resource in scope when calling the Sleep method and so it disposes of it before the recursive call:
let rec loop(n) = async {
if n < 10 then
do! async {
use t = tester()
do! Async.Sleep(1000) }
return! loop(n+1) }
Similarly, when you use for loop or other constructs that restrict the scope, the resource is disposed immediately:
let rec loop(n) = async {
for i in 0 .. 10 do
use t = tester()
do! Async.Sleep(1000) }