Is there any primitive in the langage to compose async1 then async2, akin to what parallel does for parallel execution planning ?
to further clarify, I have 2 async computations
let toto1 = Async.Sleep(1000)
let toto2 = Async.Sleep(1000)
I would like to create a new async computation made of the sequential composition of toto1 and toto2
let toto = Async.Sequential [|toto1; toto2|]
upon start, toto would run toto1 then toto2, and would end after 2000 time units
The async.Bind operation is the basic primitive that asynchronous workflows provide for sequential composition - in the async block syntax, that corresponds to let!. You can use that to express sequential composition of two computations (as demonstrated by Daniel).
However, if you have an operation <|> that Daniel defined, than that's not expressive enough to implement async.Bind, because when you compose things sequentially using async.Bind, the second computation may depend on the result of the first one. The <e2> may use v1:
async.Bind(<e1>, fun v1 -> <e2>)
If you were writing <e1> <|> <e2> then the two operations have to be independent. This is a reason why the libraries are based on Bind - because it is more expressive form of sequential composition than the one you would get if you were following the structure of Async.Parallel.
If you want something that behaves like Async.Parallel and takes an array, then the easiest option is to implement that imperatively using let! in a loop (but you could use recursion and lists too):
let Sequential (ops:Async<'T>[]) = async {
let res = Array.zeroCreate ops.Length
for i in 0 .. ops.Length - 1 do
let! value = ops.[i]
res.[i] <- value
return res }
I'm not sure what you mean by "primitive." Async.Parallel is a function. Here are a few ways to run two asyncs:
In parallel:
Async.Parallel([|async1; async2|])
or
async {
let! child = Async.StartChild async2
let! result1 = child
let! result2 = async1
return [|result1; result2|]
}
Sequentially:
async {
let! result1 = async1
let! result2 = async2
return [|result1; result2|]
}
You could return tuples in the last two. I kept the return types the same as the first one.
I would say let! and do! in an async { } block is as close as you'll get to using a primitive for this.
EDIT
If all this nasty syntax is getting to you, you could define a combinator:
let (<|>) async1 async2 =
async {
let! r1 = async1
let! r2 = async2
return r1, r2
}
and then do:
async1 <|> async2 |> Async.RunSynchronously
Related
I have a function that I want to run 5 at a time from a chunked list and then wait a second so as not to hit an Api rate limit. I know the code is wrong but it is as far as I have got.
let ordersChunked = mOrders |> List.chunkBySize 5
for fiveOrders in ordersChunked do
let! tasks =
fiveOrders
|> List.map (fun (order) -> trackAndShip(order) )
|> Async.Parallel
Async.Sleep(1000) |> ignore
trackAndShip is an async function not a task and I get a little confused about the two.
Some of the answers I have read add |> Async.RunSynchronously to the end of it - and I do that at the top level but I don't feel it is good practice here ( and I would need to convert async to Task? )
This is probably a very simple answer if you are familiar with async / tasks but I am still "finding my feet" in this area.
Unlike C#'s tasks, Async computation do not start when they are created and you have to explicitly start them. Furthermore, even if you would start the Async operation, it won't do what you expect because the sleep will not execute because you just ignore it.
From the let! in your example code, I'm going to assume you're the code snippet is inside a computation expression. So the following may be what you want
let asyncComputation =
async {
let ordersChunked = [] |> List.chunkBySize 5
for fiveOrders in ordersChunked do
let! tasks =
fiveOrders
|> List.map (fun (order) -> trackAndShip(order) )
|> Async.Parallel
do! Async.Sleep(1000) // Note the do! here, without it your sleep will be ignored
}
Async.RunSynchronously asyncComputation
Note that the Async.Sleep is now chained into the CE by using do!. The Async.RunSynchronously call is one way to start the async computation: it runs the async operation and blocks the current thread while waiting for the async operation's result. There are many other ways of running it described in the documentation.
I come from a C# background having used async/ await. I am trying to find a "less verbose" way of programming using a library.
( specifically the Microsoft Playwright library for browser automation )
let (~~) = Async.AwaitTask
let getLastPageNumber(page: IPage) =
let playwright = ~~Playwright.CreateAsync() |> Async.RunSynchronously
let browser = ~~playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions(Headless = false )) |> Async.RunSynchronously
~~page.GotoAsync("https://go.xero.com/BankRec/BankRec.aspx?accountID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&page1") |> Async.RunSynchronously |> ignore
let lastPageLink = ~~page.QuerySelectorAsync("#mainPagerEnd") |> Async.RunSynchronously
if lastPageLink = null then
//this is the last page
1
else
let lastPageNumber = ~~lastPageLink.GetAttributeAsync("href") |> Async.RunSynchronously
lastPageNumber |> int
I have shortened things a bit using the alias ~~ for Async.AwaitTask but it seems to be a lot of code to do something that was a lot easier in C#.
Async.RunSynchronously should only be used as a very last resort because it blocks a thread to perform the computation, which defeats the purpose of using async/tasks.
The F# equivalent of C#'s async/await is to use F#'s Async type, and the async computation expression. However, if you're using a .NET library which uses the .NET Task type then you can use the TaskBuilder.fs library which has a task computation expression.
Then you would write the function like this:
open FSharp.Control.Tasks
let getLastPageNumber(page: IPage) = task {
let! playwright = Playwright.CreateAsync()
let! browser = playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions(Headless = false ))
let! _ = page.GotoAsync("https://go.xero.com/BankRec/BankRec.aspx?accountID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&page1")
let! lastPageLink = page.QuerySelectorAsync("#mainPagerEnd")
if lastPageLink = null then
//this is the last page
return 1
else
let! lastPageNumber = lastPageLink.GetAttributeAsync("href")
return lastPageNumber |> int
}
Inside the computation expression (also called a builder) let! is used to await tasks.
Note that this function now returns Task<int> rather than int, so the caller would probably need to follow a similar pattern and propogate the task further.
You can read more about using async in F# here. Some of that knowledge can also be applied to the task computation expression.
F# has Async.Parallel with type signature seq<Async<'a>> -> Async<'a list>, which will take a seq of Async types, prep them to run in parallel, and output a single Async.
I was wondering if there is a similar Async.Sequential which has the same type signature, but runs each item in sequence one after the other? I feel like I must be missing something obvious?
I'd like to do something like the following:
async {
let! results =
myItems
|> List.map myAsynchronousOperation
|> Async.Sequential
... do something with results
}
I don't think there's one built in, but should be trivial to write your own:
let sequential s = async {
let results = ResizeArray()
for a in s do
let! result = a
results.Add(result)
return List.ofSeq results
}
Evaluating the items in a seq<Async<'a>> in-order and returning a list is effectively just a foldBack on the sequence where you want to evaluate the async before simply cons-ing it onto a list. While kvb's solution certainly works, you could also do it with a Seq.foldBack like so:
module Async =
let sequential s =
Seq.foldBack (fun cur acc -> async.Bind(cur, fun head ->
async.Bind(acc, fun tail -> async.Return(head :: tail)))) s (async.Return [])
With the release of F# 4.7 and FSharp.Core 4.7 Async.Sequential is now available "out-of-the-box".
Announcement: https://devblogs.microsoft.com/dotnet/announcing-f-4-7/
Source: https://github.com/dotnet/fsharp/blob/897afd3dad3bfe58a0495713e99a8094098c18c5/src/fsharp/FSharp.Core/async.fs#L1282
If one is willing to use the AsyncSeq library, then the following does what's needed in the given example:
async {
let! results =
myItems
|> AsyncSeq.ofSeq
|> AsyncSeq.mapAsync myAsynchronousOperation
|> AsyncSeq.toListAsync
... do something with results
}
Probably overkill if you're not already using AsyncSeq for other things though.
Suppose I have some C# code that takes a callback:
void DoSomething(Action<string> callback);
Now, I want to use this in F#, but wrap it in an async. How would I go about this?
// Not real code
let doSomething = async {
let mutable result = null
new Action(fun x -> result <- x) |> Tasks.DoSomething
// Wait for result to be assigned
return result
}
For example, suppose DoSomething looks like this:
module Tasks
let DoSomething callback =
callback "Hello"
()
Then the output of the following should be "Hello":
let wrappedDoSomething = async {
// Call DoSomething somehow
}
[<EntryPoint>]
let main argv =
async {
let! resultOfDoSomething = wrappedDoSomething
Console.WriteLine resultOfDoSomething
return ()
} |> Async.RunSynchronously
0
The function Async.FromContinuations is, so to say, the "lowest level" of Async. All other async combinators can be expressed in terms of it.
It is the lowest level in the sense that it directly encodes the very nature of async computations - the knowledge of what to do in the three possible cases: (1) a successful completion of the previous computation step, (2) a crash of the previous computation step, and (3) cancellation from outside. These possible cases are expressed as the three function-typed arguments of the function that you pass to Async.FromContinuations. For example:
let returnFive =
Async.FromContinuations( fun (succ, err, cancl) ->
succ 5
)
async {
let! res = returnFive
printfn "%A" res // Prints "5"
}
|> Async.RunSynchronously
Here, my function fun (succ, err, cancl) -> succ 5 has decided that it has completed successfully, and calls the succ continuation to pass its computation result to the next step.
In your case, the function DoSomething expresses only one of the three cases - i.e. "what to do on successful completion". Once you're inside the callback, it means that whatever DoSomething was doing, has completed successfully. That's when you need to call the succ continuation:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( fun res -> succ res )
)
Of course, you can avoid a nested lambda-expression fun res -> succ res by passing succ directly into DoSomething as callback. Unfortunately, you'll have to explicitly specify which type of Action to use for wrapping it, which negates the advantage:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( System.Action<string> succ )
)
As an aside, note that this immediately uncovered a hole in the DoSomething's API: it ignores the error case. What happens if DoSomething fails to do whatever it was meant to do? There is no way you'd know about it, and the whole async workflow will just hang. Or, even worse: the process will exit immediately (depending on how the crash happens).
If you have any control over DoSomething, I suggest you address this issue.
You can try something like:
let doSomething callback = async {
Tasks.DoSomething(callback)
}
If your goal is to define the callback in the method you could do something like:
let doSomething () = async {
let callback = new Action<string>(fun result -> printfn "%A" result )
Tasks.DoSomething(callback)
}
If your goal is to have the result of the async method be used in the DoSomething callback you could do something like:
let doSomething =
Async.StartWithContinuations(
async {
return result
},
(fun result -> Tasks.DoSomething(result)),
(fun _ -> printfn "Deal with exception."),
(fun _ -> printfn "Deal with cancellation."))
I've got an agent which I set up to do some database work in the background. The implementation looks something like this:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
use conn = new System.Data.SqlClient.SqlConnection("...")
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
With this I discovered that if this was called several times in some amount of time I would start getting SqlConnection objects piling up and not being disposed, and eventually I would run out of connections in the connection pool (I don't have exact metrics on how many "several" is, but running an integration test suite twice in a row could always cause the connection pool to run dry).
If I change the use to a using then things are disposed properly and I don't have a problem:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
using (new System.Data.SqlClient.SqlConnection("...")) <| fun conn ->
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
It seems that the Using method of the AsyncBuilder is not properly calling its finally function for some reason, but it's not clear why. Does this have something to do with how I've written my recursive async expression, or is this some obscure bug? And does this suggest that utilizing use within other computation expressions could produce the same sort of behavior?
This is actually the expected behavior - although not entirely obvious!
The use construct disposes of the resource when the execution of the asynchronous workflow leaves the current scope. This is the same as the behavior of use outside of asynchronous workflows. The problem is that recursive call (outside of async) or recursive call using return! (inside async) does not mean that you are leaving the scope. So in this case, the resource is disposed of only after the recursive call returns.
To test this, I'll use a helper that prints when disposed:
let tester () =
{ new System.IDisposable with
member x.Dispose() = printfn "bye" }
The following function terminates the recursion after 10 iterations. This means that it keeps allocating the resources and disposes of all of them only after the entire workflow completes:
let rec loop(n) = async {
if n < 10 then
use t = tester()
do! Async.Sleep(1000)
return! loop(n+1) }
If you run this, it will run for 10 seconds and then print 10 times "bye" - this is because the allocated resources are still in scope during the recursive calls.
In your sample, the using function delimits the scope more explicitly. However, you can do the same using nested asynchronous workflow. The following only has the resource in scope when calling the Sleep method and so it disposes of it before the recursive call:
let rec loop(n) = async {
if n < 10 then
do! async {
use t = tester()
do! Async.Sleep(1000) }
return! loop(n+1) }
Similarly, when you use for loop or other constructs that restrict the scope, the resource is disposed immediately:
let rec loop(n) = async {
for i in 0 .. 10 do
use t = tester()
do! Async.Sleep(1000) }