F# has Async.Parallel with type signature seq<Async<'a>> -> Async<'a list>, which will take a seq of Async types, prep them to run in parallel, and output a single Async.
I was wondering if there is a similar Async.Sequential which has the same type signature, but runs each item in sequence one after the other? I feel like I must be missing something obvious?
I'd like to do something like the following:
async {
let! results =
myItems
|> List.map myAsynchronousOperation
|> Async.Sequential
... do something with results
}
I don't think there's one built in, but should be trivial to write your own:
let sequential s = async {
let results = ResizeArray()
for a in s do
let! result = a
results.Add(result)
return List.ofSeq results
}
Evaluating the items in a seq<Async<'a>> in-order and returning a list is effectively just a foldBack on the sequence where you want to evaluate the async before simply cons-ing it onto a list. While kvb's solution certainly works, you could also do it with a Seq.foldBack like so:
module Async =
let sequential s =
Seq.foldBack (fun cur acc -> async.Bind(cur, fun head ->
async.Bind(acc, fun tail -> async.Return(head :: tail)))) s (async.Return [])
With the release of F# 4.7 and FSharp.Core 4.7 Async.Sequential is now available "out-of-the-box".
Announcement: https://devblogs.microsoft.com/dotnet/announcing-f-4-7/
Source: https://github.com/dotnet/fsharp/blob/897afd3dad3bfe58a0495713e99a8094098c18c5/src/fsharp/FSharp.Core/async.fs#L1282
If one is willing to use the AsyncSeq library, then the following does what's needed in the given example:
async {
let! results =
myItems
|> AsyncSeq.ofSeq
|> AsyncSeq.mapAsync myAsynchronousOperation
|> AsyncSeq.toListAsync
... do something with results
}
Probably overkill if you're not already using AsyncSeq for other things though.
Related
I have a function that I want to run 5 at a time from a chunked list and then wait a second so as not to hit an Api rate limit. I know the code is wrong but it is as far as I have got.
let ordersChunked = mOrders |> List.chunkBySize 5
for fiveOrders in ordersChunked do
let! tasks =
fiveOrders
|> List.map (fun (order) -> trackAndShip(order) )
|> Async.Parallel
Async.Sleep(1000) |> ignore
trackAndShip is an async function not a task and I get a little confused about the two.
Some of the answers I have read add |> Async.RunSynchronously to the end of it - and I do that at the top level but I don't feel it is good practice here ( and I would need to convert async to Task? )
This is probably a very simple answer if you are familiar with async / tasks but I am still "finding my feet" in this area.
Unlike C#'s tasks, Async computation do not start when they are created and you have to explicitly start them. Furthermore, even if you would start the Async operation, it won't do what you expect because the sleep will not execute because you just ignore it.
From the let! in your example code, I'm going to assume you're the code snippet is inside a computation expression. So the following may be what you want
let asyncComputation =
async {
let ordersChunked = [] |> List.chunkBySize 5
for fiveOrders in ordersChunked do
let! tasks =
fiveOrders
|> List.map (fun (order) -> trackAndShip(order) )
|> Async.Parallel
do! Async.Sleep(1000) // Note the do! here, without it your sleep will be ignored
}
Async.RunSynchronously asyncComputation
Note that the Async.Sleep is now chained into the CE by using do!. The Async.RunSynchronously call is one way to start the async computation: it runs the async operation and blocks the current thread while waiting for the async operation's result. There are many other ways of running it described in the documentation.
I come from a C# background having used async/ await. I am trying to find a "less verbose" way of programming using a library.
( specifically the Microsoft Playwright library for browser automation )
let (~~) = Async.AwaitTask
let getLastPageNumber(page: IPage) =
let playwright = ~~Playwright.CreateAsync() |> Async.RunSynchronously
let browser = ~~playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions(Headless = false )) |> Async.RunSynchronously
~~page.GotoAsync("https://go.xero.com/BankRec/BankRec.aspx?accountID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&page1") |> Async.RunSynchronously |> ignore
let lastPageLink = ~~page.QuerySelectorAsync("#mainPagerEnd") |> Async.RunSynchronously
if lastPageLink = null then
//this is the last page
1
else
let lastPageNumber = ~~lastPageLink.GetAttributeAsync("href") |> Async.RunSynchronously
lastPageNumber |> int
I have shortened things a bit using the alias ~~ for Async.AwaitTask but it seems to be a lot of code to do something that was a lot easier in C#.
Async.RunSynchronously should only be used as a very last resort because it blocks a thread to perform the computation, which defeats the purpose of using async/tasks.
The F# equivalent of C#'s async/await is to use F#'s Async type, and the async computation expression. However, if you're using a .NET library which uses the .NET Task type then you can use the TaskBuilder.fs library which has a task computation expression.
Then you would write the function like this:
open FSharp.Control.Tasks
let getLastPageNumber(page: IPage) = task {
let! playwright = Playwright.CreateAsync()
let! browser = playwright.Chromium.LaunchAsync(new BrowserTypeLaunchOptions(Headless = false ))
let! _ = page.GotoAsync("https://go.xero.com/BankRec/BankRec.aspx?accountID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx&page1")
let! lastPageLink = page.QuerySelectorAsync("#mainPagerEnd")
if lastPageLink = null then
//this is the last page
return 1
else
let! lastPageNumber = lastPageLink.GetAttributeAsync("href")
return lastPageNumber |> int
}
Inside the computation expression (also called a builder) let! is used to await tasks.
Note that this function now returns Task<int> rather than int, so the caller would probably need to follow a similar pattern and propogate the task further.
You can read more about using async in F# here. Some of that knowledge can also be applied to the task computation expression.
I get some list of data from a HTTP call. I then know what values to get for another HTTP call. I would like to have everything be asynchronous. But I need to use this data with Expecto's testCaseAsync : string -> Async<unit> -> Test. So, my goal is to get a signature like so Async<Item>[]
So, I would like to get a list of testCaseAsync.
So, I basically have something like this:
// Async<Async<Item>[]>
let getAsyncCalls =
async {
let! table = API.getTable ()
// Async<Item>[]
let items =
table.root
|> Array.map (fun x -> API.getItem x.id)
return item
}
If I run them in parallel I get:
// Async<Item[]>
let getAsyncCalls =
async {
let! table = API.getTable ()
// Item[]
let! items =
table.root
|> Array.map (fun x -> API.getItem x.id)
return item
}
So, that doesn't get me to Async<Item>[]. I'm not sure if this is possible. I would like to avoid Async.RunSynchronously for the API.getTable call since that can lead to deadlocks, right? It will most likely be called from a cached value (memoized) so I'm not sure that will make a difference.
I guess I'll keep working on it unless someone else is more clever than me :-) Thanks in advance!
In general, you cannot turn Async<Async<T>[]> into Async<T>[]. The problem is that to even get the length of the array, you need to perform some operation asynchronously, so there is no way to "lift" the array outside of the async. If you knew the length of the array in advance, then you can make this work.
The following function turns Async<'T[]> into Async<'T>[] provided that you give it the length of the array. As you figured out, the returned asyncs need to somehow share access to the one top-level async. The easiest way of doing this I can think of is to use a task. Adapting that for your use case should be easy:
let unwrapAsyncArray (asyncs:Async<'T[]>) len =
let task = asyncs |> Async.StartAsTask
Array.init len (fun i -> async {
let! res = Async.AwaitTask task
if res.Length <> len then failwith "Wrong length!"
return res.[i] }
)
In writing some code that works with a lot of nested async workflows lately I've found a pattern emerging that smells to me. A simple example:
let flip f x y = f y x
let slowInc x = async {
do! Async.Sleep 500
printfn "Here you go, %d" x
}
let verboseFun inp = async {
match List.tryFind (flip (>) 3) inp with
| Some x -> do! slowInc x
| _ -> ()
}
verboseFun [1..5] |> Async.RunSynchronously
The 'verboseFun' to me seems verbose but I can't think of a way to combine the Option and Async monads so it can be rewritten without the pattern match. I was thinking something like
let terseFun inp = async {
inp
|> List.tryFind (flip (>) 3)
|> Option.iterAsync slowInc
}
It just seemed to me that it's highly likely I just don't know what building blocks are available to achieve this.
EDIT: Extra clarification after Tomas' answer.
I was trying to adapt what would be trivial to me if everything was synchronous, e.g.,
let terseFun inp =
inp
|> List.tryFind (flip (>) 3)
|> Option.iter someSideEffectFunciton
to become part of nested async workflows. Originally I was thinking "just chuck a do! in there" so came up with
let terseFun inp = async {
inp
|> List.tryFind (flip (>) 3)
|> Option.iter (fun x -> async { do! someSideEffectFunciton x })
|> ignore
}
But it immediately smelled wrong to me because VS started demanding the ignore.
Hope this helps clarify.
The ExtCore library has a bunch of helper functions that let you work with asynchronous computations that return optional values, i.e. of type Async<'T option> and it even defines asyncMaybe computation builder for working with them.
I have not used it extensively, but from a few simple experiments I did, it looks like it is not as nicely integrated with the rest of F#'s async functionality as it perhaps could be, but if you want to go in this direction, ExtCore is probably the best library around.
The following is using the iter function from AsyncMaybe.Array (source is here). It is a bit ugly, because I had to make slowInc be of type Async<unit option>, but it is pretty close to what you asked for:
let slowInc x = async {
do! Async.Sleep 500
printfn "Here you go, %d" x
return Some ()
}
let verboseFun inp =
inp
|> List.tryFind (fun x -> 3 > x)
|> Array.ofSeq
|> AsyncMaybe.Array.iter slowInc
|> Async.Ignore
Aside, I also removed your flip function, because this is not generally recommended style in F# (it tends to make code cryptic).
That said, I think you don't really need an entire ExtCore library. It is hard to see what is your general pattern from just one example you posted, but if all your code snippets look similar to the one you posted, you can just define your own asyncIter function and then use it elsewhere:
let asyncIter f inp = async {
match inp with
| None -> ()
| Some v -> do! f v }
let verboseFun inp =
inp
|> List.tryFind (fun x -> x > 3)
|> asyncIter slowInc
The great thing about F# is that it is really easy to write these abstractions yourself and make them so that they exactly match your needs :-)
Is there any primitive in the langage to compose async1 then async2, akin to what parallel does for parallel execution planning ?
to further clarify, I have 2 async computations
let toto1 = Async.Sleep(1000)
let toto2 = Async.Sleep(1000)
I would like to create a new async computation made of the sequential composition of toto1 and toto2
let toto = Async.Sequential [|toto1; toto2|]
upon start, toto would run toto1 then toto2, and would end after 2000 time units
The async.Bind operation is the basic primitive that asynchronous workflows provide for sequential composition - in the async block syntax, that corresponds to let!. You can use that to express sequential composition of two computations (as demonstrated by Daniel).
However, if you have an operation <|> that Daniel defined, than that's not expressive enough to implement async.Bind, because when you compose things sequentially using async.Bind, the second computation may depend on the result of the first one. The <e2> may use v1:
async.Bind(<e1>, fun v1 -> <e2>)
If you were writing <e1> <|> <e2> then the two operations have to be independent. This is a reason why the libraries are based on Bind - because it is more expressive form of sequential composition than the one you would get if you were following the structure of Async.Parallel.
If you want something that behaves like Async.Parallel and takes an array, then the easiest option is to implement that imperatively using let! in a loop (but you could use recursion and lists too):
let Sequential (ops:Async<'T>[]) = async {
let res = Array.zeroCreate ops.Length
for i in 0 .. ops.Length - 1 do
let! value = ops.[i]
res.[i] <- value
return res }
I'm not sure what you mean by "primitive." Async.Parallel is a function. Here are a few ways to run two asyncs:
In parallel:
Async.Parallel([|async1; async2|])
or
async {
let! child = Async.StartChild async2
let! result1 = child
let! result2 = async1
return [|result1; result2|]
}
Sequentially:
async {
let! result1 = async1
let! result2 = async2
return [|result1; result2|]
}
You could return tuples in the last two. I kept the return types the same as the first one.
I would say let! and do! in an async { } block is as close as you'll get to using a primitive for this.
EDIT
If all this nasty syntax is getting to you, you could define a combinator:
let (<|>) async1 async2 =
async {
let! r1 = async1
let! r2 = async2
return r1, r2
}
and then do:
async1 <|> async2 |> Async.RunSynchronously