I have the following:
let getCandlesFrom (exchange: IExchange) (instrument: Instrument) (interval: TimeSpan) (fromTime: DateTime) =
let rec get (c: CandleData array) (f: DateTime) =
let now = DateTime.UtcNow
//info $"requesting {f} - {now}"
let candles = exchange.GetCandles(instrument, interval, f, now)
if candles.IsError then
failwith candles.GetError.Describe
else
//info $"received data {candles.Get.[0].Timestamp} - {candles.Get.[^0].Timestamp}"
let c = Array.append c candles.Get
if c.[^0].Timestamp < now - interval then
get c (c.[^0].Timestamp + interval)
else
c
get [||] fromTime
I would like to move the line:
let candles = exchange.GetCandles(instrument, interval, f, now)
to an async call:
let! candles = exchange.GetCandlesAsync(instrument, interval, f, now)
but if I wrap the whole function in an async block, the recursive function doesn't compile and I get this error:
DataSource.fs(14, 13): [FS0588] The block following this 'let' is unfinished. Every code block is an expression and must have a result. 'let' cannot be the final code element in a block. Consider giving this block an explicit result.
I don't see your code producing the error but you must have forgotten something (after the let) - this should work:
let getCandlesFrom (exchange: IExchange) (instrument: Instrument) (interval: TimeSpan) (fromTime: DateTime) =
let rec get (c: CandleData array) (f: DateTime) =
asnyc {
let now = DateTime.UtcNow
//info $"requesting {f} - {now}"
let! candles = exchange.GetCandlesAsync(instrument, interval, f, now)
if candles.IsError then
return (failwith candles.GetError.Describe)
else
let c = Array.append c candles.Get
if c.[^0].Timestamp < now - interval then
return! get c (c.[^0].Timestamp + interval)
else
return c
}
get [||] fromTime
notice the return and the return! for the recursive call - you need those there
Related
I need to run a process that runs something from a list - it doesn't really matter what order it runs in - but I want it to update a global "counter" when it has completed each task so that I can see the progress somewhere else ( maybe using something like signal R )
I used to do this stuff in an object-oriented way - but trying to be a little more "functional".
let doSomethingElse(value: int) = async{
// Update a global counter incrementing it by 1
return true
}
let doSomething() = async{
let values = [2; 4; 6; 8]
let! newList = values |> List.map(fun value -> doSomethingElse(value)) |> Async.Parallel
return true
}
Following what #JL0PD mentioned you could do something like the following.
[<RequireQualifiedAccess>]
module Async =
let tee (f: _ -> unit) a =
async {
let! r = a
f r
return r
}
module Counter =
let create () =
let mutable counter = 0
let increment =
fun _ ->
counter <- counter + 1
()
(increment, fun () -> counter)
let doSomethingElse(value: int) = async {
return value % 2
}
let doSomething() = async {
let values = [1; 2; 4; 6; 8; 9]
let increment, getCount = Counter.create()
let doSomethingElseWithProgress =
doSomethingElse
>> (Async.tee increment)
let! _ =
values
|> List.map doSomethingElseWithProgress
|> Async.Parallel
return getCount() = (List.length values)
}
doSomething ()
|> Async.RunSynchronously
I do recommend doing something better than a mutable counter, specially since you are dealing with parallel tasks.
In this case I'm composing doSomethingElse and increment using tee, this way doSomethingElse don't have to know (and call) an external function.
Suppose we have a library that provides a higher order function applyTest.
Can this be used with an asynchronous function asyncFunction while retaining the benefits of asynchronous code?
Can the library be designed to better support asynchronous applications without specifically providing an asynchronous version?
let applyTest f =
f 2 > 0
let syncFunction x =
x - 1
let asyncFunction x =
x - 2 |> async.Return
async {
let a = applyTest syncFunction
let b = applyTest (asyncFunction >> Async.RunSynchronously)
printfn "a = %b, b = %b" a b
}
|> Async.RunSynchronously
You would need to provide a separate async version if you didn't want to lose strong type checking, or running async computations synchronously like in your examples. Both of these things should be avoided as much as possible.
If you wanted to avoid repetition of the actual testing part (f 2 > 0) you could split this out into a function that passes the parameter 2 to the function and a function to check the value is greater than zero:
// LIBRARY CODE
let checkValue x = x > 0
// This function is generic so it can return a value or an async value
// (int -> 'a) -> 'a
let runTestFunction f = f 2
// (int -> int) -> bool
let applyTest f = f |> runTestFunction |> checkValue
// (int -> Async<int>) -> Async<bool>
let applyTestAsync f = async {
let! value = runTestFunction f // use let! to await the value
return checkValue value }
// USAGE
let syncFunction x = x - 1
let asyncFunction x = x - 2 |> async.Return
async {
let a = applyTest syncFunction
let! b = applyTestAsync asyncFunction // use let! to await the test result
printfn "a = %b, b = %b" a b
}
Another option would be to use overloaded methods. This builds on the functions defined above:
type Test =
static member Apply f = applyTest f
static member Apply f = applyTestAsync f
// USAGE
async {
let a = Test.Apply syncFunction
let! b = Test.Apply asyncFunction // We still need to consume this differently with a let!
printfn "a = %b, b = %b" a b
}
I'm trying to write a function (OnceAsync f) that ensures that an async function is run only once on a server (i.e. a multi-threaded environment). I thought it would be easy, but it became complicated quickly (locks, busy waits!!)
This is my solution, but I think it's over-engineered; there must be a better way. This should work in FSI:
let locked_counter init =
let c = ref init
fun x -> lock c <| fun () ->
c := !c + x
!c
let wait_until finished = async {
while not(finished()) do
do! Async.Sleep(1000)
}
let OnceAsync f =
// - ensure that the async function, f, is only called once
// - this function always returns the value, f()
let mutable res = None
let lock_inc = locked_counter 0
async {
let count = lock_inc 1
match res, count with
| None, 1 -> // 1st run
let! r = f
res <- Some r
| None, _ -> // nth run, wait for 1st run to finish
do! wait_until (fun() -> res.IsSome)
| _ -> () // 1st run done, return result
return res.Value
}
You can use this code to test if OnceAsync is correct:
let test() =
let mutable count = 0
let initUser id = async {
do! Async.Sleep 1000 // simulate work
count <- count + 1
return count
}
//let fmem1 = (initUser "1234")
let fmem1 = OnceAsync (initUser "1234")
async {
let ps = Seq.init 20 (fun i -> fmem1)
let! rs = ps |> Async.Parallel
printfn "rs = %A" rs // outputs: [|1; 1; 1; 1; 1; ....; 1|]
}
test() |> Async.Start
If it fits, the simplest approach overall would be to use Async.StartChild. Unlike your solution it causes the function to run even if the result is never actually used though, e.g. in the Seq.init 0 case.
//let fmem1 = OnceAsync (initUser "1234")
async {
let! fmem1 = Async.StartChild (initUser "1234")
let ps = Seq.init 20 (fun i -> fmem1)
let! rs = ps |> Async.Parallel
printfn "rs = %A" rs // outputs: [|1; 1; 1; 1; 1; ....; 1|]
} |> Async.RunSynchronously
The simplest approach most similar to yours would be to use a TaskCompletionSource as follows:
let OnceAsync f =
let count = ref 0
let tcs = TaskCompletionSource<_>()
async {
if Interlocked.Increment(count) = 1 then
let! r = f
tcs.SetResult r
return! Async.AwaitTask tcs.Task
}
A more functional approach would use a MailboxProcessor and have it cache the result after the first run, and respond with it to all subsequent requests.
let OnceAsync f =
let handler (agent: MailboxProcessor<AsyncReplyChannel<_>>) =
let rec run resultOpt =
async {
let! chan = agent.Receive()
let! result =
match resultOpt with
| None -> f
| Some result -> async.Return result
chan.Reply result
return! run (Some result)
}
run None
let mbp = MailboxProcessor.Start handler
async { return! mbp.PostAndAsyncReply id }
So I am doing a some batch computation very cpu intensive on books. And I built a tracker to track the computation of tasks. I close on a mailboxprocesser which all runs fine without parallelizaton but when I put a array.parallel.map or and async workflow the mailboxprocesser fails. I want to know why?
type timerMessage =
| Start of int
| Tick of bool
let timer = MailboxProcessor.Start(fun mbox ->
let inputloop() = async {
let progress = ref 0
let amount = ref 0
let start = ref System.DateTime.UtcNow
while true do
let! msg = mbox.Receive()
match msg with
| Start(i) -> amount := i
progress := 0
start := System.DateTime.UtcNow
| Tick(b) -> if !amount = 0 then ()
else
progress := !progress + 1
let el = System.DateTime.UtcNow - !start
let eta = int ((el.TotalSeconds/float !progress)*(float (!amount - !progress)))
let etas = (int (eta / 3600)).ToString() + ":" + (int ((eta % 3600) / 60)).ToString() + ":" + (eta % 60).ToString()
System.Console.Clear()
System.Console.Write((!progress).ToString() + "/" + (!amount).ToString() + " Completed [] Estimated Time Remaining:" + etas)
} inputloop() )
let computeBook (author :string) path =
let rs = ReadToStrings(path)
let bk = StringsToBook rs
let mt = createMatrix bk 100 10 //size 100 //span 10
let res = GetResults mt
//do stuff
timer.Post(Tick(true))
(author,path,res)
let partAopA = //clip head clip foot no word mods
let lss = seq {for x in processtree do
for y in (snd x) do
yield ((fst x),y) }
let ls = Seq.toArray lss //task list
timer.Post(Start(ls.Length)) //start counter
let compls = Array.map (fun l -> computeBook (fst l) (snd l) ) ls //Array.Parallel.map fails here the same as below async if I put async blcoks around the computbook call
//let res = compls |> Async.Parallel |> Async.RunSynchronously
writeResults compls outputfolder |> ignore
compls
What I would like to do is have a function that I can repeatedly pass a transformation function into and receive a combined transformation, the transformation function would be of the form 'a -> 'b
i.e. rather than compose a fixed workflow like this:
let input = async{ let! transform1 = transformAB input
let! transform2 = transformBC transform1
let! transform3 = transformCD transform2
return! transform3 }
I would like to be able to do this:
let combined = buildTransform(transform1).Next(transform2).Next(transform3)
So then I could simply call combined input to get the results of the workflow.
Would this be possible without hitting the value restriction, or the compiler constraining all the transformers to be the same type?
I'm not quite sure that I got your question, you need something similar to (>>) operator applied to Async?
open System
let f1 a = async { return Int32.Parse a }
let f2 a = async { return a = 10 }
let f3 a = async { return (not a).ToString() }
// async defined via workflow syntax
// string -> Async<string>
let result a = async {
let! x1 = f1 a
let! x2 = f2 x1
let! x3 = f3 x2
return x3
}
Async.RunSynchronously (result "10")
|> printfn "%s"
let (|>>) f1 f2 arg = async {
let! r = f1 arg
return! f2 r
}
// async defined with 'combine' operator
// string -> Async<string>
let combined = f1 |>> f2 |>> f3
Async.RunSynchronously (combined "10")
|> printfn "%s"