let's look at that code:
let rec doSomething () =
let d = GetSomeDataFromSomewhere()
match d with
| Some x -> x
| None -> doSomething()
so that's some form of non stop polling..
but now the following form:
let rec doSomething () =
try
let d = GetSomeDataFromSomewhereButItCouldCrash()
match d with
| Some x -> x
| None -> doSomething()
with _ ->
doSomething()
that one will lead to a stack overflow if there are a lot of exceptions.
Can someone explain the mechanics at play that make the two versions behave differently?
The issue is that the first call in your second version is not in a tail-call position. This is not entirely obvious, because the recursive call is the "last thing the function does", but the runtime still has to keep the stack frame around, because it needs to keep the associated exception handler.
let rec doSomething () =
try
let d = GetSomeDataFromSomewhereButItCouldCrash()
match d with
| Some x -> x
| None -> doSomething() // This is not a tail call!
with _ ->
doSomething() // This is a tail call
If you handle exceptions directly when calling GetSomeDataFromSomewhere and turn them into None, then you can keep the same logic, but make it tail recursive:
let rec doSomething () =
let d = try GetSomeDataFromSomewhereButItCouldCrash() with _ -> None
match d with
| Some x -> x
| None -> doSomething()
Related
Suppose I have some C# code that takes a callback:
void DoSomething(Action<string> callback);
Now, I want to use this in F#, but wrap it in an async. How would I go about this?
// Not real code
let doSomething = async {
let mutable result = null
new Action(fun x -> result <- x) |> Tasks.DoSomething
// Wait for result to be assigned
return result
}
For example, suppose DoSomething looks like this:
module Tasks
let DoSomething callback =
callback "Hello"
()
Then the output of the following should be "Hello":
let wrappedDoSomething = async {
// Call DoSomething somehow
}
[<EntryPoint>]
let main argv =
async {
let! resultOfDoSomething = wrappedDoSomething
Console.WriteLine resultOfDoSomething
return ()
} |> Async.RunSynchronously
0
The function Async.FromContinuations is, so to say, the "lowest level" of Async. All other async combinators can be expressed in terms of it.
It is the lowest level in the sense that it directly encodes the very nature of async computations - the knowledge of what to do in the three possible cases: (1) a successful completion of the previous computation step, (2) a crash of the previous computation step, and (3) cancellation from outside. These possible cases are expressed as the three function-typed arguments of the function that you pass to Async.FromContinuations. For example:
let returnFive =
Async.FromContinuations( fun (succ, err, cancl) ->
succ 5
)
async {
let! res = returnFive
printfn "%A" res // Prints "5"
}
|> Async.RunSynchronously
Here, my function fun (succ, err, cancl) -> succ 5 has decided that it has completed successfully, and calls the succ continuation to pass its computation result to the next step.
In your case, the function DoSomething expresses only one of the three cases - i.e. "what to do on successful completion". Once you're inside the callback, it means that whatever DoSomething was doing, has completed successfully. That's when you need to call the succ continuation:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( fun res -> succ res )
)
Of course, you can avoid a nested lambda-expression fun res -> succ res by passing succ directly into DoSomething as callback. Unfortunately, you'll have to explicitly specify which type of Action to use for wrapping it, which negates the advantage:
let doSometingAsync =
Async.FromContinuations( fun (succ, err, cancl) ->
Tasks.DoSomething( System.Action<string> succ )
)
As an aside, note that this immediately uncovered a hole in the DoSomething's API: it ignores the error case. What happens if DoSomething fails to do whatever it was meant to do? There is no way you'd know about it, and the whole async workflow will just hang. Or, even worse: the process will exit immediately (depending on how the crash happens).
If you have any control over DoSomething, I suggest you address this issue.
You can try something like:
let doSomething callback = async {
Tasks.DoSomething(callback)
}
If your goal is to define the callback in the method you could do something like:
let doSomething () = async {
let callback = new Action<string>(fun result -> printfn "%A" result )
Tasks.DoSomething(callback)
}
If your goal is to have the result of the async method be used in the DoSomething callback you could do something like:
let doSomething =
Async.StartWithContinuations(
async {
return result
},
(fun result -> Tasks.DoSomething(result)),
(fun _ -> printfn "Deal with exception."),
(fun _ -> printfn "Deal with cancellation."))
I'm playing around asynchronous programming and was wondering if there's a function that exists that can take a value of type 'T and transform it to an Async<'T>, similar to C#'s Task.FromResult that can take a value of type TResult and transform it to a Task<TResult> that can then be awaited.
If such a function does not exist in F#, is it possible to create it? I can kind of emulate this by using Async.AwaitTask and Task.FromResult, but can I do this by only using Async?
Essentially, I'd like to be able to do something like this:
let asyncValue = toAsync 3 // toAsync: 'T -> Async<'T>
let foo = async{
let! value = asyncValue
}
...or just async.Return
let toAsync = async.Return
let toAsync` x = async.Return x
moreover there is async.Bind (in tupled form)
let asyncBind
(asyncValue: Async<'a>)
(asyncFun: 'a -> Async<'b>) : Async<'b> =
async.Bind(asyncValue, asyncFun)
you could use them to make pretty complicated async computation without builder gist link
let inline (>>-) x f = async.Bind(x, f >> async.Return)
let requestMasterAsync limit urls =
let results = Array.zeroCreate (List.length urls)
let chunks =
urls
|> Seq.chunkBySize limit
|> Seq.indexed
async.For (chunks, fun (i, chunk) ->
chunk
|> Seq.map asyncMockup
|> Async.Parallel
>>- Seq.iteri (fun j r -> results.[i*limit+j]<-r))
>>- fun _ -> results
You can use return within your async expression:
let toAsync x = async { return x }
I am currently learning Lwt. I am interested into using asynchronous processes to replace some shell routines by OCaml routines.
Let us take a look at a simplified first attempt, where a filter is created by combining two threads running cat:
let filter_cat ()=
Lwt_process.pmap_lines ("cat", [| "cat" |])
let filter_t () =
Lwt_io.stdin
|> Lwt_io.read_lines
|> filter_cat ()
|> filter_cat ()
|> Lwt_io.write_lines Lwt_io.stdout
let () =
filter_t ()
|> Lwt_main.run
This filter somehow works but hangs up when its standard input closes instead of exiting. If I remove one of the filter_cat, it works as expected.
I am guessing that I do not compose these filters appropriately and therefore cannot join the two threads I am starting. What is the correct way to compose these filters, so that the program terminates after it reads EOF on stdin?
You can find this program together with a BSD Owl Makefile in a Github gist.
The answer to this, is that there is a little bug in Lwt. There is an internal function, monitor that which performs the piping:
(* Monitor the thread [sender] in the stream [st] so write errors are
reported. *)
let monitor sender st =
let sender = sender >|= fun () -> None in
let state = ref Init in
Lwt_stream.from
(fun () ->
match !state with
| Init ->
let getter = Lwt.apply Lwt_stream.get st in
let result _ =
match Lwt.state sender with
| Lwt.Sleep ->
(* The sender is still sleeping, behave as the
getter. *)
getter
| Lwt.Return _ ->
(* The sender terminated successfully, we are
done monitoring it. *)
state := Done;
getter
| Lwt.Fail _ ->
(* The sender failed, behave as the sender for
this element and save current getter. *)
state := Save getter;
sender
in
Lwt.try_bind (fun () -> Lwt.choose [sender; getter]) result result
| Save t ->
state := Done;
t
| Done ->
Lwt_stream.get st)
The problem is in the definition
let getter = Lwt.apply Lwt_stream.get st
When the getter process meets the end of the stream, then it is saved, but the sender is lost, which seems to prevent completion. This can be fixed by improving the definition of getter by telling it to behave as the sender when the end of the stream has been reached.
I've started learning F# and I'd like to write my own map function using tail-recursion. Here is what I have
let my_map ff list =
let rec mapAcc ff_inner list_inner acc =
match list_inner with
| [] -> acc
| front::rest -> mapAcc( ff_inner rest (ff_inner(front) :: acc) ) //error
mapAcc ff list []
It would be called like this:
let list = my_map (fun x -> x * 2) [1;2;3;4;5] // expect [2;4;6;8;10]
I get an compilation error message on the second condition that says Type mismatch. Expecting a 'a but given a 'b list -> 'a -> 'a The resulting type would be infinite when unifying ''a' and ''b list -> 'a -> 'a'
I don't know what this error message means. I'm not sure how this can be infinite if I am passing the rest in the recursive call to mapAcc.
Note: I realize I'm rebuilding the list backwards. I'm ignoring that for now.
Just remove the parenthesis when the function calls itself:
let my_map ff list =
let rec mapAcc ff_inner list_inner acc =
match list_inner with
| [] -> acc
| front::rest -> mapAcc ff_inner rest (ff_inner(front) :: acc)
mapAcc ff list []
otherwise everything contained there is interpreted as a single parameter and ff_inner as a function call with the rest as parameters.
I've got an agent which I set up to do some database work in the background. The implementation looks something like this:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
use conn = new System.Data.SqlClient.SqlConnection("...")
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
With this I discovered that if this was called several times in some amount of time I would start getting SqlConnection objects piling up and not being disposed, and eventually I would run out of connections in the connection pool (I don't have exact metrics on how many "several" is, but running an integration test suite twice in a row could always cause the connection pool to run dry).
If I change the use to a using then things are disposed properly and I don't have a problem:
let myAgent = MailboxProcessor<AgentData>.Start(fun inbox ->
let rec loop =
async {
let! data = inbox.Receive()
using (new System.Data.SqlClient.SqlConnection("...")) <| fun conn ->
data |> List.map (fun e -> // Some transforms)
|> List.sortBy (fun (_,_,t,_,_) -> t)
|> List.iter (fun (a,b,c,d,e) ->
try
... // Do the database work
with e -> Log.error "Yikes")
return! loop
}
loop)
It seems that the Using method of the AsyncBuilder is not properly calling its finally function for some reason, but it's not clear why. Does this have something to do with how I've written my recursive async expression, or is this some obscure bug? And does this suggest that utilizing use within other computation expressions could produce the same sort of behavior?
This is actually the expected behavior - although not entirely obvious!
The use construct disposes of the resource when the execution of the asynchronous workflow leaves the current scope. This is the same as the behavior of use outside of asynchronous workflows. The problem is that recursive call (outside of async) or recursive call using return! (inside async) does not mean that you are leaving the scope. So in this case, the resource is disposed of only after the recursive call returns.
To test this, I'll use a helper that prints when disposed:
let tester () =
{ new System.IDisposable with
member x.Dispose() = printfn "bye" }
The following function terminates the recursion after 10 iterations. This means that it keeps allocating the resources and disposes of all of them only after the entire workflow completes:
let rec loop(n) = async {
if n < 10 then
use t = tester()
do! Async.Sleep(1000)
return! loop(n+1) }
If you run this, it will run for 10 seconds and then print 10 times "bye" - this is because the allocated resources are still in scope during the recursive calls.
In your sample, the using function delimits the scope more explicitly. However, you can do the same using nested asynchronous workflow. The following only has the resource in scope when calling the Sleep method and so it disposes of it before the recursive call:
let rec loop(n) = async {
if n < 10 then
do! async {
use t = tester()
do! Async.Sleep(1000) }
return! loop(n+1) }
Similarly, when you use for loop or other constructs that restrict the scope, the resource is disposed immediately:
let rec loop(n) = async {
for i in 0 .. 10 do
use t = tester()
do! Async.Sleep(1000) }