Why does this IDisposable get disposed while referenced in a F# task? - .net-core

A while ago, in one of our (X)unit tests, a colleague of mine wrote:
[<Fact>]
let ``Green Flow tests`` () =
use factory = new WebAppFactory()
use client = factory.CreateClient()
Check.theGreenFlow client
|> Async.AwaitTask
|> Async.RunSynchronously
Surprised, I was wondering why my colleague forced a call to Async.RunSynchronously while XUnit works fine with Task and Async types alike.
I then tried out:
[<Fact>]
let ``Green Flow tests`` () =
use factory = new WebAppFactory()
use client = factory.CreateClient()
// this btw returns Task<unit>
Check.theGreenFlow client
And got:
Rm.Bai.IntegrationTests.RetrievalWorkflow.Green Flow tests
System.AggregateException : One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (Cannot access a disposed object.
Object name: 'IServiceProvider'.))))))
I was like "ok fair enough, scope of use ends at the bottom of the function, then released while the Task<unit> is processed by the XUnit runner".
Even though the IDisposable objects might be referenced in a function returning a Task in the example above, the runner runs the task after the end of the function and so the Dispose call has already occurred according to my understanding of https://learn.microsoft.com/en-us/dotnet/fsharp/language-reference/resource-management-the-use-keyword:
It provides the same functionality as a let binding but adds a call to Dispose on the value when the value goes out of scope. Note that the compiler inserts a null check on the value, so that if the value is null, the call to Dispose is not attempted.
[...]
when you use the use keyword, Dispose is called at the end of the containing code block
To me, the right way to avoid disposed object would have been to wrap in a computation expression like task or async computation expression:
[<Fact>]
let ``Green Flow tests`` () =
task {
use factory = new WebAppFactory()
use client = factory.CreateClient()
do! Check.theGreenFlow client
}
So that the moment when Dispose() is actually code is clearly defined.
And don't necessarily force the test to be run synchronously like in the snippet no. 1.
Unlike something like below, which still would lead to a Cannot access a disposed object. error:
[<Fact>]
let ``Green Flow tests`` () =
use factory = new WebAppFactory()
use client = factory.CreateClient()
async {
do! Check.theGreenFlow client |> Async.AwaitTask
}
Which is akin the snippet no. 2.
Is my understanding of this issue correct?

Next experiment demonstrates the flow of your second example (when there is Check.theGreenFlow client at the end). F#:
let tst(runSvc:Func<_,_>) =
printfn "[%d] start tst" Thread.CurrentThread.ManagedThreadId
use srv = {new IDisposable with
override x.Dispose() =
printfn "[%d] srv disposed" Thread.CurrentThread.ManagedThreadId }
let res: Task = runSvc.Invoke srv
printfn "[%d] end tst" Thread.CurrentThread.ManagedThreadId
res
this is analog of Green Flow tests. C#:
static async Task runSvc(IDisposable svc) {
Console.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] before runSvc");
await Task.Delay(1000);
Console.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] after runSvc");
}
static async Task Main(string[] args)
{
var task = T1.tst(runSvc);
await task;
}
runSvc plays role of Check.theGreenFlow and Main - role of XUnit test run.
The output:
[1] start tst
[1] before runSvc
[1] end tst
[1] srv disposed
[4] after runSvc
First thread starts the test and goes into Check.theGreenFlow.
At some point Check.theGreenFlow registers a continuation for asynchronous operation - in this example it is at point task.Delay.
After continuation registration, Check.theGreenFlow immediately returns with object Task (in my example, continuation is printing "after runSvc" and this part is registered for later).
When code returns from test function, service objects are disposed at the end and thread 1 awaits for result, which at some point will be created by registered continuation in the threadpool (another thread 4 executes "after runSvc"). If thread 4 accesses service objects in continuation, object disposed exception happens.
There could be race conditions. If in my example above you put Thread.Sleep(2000) before "[%d] end tst", continuation "after runSvc" will execute before disposal and there is no exception.

Related

Why doesnt my CoroutineScope stop the code until it has the Firebase data and is finished?

I created a CoroutineScope to get the data from Firebase before expanding a Card and showing that data in a listView. But the card expands while the CoroutineScope is still getting the data from Firebase and tries to display the listView with an empty list.
Here is my Expand Function inside the OnClickListener (StartPageActivity.customClassList is a List in a Object that is defined earlier):
var list : List<CustomClass> = ArrayList()
CoroutineScope(Dispatchers.IO).launch {
var customList: List<CustomClass> = ArrayList()
StartPageActivity.customClassExpandList.forEach {
if (it.title.contains(CustomClass.title)) {
customList += getFirebaseData(it.date)
if (customList.size == 12) {
list = customList
}
}
}
}
val listAdapter = MyListAdapter(context, list)
listView.adapter = listAdapter
listView.visibility = View.VISIBLE
listView.dividerHeight = 0
listView.layoutParams.height = calculateHeight(listView, 12)
Here is my getFirebaseData function:
suspend fun getFirebaseDate(date : LocalDate) : CustomClass = withContext(Dispatchers.IO){
val customClass = CustomClass("$date", date, "Empty", false)
FirebaseFirestore.getInstance().collection("users").document(FirebaseAuth.getInstance().currentUser!!.uid).collection("customClass")
.document(date.toString()).get().addOnCompleteListener { task ->
if (task.isSuccessful) {
val document = task.result
if (document.exists()) {
goal.description = document["description"].toString()
goal.title = document["tile"].toString()
}
}
}.await()
return#withContext customClass
}
The getFirebaseData function works and it returns the customClass; this is also added to my customList. But this happens while the code tries to build the expanded listView with the list, that is initiated before the CoroutineScope.
I tried to run the Code after the CoroutineScope inside that scope, but it doesn't allow that and returns an error.
I also tried adding multiple suspend functions, but that also has not fixed my problem.
I also tried putting the forEach function in a separate suspend function, but my problem still occurred.
Launching a coroutine queues it up to run in parallel or in the background, and the current function continues running without waiting for it. See here for more explanation.
To get the behavior you're expecting, you would need to use runBlocking instead of CoroutineScope(Dispatchers.IO).launch. However, this is not a viable solution for a network operation. The entire device will be completely frozen to user input and screen refreshes/animation until the coroutine is finished. If the network operation takes more than a few milliseconds, this will be unacceptable to the user, because they can't even back out of your app if they change their mind, among many other potential problems.
You must design your app to gracefully handle the fact that a network operation might take a while. This is typically done by showing some kind of loading indicator in the UI, and then you can start your coroutine, and then inside the coroutine you can update the list view when the results are ready.
Two other issues with your use of coroutines:
You should not create a new CoroutineScope like that. It will leak your fragment. Use viewLifecycleOwner.lifecycleScope.launch. And don't use Dispatchers.IO, because that will cause glitches and crashes if you start working with UI in your coroutine. You don't need Dispatchers.IO unless you're directly calling blocking code inside your coroutine, and if you do, you should wrap only those parts of the coroutine with withContext(Dispatchers.IO) { }.
In your suspend function, you should not mix using listeners and await(). That is convoluted and error prone. You're also neglecting the failure condition. If you don't use try/catch when using await(), you risk a crash or unhandled error (depending on how your coroutine scope is set up). Incidentally, you don't need to specify Dispatchers.IO here (like I mentioned above) because await() is not a blocking function. It's a suspending function. Your function should look something like this instead. I'm guessing that you are using CustomClass to communicate whether the retrieval was successful.
suspend fun getFirebaseDate(date : LocalDate) : CustomClass = try {
val document = FirebaseFirestore.getInstance()
.collection("users")
.document(FirebaseAuth.getInstance().currentUser!!.uid)
.collection("customClass")
.document(date.toString())
.await()
require(document.exists()) // Throw exception if it doesn't. Catch block will handle it.
goal.description = document["description"].toString()
goal.title = document["tile"].toString()
CustomClass("$date", date, "Empty", false)
} catch (e: Exception) {
CustomClass("$date", date, "Empty", true)
}

How to return HtmlDocument from WebBrowser using async in F#?

I'm trying to scrape a series of websites that run a bunch of javascript on the DOM before it's done loading. This means I'm using a WebBrowser instead of the friendlier WebClient. The problem I'd like to solve is to wait until the WebBrowser.DocumentCompleted event fires and then return WebBrowser.Document. I then do some post processing on the HtmlDocument but cannot get it to return yet.
The Code I Have
let downloadWebSite (address : string) =
let browser = new WebBrowser()
let browserContext = SynchronizationContext()
browser.DocumentCompleted.Add (fun _ ->
printfn "Document Loaded")
async {
do browser.Navigate(address)
let! a = Async.AwaitEvent browser.DocumentCompleted
do! Async.SwitchToContext(browserContext)
return browser.Document)
}
[downloadWebSite "https://www.google.com"]
|> Async.Parallel // there will be more addresses when working
|> Async.RunSynchronously
The Error
System.InvalidCastException: Specified cast is not valid.
at System.Windows.Forms.UnsafeNativeMethods.IHTMLDocument2.GetLocation()
at System.Windows.Forms.WebBrowser.get_Document()
at FSI_0058.downloadWebSite#209-41.Invoke(Unit _arg2) in C:\Temp\Untitled-1.fsx:line 209
at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1)
at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.FSharp.Control.AsyncResult`1.Commit()
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInAnotherThread[a](CancellationToken token, FSharpAsync`1 computation, FSharpOption`1 timeout)
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout)
at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
at <StartupCode$FSI_0058>.$FSI_0058.main#()
Stopped due to error
What I think is happening
There are several issues that make me believe that I'm accessing the WebBrowser from the wrong thread.1 2 3
Help requested
Is the use of Async.SwitchToContext(browserContext) correct here?
Could the overall approach be simplified?
Is there a concept I appear ignorant of?
How do I get the WebBrowser.Document?
The problem is in this line:
let browserContext = SynchronizationContext()
You manually created a new instance of SynchronizationContext but didn't associate it with the UI thread or any thread. That's why the program crashes when you access the browser.Document which must be accessed on the UI thread.
To solve this problem, simply use the existing SynchronizationContext which was already associated with the UI thread:
let browserContext = SynchronizationContext.Current
I assumed that the downloadWebSite function is called on the UI thread. If it is not, you can pass the context from somewhere into the function, or use a global variable.
A better design
Althought with Async.SwitchToContext you can make sure that the next line accesses and returns the document in the UI thread, but the client code which receives the document may run on a non-UI thread. A better design is to use a continuation function. Instead of directly returning a document, you can return a SomeType value produced by a continuation function passed into downloadWebSite as a parameter. By this way, the continuation function is ensured to be run on UI thread:
let downloadWebSite (address : string) cont =
let browser = new WebBrowser()
let browserContext = SynchronizationContext.Current
browser.DocumentCompleted.Add (fun _ ->
printfn "Document Loaded")
async {
do browser.Navigate(address)
let! a = Async.AwaitEvent browser.DocumentCompleted
do! Async.SwitchToContext(browserContext)
// the cont function is ensured to be run on UI thread:
return cont browser.Document }
[downloadWebSite "https://www.google.com" (fun document -> (*safely access document*))]
|> Async.Parallel
|> Async.RunSynchronously

Timeout for HttpClient doesn't work inside async block

I am trying to use HttpClient to send POST request in F#. Unfortunately I have found that timeouts in F# doesn't work. I have a code like this:
let asyncTest() = async {
let httpClient = new HttpClient()
try
let! res = httpClient.PostAsync("http://135.128.223.112", new StringContent("test"), (new CancellationTokenSource(2000)).Token) |> Async.AwaitTask
Console.WriteLine("Test 1")
with
| :? Exception as e -> Console.WriteLine("Test 2")
}
IP which you can see in code doesn't exist. I expect call to be cancelled after 2 seconds (cancellation token) but exception is never thrown ("Test 2" is never printed on screen). After PostAsync method call program control never returns into async block. Behavior is the same if I use HttpClient.Timeout property or Async.Catch instead of try block.
In C# the same code works as expected, after two seconds exception is raised:
private async Task Test()
{
var httpClient = new HttpClient();
await
httpClient.PostAsync("http://135.128.223.112", new StringContent("test"),
new CancellationTokenSource(2000).Token);
}
I know that async in F# and C# is different so I expect this be caused by some synchronization problem, deadlock ... I hope someone with deeper understanding of F# async can explain this and provide workaround.
UPDATE:
I am starting F# async work like this:
Async.Start (asyncTest())
I am running this test in console appliation and I have ReadKey() at the end so app doesn't end sooner than async.
After rading Fyodor's comment I tried to change it to RunSynchronously. Now exception is raised but still I need it work with Async.Start.
What you are seeing is a consequence of two things:
OperationCancelledException using a special path in F# async through the cancellation continuation. What it means in practice is that within the context of an async block it's an "uncatchable" exception, bypassing try-with, which works on the error continuation.
AwaitTask integrating the task you're waiting on to be part of your async block - which includes promoting task cancellation into async cancellation (OperationCancelledException instead of TaskCancelledException) and bringing your entire workflow down.
That's not an obvious behavior and it appears that it's being changed for the future versions of F#.
In the meantime, you can try something like this instead of Async.AwaitTask:
let awaitTaskCancellable<'a> (task: Task<'a>) =
Async.FromContinuations(fun (cont, econt, ccont) ->
task.ContinueWith(fun (t:Task<'a>) ->
match t with
| _ when t.IsFaulted -> econt t.Exception
| _ when t.IsCanceled ->
// note how this uses error continuation
// instead of cancellation continuation
econt (new TaskCanceledException())
| _ -> cont t.Result) |> ignore)

Chain the completion of an async function to another

I am working on a Windows Store (C++) app. This is a method that reads from the database using the web service.
task<std::wstring> Ternet::GetFromDB(cancellation_token cancellationToken)
{
uriString = ref new String(L"http://myHost:1234/RestServiceImpl.svc/attempt");
auto uri = ref new Windows::Foundation::Uri(Helpers::Trim(uriString));
cancellationTokenSource = cancellation_token_source();
return httpRequest.GetAsync(uri, cancellationTokenSource.get_token()).then([this](task<std::wstring> response)->std::wstring
{
try
{
Windows::UI::Popups::MessageDialog wMsg(ref new String(response.get().c_str()), "success");
wMsg.ShowAsync();
return response.get();
}
catch (const task_canceled&)
{
Windows::UI::Popups::MessageDialog wMsg("Couldn't load content. Check internet connectivity.", "Error");
wMsg.ShowAsync();
std::wstring abc;
return abc;
}
catch (Exception^ ex)
{
Windows::UI::Popups::MessageDialog wMsg("Couldn't load content. Check internet connectivity.", "Error");
wMsg.ShowAsync();
std::wstring abc;
return abc;
}
} , task_continuation_context::use_current());
}
I'm confused how to return the received data to the calling function. Now, I am calling this function in the constructor of my data class like this:
ternet.GetFromDB(cancellationTokenSource.get_token()).then([this](task<std::wstring> response)
{
data = ref new String(response.get().c_str());
});
I am getting a COM exception whenever I try to receive the returned data from GetFromDB(). But this one runs fine:
ternet.GetFromDB(cancellationTokenSource.get_token());
Please suggest a better way of chaining the completion of GetFromDB to other code. And how to get the returned value from inside the try{} block of GetFromDB() 's then. Please keep in mind I am a very new student of async programming.
If the continuation of the call to GetFromDB is happening on the UI thread (which I believe it will by default, assuming the call site you pasted is occurring in the UI thread), then calling get() on the returned task will throw an exception. It won't let you block the UI thread waiting for a task to finish.
Two suggestions, either of which should fix that problem. The first should work regardless, while the second is only a good option if you're not trying to get the response string to the UI thread (to be displayed, for example).
1) Write your continuations (lambdas that you pass to then) so that they take the actual result of the previous task, rather than the previous task itself. In other words, instead of writing this:
ternet.GetFromDB(...).then([this](task<std::wstring> response) { ... });
write this:
ternet.GetFromDB(...).then([this](std::wstring response) { ... });
The difference with the latter is that the continuation machinery will call get() for you (on a background thread) and then give the result to your continuation function, which is a lot easier all around. You only need to have your continuation take the actual task as an argument if you want to catch exceptions that might have been thrown by the task as it executed.
2) Tell it to run your continuation on a background/arbitrary thread:
ternet.GetFromDB(...).then([this](task<std::wstring> response) { ... }, task_continuation_context::use_arbitrary());
It won't care if you block a background thread, it only cares if you call get() on the UI thread.

Task.ContinueWith confusion

With ASP.NET 4.5 I'm trying to play with the new async/await toys. I have a IDataReader-implementing class that wraps a vendor-specific reader (like SqlDatareader). I have a simple ExecuteSql() method that operates synchronously like so:
public IDataReader ReaderForSql(string sql)
{
var cmd = NewCommand(sql, CommandType.Text);
return DBReader.ReaderFactory(cmd.ExecuteReader());
}
What I want is an Async version of this. Here's my first try:
public Task<IDataReader> ReaderForSqlAsync(string sql, CancellationToken ct)
{
var cmd = NewCommand(sql, CommandType.Text);
return cmd.ExecuteReaderAsync(ct)
.ContinueWith(t => DBReader.ReaderFactory(t.Result));
}
and I use it:
using (var r = await connection.ReaderForSqlAsync("SELECT ...", cancellationToken))
{
...
}
This works great in my limited testing so far. But after watching this Cloud9 video a few times: http://channel9.msdn.com/Events/aspConf/aspConf/Async-in-ASP-NET I got worreid about warnings they gave regarding:
ContinueWith consuming extra threadpool resources - Readerfactory is very light!
Task.Result blocking
and since I am passing a ContinuationToken to ExecuteReaderAsync() it seems cancellation is just yet another reason ExecuteReaderAsync() could fail (it's SQL after all!)
What will be the state of the task when I try to ContinueWith it? Will t.Result block? throw? do the wrong thing?
ContinueWith will use the current task scheduler (a thread pool thread) by default, but you can change that by passing TaskContinuationOptions.ExecuteSynchronously and an explicit TaskScheduler.
That said, I would make this as a first effort:
public async Task<IDataReader> ReaderForSqlAsync(string sql, CancellationToken ct)
{
var cmd = NewCommand(sql, CommandType.Text);
var readerResult = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
return DBReader.ReaderFactory(readerResult);
}
async and await handle all the ContinueWith delicacies and edge conditions in a consistent manner for you. It may be possible to complicate this code to be faster if performance testing indicates it's a serious problem.
Result blocks if the task has not completed. But in the continuation handler it already has. So it does not block. You are doing the right thing.
When you invoke Result on a faulted task (and you say this might happen) the exception is rethrown. This causes your continuation to fault with the same exception which causes the final task returned from ReaderForSqlAsync to also be faulted. This is a good thing: The entire chain of tasks faulted and all exceptions have been observed (on contrast to being swallowed). So this is best-practice, too.
Using a thread for compute-bound work is always ok. So again, you are doing the right thing using ContinueWith. You have to compute the IDataReader somewhere after all. You cannot not compute it.

Resources