I am trying to use HttpClient to send POST request in F#. Unfortunately I have found that timeouts in F# doesn't work. I have a code like this:
let asyncTest() = async {
let httpClient = new HttpClient()
try
let! res = httpClient.PostAsync("http://135.128.223.112", new StringContent("test"), (new CancellationTokenSource(2000)).Token) |> Async.AwaitTask
Console.WriteLine("Test 1")
with
| :? Exception as e -> Console.WriteLine("Test 2")
}
IP which you can see in code doesn't exist. I expect call to be cancelled after 2 seconds (cancellation token) but exception is never thrown ("Test 2" is never printed on screen). After PostAsync method call program control never returns into async block. Behavior is the same if I use HttpClient.Timeout property or Async.Catch instead of try block.
In C# the same code works as expected, after two seconds exception is raised:
private async Task Test()
{
var httpClient = new HttpClient();
await
httpClient.PostAsync("http://135.128.223.112", new StringContent("test"),
new CancellationTokenSource(2000).Token);
}
I know that async in F# and C# is different so I expect this be caused by some synchronization problem, deadlock ... I hope someone with deeper understanding of F# async can explain this and provide workaround.
UPDATE:
I am starting F# async work like this:
Async.Start (asyncTest())
I am running this test in console appliation and I have ReadKey() at the end so app doesn't end sooner than async.
After rading Fyodor's comment I tried to change it to RunSynchronously. Now exception is raised but still I need it work with Async.Start.
What you are seeing is a consequence of two things:
OperationCancelledException using a special path in F# async through the cancellation continuation. What it means in practice is that within the context of an async block it's an "uncatchable" exception, bypassing try-with, which works on the error continuation.
AwaitTask integrating the task you're waiting on to be part of your async block - which includes promoting task cancellation into async cancellation (OperationCancelledException instead of TaskCancelledException) and bringing your entire workflow down.
That's not an obvious behavior and it appears that it's being changed for the future versions of F#.
In the meantime, you can try something like this instead of Async.AwaitTask:
let awaitTaskCancellable<'a> (task: Task<'a>) =
Async.FromContinuations(fun (cont, econt, ccont) ->
task.ContinueWith(fun (t:Task<'a>) ->
match t with
| _ when t.IsFaulted -> econt t.Exception
| _ when t.IsCanceled ->
// note how this uses error continuation
// instead of cancellation continuation
econt (new TaskCanceledException())
| _ -> cont t.Result) |> ignore)
Related
I want to use a REST-call like this:
private async Task<HttpResponseMessage> SendRequestHelperAsync(HttpRequestMessage message, CancellationToken ct)
{
var response = await HttpClient.SendAsync(message, ct).ConfigureAwait(false);
return response;
}
from a non-async method. When I use it like this:
var response = SendRequestHelperAsync(message, ct).Result;
it gets stuck. When I use it directly, it gets stuck as well:
HttpResponseMessage response = HttpClient.SendAsync(message, ct).Result;
When I was looking for some solution, I came across using a task like this:
var task = Task.Run(async () => { await SendRequestHelperAsync(message, ct); });
but then how do I get the result back?
(I am using it from .Net Framework 4.8, WinForms Application.)
I want to use [async code] from a non-async method.
The best answer is "don't". Instead, go async all the way.
You could use Task.Run as a workaround; this is the thread pool hack, which will work in this specific case (warning: it does not work in every case, specifically, if the code in the Task.Run delegate accesses UI elements, then this hack will not work). The thread pool hack supports return types, as such:
var response = Task.Run(async () => await SendRequestHelperAsync(message, ct))
.GetAwaiter().GetResult();
However, the best answer is still "don't". The fact that you're deadlocking (as explained in detail on my blog) implies that this code is being run on the UI thread, and the solution is attempting to block the UI thread on a network request. Blocking the UI thread is a poor user experience.
Instead, the best solution is to redesign the UI so that no blocking is necessary. This is usually done by adding some kind of spinner or "loading..." UI element, and then updating the UI when the asynchronous operation completes.
A while ago, in one of our (X)unit tests, a colleague of mine wrote:
[<Fact>]
let ``Green Flow tests`` () =
use factory = new WebAppFactory()
use client = factory.CreateClient()
Check.theGreenFlow client
|> Async.AwaitTask
|> Async.RunSynchronously
Surprised, I was wondering why my colleague forced a call to Async.RunSynchronously while XUnit works fine with Task and Async types alike.
I then tried out:
[<Fact>]
let ``Green Flow tests`` () =
use factory = new WebAppFactory()
use client = factory.CreateClient()
// this btw returns Task<unit>
Check.theGreenFlow client
And got:
Rm.Bai.IntegrationTests.RetrievalWorkflow.Green Flow tests
System.AggregateException : One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (One or more errors occurred. (Cannot access a disposed object.
Object name: 'IServiceProvider'.))))))
I was like "ok fair enough, scope of use ends at the bottom of the function, then released while the Task<unit> is processed by the XUnit runner".
Even though the IDisposable objects might be referenced in a function returning a Task in the example above, the runner runs the task after the end of the function and so the Dispose call has already occurred according to my understanding of https://learn.microsoft.com/en-us/dotnet/fsharp/language-reference/resource-management-the-use-keyword:
It provides the same functionality as a let binding but adds a call to Dispose on the value when the value goes out of scope. Note that the compiler inserts a null check on the value, so that if the value is null, the call to Dispose is not attempted.
[...]
when you use the use keyword, Dispose is called at the end of the containing code block
To me, the right way to avoid disposed object would have been to wrap in a computation expression like task or async computation expression:
[<Fact>]
let ``Green Flow tests`` () =
task {
use factory = new WebAppFactory()
use client = factory.CreateClient()
do! Check.theGreenFlow client
}
So that the moment when Dispose() is actually code is clearly defined.
And don't necessarily force the test to be run synchronously like in the snippet no. 1.
Unlike something like below, which still would lead to a Cannot access a disposed object. error:
[<Fact>]
let ``Green Flow tests`` () =
use factory = new WebAppFactory()
use client = factory.CreateClient()
async {
do! Check.theGreenFlow client |> Async.AwaitTask
}
Which is akin the snippet no. 2.
Is my understanding of this issue correct?
Next experiment demonstrates the flow of your second example (when there is Check.theGreenFlow client at the end). F#:
let tst(runSvc:Func<_,_>) =
printfn "[%d] start tst" Thread.CurrentThread.ManagedThreadId
use srv = {new IDisposable with
override x.Dispose() =
printfn "[%d] srv disposed" Thread.CurrentThread.ManagedThreadId }
let res: Task = runSvc.Invoke srv
printfn "[%d] end tst" Thread.CurrentThread.ManagedThreadId
res
this is analog of Green Flow tests. C#:
static async Task runSvc(IDisposable svc) {
Console.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] before runSvc");
await Task.Delay(1000);
Console.WriteLine($"[{Thread.CurrentThread.ManagedThreadId}] after runSvc");
}
static async Task Main(string[] args)
{
var task = T1.tst(runSvc);
await task;
}
runSvc plays role of Check.theGreenFlow and Main - role of XUnit test run.
The output:
[1] start tst
[1] before runSvc
[1] end tst
[1] srv disposed
[4] after runSvc
First thread starts the test and goes into Check.theGreenFlow.
At some point Check.theGreenFlow registers a continuation for asynchronous operation - in this example it is at point task.Delay.
After continuation registration, Check.theGreenFlow immediately returns with object Task (in my example, continuation is printing "after runSvc" and this part is registered for later).
When code returns from test function, service objects are disposed at the end and thread 1 awaits for result, which at some point will be created by registered continuation in the threadpool (another thread 4 executes "after runSvc"). If thread 4 accesses service objects in continuation, object disposed exception happens.
There could be race conditions. If in my example above you put Thread.Sleep(2000) before "[%d] end tst", continuation "after runSvc" will execute before disposal and there is no exception.
I'm trying to scrape a series of websites that run a bunch of javascript on the DOM before it's done loading. This means I'm using a WebBrowser instead of the friendlier WebClient. The problem I'd like to solve is to wait until the WebBrowser.DocumentCompleted event fires and then return WebBrowser.Document. I then do some post processing on the HtmlDocument but cannot get it to return yet.
The Code I Have
let downloadWebSite (address : string) =
let browser = new WebBrowser()
let browserContext = SynchronizationContext()
browser.DocumentCompleted.Add (fun _ ->
printfn "Document Loaded")
async {
do browser.Navigate(address)
let! a = Async.AwaitEvent browser.DocumentCompleted
do! Async.SwitchToContext(browserContext)
return browser.Document)
}
[downloadWebSite "https://www.google.com"]
|> Async.Parallel // there will be more addresses when working
|> Async.RunSynchronously
The Error
System.InvalidCastException: Specified cast is not valid.
at System.Windows.Forms.UnsafeNativeMethods.IHTMLDocument2.GetLocation()
at System.Windows.Forms.WebBrowser.get_Document()
at FSI_0058.downloadWebSite#209-41.Invoke(Unit _arg2) in C:\Temp\Untitled-1.fsx:line 209
at Microsoft.FSharp.Control.AsyncPrimitives.CallThenInvokeNoHijackCheck[a,b](AsyncActivation`1 ctxt, FSharpFunc`2 userCode, b result1)
at Microsoft.FSharp.Control.Trampoline.Execute(FSharpFunc`2 firstAction)
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
at Microsoft.FSharp.Control.AsyncResult`1.Commit()
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronouslyInAnotherThread[a](CancellationToken token, FSharpAsync`1 computation, FSharpOption`1 timeout)
at Microsoft.FSharp.Control.AsyncPrimitives.RunSynchronously[T](CancellationToken cancellationToken, FSharpAsync`1 computation, FSharpOption`1 timeout)
at Microsoft.FSharp.Control.FSharpAsync.RunSynchronously[T](FSharpAsync`1 computation, FSharpOption`1 timeout, FSharpOption`1 cancellationToken)
at <StartupCode$FSI_0058>.$FSI_0058.main#()
Stopped due to error
What I think is happening
There are several issues that make me believe that I'm accessing the WebBrowser from the wrong thread.1 2 3
Help requested
Is the use of Async.SwitchToContext(browserContext) correct here?
Could the overall approach be simplified?
Is there a concept I appear ignorant of?
How do I get the WebBrowser.Document?
The problem is in this line:
let browserContext = SynchronizationContext()
You manually created a new instance of SynchronizationContext but didn't associate it with the UI thread or any thread. That's why the program crashes when you access the browser.Document which must be accessed on the UI thread.
To solve this problem, simply use the existing SynchronizationContext which was already associated with the UI thread:
let browserContext = SynchronizationContext.Current
I assumed that the downloadWebSite function is called on the UI thread. If it is not, you can pass the context from somewhere into the function, or use a global variable.
A better design
Althought with Async.SwitchToContext you can make sure that the next line accesses and returns the document in the UI thread, but the client code which receives the document may run on a non-UI thread. A better design is to use a continuation function. Instead of directly returning a document, you can return a SomeType value produced by a continuation function passed into downloadWebSite as a parameter. By this way, the continuation function is ensured to be run on UI thread:
let downloadWebSite (address : string) cont =
let browser = new WebBrowser()
let browserContext = SynchronizationContext.Current
browser.DocumentCompleted.Add (fun _ ->
printfn "Document Loaded")
async {
do browser.Navigate(address)
let! a = Async.AwaitEvent browser.DocumentCompleted
do! Async.SwitchToContext(browserContext)
// the cont function is ensured to be run on UI thread:
return cont browser.Document }
[downloadWebSite "https://www.google.com" (fun document -> (*safely access document*))]
|> Async.Parallel
|> Async.RunSynchronously
I am working on a Windows Store (C++) app. This is a method that reads from the database using the web service.
task<std::wstring> Ternet::GetFromDB(cancellation_token cancellationToken)
{
uriString = ref new String(L"http://myHost:1234/RestServiceImpl.svc/attempt");
auto uri = ref new Windows::Foundation::Uri(Helpers::Trim(uriString));
cancellationTokenSource = cancellation_token_source();
return httpRequest.GetAsync(uri, cancellationTokenSource.get_token()).then([this](task<std::wstring> response)->std::wstring
{
try
{
Windows::UI::Popups::MessageDialog wMsg(ref new String(response.get().c_str()), "success");
wMsg.ShowAsync();
return response.get();
}
catch (const task_canceled&)
{
Windows::UI::Popups::MessageDialog wMsg("Couldn't load content. Check internet connectivity.", "Error");
wMsg.ShowAsync();
std::wstring abc;
return abc;
}
catch (Exception^ ex)
{
Windows::UI::Popups::MessageDialog wMsg("Couldn't load content. Check internet connectivity.", "Error");
wMsg.ShowAsync();
std::wstring abc;
return abc;
}
} , task_continuation_context::use_current());
}
I'm confused how to return the received data to the calling function. Now, I am calling this function in the constructor of my data class like this:
ternet.GetFromDB(cancellationTokenSource.get_token()).then([this](task<std::wstring> response)
{
data = ref new String(response.get().c_str());
});
I am getting a COM exception whenever I try to receive the returned data from GetFromDB(). But this one runs fine:
ternet.GetFromDB(cancellationTokenSource.get_token());
Please suggest a better way of chaining the completion of GetFromDB to other code. And how to get the returned value from inside the try{} block of GetFromDB() 's then. Please keep in mind I am a very new student of async programming.
If the continuation of the call to GetFromDB is happening on the UI thread (which I believe it will by default, assuming the call site you pasted is occurring in the UI thread), then calling get() on the returned task will throw an exception. It won't let you block the UI thread waiting for a task to finish.
Two suggestions, either of which should fix that problem. The first should work regardless, while the second is only a good option if you're not trying to get the response string to the UI thread (to be displayed, for example).
1) Write your continuations (lambdas that you pass to then) so that they take the actual result of the previous task, rather than the previous task itself. In other words, instead of writing this:
ternet.GetFromDB(...).then([this](task<std::wstring> response) { ... });
write this:
ternet.GetFromDB(...).then([this](std::wstring response) { ... });
The difference with the latter is that the continuation machinery will call get() for you (on a background thread) and then give the result to your continuation function, which is a lot easier all around. You only need to have your continuation take the actual task as an argument if you want to catch exceptions that might have been thrown by the task as it executed.
2) Tell it to run your continuation on a background/arbitrary thread:
ternet.GetFromDB(...).then([this](task<std::wstring> response) { ... }, task_continuation_context::use_arbitrary());
It won't care if you block a background thread, it only cares if you call get() on the UI thread.
With ASP.NET 4.5 I'm trying to play with the new async/await toys. I have a IDataReader-implementing class that wraps a vendor-specific reader (like SqlDatareader). I have a simple ExecuteSql() method that operates synchronously like so:
public IDataReader ReaderForSql(string sql)
{
var cmd = NewCommand(sql, CommandType.Text);
return DBReader.ReaderFactory(cmd.ExecuteReader());
}
What I want is an Async version of this. Here's my first try:
public Task<IDataReader> ReaderForSqlAsync(string sql, CancellationToken ct)
{
var cmd = NewCommand(sql, CommandType.Text);
return cmd.ExecuteReaderAsync(ct)
.ContinueWith(t => DBReader.ReaderFactory(t.Result));
}
and I use it:
using (var r = await connection.ReaderForSqlAsync("SELECT ...", cancellationToken))
{
...
}
This works great in my limited testing so far. But after watching this Cloud9 video a few times: http://channel9.msdn.com/Events/aspConf/aspConf/Async-in-ASP-NET I got worreid about warnings they gave regarding:
ContinueWith consuming extra threadpool resources - Readerfactory is very light!
Task.Result blocking
and since I am passing a ContinuationToken to ExecuteReaderAsync() it seems cancellation is just yet another reason ExecuteReaderAsync() could fail (it's SQL after all!)
What will be the state of the task when I try to ContinueWith it? Will t.Result block? throw? do the wrong thing?
ContinueWith will use the current task scheduler (a thread pool thread) by default, but you can change that by passing TaskContinuationOptions.ExecuteSynchronously and an explicit TaskScheduler.
That said, I would make this as a first effort:
public async Task<IDataReader> ReaderForSqlAsync(string sql, CancellationToken ct)
{
var cmd = NewCommand(sql, CommandType.Text);
var readerResult = await cmd.ExecuteReaderAsync(ct).ConfigureAwait(false);
return DBReader.ReaderFactory(readerResult);
}
async and await handle all the ContinueWith delicacies and edge conditions in a consistent manner for you. It may be possible to complicate this code to be faster if performance testing indicates it's a serious problem.
Result blocks if the task has not completed. But in the continuation handler it already has. So it does not block. You are doing the right thing.
When you invoke Result on a faulted task (and you say this might happen) the exception is rethrown. This causes your continuation to fault with the same exception which causes the final task returned from ReaderForSqlAsync to also be faulted. This is a good thing: The entire chain of tasks faulted and all exceptions have been observed (on contrast to being swallowed). So this is best-practice, too.
Using a thread for compute-bound work is always ok. So again, you are doing the right thing using ContinueWith. You have to compute the IDataReader somewhere after all. You cannot not compute it.