override method on_finish().
Decorate this method with .gen.coroutine to make it asynchronous, it worked.
#gen.coroutine
def on_finish(self):
print("==========on_finish==============")
but use async def to make it asynchronous, it failed.
async def on_finish(self):
print("==========on_finish==============")
# RuntimeWarning: coroutine 'BaseHandler.on_finish' was never awaited
self.on_finish()
Why can't I use async def to make it asynchronous？
Coroutines have a different interface than regular functions. When overriding a method, you should make it a coroutine (or not) according to its documentation. All overridable methods in Tornado that may be coroutines say so in their documentation, and on_finish is not one of them.
What happens if you use a coroutine when the superclass doesn't expect it? As you see, this depends on the kind of coroutine. #gen.coroutine is "self-starting", so the coroutine will start, but nothing will wait for it to finish, and if it raises an exception, there is nowhere for that exception to be caught (so it will just be logged by the IOLoop). async def coroutines do not start automatically, something must await them, so if you use one where the caller is not expecting a coroutine, nothing will happen.
If you need to use a coroutine from a method where this is not allowed, you need to decide who's going to wait on the coroutine and handle its exceptions. If you don't need to catch exceptions and can just leave them for the logs, you can use IOLoop.add_callback to explicitly give this responsibility to the IOLoop:
def on_finish(self):
IOLoop.current().add_callback(self.on_finish_async)
async def on_finish_async(self):
...
Related
I'm trying to call an asynchronous method from a synchronous method like below:
1. public List<Words> Get(string code)
{
Task<List<Words>> taskWords = GetWordsAsync(code);
var result = taskWords.Result;
return result;
}
private async Task<List<Words>> GetWordsAsync(string code)
{
var result = await codeService.GetWordsByCodeAsync(code);
return result;
}
But this lead to a deadlock, await is not getting the results from the method - GetWordsByCodeAsync
I've done a bit of research and came to know that if we are calling an async method from synchronous method we should use Task.Run
When I changed the code like below, it worked:
2. public List<Words> Get(string code)
{
Task<List<Words>> taskWords = Task.Run<List<Words>>(async () => await GetWordsAsync(code);
var result = taskWords.Result;
return result;
}
private async Task<List<Words>> GetWordsAsync(string code)
{
var result = await codeService.GetWordsByCodeAsync(code);
return result;
}
But I didn't get the context, why it caused a deadlock for 1st way and 2nd one worked fine.
I'd like to know:
What's the difference between two ways?
Is second one the correct way to call async method from synchronous method?
Will using the second method also causes a deadlock at some point of time if the result is large? or is it fail proof(safe) method to use?
Also, please suggest any best practices to better do it as I have to do 5 async calls from a synchronous method - just like taskWords, I have taskSentences etc.,
Note: I don't want to change everything to async. I want to call async method from synchronous method.
I don't want to change everything to async. I want to call async method from synchronous method.
From a technical standpoint, this doesn't make sense. Asynchronous means it doesn't block the calling thread; synchronous means it does. So you understand that if you block on asynchronous code, then it's no longer truly asynchronous? The only benefit to asynchronous code is that it doesn't block the calling thread, so if you block on the asynchronous code, you're removing all the benefit of asynchrony in the first place.
The only good answer is to go async all the way. There are some rare scenarios where that's not possible, though.
What's the difference between two ways?
The first one executes the asynchronous code directly and then blocks the calling thread, waiting for it to finish. You're running into a deadlock because the asynchronous code is attempting to resume on the calling context (which await does by default), and presumably you're running this code on a UI thread or in an ASP.NET Classic request context, which only allow one thread in the context at a time. There's already a thread in the context (the one blocking, waiting for the task to finish), so the async method cannot resume and actually finish.
The second one executes the asynchronous code on a thread pool thread and then blocks the calling thread, waiting for it to finish. There is no deadlock here because the asynchronous code resumes on its calling context, which is a thread pool context, so it just continues executing on any available thread pool thread.
Is second one the correct way to call async method from synchronous method?
There is no "correct" way to do this. There are a variety of hacks, each of which has different drawbacks, and none of which work in every situation.
Will using the second method also causes a deadlock at some point of time if the result is large? or is it fail proof(safe) method to use?
It will not deadlock. What it does, though, is execute GetWordsAsync on a thread pool thread. As long as GetWordsAsync can be run on a thread pool thread, then it will work.
Also, please suggest any best practices to better do it
I have written an article on the subject. In your case, if GetWordsAsync is safe to call on a thread pool thread, then the "blocking thread pool hack" is acceptable.
Note that this is not the best solution; the best solution is to go async all the way. But the blocking thread pool hack is an acceptable hack if you must call asynchronous code from synchronous code (again, it's "acceptable" only if GetWordsAsync is safe to call on a thread pool thread).
I would recommend using GetAwaiter().GetResult() rather than Result, in order to avoid the AggregateException wrapper on error. Also, the explicit type arguments are unnecessary, and you can elide async/await in this case:
var taskWords = Task.Run(() => GetWordsAsync(code));
return taskWords.GetAwaiter().GetResult();
I want to call an async method from other libraries in tornado, like so:
class Database:
async def find_info(user_id):
pass
class TestClass(tornado.web.RequestHandler):
def get(self, id):
db = Database()
user = yield db.find_info(user_id=id)
return self.write(user.username)
But it goes to something like sleeping mode and I'll never get any result.
Which other libraries? Most async functions are written for a particular event loop (Tornado, asyncio, Twisted, etc). Different event loops don't cooperate unless you ask them to. You probably want to enable Tornado/asyncio interoperability with tornado.platform.asyncio.AsyncIOMainLoop
I have problem with method open of WebSocketHandler.
I wrapped it with gen.coroutine for using async call inside (get access to redis). But got another problem, any errors inside open does not catches.
Example:
#gen.coroutine
def open(self):
t = 8/0
self._connection_id = yield self._generate_connection_id()
self.write_message('...')
Method open is call inside WebSocketProtocol._run_callback :
def _run_callback(self, callback, *args, **kwargs):
try:
callback(*args, **kwargs)
except Exception:
app_log.error("Uncaught exception in %s",
self.request.path, exc_info=True)
self._abort()
This method hasn't any decorator, so method open return future and this exception doesn't handled.
So how can I use async method inside open, and handle exceptions?
As a general rule, coroutines can only correctly be called by other coroutines, so when overriding a method of a base class like WebSocketHandler.open, unless that method is a coroutine or is documented as "may be a coroutine" or "may return a Future", it is not safe to make it a coroutine. WebSocketHandler.open may not be a coroutine (as of Tornado 4.1).
You may spawn a coroutine from open() with IOLoop.spawn_callback:
def open(self):
IOLoop.current().spawn_callback(self.async_open)
#gen.coroutine
def async_open(self):
#...
Within async_open, you can handle errors as usual with try/except (spawn_callback also logs exceptions for you as a last resort). It is also your responsibility to deal with any timing issues that may arise between async_open and on_message or on_close, since those methods may be called before async_open completes.
I'm learning SignalR using the .Net client (not javascript), and was hoping for some clarification on how to invoke hub proxy methods in a synchronous or asynchronous manner.
Method with no return value
So far I've been doing something like this:-
myHubProxy.Invoke("DoSomething");
I've found this to be asynchronous, which is fine as it's effectively "fire-and-forget" and doesn't need to wait for a return value. A couple of questions though:-
Are there any implications with wrapping the Invoke in a try..catch block, particularly with it being asynchronous? I might want to know if the call failed.
Are there any scenarios where you would want to call a method that doesn't return a value synchronously? I've seen the .Wait() method mentioned, but I can't think why you would want to do this.
Method with return value
So far I've been using the Result property, e.g.:-
var returnValue = myHubProxy.Invoke<string>("DoSomething").Result;
Console.WriteLine(returnValue);
I'm assuming this works synchronously - after all, it couldn't proceed to the next line until a result had been returned. But how do I invoke such a method asynchronously? Is it possible to specify a callback method, or should I really be using async/await these days (something I confess to still not learning about)?
If you want to write asynchronous code, then you should use async/await. I have an intro on my blog with a number of followup resources at the end.
When you start an asynchronous operation (e.g., Invoke), then you get a task back. The Task type is used for asynchronous operations without a return value, and Task<T> is used for asynchronous operations with a return value. These task types can indicate to your code when the operation completes and whether it completed successfully or with error.
Although you can use Task.Wait and Task<T>.Result, I don't recommend them. For one, they wrap any exceptions in an AggregateException, which make your error handling code more cumbersome. It's far easier to use await, which does not do this wrapping. Similarly, you can register a callback using ContinueWith, but I don't recommend it; you need to understand a lot about task schedulers and whatnot to use it correctly. It's far easier to use await, which does the (most likely) correct thing by default.
The .Result property returns a async Task, so the server requests is still performed async.
There is not reason to hold up a thread for the duration of the call thats why you use async.
If you fire the call on the GUI thread its even more important todo it async because otherwise the GUI will not respond while the call is done
1) Yuo need to use the await keyword if you want try catch blocks to actually catch server faults. Like
try
{
var foo = await proxy.Invoke<string>("Bar");
}
catch (Exception ex)
{
//act on error
}
2) I think you ment to ask if its any reason to call it async? And yes like I said, you do not want to block any threads while the request is being made
I am attempting to implement get_current_user in the RequestHandler for Tornado, but I need the call to block while waiting on the asynchronous call to my database. Decorating the call with #tornado.web.asynchronous will not work because either way the get_current_user method returns before the async query completes and the query callback is executed.
For example:
class MyHandler(BaseHandler):
#tornado.web.asynchronous
#tornado.web.authenticated
def get(self):
self.write('example')
self.finish()
class BaseHandler(tornado.web.RequestHandler):
def get_current_user(self):
def query_cb(self, doc):
return doc or None
database.get(username='test', password='t3st', callback=query_cb)
#tornado.web.authenticated calls get_current_user, but always receives "None" because the BaseHandler does not have time to respond. Is there a way, using tornado, to temporarily block for a call such as the one above?
Do a blocking database operation instead of the non blocking described above (There is a blocking mysql lib shipped with tornado).
From the Tornado wiki page about threads and concurrency:
"Do it synchronously and block the IOLoop. This is most appropriate for things like memcache and database queries that are under your control and should always be fast. If it's not fast, make it fast by adding the appropriate indexes to the database, etc."
https://github.com/facebook/tornado/wiki/Threading-and-concurrency
How about having get_current_user return a Future that you signal when the asynchronous response from your database is returned?
class BaseHandler(tornado.web.RequestHandler):
def get_current_user(self):
future = Future()
def query_cb(user):
future.set_result(user or None)
database.get(username='test', password='t3st', callback=query_cb)
return future
class MainHandler(BaseHandler):
#gen.coroutine
def get(self):
user = yield self.get_current_user()
self.write('user: ' + user)
# ... actual request processing
I thought Tornado allowed you to make either blocking or non-blocking requests.
Here is Tornado being used for both: https://bitbucket.org/nephics/tornado-couchdb/src/147579581b47/couch.py
Disclaimer: I know very little of Python and Tornado.