I am attempting to implement get_current_user in the RequestHandler for Tornado, but I need the call to block while waiting on the asynchronous call to my database. Decorating the call with #tornado.web.asynchronous will not work because either way the get_current_user method returns before the async query completes and the query callback is executed.
For example:
class MyHandler(BaseHandler):
#tornado.web.asynchronous
#tornado.web.authenticated
def get(self):
self.write('example')
self.finish()
class BaseHandler(tornado.web.RequestHandler):
def get_current_user(self):
def query_cb(self, doc):
return doc or None
database.get(username='test', password='t3st', callback=query_cb)
#tornado.web.authenticated calls get_current_user, but always receives "None" because the BaseHandler does not have time to respond. Is there a way, using tornado, to temporarily block for a call such as the one above?
Do a blocking database operation instead of the non blocking described above (There is a blocking mysql lib shipped with tornado).
From the Tornado wiki page about threads and concurrency:
"Do it synchronously and block the IOLoop. This is most appropriate for things like memcache and database queries that are under your control and should always be fast. If it's not fast, make it fast by adding the appropriate indexes to the database, etc."
https://github.com/facebook/tornado/wiki/Threading-and-concurrency
How about having get_current_user return a Future that you signal when the asynchronous response from your database is returned?
class BaseHandler(tornado.web.RequestHandler):
def get_current_user(self):
future = Future()
def query_cb(user):
future.set_result(user or None)
database.get(username='test', password='t3st', callback=query_cb)
return future
class MainHandler(BaseHandler):
#gen.coroutine
def get(self):
user = yield self.get_current_user()
self.write('user: ' + user)
# ... actual request processing
I thought Tornado allowed you to make either blocking or non-blocking requests.
Here is Tornado being used for both: https://bitbucket.org/nephics/tornado-couchdb/src/147579581b47/couch.py
Disclaimer: I know very little of Python and Tornado.
Related
override method on_finish().
Decorate this method with .gen.coroutine to make it asynchronous, it worked.
#gen.coroutine
def on_finish(self):
print("==========on_finish==============")
but use async def to make it asynchronous, it failed.
async def on_finish(self):
print("==========on_finish==============")
# RuntimeWarning: coroutine 'BaseHandler.on_finish' was never awaited
self.on_finish()
Why can't I use async def to make it asynchronous？
Coroutines have a different interface than regular functions. When overriding a method, you should make it a coroutine (or not) according to its documentation. All overridable methods in Tornado that may be coroutines say so in their documentation, and on_finish is not one of them.
What happens if you use a coroutine when the superclass doesn't expect it? As you see, this depends on the kind of coroutine. #gen.coroutine is "self-starting", so the coroutine will start, but nothing will wait for it to finish, and if it raises an exception, there is nowhere for that exception to be caught (so it will just be logged by the IOLoop). async def coroutines do not start automatically, something must await them, so if you use one where the caller is not expecting a coroutine, nothing will happen.
If you need to use a coroutine from a method where this is not allowed, you need to decide who's going to wait on the coroutine and handle its exceptions. If you don't need to catch exceptions and can just leave them for the logs, you can use IOLoop.add_callback to explicitly give this responsibility to the IOLoop:
def on_finish(self):
IOLoop.current().add_callback(self.on_finish_async)
async def on_finish_async(self):
...
I want to call an async method from other libraries in tornado, like so:
class Database:
async def find_info(user_id):
pass
class TestClass(tornado.web.RequestHandler):
def get(self, id):
db = Database()
user = yield db.find_info(user_id=id)
return self.write(user.username)
But it goes to something like sleeping mode and I'll never get any result.
Which other libraries? Most async functions are written for a particular event loop (Tornado, asyncio, Twisted, etc). Different event loops don't cooperate unless you ask them to. You probably want to enable Tornado/asyncio interoperability with tornado.platform.asyncio.AsyncIOMainLoop
I have problem with method open of WebSocketHandler.
I wrapped it with gen.coroutine for using async call inside (get access to redis). But got another problem, any errors inside open does not catches.
Example:
#gen.coroutine
def open(self):
t = 8/0
self._connection_id = yield self._generate_connection_id()
self.write_message('...')
Method open is call inside WebSocketProtocol._run_callback :
def _run_callback(self, callback, *args, **kwargs):
try:
callback(*args, **kwargs)
except Exception:
app_log.error("Uncaught exception in %s",
self.request.path, exc_info=True)
self._abort()
This method hasn't any decorator, so method open return future and this exception doesn't handled.
So how can I use async method inside open, and handle exceptions?
As a general rule, coroutines can only correctly be called by other coroutines, so when overriding a method of a base class like WebSocketHandler.open, unless that method is a coroutine or is documented as "may be a coroutine" or "may return a Future", it is not safe to make it a coroutine. WebSocketHandler.open may not be a coroutine (as of Tornado 4.1).
You may spawn a coroutine from open() with IOLoop.spawn_callback:
def open(self):
IOLoop.current().spawn_callback(self.async_open)
#gen.coroutine
def async_open(self):
#...
Within async_open, you can handle errors as usual with try/except (spawn_callback also logs exceptions for you as a last resort). It is also your responsibility to deal with any timing issues that may arise between async_open and on_message or on_close, since those methods may be called before async_open completes.
I'm trying to get my head around Tornado. I'm writing a chat application backed by mongodb and I'm using motor for non-blocking access to it.
What I'm trying to achieve is:
Create a decorator that uses motor to asynchronously pull the user's record from mongo
Validate their credentials (username & token)
Create another decorator that checks that the user_id retrieved in 1. above is permitted access to the chat room. This requires another asynchronous call to mongo with motor to retrieve the 'ChatRoom' record.
Subscribe to the chat room if all is OK
I have decorator 1. working (basically taken from http://tornadogists.org/5251927/):
def authenticated_async(f):
#functools.wraps(f)
#gen.engine
def wrapper(self, *args, **kwargs):
self.current_user = yield gen.Task(self.get_current_user_async)
if self.current_user:
logging.info("User '%s' (%s) successfully authenticated" %
(self.current_user['username'],
self.current_user['_id']))
f(self, *args, **kwargs)
else:
raise tornado.web.HTTPError(401, "User not authenticated, "
"aborting")
return wrapper
The trouble is that for the second decorator, I need to access self.current_user. Because this is set in an asynchronous callback, it's not available when I get into my validation decorator (i.e the validation decorator is called before the auth decorator completes).
Is it just not possible for me to use decorators in this way with asynchronous functions? Do I just need to call the validation method inside the above method after making sure that self.current_user is True so it's more like a callback?
I'd ideally like to have my methods in my Handler wrapped with both of these decorators so I can reuse them elsewhere, i.e.:
class ChatSocketHandler(tornado.websocket.WebSocketHandler):
#gen.coroutine
#validate_invitation_access_async
#authenticated_async
def open(self, invitation_id):
# do stuff here...
Update
In fact, there is no dependency. user_id is provided as a parameter, and that could be used to run both decorators in parallel - one to confirm authentication, the other to see whether the user with that ID is allowed access to the room. The open() method would then only proceed if self.auth_check == True and self.room_check == True.
Could open() ever be called before the async decorators complete though?
You need to switch the order of the decorators so your validate_invitation_access_async wrapper has access to current_user:
#authenticated_async
#validate_invitation_access_async
def open(self, invitation_id):
# do stuff here...
Then the wrapper inside validate_invitation_access_async is the "f" in authenticated_async: it's called after self.current_user is set. Note that you don't need an additional gen.engine decorator, all the wrappers are already engines. Your wrapper could be like:
def validate_invitation_access_async(f):
#gen.engine
def wrapper(self, *args, **kwargs):
# ... use Motor to see if user is authorized ...
# if not, log and redirect. otherwise:
f(self, *args, **kwargs)
You should also update your code to use Tornado 3's gen.coroutine instead of gen.engine: it's much cleaner. But I leave that as an exercise.
I'm learning SignalR using the .Net client (not javascript), and was hoping for some clarification on how to invoke hub proxy methods in a synchronous or asynchronous manner.
Method with no return value
So far I've been doing something like this:-
myHubProxy.Invoke("DoSomething");
I've found this to be asynchronous, which is fine as it's effectively "fire-and-forget" and doesn't need to wait for a return value. A couple of questions though:-
Are there any implications with wrapping the Invoke in a try..catch block, particularly with it being asynchronous? I might want to know if the call failed.
Are there any scenarios where you would want to call a method that doesn't return a value synchronously? I've seen the .Wait() method mentioned, but I can't think why you would want to do this.
Method with return value
So far I've been using the Result property, e.g.:-
var returnValue = myHubProxy.Invoke<string>("DoSomething").Result;
Console.WriteLine(returnValue);
I'm assuming this works synchronously - after all, it couldn't proceed to the next line until a result had been returned. But how do I invoke such a method asynchronously? Is it possible to specify a callback method, or should I really be using async/await these days (something I confess to still not learning about)?
If you want to write asynchronous code, then you should use async/await. I have an intro on my blog with a number of followup resources at the end.
When you start an asynchronous operation (e.g., Invoke), then you get a task back. The Task type is used for asynchronous operations without a return value, and Task<T> is used for asynchronous operations with a return value. These task types can indicate to your code when the operation completes and whether it completed successfully or with error.
Although you can use Task.Wait and Task<T>.Result, I don't recommend them. For one, they wrap any exceptions in an AggregateException, which make your error handling code more cumbersome. It's far easier to use await, which does not do this wrapping. Similarly, you can register a callback using ContinueWith, but I don't recommend it; you need to understand a lot about task schedulers and whatnot to use it correctly. It's far easier to use await, which does the (most likely) correct thing by default.
The .Result property returns a async Task, so the server requests is still performed async.
There is not reason to hold up a thread for the duration of the call thats why you use async.
If you fire the call on the GUI thread its even more important todo it async because otherwise the GUI will not respond while the call is done
1) Yuo need to use the await keyword if you want try catch blocks to actually catch server faults. Like
try
{
var foo = await proxy.Invoke<string>("Bar");
}
catch (Exception ex)
{
//act on error
}
2) I think you ment to ask if its any reason to call it async? And yes like I said, you do not want to block any threads while the request is being made