I'm trying to get my head around Tornado. I'm writing a chat application backed by mongodb and I'm using motor for non-blocking access to it.
What I'm trying to achieve is:
Create a decorator that uses motor to asynchronously pull the user's record from mongo
Validate their credentials (username & token)
Create another decorator that checks that the user_id retrieved in 1. above is permitted access to the chat room. This requires another asynchronous call to mongo with motor to retrieve the 'ChatRoom' record.
Subscribe to the chat room if all is OK
I have decorator 1. working (basically taken from http://tornadogists.org/5251927/):
def authenticated_async(f):
#functools.wraps(f)
#gen.engine
def wrapper(self, *args, **kwargs):
self.current_user = yield gen.Task(self.get_current_user_async)
if self.current_user:
logging.info("User '%s' (%s) successfully authenticated" %
(self.current_user['username'],
self.current_user['_id']))
f(self, *args, **kwargs)
else:
raise tornado.web.HTTPError(401, "User not authenticated, "
"aborting")
return wrapper
The trouble is that for the second decorator, I need to access self.current_user. Because this is set in an asynchronous callback, it's not available when I get into my validation decorator (i.e the validation decorator is called before the auth decorator completes).
Is it just not possible for me to use decorators in this way with asynchronous functions? Do I just need to call the validation method inside the above method after making sure that self.current_user is True so it's more like a callback?
I'd ideally like to have my methods in my Handler wrapped with both of these decorators so I can reuse them elsewhere, i.e.:
class ChatSocketHandler(tornado.websocket.WebSocketHandler):
#gen.coroutine
#validate_invitation_access_async
#authenticated_async
def open(self, invitation_id):
# do stuff here...
Update
In fact, there is no dependency. user_id is provided as a parameter, and that could be used to run both decorators in parallel - one to confirm authentication, the other to see whether the user with that ID is allowed access to the room. The open() method would then only proceed if self.auth_check == True and self.room_check == True.
Could open() ever be called before the async decorators complete though?
You need to switch the order of the decorators so your validate_invitation_access_async wrapper has access to current_user:
#authenticated_async
#validate_invitation_access_async
def open(self, invitation_id):
# do stuff here...
Then the wrapper inside validate_invitation_access_async is the "f" in authenticated_async: it's called after self.current_user is set. Note that you don't need an additional gen.engine decorator, all the wrappers are already engines. Your wrapper could be like:
def validate_invitation_access_async(f):
#gen.engine
def wrapper(self, *args, **kwargs):
# ... use Motor to see if user is authorized ...
# if not, log and redirect. otherwise:
f(self, *args, **kwargs)
You should also update your code to use Tornado 3's gen.coroutine instead of gen.engine: it's much cleaner. But I leave that as an exercise.
Related
override method on_finish().
Decorate this method with .gen.coroutine to make it asynchronous, it worked.
#gen.coroutine
def on_finish(self):
print("==========on_finish==============")
but use async def to make it asynchronous, it failed.
async def on_finish(self):
print("==========on_finish==============")
# RuntimeWarning: coroutine 'BaseHandler.on_finish' was never awaited
self.on_finish()
Why can't I use async def to make it asynchronous？
Coroutines have a different interface than regular functions. When overriding a method, you should make it a coroutine (or not) according to its documentation. All overridable methods in Tornado that may be coroutines say so in their documentation, and on_finish is not one of them.
What happens if you use a coroutine when the superclass doesn't expect it? As you see, this depends on the kind of coroutine. #gen.coroutine is "self-starting", so the coroutine will start, but nothing will wait for it to finish, and if it raises an exception, there is nowhere for that exception to be caught (so it will just be logged by the IOLoop). async def coroutines do not start automatically, something must await them, so if you use one where the caller is not expecting a coroutine, nothing will happen.
If you need to use a coroutine from a method where this is not allowed, you need to decide who's going to wait on the coroutine and handle its exceptions. If you don't need to catch exceptions and can just leave them for the logs, you can use IOLoop.add_callback to explicitly give this responsibility to the IOLoop:
def on_finish(self):
IOLoop.current().add_callback(self.on_finish_async)
async def on_finish_async(self):
...
An API I'm writing accepts two arguments via URL - e.g /api/targets/foo/bar
app = webapp2.WSGIApplication([
('/api/targets/(\w*?)/(\w*?)$', MainPage),
], debug=True)
This fires off a GET handler:
def get(self, url_1, url_2):
#do some stuff
The cool thing here is that I can reference url_1 (foo) and url_2 (bar) inside my GET handler and they're defined for me.
I have another page which accepts both GET and POST requests. About 90% of what happens is the same in either case, so I've opted to use def dispatch() instead of having two separate handlers.
The problem here is that (even though I still have the regex blocks enclosed in my webapp2.WSGIApplication initialisation) they are no longer passed to the request handler, so I have to define them like this:
url_1= self.request.url.split("/")[3]
url_2= self.request.url.split("/")[4]
Which makes me feel like a peon. If I tell dispatch() to expect those two params, they never arrive - what do I need to do to mimic the get behaviour in dispatch?
It would be shorter to use:
arg_one = self.request.route_args[0]
arg_two = self.request.route_args[1]
Take a look at the webapp2 docs for the Request object, specifically the route_args and route_kwargs towards the bottom of the Request section.
This is an interesting use case. If I were you, I would keep get() and post() separate. If get() and post() share code, then I would move that code to a method of the RequestHandler that can be called from both get() and post(), or move the shared code into another class (I find a lot of my RequestHandlers just call methods on my model classes).
If you still want one method to replace both get() and post(), then instead of using dispatch(), I recommend you set the handler_method for the Route (see here in the docs). I would advise against handling these requests entirely in the dispatch() method, as that is supposed to, at some point, call the "handler method", which defaults to GET/POST/etc. based on the HTTP method, but you can set your own handler method to handle GET and POST and whatever else.
If you set the handler_method, it would look like this:
# note that handler_method is the method name as a string
app = webapp2.WSGIApplication([
('/api/targets/(\w*?)/(\w*?)$', MainPage, handler_method='handle_request'),
], debug=True)
In your handler:
class MainPage(webapp2.RequestHandler):
# note that your method signature MUST have args for the route args,
# or else an exception will occur
def handle_request(self, arg_one, arg_two):
# your code here
if self.request.method == 'GET':
# do GET stuff
elif self.request.method == 'POST':
# do POST stuff
This is nice because it leaves dispatch() unchanged. I like to think of dispatch() as pre- and post-processing to occur before/after calling get()/post()/whatever handler method you specify.
I want to call an async method from other libraries in tornado, like so:
class Database:
async def find_info(user_id):
pass
class TestClass(tornado.web.RequestHandler):
def get(self, id):
db = Database()
user = yield db.find_info(user_id=id)
return self.write(user.username)
But it goes to something like sleeping mode and I'll never get any result.
Which other libraries? Most async functions are written for a particular event loop (Tornado, asyncio, Twisted, etc). Different event loops don't cooperate unless you ask them to. You probably want to enable Tornado/asyncio interoperability with tornado.platform.asyncio.AsyncIOMainLoop
I'm trying to use Twisted's HTTP basic authentication to control access to some protected resources.
According to some articles, it is necessary to use three important concepts: Realm, Portal and avatar. Now I'm wondering if the Realm and avatar is one to one correspondence.
Let's look at an example
import sys
from zope.interface import implements
from twisted.python import log
from twisted.internet import reactor
from twisted.web import server, resource, guard
from twisted.cred.portal import IRealm, Portal
from twisted.cred.checkers import InMemoryUsernamePasswordDatabaseDontUse
class GuardedResource(resource.Resource):
"""
A resource which is protected by guard
and requires authentication in order
to access.
"""
def getChild(self, path, request):
return self
def render(self, request):
return "Authorized!"
class SimpleRealm(object):
"""
A realm which gives out L{GuardedResource} instances for authenticated
users.
"""
implements(IRealm)
def requestAvatar(self, avatarId, mind, *interfaces):
if resource.IResource in interfaces:
return resource.IResource, GuardedResource(), lambda: None
raise NotImplementedError()
def main():
log.startLogging(sys.stdout)
checkers = [InMemoryUsernamePasswordDatabaseDontUse(joe='blow')]
wrapper = guard.HTTPAuthSessionWrapper(
Portal(SimpleRealm(), checkers),
[guard.DigestCredentialFactory('md5', 'example.com')])
reactor.listenTCP(8889, server.Site(
resource = wrapper))
reactor.run()
if __name__ == '__main__':
main()
Of course I know the SimpleRealm is used to return the corresponding resource, e.g. GuardedResource in above example. However, I don't know what to do when there lots of resources to be guarded. For example, I have GuardedResource1, GuardedResource2 and GuardedResource3, maybe they need the same or different number of parameters when they are initialized; If so, is it necessary to implement SimpleRealm1, SimpleRealm2 and SimpleRealm3, respectively?
Someone asked this same question on the Twisted mailing list, with very similar code samples - http://twistedmatrix.com/pipermail/twisted-python/2015-December/030042.html - so I'll refer you to my answer there: http://twistedmatrix.com/pipermail/twisted-python/2015-December/030068.html
Rather than thinking of a resource as always existing and just needing to have a lock on it or not, consider the more flexible model (the one that cred actually implements) where a single Avatar object (in this case: the top IResource returned from SimpleRealm) is the top level of "everything the user has access to". In other words, 'GuardedResource' should have a 'getChild' method which makes the determination if the user they represent (really, at least the avatarId should be supplied to GuardedResource.init) has access to other resources, and return them if so, and appropriate errors if not.
Even the resources available to a not-logged-in user (see twisted.cred.credentials.Anonymous) is just another avatar, the one served up to unauthenticated people.
So, if you have https://myapp.example.com/a/b/secure/c/d https://myapp.example.com/a/b/secure/c/d, https://myapp.example.com/a/b/secure https://myapp.example.com/a/b/secure would be the guarded resource, and then SecureResource.getChild("c", ...) would return "c", which would in turn return "d" if the logged-in user has access to it.
Hopefully this answer worked for you on the list :-).
I am attempting to implement get_current_user in the RequestHandler for Tornado, but I need the call to block while waiting on the asynchronous call to my database. Decorating the call with #tornado.web.asynchronous will not work because either way the get_current_user method returns before the async query completes and the query callback is executed.
For example:
class MyHandler(BaseHandler):
#tornado.web.asynchronous
#tornado.web.authenticated
def get(self):
self.write('example')
self.finish()
class BaseHandler(tornado.web.RequestHandler):
def get_current_user(self):
def query_cb(self, doc):
return doc or None
database.get(username='test', password='t3st', callback=query_cb)
#tornado.web.authenticated calls get_current_user, but always receives "None" because the BaseHandler does not have time to respond. Is there a way, using tornado, to temporarily block for a call such as the one above?
Do a blocking database operation instead of the non blocking described above (There is a blocking mysql lib shipped with tornado).
From the Tornado wiki page about threads and concurrency:
"Do it synchronously and block the IOLoop. This is most appropriate for things like memcache and database queries that are under your control and should always be fast. If it's not fast, make it fast by adding the appropriate indexes to the database, etc."
https://github.com/facebook/tornado/wiki/Threading-and-concurrency
How about having get_current_user return a Future that you signal when the asynchronous response from your database is returned?
class BaseHandler(tornado.web.RequestHandler):
def get_current_user(self):
future = Future()
def query_cb(user):
future.set_result(user or None)
database.get(username='test', password='t3st', callback=query_cb)
return future
class MainHandler(BaseHandler):
#gen.coroutine
def get(self):
user = yield self.get_current_user()
self.write('user: ' + user)
# ... actual request processing
I thought Tornado allowed you to make either blocking or non-blocking requests.
Here is Tornado being used for both: https://bitbucket.org/nephics/tornado-couchdb/src/147579581b47/couch.py
Disclaimer: I know very little of Python and Tornado.