Simplest tornado.gen example - asynchronous

I am trying to use Tornado's sync-style 'gen' tool to run a simple echo function, in a non-blocking style:
import tornado.web
import tornado.gen
import logging
def echo(message):
return message
#tornado.gen.engine
def runme():
response = yield tornado.gen.Task(echo, 'this is a message')
logging.warn(response)
runme()
As far as I can tell this code isn't significantly different to the demo code in the docs, minus the unnecessary request handler stuff - I'm not handling any HTTP requests, AFAICT that's orthagonal to running something asynchronously. Yet this always fails with:
Traceback (most recent call last):
File "./server.py", line 46, in <module>
runme()
TypeError: wrapper() takes at least 1 argument (0 given)
Exactly where am I missing the argument? How can I make Tornado run this function asynchronously?

Task doesn't actually make a callback for the function being run, and start the callback when the function returns, as I originally thought.
I need to create a callback in the task being run myself, and invoke it, i.e.:
import tornado.web
import tornado.gen
import logging
def echo(message, callback=None):
callback(message)
#tornado.gen.engine
def runme():
response = yield tornado.gen.Task(echo, 'this is a message')
logging.warn(response)
runme()

Related

RuntimeError: asyncio.run() cannot be called from a running event loop

I am running the simplest of examples on asyncio:
import asyncio
async def main():
print("A")
await asyncio.sleep.sleep(1)
print("B")
asyncio.run(main())
and I get a runtime error:
RuntimeError: asyncio.run() cannot be called from a running event loop
I am using Spyder (Python 3.9) on an M1 Mac (...if that matters).
the outcome expected is:
A
B
Process finished with exit code 0
But for the ".sleep.sleep" this code is fine - "event loop already running" is certainly not an issue for a standalone script with this code.
Maybe you are running it in as a notebook cell, with some asyncio state already set-up?
In a bash terminal, I pasted your code as is, and just replaced the incorrect function name:
[gwidion#fedora tmp01]$ cat >bla42.py
import asyncio
async def main():
print("A")
await asyncio.sleep.sleep(1)
print("B")
asyncio.run(main())
[gwidion#fedora tmp01]$ python bla42.py
A
Traceback (most recent call last):
[...]
File "/home/gwidion/tmp01/bla42.py", line 5, in main
await asyncio.sleep.sleep(1)
AttributeError: 'function' object has no attribute 'sleep'
[gwidion#fedora tmp01]$ python -c 'open("bla43.py", "w").write(open("bla42.py").read().replace(".sleep.sleep", ".sleep"))'
[gwidion#fedora tmp01]$ python bla43.py
A
B
[gwidion#fedora tmp01]$

Unit test for exception raised in custom GNU radio python block

I have created a custom python sync block for use in a gnuradio flowgraph. The block tests for invalid input and, if found, raises a ValueError exception. I would like to create a unit test to verify that the exception is raised when the block indeed receives invalid input data.
As part of the python-based qa test for this block, I created a flowgraph such that the block receives invalid data. When I run the test, the block does appear to raise the exception but then hangs.
What is the appropriate way to test for this? Here is a minimal working example:
#!/usr/bin/env python
import numpy as np
from gnuradio import gr, gr_unittest, blocks
class validate_input(gr.sync_block):
def __init__(self):
gr.sync_block.__init__(self,
name="validate_input",
in_sig=[np.float32],
out_sig=[np.float32])
self.max_input = 100
def work(self, input_items, output_items):
in0 = input_items[0]
if (np.max(in0) > self.max_input):
raise ValueError('input exceeds max.')
validated_in = output_items[0]
validated_in[:] = in0
return len(output_items[0])
class qa_validate_input (gr_unittest.TestCase):
def setUp (self):
self.tb = gr.top_block ()
def tearDown (self):
self.tb = None
def test_check_valid_data(self):
src_data = (0, 201, 92)
src = blocks.vector_source_f(src_data)
validate = validate_input()
snk = blocks.vector_sink_f()
self.tb.connect (src, validate)
self.tb.connect (validate, snk)
self.assertRaises(ValueError, self.tb.run)
if __name__ == '__main__':
gr_unittest.run(qa_validate_input, "qa_validate_input.xml")
which produces:
DEPRECATED: Using filename with gr_unittest does no longer have any effect.
handler caught exception: input exceeds max.
Traceback (most recent call last):
File "/home/xxx/devel/gnuradio3_8/lib/python3.6/dist-packages/gnuradio/gr/gateway.py", line 60, in eval
try: self._callback()
File "/home/xxx/devel/gnuradio3_8/lib/python3.6/dist-packages/gnuradio/gr/gateway.py", line 230, in __gr_block_handle
) for i in range(noutputs)],
File "qa_validate_input.py", line 21, in work
raise ValueError('input exceeds max.')
ValueError: input exceeds max.
thread[thread-per-block[1]: <block validate_input(2)>]: SWIG director method error. Error detected when calling 'feval_ll.eval'
^CF
======================================================================
FAIL: test_check_valid_data (__main__.qa_validate_input)
----------------------------------------------------------------------
Traceback (most recent call last):
File "qa_validate_input.py", line 47, in test_check_valid_data
self.assertRaises(ValueError, self.tb.run)
AssertionError: ValueError not raised by run
----------------------------------------------------------------------
Ran 1 test in 1.634s
FAILED (failures=1)
The top_block's run() function does not call the block's work() function directly but starts the internal task scheduler and its threads and waits them to finish.
One way to unit test the error handling in your block is to call the work() function directly
def test_check_valid_data(self):
src_data = [[0, 201, 92]]
output_items = [[]]
validate = validate_input()
self.assertRaises(ValueError, lambda: validate.work(src_data, output_items))

Running all python unittests from different modules as a suite having parameterized class constructor

I am working on a design pattern to make my python unittest as a POM, so far I have written my page classes in modules HomePageObject.py,FilterPageObject.py, my base class (for common stuff)TestBase in BaseTest.py, my testcase modules are TestCase1.py and TestCase2.py and one runner module runner.py.
In runner class i am using loader.getTestCaseNames to get all the tests from a testcase class of a module. In both the testcase modules the name of the test class is same 'Test' and also the method name is same 'testName'
Since the names are confilicting while importing it in runner, only one test is getting executed. I want python to scan all the modules that i specify for tests in them and run those even the name of classes are same.
I got to know that nose might be helpful in this, but not sure how i can implement it here. Any advice ?
BaseTest.py
from selenium import webdriver
from selenium.webdriver.support.wait import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver import ChromeOptions
import unittest
class TestBase(unittest.TestCase):
driver = None
def __init__(self,testName,browser):
self.browser = browser
super(TestBase,self).__init__(testName)
def setUp(self):
if self.browser == "firefox":
TestBase.driver = webdriver.Firefox()
elif self.browser == "chrome":
options = ChromeOptions()
options.add_argument("--start-maximized")
TestBase.driver = webdriver.Chrome(chrome_options=options)
self.url = "https://www.airbnb.co.in/"
self.driver = TestBase.getdriver()
TestBase.driver.implicitly_wait(10)
def tearDown(self):
self.driver.quit()
#staticmethod
def getdriver():
return TestBase.driver
#staticmethod
def waitForElementVisibility(locator, expression, message):
try:
WebDriverWait(TestBase.driver, 20).\
until(EC.presence_of_element_located((locator, expression)),
message)
return True
except:
return False
TestCase1.py and TestCase2.py (same)
from airbnb.HomePageObject import HomePage
from airbnb.BaseTest import TestBase
class Test(TestBase):
def __init__(self,testName,browser):
super(Test,self).__init__(testName,browser)
def testName(self):
try:
self.driver.get(self.url)
h_page = HomePage()
f_page = h_page.seachPlace("Sicily,Italy")
f_page.selectExperience()
finally:
self.driver.quit()
runner.py
import unittest
from airbnb.TestCase1 import Test
from airbnb.TestCase2 import Test
loader = unittest.TestLoader()
test_names = loader.getTestCaseNames(Test)
suite = unittest.TestSuite()
for test in test_names:
suite.addTest(Test(test,"chrome"))
runner = unittest.TextTestRunner()
result = runner.run(suite)
Also even that one test case is getting passed, some error message is coming
Ran 1 test in 9.734s
OK
Traceback (most recent call last):
File "F:\eclipse-jee-neon-3-win32\eclipse\plugins\org.python.pydev.core_6.3.3.201805051638\pysrc\runfiles.py", line 275, in <module>
main()
File "F:\eclipse-jee-neon-3-win32\eclipse\plugins\org.python.pydev.core_6.3.3.201805051638\pysrc\runfiles.py", line 97, in main
return pydev_runfiles.main(configuration) # Note: still doesn't return a proper value.
File "F:\eclipse-jee-neon-3-win32\eclipse\plugins\org.python.pydev.core_6.3.3.201805051638\pysrc\_pydev_runfiles\pydev_runfiles.py", line 874, in main
PydevTestRunner(configuration).run_tests()
File "F:\eclipse-jee-neon-3-win32\eclipse\plugins\org.python.pydev.core_6.3.3.201805051638\pysrc\_pydev_runfiles\pydev_runfiles.py", line 773, in run_tests
all_tests = self.find_tests_from_modules(file_and_modules_and_module_name)
File "F:\eclipse-jee-neon-3-win32\eclipse\plugins\org.python.pydev.core_6.3.3.201805051638\pysrc\_pydev_runfiles\pydev_runfiles.py", line 629, in find_tests_from_modules
suite = loader.loadTestsFromModule(m)
File "C:\Python27\lib\unittest\loader.py", line 65, in loadTestsFromModule
tests.append(self.loadTestsFromTestCase(obj))
File "C:\Python27\lib\unittest\loader.py", line 56, in loadTestsFromTestCase
loaded_suite = self.suiteClass(map(testCaseClass, testCaseNames))
TypeError: __init__() takes exactly 3 arguments (2 given)
I did this by searching for all the modules of test classes with a pattern and then used __import__(modulename) and called its Test class with desired parameters,
Here is my runner.py
import unittest
import glob
loader = unittest.TestLoader()
suite = unittest.TestSuite()
test_file_strings = glob.glob('Test*.py')
module_strings = [str[0:len(str)-3] for str in test_file_strings]
for module in module_strings:
mod = __import__(module)
test_names =loader.getTestCaseNames(mod.Test)
for test in test_names:
suite.addTest(mod.Test(test,"chrome"))
runner = unittest.TextTestRunner()
result = runner.run(suite)
This worked but still looking for some organized solutions.
(Not sure why second time its showing Ran 0 tests in 0.000s )
Finding files... done.
Importing test modules ... ..done.
----------------------------------------------------------------------
Ran 2 tests in 37.491s
OK
----------------------------------------------------------------------
Ran 0 tests in 0.000s
OK

About tornado.gen.Task's usage - asynchronous request

below is my source:
class Get_Salt_Handler(tornado.web.RequestHandler):
#tornado.web.asynchronous
#tornado.gen.coroutine
def get(self):
#yield tornado.gen.Task(tornado.ioloop.IOLoop.instance().add_timeout, time.time() + 5)
yield tornado.gen.Task(self.get_salt_from_db, 123)
self.write("when i sleep 5s")
def get_salt_from_db(self, params):
print params
and I run it; the console reported that:
TypeError: get_salt_from_db() got an unexpected keyword argument 'callback'
and I don't know why?
gen.Task is used to adapt a callback-based function to the coroutine style; it cannot be used to call synchronous functions. What you probably want is a ThreadPoolExecutor (standard in Python 3.2+, available with pip install futures on Python 2):
# global
executor = concurrent.futures.ThreadPoolExecutor(NUM_THREADS)
#gen.coroutine
def get(self):
salt = yield executor.submit(self.get_salt_from_db)

servlet filter in jython

Based on this java example, I made the following servlet filter in jython (exact code):
from javax.servlet import Filter
from javax.servlet.http import HttpServletRequest
class HttpServletRequestWrapper(HttpServletRequest):
def init(self, request):
self.originalURL = self.getRequestURL()
pathi = self.originalURL.find('/', 10) # find start of path
qsi = self.originalURL.find('?', pathi) # find start of qs if any
qs = self.originalURL[qsi:] if qsi > -1 else ''
self.newURL = self.originalURL[:pathi] + '/ccc/jope.py' + qs
def getRequestURL(self):
return self.newURL
class Route2Jope(Filter):
def init(self, config):
pass
def doFilter(self, request, response, chain):
wrapped = HttpServletRequestWrapper(request)
chain.doFilter(wrapped, response)
However, I am getting the error message:
Traceback (most recent call last):
File "c:\CCC\webapps\ccc\WEB-INF\pyfilter\Route2Jope.py", line 24, in doFilter
wrapped = HttpServletRequestWrapper(request)
TypeError: org.python.proxies.__main__$HttpServletRequestWrapper$2(): expected 0 args; got 1
org.python.core.Py.TypeError(Py.java:259)
org.python.core.PyReflectedFunction.throwError(PyReflectedFunction.java:209)
org.python.core.PyReflectedFunction.throwArgCountError(PyReflectedFunction.java:262)
org.python.core.PyReflectedFunction.throwError(PyReflectedFunction.java:319)
org.python.core.PyReflectedConstructor.__call__(PyReflectedConstructor.java:177)
org.python.core.PyObject.__call__(PyObject.java:419)
org.python.core.PyMethod.instancemethod___call__(PyMethod.java:237)
org.python.core.PyMethod.__call__(PyMethod.java:228)
org.python.core.PyMethod.__call__(PyMethod.java:223)
org.python.core.Deriveds.dispatch__init__(Deriveds.java:19)
org.python.core.PyObjectDerived.dispatch__init__(PyObjectDerived.java:1112)
org.python.core.PyType.type___call__(PyType.java:1713)
org.python.core.PyType.__call__(PyType.java:1696)
org.python.core.PyObject.__call__(PyObject.java:461)
org.python.core.PyObject.__call__(PyObject.java:465)
org.python.pycode._pyx1.doFilter$6(c:\CCC\webapps\ccc\WEB-INF\pyfilter\Route2Jope.py:25)
org.python.pycode._pyx1.call_function(c:\CCC\webapps\ccc\WEB-INF\pyfilter\Route2Jope.py)
org.python.core.PyTableCode.call(PyTableCode.java:167)
org.python.core.PyBaseCode.call(PyBaseCode.java:307)
org.python.core.PyBaseCode.call(PyBaseCode.java:198)
org.python.core.PyFunction.__call__(PyFunction.java:482)
org.python.core.PyMethod.instancemethod___call__(PyMethod.java:237)
org.python.core.PyMethod.__call__(PyMethod.java:228)
org.python.core.PyMethod.__call__(PyMethod.java:218)
org.python.core.PyMethod.__call__(PyMethod.java:213)
org.python.core.PyObject._jcallexc(PyObject.java:3626)
org.python.proxies.__main__$Route2Jope$3.doFilter(Unknown Source)
org.python.util.PyFilter.doFilter(PyFilter.java:80)
I think it's telling me I should not pass the parameter 'request', but it does not make sense to me. Maybe I am overlooking some mapping issue between python ad java classes? Suggestions?
The name of your constructor must be __init__, not init. :)

Resources