I have a large number of test cases, in which several test cases are interdependent. Is it possible that while a later test case is getting executed you can find out the status of a previously executed test case?
In my case, the 99th test case depends on the status of some prior test cases and thus, if either the 24th or the 38th fails I would like the 99th test case NOT to get executed at all and thus save me a lot of time.
Kindly, explain with some example if possible. Thanks in advance!
Robot is very extensible, and a feature that was introduced in version 2.8.5 makes it easy to write a keyword that will fail if another test has failed. This feature is the ability for a library to act as a listener. With this, a library can keep track of the pass/fail status of each test. With that knowledge, you can create a keyword that fails immediately if some other test fails.
The basic idea is, cache the pass/fail status as each test finishes (via the special _end_test method). Then, use this value to determine whether to fail immediately or not.
Here's an example of how to use such a keyword:
*** Settings ***
Library /path/to/DependencyLibrary.py
*** Test Cases ***
Example of a failing test
fail this test has failed
Example of a dependent test
[Setup] | Require test case | Example of a failing test
log | hello, world
Here is the library definition:
from robot.libraries.BuiltIn import BuiltIn
class DependencyLibrary(object):
ROBOT_LISTENER_API_VERSION = 2
ROBOT_LIBRARY_SCOPE = "GLOBAL"
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
self.test_status = {}
def require_test_case(self, name):
key = name.lower()
if (key not in self.test_status):
BuiltIn().fail("required test case can't be found: '%s'" % name)
if (self.test_status[key] != "PASS"):
BuiltIn().fail("required test case failed: '%s'" % name)
return True
def _end_test(self, name, attrs):
self.test_status[name.lower()] = attrs["status"]
To solve this problem I'm using something like this:
Run Keyword if '${PREV TEST STATUS}'=='PASSED' myKeyword
so maybe this will be usable also for you.
Related
I need to set a list inside a test case and use this list in [SETUP] to pass this list to a python script, how can I achieve this?
TEST-List
#{lst} Create List a b
#{tmp} Set Test Variable #{lst}
[Setup] Receive List ${tmp} ${another_var}
When I try the code above, I got this error:
Variable '{${tmp}' not found.
[Setup] setting is used for performing actions before a test case. Its purpose is to set up a state for your test. That implies that it happens (executes) before test steps, regardless of where you type it.
In your case, [Setup] Receive List ${tmp} ${another_var} is executed first, and ${tmp} variable has not been declared yet.
The solution might be to move the declaration of ${tmp} to a suite level.
In your example, the code in [Setup] is run before any other code in the test. Therefore, #{lst} and #{tmp} are undefined at the time that it runs.
The simplest solution is to create a local keyword that performs everything you need in the setup, and then call that keyword from [Setup].
Example
*** Keywords ***
Initialize test
#{lst} Create List a b
#{tmp} Set Test Variable #{lst}
Receive List ${tmp} ${another_var}
*** Test Cases ***
TEST-List
[Setup] Initialize test
# ... rest of your test goes here ...
Is there any possible to change testcase name with variable like below?
(I don't want to change name from python side)
*** Variables ***
${country} US
*** Test Cases ***
test_${country}
As far as I know, it isn't possible to use a variable inside a test case name. It follows the same logic as normal Python functions, so normally, it isn't possible.
Instead, you can use the variable in the setup or in the test case directly to modify it's behaviour.
If you want to generate test cases based on a variable, you can write a (python) script that can generate the needed file/test cases with the corresponding values. Or, even better, use an Model-Based Testing tool to produce them.
Yes, you can. The way you have shown it should work. Are you facing any issue with that? If yes, pls provide the detailed error.
Yes this is supported.
example:-
*** Test Cases ***
Test title ${name}
[Tags] DEBUG
Log Welcome ${name}
output:-
robot --variable name:sample eg.robot
I'm writing some tests which rely on an external binary being present. Having a test case for ensuring binary is present and functioning seems useful. I want multiple tests, expressing:
Is binary found by subprocess.Popen?
Can it do basic X?
Can it do basic Y?
I want a pattern to skip the basic tests X and Y if the first one fails. It would also be good to test several binaries in the same TestCase, with different kinds of X and Y.
Some approaches come to mind:
Decorate test_X() and test_Y() with #unittest.skipIf(basic failed), either through checking TestResults or setting a flag
Have test_basic() call TestResults.addSkip(test_binary_X) upon failure (but this doesn't appear to be exposed)
Put test_basic() into one TestCase and the others into another
Make test_basic() part of setUp(), so it's run for each of X and Y
Unittest doesn't run tests in the order written, so it may be hard to ensure sequence.
# Something like:
class TestBasics(unittest.TestCase):
test_basic(self):
try:
subprocess.Popen(["binary"], stdout=None, stderr=None)
catch OsError as e:
# fail depending on the error
#unittest.skipIf( test binary in path failed )
test_X(self):
# assert
#unittest.skipIf( test binary in path failed )
test_Y(self):
# assert
Is it possible to write Robot Framework tests in Python instead of the .txt format?
Behind the scenes it looks like the .txt test get converted into Python by pybot so I'm hoping that this is simply a matter of importing the right library and inheriting from the right class but I haven't been able to figure out how to do that.
(We already have a bunch of suites and have keywords written in both formats but sometimes the RF syntax makes it very difficult to do things that are simple in Python. I understand it would be possible to just write a Python keyword for each test plus 'wrap' setup and teardown functions the same way, but that seems cumbersome.)
Robot does not convert your test cases to python behind the scenes before running them. Instead, it parses the test cases, then iterates over each keyword, calling the code that implements the keyword. There isn't ever a stage where there's a completely pure python representation of a test case.
It is not possible to write tests in python, and have those tests run alongside traditional robot tests by the provided test runner. Like you said in your question, your only option is to put all of your logic for a single test case in a single keyword, and call that keyword from a test case.
It is possible to create and execute tests in python solely via the published API. This might not be what you're really asking for, because ultimately you're still creating keywords, you're just creating them via python.
from robot.api import TestSuite
suite = TestSuite('Activate Skynet')
suite.imports.library('OperatingSystem')
test = suite.tests.create('Should Activate Skynet', tags=['smoke'])
test.keywords.create('Set Environment Variable', args=['SKYNET', 'activated'], type='setup')
test.keywords.create('Environment Variable Should Be Set', args=['SKYNET'])
The above example was taken from here:
http://robot-framework.readthedocs.org/en/2.8.1/autodoc/robot.running.html
Well, you should not care if your python code represents tests or keywords as long as you code the logic of the tests in python.
The best you can do is to keep some html tables in robot format. Each line would be a call for a keyword. The keyword could be implemented in python, and, logically, represents a whole test (although in robot terminology it is still a "keyword").
This post shows how you can have access to the robot context from your python code.
robot variables
BuiltIn().get_variable_value("${USERNAME}")
java keywords
from com.mycompany.myproject.testtools import LoginRobotKeyword
LoginRobotKeywords().login(user, pwd)
robot keywords
BuiltIn().run_keyword("check user connected", user)
Robotframework does not support writting test cases in python directly. I've submitted an enhancement PR, check it here
https://github.com/robotframework/robotframework/issues/3128
But I've tried to do that by moving all the test cases logic to python code, and make RF test cases just a entry point to them.
Here is an example.
We could create a python file to include all testing logic and setup/teardown logic, like this
# *** case0001.py *****
from SchoolClass import SchoolClass
schCla = SchoolClass()
class case0001:
def steps(self):
print('''\n\n***** step 1 **** add school class \n''')
self.ret1 = schCla.add_school_class('grade#1', 'class#1', 60)
assert self.ret1['retcode'] == 0
print('''\n\n***** step 2 **** list school class to check\n''')
ret = schCla.list_school_class(1)
schCla.classlist_should_contain(ret['retlist'],
'grade#1',
'class#1',
60,
self.ret1['id'])
def setup(self):
pass
def teardown(self):
schCla.delete_school_class(self.ret1['id'])
And then we creat a Robot file. In which all RF test cases are in the same form and just work as entry points to python test cases above.
like this
*** Settings ***
Library cases/case0001.py WITH NAME C000001
Library cases/case0002.py WITH NAME C000002
*** Test Cases ***
add class - tc000001
[Setup] C000001.setup
C000001.steps
[Teardown] C000001.teardown
add class - tc000002
[Setup] C000002.setup
C000002.steps
[Teardown] C000002.teardown
You could see, in this way, the RF testcases are similar. We could even create a tool to auto generate them by scanning Python testcases.
In a robot framework test case I set a variable and then do a process.
Because the setting of the variable is not a very interesting bit of information, I don't want to include that in my report.
| Verifying STUFF |
| | ${endpoint}= | set variable | STUFF
| | Verify
My report contains this:
KEYWORD: ${endpoint} = BuiltIn.Set Variable STUFF
But I would rather not have it there. How can I tell Robot Framework to just not log that line?
------edit------
It looks like this should do it:
pybot --removekeywords NAME:SetVariable testcase.txt
But the Set Variable keywords are still there.
(And yes, I upgraded my robot framework to 2.8.3 to take advantage of this function.)
The best you can do is to use
Set Log Level NONE
but it will still log all the keyword calls, just not anything inside those.
Or if you call a python function which calls another function, then the call to the second function is not logged.
Like this:
*** Settings ***
Library lib.py
*** Test Cases ***
demo
Set Log Level NONE
${a} foo
xyzzy
*** Keywords ***
xyzzy
qwerty
qwerty
No Operation
Log 123
and lib.py being like this:
def foo():
abc = bar()
return abc
def bar():
c = 1
print c
return c
The problem is that when you assign a variable like ${var} = Keyword, the name of the keyword in Robot Framework outputs is ${var} = Keyword, not Keyword as you would expect. If your keyword is from a library or a resource file, its name will also be included like ${var} = MyLibrary.Keyword. The latter is a feature but the former is a bug that is hopefully fixed in RF 2.9.
An easy workaround for the keyword name, for now, including the variable name is using wildcards. Something like this ought to work for you:
--RemoveKeywords 'name:* = BuiltIn.Set Variable'
You can use --removekeywords or --flattenkeywords option on pybot to remove the content of keyword So if you have e.g. keyword "foo" that contains lot's of logging keywords, you can set "--flattenkeywords name:foo" option to pybot, and In the log you'll only see the primary keyword, but no keywords inside it.
http://robotframework.googlecode.com/hg/doc/userguide/RobotFrameworkUserGuide.html?r=2.8.3#removing-and-flattening-keywords
If you use a python library, the following monkey-patching works for me:
from robot.libraries.BuiltIn import BuiltIn
from robot.output.logger import LOGGER
import types
def _nothing(*args, **kwargs):
pass
def disable_keyword_logging(self):
self._logging_methods = (LOGGER.start_keyword, LOGGER.end_keyword)
LOGGER.start_keyword = types.MethodType(_nothing,LOGGER)
LOGGER.end_keyword = types.MethodType(_nothing,LOGGER)
def enable_keyword_logging(self):
LOGGER.start_keyword, LOGGER.end_keyword = self._logging_methods
Then when this script is run:
Disable keyword logging
Log Hello world
Enable keyword logging
the "Log" keyword is not logged to the output but the output is.
If you really want nothing (also no debug/info/warn information logged by the called keywords), you will still have to set the log level to "NONE".
Robot Framework doesn't log "global" variables as part of a variable table. Global is in quotation marks because Set Global Variable actually is logged, but if you initialize your variable like so...
*** Variables ***
${endpoint} stuff
*** Keywords ***
...then it will not be in the Log. Additionally, if you don't want anyone to see the variable at all if they're just looking at the front end of your testing suite, you can bury it in a Resource file and the call the Resource file.
Robot Framework logs your Set Variable keywords and results because Set Variable implies that you're setting a variable dynamically and might be setting it based on the result of a keyword, in which case you'd probably want to know what the result of the keyword is. If you're just creating a static variable, then no extra work beyond the table is required. Is a dynamic variable a required part of your code?