I am testing a GUI application with robot framework and AutoItLibrary. I would like the test to take screenshot after any failure in test, but not in the test teardown, but right after the error occurs, as the steps in between the failure and the teardown usually make the screen worth nothing. Is there a way to do that in robot? All I could get so far is a screenshot in the end of a test...
Thanks in advance!
AutoIt lib has CaptureScreenOnError (defaults to False) argument so when importing the lib, set it to True. See https://robotframework-autoitlibrary.googlecode.com/svn/tags/robotframework-AutoItLibrary-1.1/doc/AutoItLibrary.html
It will not take screenshot if the error happens in non-AutoIt lib keyword. AFAIK there isn't any nice and sure way to take screenshot after any failure.
Run Keyword If Test Failed Take Screenshot
in every test teardown will handle most situation, but if the error happens in a keyword that has a teardown, the keyword teardown will be done before the test teardown. And "Run Keyword If Test Failed" will only work in test teardown.
Related
I try to find a way to execute a keyword after the test suite. Something like Suite Setup for after the execution.
There is a way to do that?
Suite Teardown defined in the Settings section - it will be the last thing ran for the suite, after the last case's teardown.
It is explained in details in the user guide - http://robotframework.org/robotframework/latest/RobotFrameworkUserGuide.html#suite-setup-and-teardown
As shown in below robot file, I have three testcases. I want to stop the test execution, if TESTCASE1 fails. TESTCASE2 should be executed only if TESTCASE1 passes.
*** Settings ***
Library pythonLib
*** Test cases ***
TESTCASE1
boot device
TESTCASE2
configure device
TESTCASE3
restart device
Is there any keyword or custom way to do this ?
There is a command line option for this, if you want the behavior that robot should stop running as soon as any test fails. This option is --exitonfailure. From the robot framework user guide, in a section titled Stopping when the first test fails:
If option --exitonfailure is used, test execution stops immediately if
any critical test fails. Also the remaining tests are marked as
failed.
You might also want to take a look at this answer to the question Automatic failing/non-execution of interdependent tests in Robot Framework, which shows how to write a keyword to implement dependencies between test cases.
There are multiple ways to get the job done, each appropriate for different situations.
--exitonfailure
The command line option --exitonfailure aborts the test run after any testcase fails, unless it is marked noncritical.
Fatal Error
You might only want to abort if exactly TESTCASE1 fails. The Fatal Error keyword exist just for this purpose:
TESTCASE1
${passed}= Run Keyword And Return Status boot device
Run Keyword If not ${passed} Fatal Error
If this seems clunky to you, you can throw fatal errors directly from Python/Java.
Suite Setup
These tools will get the job done, and are appropriate in some cases. Although in the asker's case, I observe:
The first testcase must pass for any other tests to run.
It runs a keyword called boot device.
To me that is not a testcase. That is a setup. If you need a setup to run once before a suite of testcases, you should specify it as a Suite Setup.
***Settings***
Suite Setup boot device
--pause_on_failure, this will stop the execution, whenever script hit the error. Script execution won't be resumed unless you explicitly start.
I'm new to using robot framework and I'm struggling to get my teardown to work.
It currently looks like:
[Teardown] run keyword if any tests failed KeyFail
When I run the program with code like this, I get the error: Keyword 'Run Keyword If Any Tests Failed' can only be used in suite teardown.
I can change it so that I put it inside it's own test case, however I then get the error that: Test Case contains no keywords.
Please advise me as to what I'm doing wrong. It would be appreciated. Thanks.
Edit:
***Keywords***
Generation
(Some stuff)
KeyFail
log to console Error report being sent.
***Test Cases***
Requires successful generation of file
Generation
Teardown Case
[Teardown] run keyword if any tests failed KeyFail
Edit: And how to fix this problem. Thanks
It looks like you have defined it in the test case teardown instead of the test suite teardown. You can change it to use the Test teardown instead.
Edit: Here are two solutions:
1. Change your keyword to the TEST specific one, Run Keyword If Test Failed which applies to the last executed test, and can only be used in a test teardown.
2. The second is to use Suite Setups / teardowns. These apply to ALL test cases that you run. Like this:
***Settings***
Suite Setup Your Test Setup Keyword
Suite Teardown run keyword if any tests failed KeyFail
***Keywords***
Generation
(Some stuff)
KeyFail
log to console Error report being sent.
***Test Cases***
Requires successful generation of file
Generation
Teardown Case
Stuff to do
# teardown is automatic, and does not need to be called.
How to pause the script execution on failures and continue the executions, after performing some operations in Robot framework?
please suggest me.
You can use the Pause Execution keyword from the Dialogs library, which will open a dialog and pause robot until the dialog is dismissed.
Unfortunately, robot has no way of automatically running a keyword on any failure. You'll have to manage that yourself. For example, you could use Run Keyword If Test Failed in a test teardown. Of course, you won't be able to continue that particular test, but it will let you pause before the next test.
If you need to allow the current test to continue after a failed keyword there are many keywords that can help you, such as Run keyword and continue on failure and all of the other Run keyword if... and Run keyword unless... keywords.
Sometimes a test fails because of infrastructure failures, for example a network outage that does not indicate a regression. Is there any alternative in robotframework to PASS / FAIL? Something like ERROR?
Robot Framework offers only PASS or FAIL, nothing like ERROR.
I can see 2 strategies though to handle those intermittent problems (that most of us are facing).
1) use the "wait until keywords succeeds" keyword. For example, if you have to do a GET via REST on a remote server that could be unreachable for some network reason, then instead of
Get MyURL
you could do
wait until keywords succeeds Get http://example.com
and even better option would be to create a custom keyword for that
*** keywords ***
Get_until_succeeds
[Arguments] ${url}
wait until keywords succeeds Get ${url}
So then you just have to call:
Get_until_succeeds http://example.com
2) use the "--rerunfailed" option or Robot Framework that allows you to re-run the tests that failed. The way to use it is to first launch your suite the usual way:
pybot tests
And then give the output.xml of the previous execution as input of another round:
pybot --rerunfailed output.xml tests
(you can then merge the 2 reports and get a single nice report)