I run my robotframework test suite as a teamcity/jenkins build with two simple steps as below
build step #1: pybot
build step #2: pybot --rerunfailed Results\output.xml
When all the tests in step-1 are passed, the build fails because step-2 (--rerunfailed) triggers an error ( [ ERROR ] Collecting failed tests from 'Results\output.xml' failed: All tests passed.) .
Could someone please suggest how to ignore or overcome this error, so that I can show the build as passed in this case ?
I had similar Problem, i fixed it this way:
robot -d %ResultPath% %TestSuitName% || robot --rerunfailed output.xml --output output1.xml -l log.html -r report.html TestSuitName || rebot --rerunmerge --output output.xml -l log.html -r report.html output.xml output1.xml
Uses || to run those command simultaneously and it will work.
Make build step #2 dependent on build step #1 failing. That is, only run pybot --rerunfailed if the first pybot exited with a non-zero exit status.
The simplest way to do this is to create a custom test runner in bash or python or powershell that does both the running of pybot and re-running of pybot as a single step. You then configure this shell script as a single step.
Another way to do it is to have your second build step either look at the return code of the previous step (if possible), or scans the output.xml to see if there are failures. If there are no failures, it returns without doing any work.
There's robot:skip-on-failure tag. It it sets then a test case will skip for failure and perform if it's ok. It takes RobotFramework 5.0.
Related
I am using pycharm terminal to execute test suite
robot -d Output --loglevel TRACE TestCases\TC001Suite.robot
Within the TC001Suite, there are 10 cases.
I want to rerun failed cases for this test suite only.
I have tried --runfailed but it cannot be executed.
the option -d means that you want to output to directory Output. Maybee your xml file is in different director and when you rerun the test it fails
robot -d Output --loglevel TRACE TestCases\TC001Suite.robot
Dont change the directory and just specify the name of xml file. The
robot will put the output in your current dir.
robot --output original.xml --loglevel TRACE TestCases\TC001Suite.robot # first execute all tests
robot --rerunfailed original.xml --output rerun.xml --loglevel TRACE TestCases\TC001Suite.robot # then re-execute failing
you can then merge the tests. You can read about it in the documentation
I am using RIDE 1.3 using robotframework 2.8.7 running on Python 2.7.6. To execute my test I use jybot. The Jython version is Jython 2.5.3. I have an output XML generate from a previous run with failed cases. Now I need to selectively execute only the failed cases from the XML.
I have tried using the Command -R --rerunfailed [xml_path] by inserting it in the arguments tab of RIDE. But this command is unrecognised by the editor so I use -R [xml_path] instead which does not throw any errors.
Finally, when I start executing the suite I expect only the failed results from the suite to run but the entire suite runs as if the command has no effect. The suite from previous run and current run is same but the suite is large and output.xml generated is huge (~100 MB).
How can I execute only failed cases?
The robot scripts when ran on RIDE, generate output.xml, report.html etc files, once run is over.
Is there any way available to view logs when script is still running? (When I use pause on failure)
Also some times I had to Stop/Abort the run in middle, and no logs are generated in such cases.
Kindly help,
Thanks in advance
As for first part — RIDE runs tests adding own listener, providing more verboseness of the output and pausing/resuming functionality.The easiest thing is to run tests not from RIDE, but from console using robot/pybot script. In this case much less logs are written to output (though it doesn't provide pause/resume functionality).
For second part — robot (RIDE starts robot script — you can see it in execution log: command: pybot.bat...) generates output.xml file not after but during execution, so generated output.xml is not valid until test is finished. After normal execution rebot tool generating log.html automatically. So generally it's possible to take following steps:
"Fix" your incomplete output.xml file after execution stop with fixml. output.xml location for RIDE execution can be found in the very same execution log of yours (e.g. ...\appdata\local\temp\RIDEv_0yrp.d\ in my case)
Run rebot stand-alone: rebot output.xml --log log.html --report report.html. Rebot options description you can check using rebot --help (as usual)
Please also note that directory where RIDE output files are stored is temporary — exists only when RIDE is started. You will lose your output on exiting RIDE
I'm using RIDE 1.5 so maybe my answer is not valid for other versions
In RIDE, under Run Tab , when you are running the scripts , you have a option, show message log, it will shows the runtime log.
Try this out.
I am trying to run some robot framework tests on sauce labs. I am not able to pass a variable file to my pybot command.
When I run my tests locally I use this command :
pybot -vbrowser:firefox -vbaseur --variablefile ../VariableFiles/superdesk.py mytest.robot
On sauce labs I need to pass some other variables: sauce username, key...
pybot -v browser:firefox -v baseurl:http://myurl.fr -v sauce_apikey:mykey -v sauce_platform:linux -v sauce_username:myusername mytest.robot
How could I pass a variable to the second command as as soon as I pass --variable filemy tests run locally and not on sauce labs.
I have tried this command
pybot -v browser:firefox -v baseurl:http://myurl.fr -v sauce_apikey:mykey -v sauce_platform:linux -v sauce_username:myusername --variablefile myvarfile.py mytest.robot
When running the command above the variable file is not taken into account. My tests run with default variables
The literal answer to your question is "you pass a variable file to a test the same way no matter whether you are using saucelabs or not".
Unless robot is throwing an error, your variable file is being passed to robot when you include --variablefile myvarfile.py. You can write a simple test to verify that, by having the test log the values from the variable file.
If you are seeing different behaviours, the behaviours must be in your test cases, or in your own variable file. There is no feature in robot that behaves differently when running on saucelabs or not.
I am new to robot framework and wanted to see if i can run test cases without RIDE. I want to create test suite and run test cases sequentially without using RIDE. I went through documentation but could not understand it.
Ex: Test Suite
Test Case 1
Test Case 2
Test Case 3
i would like to put my references to all my resource files to test suite and run all test cases. I can do this using RIDE but wanted to know if i can do this without using it. Do i need to create batch file to do this or any other method to run? Any example will help me. Thank you for advance.
When you install robot, you also installed a program called robot (or pybot in older versions), which is the official robot test runner.
If you have a test suite named "my_tests.robot", you can open up a command prompt (bash on *nix, powershell or command.exe on windows) and type the following command (assuming that robot is in your PATH environment variable, which it probably is):
$ robot my_tests.robot
If you have a collection of suites in a folder, you can give pybot the name of the folder rather than the name of a test file.
To see a list of all robot command line options, use the --help option:
$ robot --help
For more information see Starting test execution in the robot framework users guide.
if you use sublime-text/bracket/Atom/Emacs/vim ... you also have some plug-in :
http://robotframework.org/#tools
Show the path of test suite
exmp: Location of my 'catiav6' test script_
cd C:\Users....\Desktop\catiav6
Run following command from cmd
For single test case:
pybot --test 'Name_of_single_test_case' 'Name_of_test_suite'
For all test cases:
pybot --test * 'Name_of_test_suite'