I have recently started working on a program that should not be changed much and there are problems that it closes properly with Robot Framework. The method that came to my mind now was that I could get the reports before the test end.
So I have this Question:
Is there a possibility or keyword in Robotframework that I can use to get reports before the test is done?
Is there a possibility or keyword in Robotframework that I can use to get reports before the test is done?
No, there is not. Robot creates the reports as an in-memory xml document. It doesn't write the data to disk until the tests are finished. It then runs a post-processing step to convert them to html.
As #Bryan wrote, there is no way to get report.html during robot runtime as it is generated in postrun step.
However, you may use listener to get feedback from robot about execution status. For instance, in RED Robot Editor IDE, it is used to populate Execution View (see screenshot at the bottom):
http://nokia.github.io/RED/help/user_guide/launching/ui_elements.html
More about listener API: https://github.com/robotframework/robotframework/blob/master/doc/userguide/src/ExtendingRobotFramework/ListenerInterface.rst
Related
In my RF tests in few places I have added logging by
log to console Scanning first UID ${pack_uids}[0]
But when I open HTML log file, I expected to see a value, which was used in this test. Instead I have exactly the text.
While Robot Framework can work with ${pack_uids}[0] stored as a variable, how can I get variable value, instead of plain text? I tried also to use Log only, same issue.
Similar case, when I try to log to file from Python file by running
logger.console("Some foo")
But as a result, this is not presented in log.html file at all.
And a side question, is it possible to get logs produced by each pabot thread in some file, like it's created in the console when running a single robot command?
Another option is to set the loglevel to INFO
In the command line use --loglevel option or the Set Log Level keyword
Of course I found it out just after placing the question... It's really important to have Log only, and the details are presented in a new line
In Python file use
logger.write("Log you want to include" + str(variable))
And it's working all fine. Just mind you need to expand the logging row
my management would like to have a simple switch to trigger the generated ROBOT test reports, to get from one test execution two reports: one comprehensive detailed report (xml, html) and one management-level report with general info and without many technical details.
Is there a standard ROBOT mechanism to generate two different reports at once?
How would you do?
Thank you for your suggestions!
Robot has the built-in ability to generate four types of outputs:
log.html is a detailed low level log of test execution, showing the details of every suite, test case, and keyword (parameters, results, duration)
report.html is a more high level overview of test execution
output.xml is a detailed log of all of the data used to generate the other reports
xUnit is an XUnit-compatible file that can be processed by many xUnit-compatibile tools
The log.html, report.html, and output.xml files are all generated by default. Generating the xUnit output requires the use of a command line option.
If none of those meet your need, there is an API for reading and processing the output.xml file which you can use to generate a custom report. The format of the output.xml file is very simple and easy to parse, so you can also use just about any xml parsing tool you want to parse the results and generate your own report.
All of this information is available in the robot frame user guide, in a section titled Created outputs.
I am currently trying to find a way to start a test run from the UI or command line following this reference. With the command line tool TCM (TFS 2017 and earlier) you could start a test run and provide an alternate build drop location through the switch "\BuildDir", if not provided it would look in the builds drop location stored in TFS.
I am looking for a way to do this in the new way of testing using Test Hub.
I have done a lot of searching but to no avail.
Any help would be very much welcome.
You could use Rest API to achieve the requirement:
POST https://{accountName}.visualstudio.com/{project}/_apis/test/runs?api-version=5.0-preview.2
For the request body there is
buildDropLocation Drop location of the build used for test run.
I am trying to find a reasonable approach to getting a code coverage report for code that is called from within a test via HTTP. Basically I am testing my own API the way it is supposed to be called but because of that PHPUnit/Xdebug are unaware of the execution of the code within the same codebase.
Basically what I want to achieve is already done using the PHPUnit Selenium extension but I don't run Selenium, I call the code through an OAuth2 Client which in turn uses curl.
Is it be possible to call my API with a GET-parameter that triggers a code coverage report and to have PHPUnit read that report and merge it with the other code coverage? Is there a project that already does that or do I have to resort to writing my own PHPUnit extension?
OP says the problem is that Xdebug-based code coverage collection, won't/can't collect coverage data because Xdebug isn't enabled in all (PHP) processes that execute the code.
There would seem to only be two ways out of this.
1) Find a way to enable Xdebug in all processes invoked. I don't know how to do this, but I would expect there to be some configuration parameter for the PHP interpreter to cause this. I also can't speak to whether separate Xdebug-based coverage reports can be merged into one. One the face of it, the raw coverage data is abstractly just a set of "this location got executed" signals, so merging should just be a set union. How these sets are collected and encoded may make this more problematic.
2) Find a coverage solution that doesn't involve Xdebug, so whether Xdebug is enabled or not is irrelevant. My company's (see bio) PHP Test Coverage Tool does not use Xdebug, so it can collect the test coverage data you want without an issue. You can download it and try it; there's a built in-example of test coverage collection triggered exactly by HTTP requests. The tool has a built-in ability to merge separate test-coverage runs into an integrated result. (I'd provide a direct link, but some SO people are virulently against this).
From my Eclipse plugin, I want to execute a command and show the results in the Console view (and later do some formatting and hyperlinking and pattern matching, which is done via the org.eclipse.ui.console.consolePatternMatchListeners extension point AFAIK). My question is how to do this? In plain Java, I would use a ProcessBuilder. Do I have to do this and bind the stdout/stderr somehow to a newly created console page or is there another way? Any pointers/experiences are appreciated.
Using Eclipse FAQ, this SO question and the ProcessBuilder, I managed to lazily create my console, show it and echo the input stream of the process to the console's message stream (instead of System.out).