I have some tests that are passing locally but failing in CI. I'm trying to troubleshoot using the command php ./vendor/bin/phpunit -d memory_limit=512m --debug --verbose however no amount of var_dump(), throwing exceptions or anything will give me any debug output. I know it's the output buffer hiding the output. I've looked at phpunit's --help and don't see anymore flags I could possibly add that will show me the output. Looking at the TestCase source code I'm not seeing a clear way to turn off this buffering. I'm also confused why locally I do see the debug output...
Related
Since a few days my pycharm debug console is systematically polluted by teamcity infos, e.g. when i want to print out a variable value, the following will be displayed:
>>> df
##teamcity[testStdOut timestamp='2017-11-02T15:55:00.670' flowId='test_pricing_36_weight_provider.TestPricing36WeightProvider.test_add_weights' locati...
Any hint how to deactivate this teamcity thing ?
This is a PyCharm/IntelliJ bug.
PyCharm replaces stdout with its own object.
They do a bunch of screwing around with objects that get printed, resulting in:
some object are not printed to the prompt
some programs cannot be run if they expect stdout to be a file object and not a StringIO.
This is a long standing bug with PyCharm - there's really no excuse for it in a piece of software with a $500/yr license fee. Being able to debug my program in my IDE out of the box is way up there in my list of requirements.
The work around they recommend is to set:
JB_DISABLE_BUFFERING=1
... which you can do for all unit tests.
See: no output in console for unittests in pycharm 2017
Any chance you've installed a teamcity test runner or debugger of some kind in your pip requirements? The ##teamcity bit is a service message used to report output to a TeamCity build log.
Check this for more info about TC service messages: https://confluence.jetbrains.com/display/TCD9/Build+Script+Interaction+with+TeamCity
The robot scripts when ran on RIDE, generate output.xml, report.html etc files, once run is over.
Is there any way available to view logs when script is still running? (When I use pause on failure)
Also some times I had to Stop/Abort the run in middle, and no logs are generated in such cases.
Kindly help,
Thanks in advance
As for first part — RIDE runs tests adding own listener, providing more verboseness of the output and pausing/resuming functionality.The easiest thing is to run tests not from RIDE, but from console using robot/pybot script. In this case much less logs are written to output (though it doesn't provide pause/resume functionality).
For second part — robot (RIDE starts robot script — you can see it in execution log: command: pybot.bat...) generates output.xml file not after but during execution, so generated output.xml is not valid until test is finished. After normal execution rebot tool generating log.html automatically. So generally it's possible to take following steps:
"Fix" your incomplete output.xml file after execution stop with fixml. output.xml location for RIDE execution can be found in the very same execution log of yours (e.g. ...\appdata\local\temp\RIDEv_0yrp.d\ in my case)
Run rebot stand-alone: rebot output.xml --log log.html --report report.html. Rebot options description you can check using rebot --help (as usual)
Please also note that directory where RIDE output files are stored is temporary — exists only when RIDE is started. You will lose your output on exiting RIDE
I'm using RIDE 1.5 so maybe my answer is not valid for other versions
In RIDE, under Run Tab , when you are running the scripts , you have a option, show message log, it will shows the runtime log.
Try this out.
The below is the code present in parallel_tests.txt
*** Settings ***
Library Parallel
*** Test Cases ***
Runner
Run Parallel Tests Hello World
Hello
[Tags] parallel
Log Hello123
World
[Tags] parallel
Log World123
I am trying to execute this test case file
from terminal using jybot as below:
jybot parallel_tests.txt;
I am getting the following error:
No keyword with name 'Run Parallel Tests' found.
How to execute both test cases Hello and World ,in parallel in robot framework using parallel library.
Slightly different use case but have you tried using pabot?
I haven't gone down the rabbit hole with this one so don't have an in-depth understanding but I do use it for cross-browser tests via sauceLabs. It will work differently from how you want in that it does parallel runs but by test suite rather than test case level. Perhaps this is configurable? Unfortunately as mentioned I just know enough to get it working for my needs and haven't done a deep dive.
Any questions let me know and I'll try to help.
Here's a quick example as requested, I've stripped lots of our stuff which is useless in your case but left some in so you can see how to build up. I changed the processes parameter from %processes% to 2, the end result is the same, in my case these values are coming from a teamcity build configuration:
pabot --processes 2 --variable environment:%environment% --name OS_X_10.9-Firefox --variable browser:%browser% --listener robot-tc-listener --include %includetags% --exclude %exclusiontags% --outputdir %PabotResults_DIR% --output output.xml --report report.html --log log.html tests
Which version of RobotFramework are you using? The Parallel library was removed in 2.5 (see this ticket, though there may be distant plans of reintroducing it).
I tried to reproduce your issue using this file with RF version 2.8, but there are several compatibility problems, as expected.
Are you sure that the library has been loaded properly? When it fails, in addition to the error, you will also get a message telling you that the keywords are unavailable.
When running "Debug" in phpstorm on my project, for some reason it's started having this weird problem where phpstorm doesn't really pick up the test results, even though phpunit is not throwing an error.
Phpunit will generate a report in the phpstorm console that looks something like this:
[37;41m[2KFAILURES!
[0m[37;41m[2KTests: 3, Assertions: 6, Failures: 1.
[0m[2KGenerating code coverage report in Clover XML format ... done
Process finished with exit code 1
I'm guessing phpstorm can't interpret these results? It was working before and it would be nice to have full integration again.
Any help would be great
It looks like phpunit colorizes its output. Could that be the problem? Make sure not to run it with --colors
The command line documentation mentions an argument -x for debug output.
However I cannot find anything in the ant task documentation.
Is there a way to activate it there as well?
Use the standard ant -d switch to turn on the debug output.