In my project I have a PHPUnit configuration with whitelisting for my code coverage and also some directories excluded. When executing PHPUnit with this configuration and output the code coverage, I only get the whitelisted files in the code coverage.
When I do the same within PhpStorm, it shows code coverage of all files instead of only the whitelisted files.
An example of my PHPUnit configuration:
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">../../../../Classes/*</directory>
<exclude>
<directory suffix=".php">../../../../Tests</directory>
</exclude>
</whitelist>
</filter>
See the screenshot: it shows that 0% of my files are tested in the Test folder, what is correct because I excluded that folder. But when excluded I don't want to see it as well.
AFAIK you cannot do anything about it -- it's how PhpStorm works right now. If clover coverage report mentions any file then parent folder of such file will have coverage info (0% in our case).
https://youtrack.jetbrains.com/issue/WI-16792 -- watch this ticket (star/vote/comment) to get notified on any progress.
Related
I have a directory tests where I store all my tests.
Hierarchy of tests dirrectory
tests->
ApplicationTests->
IntegrationTests->
Factory->
Service->
UnitTests->
How can I make phpunit launch tests only from for example Service directory, instead of the whole tests directory? I read about #Groups, but I think that's not what I'm looking for and it would be best if I would not need to edit config files, but some how, do it from the command line.
Found a quite easy way, even with config editing.
phpunit.xml
See: The XML Configuration File Phpunit Docs and Organizing Tests Phpunit Docs<
<testsuite name="Service">
<directory>tests/IntegrationTests/Service</directory>
</testsuite>
Command line:
php ./vendor/bin/phpunit --testdox --testsuite Service
I'm playing for a week with phpUnit.
I'm slowly going forward with documentation at:
https://phpunit.readthedocs.io
At this point I'm at code Coverage. I've managed to generate small test --coverage-html (via console). I want everything to work via phpStorm.
I'm struggling with inclusion paths. I can see errors in console, but these aren't to helpful at all.
This is how my console output looks like:
This is the only place I use this file in
This is how folder structure for tested and displayed (in console) file looks like
|- dir:Boostrap
|- dir:Coverage
|- dir:Database
|- dir:Interfaces
|- dir:Methods
|---- file: BasicCalculations.php (line 3 inclusion)
|- dir:Tests
|---- file:DataDisplayingTest.php (file that I'm testing)
|---- dir:Data Providers
|-------- file:BasicCalculationDataProvider.php (line 4 inclusion)
What I've tried/What I've made so far
Cannot find PHPUnit in include path phpstorm (I'm not doing this via composer/vendor so this is not helping, my remote machine is for all the debugging, calculations etc).
PHPUnit test suite include path - this gave me idea to play around with bootstrap file, where I've included all the required files, but again this worked only via manually running tests in console - I want to make it work in phpStorm remotely.
PHPUnit's whitelist and blacklist seem to be ignored . At this point situation looks like this
without processUncoveredFilesFromWhitelist="true" I've got no inclusion error, but I got just more errors after that:
It looks like it's trying to work as console gets green for a moment and I've got the Coverage panel now, but I bet this errors shouldn't be there. I can't be sure If coverage report is being displayed correctly at all.
PHPUnit error "Failed to open stream: No such file or directory" - I've tried the DIR as You can see,
https://github.com/sebastianbergmann/phpunit/issues/1932 - this helped me a bit, now I know I can/have to import xml, and this way I don’t have to generate coverage report remotely via ssh
This is how my phpunit.xml looks like:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="phpunit.xsd"
cacheResult="true"
verbose="true">
<filter>
<whitelist processUncoveredFilesFromWhitelist="true">
<directory>/var/www/html/phpUnit</directory>
</whitelist>
</filter>
<php>
<includePath>/var/www/html/phpUnit</includePath>
</php>
</phpunit>
I've played around with directory/incluedPath, tried variation like:
- /var/www/html/phpUnit
- /var/www/html/phpUnit/
- .
- ./
- <file>pointed/tested/file/here/</file>
I'm working with:
phpStorm
phpUnit 7.x Remotely
php 7.x Remotely
xdebug Remotely
To be more clear:
what am I doing wrong?
how can I deal with inclusions problems?
what is causing all this inclusion path problems?
I’ve manage to solve all the problems I had. I’ve used some of the informations provided in the links I’ve pointed above.
First of all
Include errors
PhpUnit xml uses directive includePath, which in my case looked like this:
<php>
<includePath>/var/www/html/phpUnit</includePath>
</php>
Generally at this point the problem is with… existence of includePath in xml file. This attribute changes the inclusion path.
So lets say that You got project structure like that:
- dir: Classes
–- dir: A
–-- file: A.php class: A (extends B)
–- dir: B
–-- file: B.pphp class: B
-file: index.php
So from the look of file A.php You would need to include B.php like that:
../B/B.php
Since the working directory is
/var/www/html/phpUnit/Classes/
But now since You’ve set up inclusion path to:
var/www/html/phpUnit
File A, tries to load file B from the perspective of phpUnit folder, and it’s kinda looking for file in:
var/www/html
Not having this directive is not solving the problem as phpUnit seems to use some other default path.
I’ve solved this problem by changing the way I include files in project:
Just instead using:
include_once '../Interfaces/BasicCalculationsInterface.php';
I’ve started doing it like this:
include_once __DIR__.'/../Interfaces/BasicCalculationsInterface.php';
This way:
Single files tests work fine
Project itself works fine
phpStorm detects methods, classes etc in included file
Group tests work well too
Writing file index.html permission denied
I’ve stumbled upon this problem as well. This actually is some kind of phpStorm issue, which I don’t know how to solve permanently but I’ve dealt with it for the xml file from which I can run all my tests.
Basically phpStorm added some default configurations for executed tests.
In menu go to
Run/Edit Configurations
Take a look at field Test Runner options.
In my case phpStorm added
--coverage-html /
Everything would be fine, but I use Ubuntu on laptop as remote host, and phpStorm tries this way to create files in / directory for which there is no writing permission. Changing this to some writeable folder or removing this line solved the problem.
And that’s all, this is how my xml file looks like at this point (just in case someone would like to have something to look at)
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="phpunit.xsd"
verbose="true">
<filter>
<whitelist addUncoveredFilesFromWhitelist="false">
<directory suffix=".php">/var/www/html/phpUnit</directory>
</whitelist>
</filter>
<logging>
<log type="coverage-clover" target="/var/www/html/phpunit/coverage.xml" lowUpperBound="35" highLowerBound="70"/>
<log type="coverage-html" target="/var/www/html/phpUnit/phpStormCoverageTest" lowUpperBound="35"
highLowerBound="70"/>
</logging>
<testsuites>
<testsuite name="allTests">
<directory suffix="Test.php">/var/www/html/phpUnit/Tests</directory>
</testsuite>
</testsuites>
</phpunit>
Preview of working html/phpStorm coverage
I would like to have different configuration files for PHPUnit for my
Unit Tests: no database available, no cache, no everything
integration tests: everything is there
AND
to be able to execute these tests in PhpStorm all together or separated per file with automatically the right configuration.
There are several opportunities, which do not work for all needs:
Option 1:
Multiple phpunit.xml files
In PhpStorm can only set one PHPUnit default configuration, for executing a single test file, this will not work.
Option 2: Using a PHPUnit_Framework_BaseTestListener::startTestSuite($suite).
This works with a single phpunit.xml and if you are just executing the whole test suite, this works. But when you want to execute a single test file in PhpStorm, you do not have a $suite available and can not load the right configuration.
How do you handle different test configurations with PhpStorm?
Because there seems to be no offical possiblity on PHPStorm, I figured out a way by using a PHPUnit TestListener:
https://sebastianviereck.de/run-phpunit-unit-integrations-test-configurations-phpstorm/
PHPStorm can handle unit tests and integration tests in the same phpunit.xml file
Create two separate test suites in the same xml file. Each suite must have a distinct name in the name attribute.
Create two separate configurations for the tests
in PHPStorm (one for the unit tests and one for the integration tests) and indicate the same xml configuration file path for both tests
In the "Test Runner options" field, indicate the test suite name (the same as the one in the xml file) like this: --testsuite unit_tests
<testsuites>
<testsuite name="unit_tests">
<directory>/foo</directory>
</testsuite>
<testsuite name="integration_tests">
<directory>/bar</directory>
</testsuite>
</testsuites>
Once you have done that you can run unit tests and integration tests separately and still have them configured in the same xml file
Some of my testcases use a custom testing library. Also these testcases are very slow. So I would like to run them only in the build server and not in my local. I want to run the other tests locally.
Following is the directory structure. The ones inside the slow directory are the slow test cases that should be excluded.
/tests/unit-tests/test-1.php
/tests/unit-tests/test-2.php
/tests/unit-tests/slow/test-1.php
/tests/unit-tests/slow/test-2.php
/tests/unit-tests/foo/test-1.php
/tests/unit-tests/bar/test-2.php
I tried creating groups using #group annotation. This works, but the issue is that these test files are getting loaded (tests not executed though). Since they require the test library which is not installed locally, it is giving an error.
What is the best way to create the phpunit.xml configuration such that these slow tests are excluded (and not even loaded) by default and can be executed if needed?
There are 2 options:
In your phpunit.xml create 2 test suits - one for CI server and one for local development
<testsuites>
<testsuite name="all_tests">
<directory>tests/unit-tests/*</directory>
</testsuite>
<testsuite name="only_fast_tests">
<directory>tests/unit-tests/*</directory>
<!-- Exclude slow tests -->
<exclude>tests/unit-tests/slow</exclude>
</testsuite>
</testsuites>
So on CI server you can run
phpunit --testsuite all_tests
And locally
phpunit --testsuite only_fast_tests
Obviously, you can name test suites as you want.
I think preferable way is:
Create phpunit.xml.dist and configure default execution of phpunit (for CI server and all those who just cloned repository)
Modify phpunit.xml by configuring your local execution of phpunit
(by adding <exclude>tests/unit-tests/slow</exclude> to default
testsuite)
Exclude phpunit.xml from version control.
From the docs:
If phpunit.xml or phpunit.xml.dist (in that order) exist in the
current working directory and --configuration is not used, the
configuration will be automatically read from that file.
Some links:
The XML Configuration File. Test Suites
How to run a specific phpunit xml testsuite?
As far as I understand one can exclude files and directories from code coverage by name but is it possible to exclude by matching a pattern ?
I ask this because I don't want to mention the path to all my view sub-folders one by one in my phpunit.xml. Something like <exclude>*/view</exclude> perhaps ? By this I mean "exclude all view folders wherever you reach them".
Yes, you can specify file patterns to exclude from Code Coverage. Here is a sample of one of my PHPUnit.xml.dist files.
<!-- Add files not covered with tests into Code Coverage Analysis -->
<filter>
<whitelist processUncoveredFilesFromWhitelist="true">
<directory suffix=".class">.</directory>
<directory suffix=".fn">.</directory>
<directory suffix=".php">.</directory>
<exclude>
<directory>ExternalLibraries</directory>
</exclude>
</whitelist>
</filter>
The PHPUnit Manual for Code Coverage and then the Including/Excluding files for Code Coverage sections of the manual will show you additional options to specify the file patterns by directory or name.