How to get a simple phpunit codecoverage summary in text output? - phpunit

I use PHPUnit 9.5.7 on php 8.0.3
I would like to have a checker for minimum code coverage of lines as a git hook.
I have seen in online examples a simple 3 line output as a summary after running the tests. How do I get this output? I searched several articles and the config documentation but didn't find anything.
My aim is to deny the commit if a minimal coverage of lines isn't achieved. So if you have other ideas to get this done I am open.
Here is my current configuration file:
<?xml version="1.0" encoding="UTF-8"?>
<!-- https://phpunit.readthedocs.io/en/latest/configuration.html -->
<phpunit
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="tests/bootstrap.php"
stopOnError="false"
stopOnFailure="false"
stopOnIncomplete="false"
stopOnSkipped="false"
cacheResult="false"
printerClass="Codedungeon\PHPUnitPrettyResultPrinter\Printer"
>
<coverage>
<include>
<directory>src</directory>
</include>
</coverage>
<php>
<ini name="error_reporting" value="-1"/>
<server name="KERNEL_CLASS" value="\App\Kernel" />
<server name="APP_ENV" value="test" force="true"/>
<server name="SHELL_VERBOSITY" value="-1"/>
<server name="SYMFONY_PHPUNIT_REMOVE" value=""/>
<env name="SYMFONY_DEPRECATIONS_HELPER" value="weak"/>
<env name="CORS_ALLOW_ORIGIN" value="^https?://(localhost|127\.0\.0\.1)(:[0-9]+)?$"/>
</php>
<extensions>
<extension class="DAMA\DoctrineTestBundle\PHPUnit\PHPUnitExtension"/>
</extensions>
<testsuites>
<testsuite name="Project Test Suite">
<directory>tests</directory>
</testsuite>
</testsuites>
</phpunit>

What you refer to is the phpunit text-output for coverage:
--coverage-text=<file> Generate code coverage report in text format [default: standard output]
An alternative to it is a small script that you can run after running the tests with coverage to check on the gathered data.
Here an excerpt from a composer.json {"script": {}} section:
"unit-test": [
"#phpunit --log-junit build/log/junit.xml --coverage-clover build/log/clover.xml test/unit",
"#php -f lib/build/coverage-checker.php -- build/log/clover.xml"
],
Taken from an existing project which has the script:
https://github.com/ktomk/pipelines/blob/master/lib/build/coverage-checker.php
It is a maintained version that originated in this blog-post:
http://ocramius.github.io/blog/automated-code-coverage-check-for-github-pull-requests-with-travis/
The percentage to check for by default is 100%. You can pass it as the second positional argument if you would like to lower it.

Related

Phpunit starts all the tests or not starts all the tests, ignoring the config

My project contains 2 packages, and I want to run tests in only one of them. Used symfony 3.3 and phpunit 6.3.0
phpunit.xml.dist
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/6.3/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="./src/CoreBundle/Tests/autoloadWithIsolatedDatabase.php"
>
<php>
<ini name="error_reporting" value="-1" />
<server name="KERNEL_CLASS" value="AppKernel" />
</php>
<testsuites>
<testsuite name="App">
<directory>src/AppBundle/Tests</directory>
</testsuite>
</testsuites>
<filter>
<whitelist>
<directory>src</directory>
<exclude>
<directory>src/*Bundle/Resources</directory>
<directory>src/*Bundle/Tests</directory>
<directory>src/*/*Bundle/Resources</directory>
<directory>src/*/*Bundle/Tests</directory>
<directory>src/*/Bundle/*Bundle/Resources</directory>
<directory>src/*/Bundle/*Bundle/Tests</directory>
</exclude>
</whitelist>
</filter>
</phpunit>
And structure of the project
This configuration will run all the tests from AppBundle and CoreBundle (in the second there are no tests), and if you change
<directory>src/AppBundle/Tests</directory>
to
<directory>src/CoreBundle/Tests</directory>
then there will be no tests at all. I can not understand what's wrong
Let's start of by how your phpunit.xml.dist is configured. You have one test suite defined:
<testsuites>
<testsuite name="App">
<directory>src/AppBundle/Tests</directory>
</testsuite>
</testsuites>
This is the place phpunit will look into for tests. They have to conform to the usual conventions like having a file name ending in Test and each test method must be prefixed with a test.
Also from your screenshot I can gather that you have a top level tests/ folder (right next to app/, src/, etc.). This is probably where your other tests are placed in.
The second folder is where you should also place your tests from the AppBundle if you follow the best practices: https://symfony.com/doc/current/best_practices/tests.html
I think this was established sometime during the 3.x release cycle.
In theory you should be able to copy src/AppBundle/Tests to tests/AppBundle and hopefully everything still works. Now you can update your test suite configuration to:
<testsuites>
<testsuite name="App">
<directory>tests/</directory>
</testsuite>
</testsuites>
Your filter can stay in place as src/CoreBundle/Tests does not actually contain test-classes, only helpers used for tests.
Now that you have all tests in one big tests folder separated by bundle you might want to do a search on this folder for classes extending PHPUnit_Framework_TestCase. Since PHPUnit 6.0 introduced namespaces those need to be updated with PHPUnit\Framework\TestCase otherwise PHPUnit will ignore those tests.

PHPUnit Code Coverage not working and always saying incorrect whitelist config

Ich checked on several sources on the network and also here on stack overflow but couldn't solve it so far.
Heres my config:
<?xml version="1.0" encoding="UTF-8"?>
<!-- https://phpunit.de/manual/current/en/appendixes.configuration.html -->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/6.0/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="../../src/test.bootstrap.php"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnError="false"
stopOnFailure="false"
stopOnIncomplete="false"
stopOnSkipped="false"
stopOnRisky="false"
verbose="true"
>
<!--printerFile="vendor/whatthejeff/emoji-phpunit-resultprinter/src/Emoji/PHPUnit/ResultPrinter.php"-->
<!--printerClass="Emoji\PHPUnit\ResultPrinter"-->
<php>
<ini name="error_reporting" value="-1" />
<server name="KERNEL_DIR" value="src/" />
<env name="BOOTSTRAP_CLEAR_CACHE_ENV" value="testing"/>
</php>
<loggin>
<log type="coverage-html" target="build/coverage"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
<log type="coverage-crap4j" target="build/logs/crap4j.xml"/>
<log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
</loggin>
<testsuites>
<testsuite name="Project Test Suite">
<directory>src/*/*Bundle/Tests</directory>
<directory>src/*/Bundle/*Bundle/Tests</directory>
</testsuite>
</testsuites>
<filter>
<whitelist>
<directory>../src</directory>
<exclude>
<directory>src/*/*Bundle/Resources</directory>
<directory>src/*/*Bundle/Tests</directory>
<directory>src/*/Bundle/*Bundle/Resources</directory>
<directory>src/*/Bundle/*Bundle/Tests</directory>
</exclude>
</whitelist>
</filter>
</phpunit>
Due to the Build Server limitations I need to call phpunit with the following command:
./build/bin/phpunit.phar -c build/config/phpunit.xml --coverage-clover build/logs/clover.xml --coverage-crap4j build/logs/crap4j.xml --coverage-html build/coverage --coverage-xml build/logs/coverage.xml --log-junit build/logs/junit.xml --testdox-html build/testdox/testdox.html --testdox-xml build/logs/testdox.xml src/ -v
Everything works except code coverage heres the Result Output:
PHPUnit 6.1.4 by Sebastian Bergmann and contributors.
Runtime: PHP 7.1.5-1+0~20170522123046.25+jessie~1.gbpb8686b with Xdebug 2.6.0-dev
Configuration: /home/testcase/build-directories/build/config/phpunit.xml
Error: Incorrect whitelist config, no code coverage will be generated.
.................. 18 / 18 (100%)
Time: 945 ms, Memory: 6.00MB
OK (18 tests, 68 assertions)
I'm a little bit irritated as unit tests itself are working.
The Reds Message can be ignored (Hope so) as the Redis server is not yet fully configured.
I'm using oh-unit 6.1.4
Please give me a clue where the error is in my configuration
Try to change the whitelist deirectory into you phpunit with this:
<whitelist>
<directory suffix=".php">./src</directory>
<exclude>
<directory>./src/*/*Bundle/Resources</directory>
<directory>./src/*/*Bundle/Tests</directory>
<directory>./src/*/Bundle/*Bundle/Resources</directory>
<directory>./src/*/Bundle/*Bundle/Tests</directory>
</exclude>
</whitelist>
The directory given in the filter/whitelist/directory XML node is relative to the directory that contains the phpunit configuration file.
I think you should use ../../src.
I have the same issue, so I have update the whitelist directory path in phpunit.xml
<whitelist>
<directory>./web/core/includes</directory>
<directory>./web/core/lib</directory>
<!-- Extensions can have their own test directories, so exclude those. -->
<directory>./web/core/modules</directory>
<exclude>
<directory>./web/core/modules/*/src/Tests</directory>
<directory>./web/core/modules/*/tests</directory>
</exclude>
<directory>./web/modules</directory>
<exclude>
<directory>./web/modules/*/src/Tests</directory>
<directory>./web/modules/*/tests</directory>
<directory>./web/modules/*/*/src/Tests</directory>
<directory>./web/modules/*/*/tests</directory>
</exclude>
<directory>./web/sites</directory>
</whitelist>

"No whitelist configured...". But thats not true. <whitelist> defined in Symfony app\phpunit.xml.dist. (This error occur only in Netbeans, not in cmd)

(I'm beginner in PHPUnit)
In Netbeans I try to code-coverage PHPUnit in Symfony2.8 project, but it throws error:
"C:\wamp\www\treningPHPUnitSymfony2.8\bin\phpunit.bat" "--colors" "--log-junit" "C:\Users\chiny\AppData\Local\Temp\nb-phpunit-log.xml" "--coverage-clover" "C:\Users\chiny\AppData\Local\Temp\nb-phpunit-coverage.xml" "C:\Program Files\NetBeans 8.1\php\phpunit\NetBeansSuite.php" "--" "--run=C:\wamp\www\treningPHPUnitSymfony2.8\src\TreningBundle\Tests\Utils\CalculatorTest.php"
PHPUnit 5.3.4 by Sebastian Bergmann and contributors.
Error: No whitelist configured, no code coverage will be generated
................I 17 / 17 (100%)
Time: 531 ms, Memory: 4.00MB
OK, but incomplete, skipped, or risky tests!
Tests: 17, Assertions: 16, Incomplete: 1.
Done.
But i have Symfony default app\phpunit.xml.dist with <whitelist> defined :
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/4.8/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="autoload.php"
>
<php>
<ini name="error_reporting" value="-1" />
<!--
<server name="KERNEL_DIR" value="/path/to/your/app/" />
-->
</php>
<testsuites>
<testsuite name="Project Test Suite">
<directory>../src/*/*Bundle/Tests</directory>
<directory>../src/*/Bundle/*Bundle/Tests</directory>
<directory>../src/*Bundle/Tests</directory>
</testsuite>
</testsuites>
<filter>
<whitelist>
<directory>../src</directory>
<exclude>
<directory>../src/*Bundle/Resources</directory>
<directory>../src/*Bundle/Tests</directory>
<directory>../src/*/*Bundle/Resources</directory>
<directory>../src/*/*Bundle/Tests</directory>
<directory>../src/*/Bundle/*Bundle/Resources</directory>
<directory>../src/*/Bundle/*Bundle/Tests</directory>
</exclude>
</whitelist>
</filter>
</phpunit>
I got:
Symfony2.8, PHPUnit5.3.4, Netbeans 8.1
edit
But in windows console command
phpunit -c app/ src/TreningBundle/ --coverage-html=cov/ works great, generate coverage.
Unfortunately the "No whitelist configured, no code coverage will be generated" message is also printed when the whitelist configuration is invalid, see https://github.com/sebastianbergmann/phpunit/issues/2049 for details.

Speeding up PHPUnit startup when code coverage is not required

I am using PHPUnit in my Codeception unit tests. I am not interested in code coverage yet, so I would like to completely disable it, especially because it delays my tests by 8..12 seconds. This becomes annoying when tests are configured to be run automatically when files change.
I debugged PHPUnit code to see why it is starting up so long and found out that it spends up to 12 seconds inside getCodeCoverageFilter looping through getBlacklistedDirectories and collecting filenames calling addDirectoryToBlacklist.
Is there any way to disable processing getCodeCoverageFilterin Codeception or PHPUnit itself without directly hacking its code?
Here is my current phpunit.xml at the root of my Laravel 5 project:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
backupStaticAttributes="false"
bootstrap="bootstrap/autoload.php"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false"
syntaxCheck="false">
<testsuites>
<testsuite name="Application Test Suite">
<directory>./tests/</directory>
</testsuite>
</testsuites>
<filter>
<blacklist>
<directory>./vendor/</directory>
<directory>./database/</directory>
<directory>./public/</directory>
<directory>./resources/</directory>
<directory>./storage/</directory>
<directory>./tests/</directory>
</blacklist>
<whitelist>
<directory suffix=".php">app/</directory>
</whitelist>
</filter>
<php>
<env name="APP_ENV" value="testing"/>
<env name="CACHE_DRIVER" value="array"/>
<env name="SESSION_DRIVER" value="array"/>
<env name="QUEUE_DRIVER" value="sync"/>
</php>
</phpunit>
Just remove the line
<log type="coverage-html" target="coverage"/>
from your phpunit.xml
In fact, I use two xml files.
One standard phpunit.xml that is used on a remote code inspection service, and one that I specifically named phpunit_no_code_coverage.xml without code coverage, that I use locally while developing.
You can specify which xml file to use via phpunit's c flag, e.g:
./phpunit -c tests/phpunit_no_code_coverage.xml --testsuite suite_name
The result is rather huge, my testsuite runs now rather fast, on average taking ~15 seconds, whereas it before took 110 seconds.
According to codeception documentation, the code coverage is enabled in codeception.yml. Try:
coverage:
enabled: false
Or removing the key.

How to exclude unused libraries from code coverage report?

There's something I apparently don't understand about PHPUnit's code coverage whitelisting.
I have the following filter for a couple of folders:
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
</whitelist>
<whitelist addUncoveredFilesFromWhitelist="false">
<directory suffix=".php">./lib</directory>
</whitelist>
</filter>
"app" - I want to know about anything not covered in here so I've enabled addUncoveredFilesFromWhitelist for the core application. (this works as expected)
"lib" - I want to see what was covered in files used from here but there will be a lot that isn't used, so I've disabled addUncoveredFilesFromWhitelist for the library files. (this I have a problem with)
What happens is that library files not included at runtime show up in the reports as unexecuted code. I can verify the code is not included during test by adding lines that would throw fatal errors.
This result seems contradictory to what the PHPUnit docs say, but no doubt I'm doing it wrong. Can anyone explain how I can include my executed library code, but only that which was required during test?
You can use the filter node in the phpunit.xml. I have used the following file in a project. It whitelist's local files:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
backupStaticAttributes="false"
syntaxCheck="false"
bootstrap="tests/bootstrap.php"
>
<php>
<includePath>lib/php</includePath>
</php>
<filter>
<whitelist>
<directory suffix=".php">lib/php/</directory>
</whitelist>
</filter>
<testsuites>
<testsuite name="Jm_Log">
<directory suffix="Test.php">tests</directory>
</testsuite>
</testsuites>
<logging>
<log type="coverage-html" target="build/coverage" title="jam"
charset="UTF-8" yui="true" highlight="true"
lowUpperBound="35" highLowerBound="70"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
<log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
</logging>
</phpunit>
Note: You can also use a blacklist.

Resources