If you have multiple test suites configured in phpunit.xml how do you run more than one test suite but not all of them from the command line?
phpunit.xml
<?xml version="1.0" encoding="utf-8"?>
<phpunit
backupGlobals="false"
backupStaticAttributes="false"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="true"
syntaxCheck="true"
bootstrap="tests/bootstrap.php">
<testsuites>
<testsuite name="Unit">
<directory suffix="Test.php">tests/unit</directory>
</testsuite>
<testsuite name="Integration">
<directory suffix="Test.php">tests/integration</directory>
</testsuite>
<testsuite name="Acceptance">
<directory suffix="Test.php">tests/acceptance</directory>
</testsuite>
</testsuites>
<logging>
<log type="coverage-html" target="build/coverage"/>
<log type="testdox-html" target="build/requirements.html"/>
</logging>
<filter>
<whitelist>
<directory suffix=".php">src</directory>
</whitelist>
</filter>
</phpunit>
example
phpunit --testsuite Unit|Integration but not Acceptance
Since PHPUnit 6.0 you can specify a comma delimited list of suites:
--testsuite suite1,suite2
Original answer for versions < PHPUnit 6.0
It doesn't work as you expect it to i.e. (where <pattern> is not an actual pattern):
--testsuite <pattern> Filter which testsuite to run.
You can choose a test suite to run but you can't use a pattern to filter which ones to run.
A better description would be --testsuite <name> Which testsuite to run.
Issue report https://github.com/sebastianbergmann/phpunit/issues/2273, which was fixed in version 6.0.
Related
When run command phpunit --bootstrap="TestHelper.php" --coverage-clover coverage.xml --whitelist="../../../" --debug --log-junit result.xml ./
It generates result.xml but console get stuck after this line.
Generating code coverage report in Clover XML format ...
Note : I already checked previous quetion on stackoverflow.
My phpunit.xml :
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="./TestHelper.php"
backupGlobals="false"
backupStaticAttributes="false"
verbose="true"
colors="false"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false"
syntaxCheck="true">
<testsuite name="Phalcon - Testsuite">
<directory>./</directory>
</testsuite>
<filter>
<whitelist processUncoveredFilesFromWhitelist="false">
<directory suffix=".php">../app/repositories/Account</directory>
<directory suffix=".php">../app/repositories/Call</directory>
<directory suffix=".php">../app/repositories/Credit</directory>
<directory suffix=".php">../app/repositories/User</directory>
</whitelist>
</filter>
<logging>
<log type="coverage-clover" target="./coverage.xml"/>
<log type="junit" target="./results.xml" logIncompleteSkipped="false"/>
</logging>
</phpunit>
I am trying to define my phpunit.xml configuration with a custom test suite, which loads all Unit tests in desired folder recursively.
Folder path examples:
tests/path/Unit/MyTest.php
tests/path/path/Unit/MyTest.php
tests/path/path/path/Unit/MyTest.php
tests/path/path/path/path/Unit/MyTest.php
...
tests/{.../}/Unit/MyTest.php
I can define my suite like this:
<testsuites>
<testsuite name="Project Unit Test Suite">
<directory>tests/*/Unit</directory>
<directory>tests/*/*/Unit</directory>
<directory>tests/*/*/*/Unit</directory>
<directory>tests/*/*/*/*/Unit</directory>
...
</testsuite>
</testsuites>
Is there a way to iterate over all sub-folders that my expression would be only a single line?
I should use ** in this case. I'd try to name directories the same inside different sub-directories, so that ** would work everywhere.
<?xml version="1.0"?>
<phpunit colors="true" backupGlobals="false">
<testsuites>
<testsuite name="Test Suite">
<directory>tests</directory>
<exclude>**/Functional</exclude>
</testsuite>
</testsuites>
</phpunit>
or if you'd rather include things:
<?xml version="1.0"?>
<phpunit colors="true" backupGlobals="false">
<testsuites>
<testsuite name="Test Suite">
<directory>**/Unit</directory>
<exclude>*</exclude>
</testsuite>
</testsuites>
</phpunit>
Ich checked on several sources on the network and also here on stack overflow but couldn't solve it so far.
Heres my config:
<?xml version="1.0" encoding="UTF-8"?>
<!-- https://phpunit.de/manual/current/en/appendixes.configuration.html -->
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="http://schema.phpunit.de/6.0/phpunit.xsd"
backupGlobals="false"
colors="true"
bootstrap="../../src/test.bootstrap.php"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnError="false"
stopOnFailure="false"
stopOnIncomplete="false"
stopOnSkipped="false"
stopOnRisky="false"
verbose="true"
>
<!--printerFile="vendor/whatthejeff/emoji-phpunit-resultprinter/src/Emoji/PHPUnit/ResultPrinter.php"-->
<!--printerClass="Emoji\PHPUnit\ResultPrinter"-->
<php>
<ini name="error_reporting" value="-1" />
<server name="KERNEL_DIR" value="src/" />
<env name="BOOTSTRAP_CLEAR_CACHE_ENV" value="testing"/>
</php>
<loggin>
<log type="coverage-html" target="build/coverage"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
<log type="coverage-crap4j" target="build/logs/crap4j.xml"/>
<log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
</loggin>
<testsuites>
<testsuite name="Project Test Suite">
<directory>src/*/*Bundle/Tests</directory>
<directory>src/*/Bundle/*Bundle/Tests</directory>
</testsuite>
</testsuites>
<filter>
<whitelist>
<directory>../src</directory>
<exclude>
<directory>src/*/*Bundle/Resources</directory>
<directory>src/*/*Bundle/Tests</directory>
<directory>src/*/Bundle/*Bundle/Resources</directory>
<directory>src/*/Bundle/*Bundle/Tests</directory>
</exclude>
</whitelist>
</filter>
</phpunit>
Due to the Build Server limitations I need to call phpunit with the following command:
./build/bin/phpunit.phar -c build/config/phpunit.xml --coverage-clover build/logs/clover.xml --coverage-crap4j build/logs/crap4j.xml --coverage-html build/coverage --coverage-xml build/logs/coverage.xml --log-junit build/logs/junit.xml --testdox-html build/testdox/testdox.html --testdox-xml build/logs/testdox.xml src/ -v
Everything works except code coverage heres the Result Output:
PHPUnit 6.1.4 by Sebastian Bergmann and contributors.
Runtime: PHP 7.1.5-1+0~20170522123046.25+jessie~1.gbpb8686b with Xdebug 2.6.0-dev
Configuration: /home/testcase/build-directories/build/config/phpunit.xml
Error: Incorrect whitelist config, no code coverage will be generated.
.................. 18 / 18 (100%)
Time: 945 ms, Memory: 6.00MB
OK (18 tests, 68 assertions)
I'm a little bit irritated as unit tests itself are working.
The Reds Message can be ignored (Hope so) as the Redis server is not yet fully configured.
I'm using oh-unit 6.1.4
Please give me a clue where the error is in my configuration
Try to change the whitelist deirectory into you phpunit with this:
<whitelist>
<directory suffix=".php">./src</directory>
<exclude>
<directory>./src/*/*Bundle/Resources</directory>
<directory>./src/*/*Bundle/Tests</directory>
<directory>./src/*/Bundle/*Bundle/Resources</directory>
<directory>./src/*/Bundle/*Bundle/Tests</directory>
</exclude>
</whitelist>
The directory given in the filter/whitelist/directory XML node is relative to the directory that contains the phpunit configuration file.
I think you should use ../../src.
I have the same issue, so I have update the whitelist directory path in phpunit.xml
<whitelist>
<directory>./web/core/includes</directory>
<directory>./web/core/lib</directory>
<!-- Extensions can have their own test directories, so exclude those. -->
<directory>./web/core/modules</directory>
<exclude>
<directory>./web/core/modules/*/src/Tests</directory>
<directory>./web/core/modules/*/tests</directory>
</exclude>
<directory>./web/modules</directory>
<exclude>
<directory>./web/modules/*/src/Tests</directory>
<directory>./web/modules/*/tests</directory>
<directory>./web/modules/*/*/src/Tests</directory>
<directory>./web/modules/*/*/tests</directory>
</exclude>
<directory>./web/sites</directory>
</whitelist>
There's something I apparently don't understand about PHPUnit's code coverage whitelisting.
I have the following filter for a couple of folders:
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">./app</directory>
</whitelist>
<whitelist addUncoveredFilesFromWhitelist="false">
<directory suffix=".php">./lib</directory>
</whitelist>
</filter>
"app" - I want to know about anything not covered in here so I've enabled addUncoveredFilesFromWhitelist for the core application. (this works as expected)
"lib" - I want to see what was covered in files used from here but there will be a lot that isn't used, so I've disabled addUncoveredFilesFromWhitelist for the library files. (this I have a problem with)
What happens is that library files not included at runtime show up in the reports as unexecuted code. I can verify the code is not included during test by adding lines that would throw fatal errors.
This result seems contradictory to what the PHPUnit docs say, but no doubt I'm doing it wrong. Can anyone explain how I can include my executed library code, but only that which was required during test?
You can use the filter node in the phpunit.xml. I have used the following file in a project. It whitelist's local files:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit backupGlobals="false"
backupStaticAttributes="false"
syntaxCheck="false"
bootstrap="tests/bootstrap.php"
>
<php>
<includePath>lib/php</includePath>
</php>
<filter>
<whitelist>
<directory suffix=".php">lib/php/</directory>
</whitelist>
</filter>
<testsuites>
<testsuite name="Jm_Log">
<directory suffix="Test.php">tests</directory>
</testsuite>
</testsuites>
<logging>
<log type="coverage-html" target="build/coverage" title="jam"
charset="UTF-8" yui="true" highlight="true"
lowUpperBound="35" highLowerBound="70"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
<log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
</logging>
</phpunit>
Note: You can also use a blacklist.
I get the message no tests executed when i try to do
phpunit
or
phpunit -c phpunit.xml
on the other hand, if i do
phpunit -c phpunit.xml ./tests
It works. But this is a problem considering that some tool that I'm using does not handle this well.
Directory structur
code
tests/
phpunit.xml
autoloader.php
And here is the config file
<phpunit backupGlobals="false"
backupStaticAttributes="false"
colors="true"
convertErrorsToExceptions="true"
convertNoticesToExceptions="true"
convertWarningsToExceptions="true"
processIsolation="false"
stopOnFailure="false"
syntaxCheck="true"
bootstrap="./tests/bootstrap.php"
>
<testsuites>
<testsuite name="StdTestSuite">
<directory>
tests/
</directory>
</testsuite>
</testsuites>
</phpunit>
The problem was the stupidest thing ever.
You can't have whitespace inside the tag.
So what you need to have in you phpunit.xml file is this
...
<testsuite name="StdTestSuite">
<directory>tests/</directory>
</testsuite>
...
Then running
phpunit
Should work