PHPUnit passing a test with no assertions within config - phpunit

I want to pass tests which get the following: "This test did not perform any assertions"
I know I could add something like assertTrue(true) however is it possible to add something to the config to make these tests pass cleaner?
I'm pretty sure this only happens since version PHPUnit 3.5.0 with the introduction of
--strict

Edit: You've got a few choices depending on which version you're using, whether you want to ignore all risky tests or just a few, and if you want it to be permanent or temporary.
Prior to 5.6, if you didn't want to add bogus assertions to all your tests, you had to avoid passing --strict to PHPUnit or add strict="false" to your phpunit.xml. The point of this option is to "Mark a test as incomplete if no assertions are made."
At some point, PHPUnit added the related --dont-report-useless-tests command-line switch and beStrictAboutTestsThatDoNotTestAnything="false" config option. I haven't checked if they are replacements or additional fine-grained versions.
The above options affect all risky tests. Using them will leave you open to accidentally writing tests without assertions. The following new options are safer as you have to purposefully mark each risky test that you wish to allow.
PHPUnit 5.6 added the #doesNotPerformAssertions annotation to mark individual test cases as "not risky" even though they perform no assertions.
/**
* #doesNotPerformAssertions
*/
public function testWithoutAsserting() {
$x = 5;
}
PHPUnit 7.2 introduced TestCase::expectNotToPerformAssertions() which does the same thing.
public function testWithoutAsserting() {
$this->expectNotToPerformAssertions();
$x = 5;
}

Use the #doesNotPerformAssertions annotation:
/**
* #doesNotPerformAssertions
*/
public function testCodeWithoutUsingAssertions()
{
// do stuff...
}

With PHPUnit 7.2 you get another alternative:
public function testCodeWithoutUsingAssertions()
{
$this->expectNotToPerformAssertions();
// do stuff...
}
See also https://github.com/sebastianbergmann/phpunit/pull/3042.

Make use of $this->addToAssertionCount(1). See below.
class NoAssertTest extends PHPUnit_Framework_TestCase
{
function testWithoutAssertions() {
$x = 5;
// Increment the assertion count to signal this test passed.
// This is important if you use a #depends on this test
$this->addToAssertionCount(1);
}
}

If you have PHP SimpleTest, another approach would be to use:
$this->pass();
This will mark the test as completed and passed.
On the other hand, for a test that you want to fail, you can use:
$this->fail();
For example:
if (someComplicatedLogicValidation) {
// Do more stuff and asserts
} else{
$this->fail();
}
I tried this in PHP 5.5 and works:
function testThatWorks() {
$this->pass();
}
function testThatFails() {
$this->fail();
}
Output:
1) Fail at [...ExampleTest.unit.php line 23]
in testThatFails
in ExampleTest
in .../ExampleTest.unit.php
fail in 3.02s
Maybe the method pass can still be easily implemented in PHPUnit. Source from SimpleTest:
function pass($message = "Pass") {
if (! isset($this->reporter)) {
trigger_error('Can only make assertions within test methods');
}
$this->reporter->paintPass(
$message . $this->getAssertionLine());
return true;
}

In my case, the test has no assertion because I'm just checking everything using prophecy lib to mock and check that methods are being called as expected...
PHPUnit by default expects that test has at least one assertion, and warn you to put it.
Just adding this line in any part of your test is enough.
$this->expectNotToPerformAssertions();
But also you can design a test that in some cases have assertions or not
public function testCodeNoAssertions()
{
if ($something === true) {
$this->expectNotToPerformAssertions();
}
// do stuff...
if ($somethingElse === true) {
//do any assertion
}
}
In conclusion, It's not a bug... It's a feature.

PHPUnit 7.2+. $this->expectNotToPerformAssertions() can be used, if you have conditions, that might disable assertions. Seems it is not in the official docs, but can be found here: https://github.com/sebastianbergmann/phpunit/pull/3042/files
public function testCodeNoAssertions()
{
$this->expectNotToPerformAssertions();
}
At least PHPUnit 7.0+. #doesNotPerformAssertions can be used. Current docs do not state when this was added. https://phpunit.readthedocs.io/en/7.0/annotations.html#doesnotperformassertions
/**
* #doesNotPerformAssertions
*/
public function testCodeWithoutUsingAssertions()
{
}
If your test is simply incomplete, you should be using $this->markTestIncomplete() https://phpunit.readthedocs.io/en/7.0/incomplete-and-skipped-tests.html
public function testCodeThatIsIncomplete()
{
$this->markTestIncomplete('Implementation of this test is not complete.');
}
If you want to temporarily disable a test $this->markTestSkipped() should be used. https://phpunit.readthedocs.io/en/7.0/incomplete-and-skipped-tests.html#skipping-tests
public function testCodeThatIsDisabledTemporarily()
{
$this->markTestSkipped('This test is disabled to test, if it is interfering with other tests.');
}

I created a superclass and placed SimpleTest -like function there:
/**
* Pass a test
*/
public function pass()
{
//phpunit is missing pass()
$this->assertTrue(true);
}
Then you can call it
$this->pass();

Related

PHPUnit: how to use assertions outside of the "test..." methods?

I have the following code:
private function registerShutdownFunction(): void
{
register_shutdown_function(function () {
$this->dropDatabasesAndUsersIfExist();
});
}
And this code:
private function dropDatabasesAndUsersIfExist(): void
{
// some code for deletion of the databases...
foreach ($connections as $connection) {
$this->assertNotContains($connection, $databases);
}
}
But dropDatabasesAndUsersIfExist is not a "test..." method. And phpunit ignores assertions outside of the test methods.
And seems there are problems may occur, because this shutdown function running directly before the die of the script...
You can use PHPUnit's Assert class outside of test cases if that is really what you want to do:
PHPUnit\Framework\Assert::assertNotContains($connection, $databases);
Edit: After reading your question one more time I'm not really sure if my answer helps you. If I got you right, you are already using the assertion but it did not behave as you'd expect it. My guess is that you want the whole test run to fail if any of the assertions in dropDatabasesAndUsersIfExist was not met.
One solution could be to move the checks you are doing in dropDatabasesAndUsersIfExist to a separate test class that should be executed last. You can achieve this by appending another test suite with the new class right after your test suite(s).

Testng assertj report and continue

I'm using AssertJ to test web using fluentlenium and extent reports for reporting the results.
I asked before the question but forgot to mention the use of AssertJ.
The provided answer was to extend soft assert and that it has onAssertFailure function.
Is there anything like this for AssertJ soft assertions? Or is there another solution to bypass it?
In the next AssertJ version (2.5.0) you will have access to all soft assertions errors (see this commit).
Hope it helps
In a future release of assertJ a method wasSuccess() is added (as can be seen on git history), but it is not yet available in the current release.
When this method is added you can do something like this:
public class AssertjSoftAssert extends SoftAssertions {
private void checkFailure() {
if(!wasSuccess()) {
onFailure();
}
}
private void onFailure() {
//doFailureStuff
}
#Override
public BigDecimalAssert assertThat(BigDecimal actual) {
BigDecimalAssert assertion = super.assertThat(actual);
checkFailure();
return assertion;
}
#Override
public BooleanAssert assertThat(boolean actual) {
BooleanAssert assertion = super.assertThat(actual);
checkFailure();
return assertion;
}
}
Do note, however, that you will have to override EVERY assertion method in the SoftAssertions class like I've shown you with the examples here. And also if new Assertions are added to the SoftAssertions class you will have to override those as well. This is the best solution I could find right now, but won't work until assertj is updated either.
EDIT: Actually I am not sure this would even work because I am not sure wasSuccess() will return true after every successvul softassert or only after throwing assertAll() but I can't test this obviously as the feature isn't out yet.
Bonus: The commit that added wasSuccess()

Always run Unit Tests in Release mode

I've got a method that contains logic that varies between release mode and debug mode. It is not advanced at all, but i still want a unit test for it, since my application will be used in a bigger picture and i want to redirect the user to other sites if it is not used in release mode.
And now to my question, is there any way to force the unit test to run in Release Mode? I don't want to manually change build configuration for every time i want to run my unit tests.
Instead of running your unit tests in Release mode you can create a test seam so you can control which behavior you want to elicit. You might be able to something like this:
public class Foo {
public int Bar() {
if (IsDebugModeEnabled()) {
return 1;
} else {
return 0;
}
}
public boolean IsDebugModeEnabled() {
#if DEBUG
return true;
#else
return false;
#endif
}
}
This way you have a couple of options to test both paths of your logic. You can create a subclass Foo and override IsDebugModeEnabled or use a partial mock to directly set the return value.

How To Ensure an #Test Method Always Runs Before Any Others Regardless of Class, Suite or Group?

My reading of the TestNG docs suggests that if I have a test method marked like this:
#BeforeSuite(alwaysRun = true)
#Test
public void MyTestMethod { ... }
then MyTestMethod would run before any other test defined anywhere, regardless of class, suite or group. But that does not seem to be the case.
Is there a way to make a test method run unconditionally before everything else? (And that if it fails no other tests will be run.)
Edit:
The test class:
class Suite_Setup
extends BaseTestSuite
{
#BeforeSuite(alwaysRun = true)
def setup() {
System.out.println("Conducting test suite setup...")
// Verify that the internal API is configured properly and that the API host is available...
new Action(ApiHostname, new BasicCookieStore)
}
}
Edit:
The answer:
We generate our own TestNG.xml files (automatically) and the #BeforeSuite method was not being included in it. Once it was included, #BeforeSuite had the expected effect.
However, it does appear that both #BeforeSuite (and presumably other #Before... and #After... annotations) can be mixed with #Test and rather than inhibiting the execution of the annotated method, they cause it to run more than once!
Also, I remiss in not indicating which version of TestNG I'm using. It's 6.2.
Try using groups either on class level or on method level.
In my case, I have a set of smoke tests that need to run before everything and if any of those fail, no other tests should run.
#Test(groups="smoke")
public class SmokeTests {
public void test1 {
//
}
...
}
And all other classes with tests are:
#Test(dependsOnGroups = "smoke")
public class OtherTests {
public void test1 {
//
}
...
}
I think that the answer is
to separate the configuration-methods from the test-methods,
to use #BeforeSuite for the method to be executed befor all tests in the suite (for example to get an EntityManager)
to use dependsOnMethods to decide in which order the tests shall be run
In the following example the testRemove will run after testInsert, testReadAll and testUpdate have run.
testReadAll will run after testInsert has run.
testUpdate will run after testInsert has run.
Note that there is no dependency between testReadAll and testUpdate.
#Test
public void testInsert()
{..}
#Test(dependsOnMethods={"testInsert"})
public void testUpdate()
{..}
#Test(dependsOnMethods={"testInsert"})`
public void testReadAll()`
{..}
#Test(dependsOnMethods={"testInsert", "testUpdate", "testReadAll"})`
public void testRemove()`
{..}
Remove #Test, a method cannot be both a configuration and a test method.
#Test (alwaysRun=True)
makes the test always run irrespective of methods or groups it depends on and even it is failed
Is there a way to mark a test method so it will unconditionally be run before everything else?
Just try to assign to target test the lowest priority value.
#Test(priority = -2147483648)
public void myTest() {
...
}
You can read more about TestNG test priority over there.
And that if it fails no other tests will be run.
You need to make other tests depending on this test method by using one of the following options:
Assign to your first method some group and mark other tests by dependsOnGroup.
Mark other tests by dependsOnMethod.
If you will use one of the dependency options you do not need to provide priority for your first method.

Symfony2: Testing entity validation constraints

Does anyone have a good way to unit test an entity's validation constraints in Symfony2?
Ideally I want to have access to the Dependency Injection Container within the unit test which would then give me access to the validator service. Once I have the validator service I can run it manually:
$errors = $validator->validate($entity);
I could extend WebTestCase and then create a client to get to the container as per the docs however it doesn't feel right. The WebTestCase and client read in the docs as more of a facility to test actions as a whole and therefore it feels broken to use it to unit test an entity.
So, does anyone know how to either a) get the container or b) create the validator inside a unit test?
Ok since this got two votes I guess other people are interested.
I decided to get my shovel out and was pleasantly surprised (so far anyway) that this wasn't at all difficult to pull off.
I remembered that each Symfony2 component can be used in a stand alone mode and therefore that I could create the validator myself.
Looking at the docs at: https://github.com/symfony/Validator/blob/master/ValidatorFactory.php
I realised that since there was a ValidatorFactory it was trivial to create a validator (especially for validation done by annotations which I am, although if you look at the docblock on the page I linked above you'll also find ways to validate xml and yml).
First:
# Symfony >=2.1
use Symfony\Component\Validator\Validation;
# Symfony <2.1
use Symfony\Component\Validator\ValidatorFactory;
and then:
# Symfony >=2.1
$validator = Validation::createValidatorBuilder()->enableAnnotationMapping()->getValidator();
# Symfony <2.1
$validator = ValidatorFactory::buildDefault()->getValidator();
$errors = $validator->validate($entity);
$this->assertEquals(0, count($errors));
I hope this helps anyone else whose conscience wouldn't allow them to just use WebTestCase ;).
We end up rolling your own base test case to access the dependency container from within a test case. Here the class in question:
<?php
namespace Application\AcmeBundle\Tests;
// This assumes that this class file is located at:
// src/Application/AcmeBundle/Tests/ContainerAwareUnitTestCase.php
// with Symfony 2.0 Standard Edition layout. You may need to change it
// to fit your own file system mapping.
require_once __DIR__.'/../../../../app/AppKernel.php';
class ContainerAwareUnitTestCase extends \PHPUnit_Framework_TestCase
{
protected static $kernel;
protected static $container;
public static function setUpBeforeClass()
{
self::$kernel = new \AppKernel('dev', true);
self::$kernel->boot();
self::$container = self::$kernel->getContainer();
}
public function get($serviceId)
{
return self::$kernel->getContainer()->get($serviceId);
}
}
With this base class, you can now do this in your test methods to access the validator service:
$validator = $this->get('validator');
We decided to go with a static function instead of the class constructor but you could easily change the behavior to instantiate the kernel into the constructor directly instead of relying on the static method setUpBeforeClass provided by PHPUnit.
Also, keep in mind that each single test method in you test case won't be isolated fro, each others because the container is shared for the whole test case. Making modification to the container may have impact on you other test method but this should not be the case if you access only the validator service. However, this way, the test cases will run faster because you will not need to instantiate and boot a new kernel for each test methods.
For the sake of reference, we find inspiration for this class from this blog post. It is written in French but I prefer to give credit to whom it belongs :)
Regards,
Matt
I liked Kasheens answer, but it doesn't work for Symfony 2.3 anymore.
There are little changes:
use Symfony\Component\Validator\Validation;
and
$validator = Validation::createValidatorBuilder()->getValidator();
If you want to validate Annotations for instance, use enableAnnotationMapping() like below:
$validator = Validation::createValidatorBuilder()->enableAnnotationMapping()->getValidator();
the rest stays the same:
$errors = $validator->validate($entity);
$this->assertEquals(0, count($errors));
With Symfony 2.8, it seems that you can now use the AbstractConstraintValidatorTest class this way :
<?php
namespace AppBundle\Tests\Constraints;
use Symfony\Component\Validator\Tests\Constraints\AbstractConstraintValidatorTest;
use AppBundle\Constraints\MyConstraint;
use AppBundle\Constraints\MyConstraintValidator;
use AppBundle\Entity\MyEntity;
use Symfony\Component\Validator\Validation;
class MyConstraintValidatorTest extends AbstractConstraintValidatorTest
{
protected function getApiVersion()
{
return Validation::API_VERSION_2_5;
}
protected function createValidator()
{
return new MyConstraintValidator();
}
public function testIsValid()
{
$this->validator->validate(null, new MyEntity());
$this->assertNoViolation();
}
public function testNotValid()
{
$this->assertViolationRaised(new MyEntity(), MyConstraint::SOME_ERROR_NAME);
}
}
You have got a good sample with the IpValidatorTest class
The answer in https://stackoverflow.com/a/41884661/4560833 has to be changed a little for Symfony 4:
Use ConstraintValidatorTestCase instead of AbstractConstraintValidatorTest.
Answer (b): Create the Validator inside the Unit Test (Symfony 2.0)
If you built a Constraint and a ConstraintValidator you don't need any DI container at all.
Say for example you want to test the Type constraint from Symfony and it's TypeValidator. You can simply do the following:
use Symfony\Component\Validator\Constraints\TypeValidator;
use Symfony\Component\Validator\Constraints\Type;
class TypeValidatorTest extends \PHPUnit_Framework_TestCase
{
function testIsValid()
{
// The Validator class.
$v = new TypeValidator();
// Call the isValid() method directly and pass a
// configured Type Constraint object (options
// are passed in an associative array).
$this->assertTrue($v->isValid(5, new Type(array('type' => 'integer'))));
$this->assertFalse($v->isValid(5, new Type(array('type' => 'string'))));
}
}
With this you can check every validator you like with any constraint configuration. You neither need the ValidatorFactory nor the Symfony kernel.
Update: As #psylosss pointed out, this doesn't work in Symfony 2.5. Nor does it work in Symfony >= 2.1. The interface from ConstraintValidator got changed: isValid was renamed to validate and doesn't return a boolean anymore. Now you need an ExecutionContextInterface to initialize a ConstraintValidator which itself needs at least a GlobalExecutionContextInterface and a TranslatorInterface... So basically it's not possible anymore without way too much work.
I don't see a problem with the WebTestCase. If you don't want a client, don't create one ;) But using a possibly different service than your actual application will use, that's a potential pit fall. So personally, I've done like this:
class ProductServiceTest extends Symfony\Bundle\FrameworkBundle\Test\WebTestCase
{
/**
* Setup the kernel.
*
* #return null
*/
public function setUp()
{
$kernel = self::getKernelClass();
self::$kernel = new $kernel('dev', true);
self::$kernel->boot();
}
public function testFoo(){
$em = self::$kernel->getContainer()->get('doctrine.orm.entity_manager');
$v = self::$kernel->getContainer()->get('validator');
// ...
}
}
It's less DRY than Matt answer -- as you'll repeat the code (for each test class) and boot the kernel often (for each test method), but it's self-contained and require no extra dependencies, so it depends on your needs. Plus I got rid of the static require.
Also, you're sure to have the same services that your application is using -- not default or mock, as you boot the kernel in the environnement that you wish to test.
If people still read this one in 2023, prefer to inject the ValidatorInterface for Symfony > 3 / 4.
use Symfony\Component\Validator\Validator\ValidatorInterface;
// ...
$this->validator->validate($myEntity);

Resources