I am using xUnit for integration testing. and for that, I use a localdb instance for it. With that being said, I would like to initiate DB instance once with some pre-defined data and of course I would that stay true for all test cases. I could write each test cases isolated so they are not running into each other however I would like to only create DB instance once.
I followed xunit constructor runs before each test and the code looks like
//similar to base class
public class DatabaseFixture : IDisposable
{
public SqlConnection Db { get; private set; }
public DatabaseFixture()
{
InitialDB();
}
public InitialDB()
{
CreateDBInstance();
CreateDBSchemas();
InitDBMetaData();
}
public void Dispose()
{
// clean up test data from the database
CleanUpDB();
}
}
//Class where you want to use shared class instance
public class MyDatabaseTests : IClassFixture<DatabaseFixture>
{
DatabaseFixture dbFixture;
public MyDatabaseTests(DatabaseFixture fixture)
{
this.dbFixture = fixture;
}
// write tests, using dbFixture.Db to get access to the SQL Server
}
The problem I am facing is I noticed this DBFixture being called everytime per test case. I thought with iClassFixture it is only called once. which brings problem when test cases run in parallel because it is trying to cleanup db while other test trying to access it and also multiple test cases would try to create the db at the same time which causes error. https://xunit.net/docs/shared-context.html
Can anyone shed light on why it is not working?
I stumbled across the same problem and it was an issue with visual studio:
https://github.com/xunit/xunit/issues/2347#issuecomment-983586580
Right clicking the class file which contains the tests to start the test runner, may lead to this behaviour.
You need to use Collection Fixtures instead.
https://xunit.net/docs/shared-context#collection-fixture
When to use: when you want to create a single test context and share it among tests in several test classes, and have it cleaned up after all the tests in the test classes have finished.
Related
My reading of the TestNG docs suggests that if I have a test method marked like this:
#BeforeSuite(alwaysRun = true)
#Test
public void MyTestMethod { ... }
then MyTestMethod would run before any other test defined anywhere, regardless of class, suite or group. But that does not seem to be the case.
Is there a way to make a test method run unconditionally before everything else? (And that if it fails no other tests will be run.)
Edit:
The test class:
class Suite_Setup
extends BaseTestSuite
{
#BeforeSuite(alwaysRun = true)
def setup() {
System.out.println("Conducting test suite setup...")
// Verify that the internal API is configured properly and that the API host is available...
new Action(ApiHostname, new BasicCookieStore)
}
}
Edit:
The answer:
We generate our own TestNG.xml files (automatically) and the #BeforeSuite method was not being included in it. Once it was included, #BeforeSuite had the expected effect.
However, it does appear that both #BeforeSuite (and presumably other #Before... and #After... annotations) can be mixed with #Test and rather than inhibiting the execution of the annotated method, they cause it to run more than once!
Also, I remiss in not indicating which version of TestNG I'm using. It's 6.2.
Try using groups either on class level or on method level.
In my case, I have a set of smoke tests that need to run before everything and if any of those fail, no other tests should run.
#Test(groups="smoke")
public class SmokeTests {
public void test1 {
//
}
...
}
And all other classes with tests are:
#Test(dependsOnGroups = "smoke")
public class OtherTests {
public void test1 {
//
}
...
}
I think that the answer is
to separate the configuration-methods from the test-methods,
to use #BeforeSuite for the method to be executed befor all tests in the suite (for example to get an EntityManager)
to use dependsOnMethods to decide in which order the tests shall be run
In the following example the testRemove will run after testInsert, testReadAll and testUpdate have run.
testReadAll will run after testInsert has run.
testUpdate will run after testInsert has run.
Note that there is no dependency between testReadAll and testUpdate.
#Test
public void testInsert()
{..}
#Test(dependsOnMethods={"testInsert"})
public void testUpdate()
{..}
#Test(dependsOnMethods={"testInsert"})`
public void testReadAll()`
{..}
#Test(dependsOnMethods={"testInsert", "testUpdate", "testReadAll"})`
public void testRemove()`
{..}
Remove #Test, a method cannot be both a configuration and a test method.
#Test (alwaysRun=True)
makes the test always run irrespective of methods or groups it depends on and even it is failed
Is there a way to mark a test method so it will unconditionally be run before everything else?
Just try to assign to target test the lowest priority value.
#Test(priority = -2147483648)
public void myTest() {
...
}
You can read more about TestNG test priority over there.
And that if it fails no other tests will be run.
You need to make other tests depending on this test method by using one of the following options:
Assign to your first method some group and mark other tests by dependsOnGroup.
Mark other tests by dependsOnMethod.
If you will use one of the dependency options you do not need to provide priority for your first method.
How do you prevent a CakePHP 2.0 test case, which extends CakeTestCase (uses PHPUnit), from reloading fixtures between tests?
Background: There is a set of integration tests which we have written with CakePHP 2.0 using PHPUnit. We have extended the test case class from the standard CakeTestCase. For this set of tests, we have a bunch of fixtures setup to populate the data from the database. Naturally, these tests take a long time to run. Basically, all of the time is coming from Cake unloading and re-loading all of the fixtures between tests.
All of the tests act as READ only. We are just issuing find calls to the database and testing the logic among a set of class interactions based on those results. In fact, the tests can be boiled down to:
class ALongRunningTest extends CakeTestCase {
public $fixtures = array('app.class1', 'app.class2', ... 'app.class8');
/**
* #dataProvider provider
* #test
*/
public function checkCompositionLogic($val1, $val2, $val3) {
// internally calls class1 and class3
$data = $this->ModelX->generateComplexStructure($val1);
// internally calls other classes & models, which touch the
// other loaded fixtures
$results = $this->ModelY->checkAllWhichApply($val2, $data);
$this->assertEquals($val3, $results);
}
public function provider() {
return array(
array(stuff, stuff1, stuff2),
array(x_stuff, x_stuff1, x_stuff2),
array(y_stuff, y_stuff1, y_stuff2),
array(z_stuff, z_stuff1, z_stuff2),
array(a_stuff, a_stuff1, a_stuff2),
// More test cases
);
}
}
I've not been able to find anything on how to prevent this. I saw in the CakeTestCase class a public variable autoFixtures with a comment that says if you change it to false it won't load the fixtures. It makes a note stating that you have to load them manually. However, I see no documentation on how to actually load them manually.
Strictly speaking, CakePHP is correct in the way that it works. Tests shouldn't be dependent upon each other, so that database is reset between each test case. You could even argue that it should be reset between each test method, but the overhead would be even more noticeable.
However, since you're doing read only actions on the database, you could remove all the references to fixtures in your test cases and set up your database entries before you run the test suite (e.g. import it from an SQL file).
Or you could create a custom test suite that adds a whole load of data, e.g:
class AllTest extends CakeTestSuite {
public static function suite() {
self::loadDB();
$suite = new CakeTestSuite('All tests');
$suite->addTestDirectoryRecursive(TESTS . 'Case');
return $suite;
}
public static function loadDB() {
//Do some set up here using your models
}
}
The advantage of doing it that was is that if you ever had to introduce tests that do write to the database, you could run them under a separate test suite.
I am new in unit testing. I want to unit test my ASP.NET/MVC application using built-in MS Test.
To perform many tests i need to use temporary user accounts. Where is the best place to put creating/dropping code? I tried to create users in [TestInitialize()] method and put results of type User into class field but it doesnt remain to the moment of the second test s starting :(
Or should i create temp user account in every test method? This doesntlook well...
Thank you!
I would normally generate the user accounts into a property of the test during the method with the attribute setup, and delete them in the method marked teardown
public UserAccount account { get; set; }
[SetUp]
public void SetUp
{
// Set up your accounts here
}
[TearDown]
public void TearDown()
{
account.Delete();
}
The attributes ensure that these fixtures are run before and after the test methods.
Based on this article, I've written a custom class which implements the Watin.Core.interfaces.IFindByDefaultFactory, but I don't think I'm correctly assigning it to the watin settings, because it is never used.
Basically, Where/when should I assign to the Settings.FindByDefaultFactory? I've tried in my test Setup, and the text fixture's constructor, but neither seem to cause my custom class to be used. The tests still run and work, but I have to use the full asp.net ID's.
I'm using Watin 2.0.15.928 in VS2008 from nUnit 2.5.2.9222. I am running visual studio as administrator, and tests run sucessfully as long as I don't rely on my custom find logic.
Here's what the start of my text fixture looks like, where I set the FindByDefaultFactory
namespace Fundsmith.Web.Main.BrowserTests
{
[TestFixture]
class WatinHomepageTests
{
private IE _ie;
[SetUp]
public void Setup()
{
Settings.FindByDefaultFactory = new FindByAspIdFactory();
_ie = new IE("http://localhost/somepage.aspx");
}
//etc etc...
And this is what my custom Find By Default factory looks like (simplified), unfortunately, it's never called.
using System.Text.RegularExpressions;
using WatiN.Core;
using WatiN.Core.Constraints;
using WatiN.Core.Interfaces;
namespace Fundsmith.Web.Main.BrowserTests
{
public class FindByAspIdFactory : IFindByDefaultFactory
{
public Constraint ByDefault(string value)
{
// This code is never called :(
// My custom find by id code to cope with asp.net webforms ids...
return Find.ById(value);
}
public Constraint ByDefault(Regex value)
{
return Find.ById(value);
}
}
}
Edit: Extra information after the fact.
Based on me fuguring this out, (see answer below), It turns out that the way I was consuming Watin to find the elements was wrong. I was explicitly calling Find.ById, rather than letting the default action occur. So I'd reassigned the default but was then failing to use it!
[Test]
public void StepOneFromHomepageShouldRedirectToStepTwo()
{
_ie.TextField(Find.ById("textBoxId")).TypeText("100");
//Other test stuff...
}
Right, I've figured this one out, and it was me being an idiot and explicitly calling the Find.ById method, rather than letting the default action occur. It seems the test setup is a fine place to set the FindByDefaultFactory.
ie, I was doing this (wrong):
[Test]
public void StepOneFromHomepageShouldRedirectToStepTwo()
{
_ie.TextField(Find.ById("textBoxId")).TypeText("100");
//Other test stuff...
}
When I should have been simply doing this. (Without the explicit "Find.ById")
[Test]
public void StepOneFromHomepageShouldRedirectToStepTwo()
{
_ie.TextField("textBoxId").TypeText("100");
//Other test stuff...
}
Not only was this me being stupid, but I didn't include this in my original question, so it would have been impossible for anyone else to figure it out for certain. Double slaps for me.
I have a simple page with a Grid that I'm binding a collection of objects to. I also have some simple functionality on the grid to edit and save rows. I would like to write unit tests for this page, but it's not really making sense to me.
For example:
Private Sub LoadGrid()
'Populate Collection
grid.datasource = MyCollection
grid.databind()
end sub
I guess a Sub really doesn't need a unit test, but what if this were a function that returned true when the grid had been loaded. How do you write a unit test for this? What other test should be done on a simple web page like this?
As always, thanks to everyone who give any sort of input.
How do you write a unit test for this?
The first step is actually making your form testable. Have a look at this page for separating UI and BL layers, there are about a bajillion different ways to implement MVC, MVP, and all of its variants, and there's no One True Way™ to do it. So long as your code is sane and consistent, other people will be able to work on your code.
I personally find the following pattern works in most cases for testing UIs:
Create an interface representing your Model.
Create a class for your controller which handles all the updates to the model.
Your view should listen to changes to the model.
So in the end, you end up with something like this (sorry, my VB-fu is rusty, writing this in C# instead):
interface IProductPageModel
{
int CurrentPage { get; set; }
int ItemsPerPage { get; set; }
DataSet ProductDataSet { get; set; }
}
class ProductPageController
{
public readonly IProductPageModel Model;
public ProductPageController(IProductPageModel model)
{
this.Model = model;
}
public void NavigateTo(int page)
{
if (page <= 0)
throw new ArgumentOutOfRangeException("page should be greater than 0");
Model.CurrentPage = page;
Model.ProductDataSet = // some call to retrieve next page of data
}
// ...
}
This is concept code, of course, but you can see how its very easy to unit test. In principle, you could re-use the same controller code in for desktop apps, silverlight, etc since your controller doesn't depend directly on any particular view implementation.
And finally on your form side, you'd implement your page similar to:
public class ProductPage : Page, IProductPageModel
{
ProductPageController controller;
public ProductPage()
{
controller = new ProductPageController(this);
}
public DataSet ProductDataSet
{
get { return (DataSet)myGrid.DataSource; }
set { myGrid.DataSource = value; myGrid.DataBind(); }
}
protected void NavigateButton_OnCommand(object sender, CommandEventArgs e)
{
controller.NavigateTo(Convert.ToInt32(e.CommandArgument));
}
}
Here there's no real distinction between view and model -- they're the same entity. The idea is to make your code-behind as "stupid" as possible, so that as much testable business logic is contained in the controller as possible.
What other test should be done on a
simple webpage like this?
You'd want tests for any sort of form validation, you want to make sure you're throwing exceptions in exceptional cases, ensuring that your controller methods update your model in an expected way, and so on.
Juliet is right.
The line of code where you said
'Populate Collection
that is the testable part. You can do assertions on if the collection is null, if it has items, if it has exactly 42 items. But that would be an integration test.
If you can isolate all the calls to the database (the part that returns a datareader), then return a empty, fake DbDataReader, then you can test everything inbetween the UI and the database.
Tests that spin up a browser and verify that a table is render, similarly is a integration test that will depend on having IIS up and working (and a DB up and working, unless you have a repository you can fake)
If you are just getting started, I would look for all the easy to test code first, such as methods that do have dependencies on the database, then move on the tricker tests that require mocking/stubbing/faking database servers, etc.