I'm writing some functional tests for an API in Symfony that depend on having data in the database. It seems the generally accepted method for doing this is to use Fixtures and load the fixtures before testing.
It seems pretty daunting and impractical to create a robust library of Fixture classes that are suitable for all my tests. I am using the LiipFunctionalTestBundle so I only load the Fixtures I need, but it only makes things slightly easier.
For instance, for some tests I may need 1 user to exist in the database, and other tests I may need 3. Beyond that, I may need each of those users to have slightly different properties, but it all depends on the test.
I'd really like to just create the data I need for each test on-demand as I need it. I don't want to pollute the database with any data I don't need which may be a side effect of using Fixtures.
My solution is to use the container to get access to Doctrine and set up my objects in each test before running assertions.
Is this a terrible decision for any reason which I cannot foresee? This seems like a pretty big problem and is making writing tests a pain.
Another possibility it to try and use this port of Factory Girl for PHP, but it doesn't seem to have a large following, even though Factory Girl is used so widely in the Ruby community to solve this same problem.
The biggest reason to fixture test data in a single place (rather than in each test as needed) is that it allows you to isolate test failures from BC-breaking schema changes to only tests that are affected by the changes directly.
A simple example:
Suppose you have a User class, in which you have a required field name, for which you provide getName() and setName(). You write your functional tests without fixtures (each test creating Users as needed) and all is well.
Sometime down the road, you decide you actually need firstname and lastname fields instead of just name. Naturally you change the schema and replace getName/setName with new get and set methods for these, and go ahead and run your tests. Test fail all over the place (anything that sets up a User), since even tests that don't use the name field have called setName() during the setup, and now they need to be changed.
Compare to using a fixture, where the User classes needed for testing are all created in one fixture. After making your breaking change, you update the fixture (in one place / class) to setup the Users properly, and run your tests again. Now the only failures you get should be directly related to the change (ie tests that use getName() for something, and every other test that doesn't care about the name field proceeds as normal.
For this reason it's highly preferred to use fixtures for complicated functional tests where possible, but if your particular schema / testing needs make it really inconvenient you can set entities up manually in tests, but I'd do my best to avoid doing so except where you really need to.
Related
We use flyway since years to maintain our DB scripts, and it does a wonderful job.
However there is one single situation where I am not really happy - possibly someone out there has a solution:
In order to reduce the number of scripts required (and also in order to keep overview about "where" our procedures are defined) I'd like to implement our functions/procedures in one script. Every time a procedure changes (or a new one is developed) this script shall be updated - repeatable scripts sound perfect for this purpose, but unfortunately they are not.
The drawback is, that a new procedure cannot be accessed by non-repeatable scripts, as repeatable scripts are executed last, so the procedure does not exist when the non-repeatable script executes.
I hoped I can control this by specifying different locations (e.g. loc_first containing the repeatables I want to be executed first, loc_normal for the standard scripts and the repeatables to be executed last).
Unfortunately the order of locations has no impact on execution order ;-(
What's the proper way to deal with this situation? Right now I need to specify the corresponding procedures in non-repeatable scripts, but that's exactly what I'd like to avoid ....
I found a workaround on my own: I'm using flyway directly with maven (the same would work in case you use the API of course). Each stage of my maven script has its own profile (specifiying URL etc.)
Now I create two profiles for every stage - so I have e.g. dev and devProcs.
The difference between these two maven profiles is, that the "[stage]Procs" profile operates on a different location (where only the repeatable scripts maintaining procedures are kept). Then I need to execute flyway twice - first with [stage]Procs then with [stage].
To me this looks a bit messy, but at least I can maintain my procedures in a repeatable script this way.
According to flyway docs, Repeatable migrations ALWAYS execute after versioned migration.
But, I guess, you can use Flyway callbacks. Looks like, beforeMigrate.sql callback is exactly what you need.
I have two different projects in two separate solution files. One is a test project containing my unit test methods and other is the main project. I know I need to create an instance of the individual GUI form first in order to access the methods in but not sure how to do this.
Also, I have certain variable values which get initiated only when I run the entire original application. These values are used in many of the methods in the main project. Without them I will just get null values and the test method would just fail. Is there any way to get values for those variables without running the application? I tried to place the logic which fetches the value for those variables in my test method and then call the actual method but this still doesn't work. How do I resolve this problem?
The problems you have run into are quite common for trying to write unit tests for an application that is not well written with regards for testability.
Usually the code you write test for, does not reside in forms or UI classes.
The business logic should be located in separated classes within your main project. The business logic can then be called from your UI and from your unit test too.
So you have to move any business logic to separate classes first.
The next thing you need to do is remove any dependencies from the newly created classes to other classes which prevent you from writing a unit test and replace those dependencies with interfaces.
For instance you can change your application so that every environment variable which you mention in your question, is retrieved by special classes.
If you create interfaces for this classes you can setup a so called Mock ( a fake version of your real class) in you unit test which you can configure in the test to behave as desired for the particular test scenario.
Some general advice:
Often refactoring the code to be tested (i.e. making it better testable) takes more time than writing the unit test itself. There are whole books about refactoring so called "legacy" code.
Refactoring an application for testability often means to de-couple dependencies between concrete classes and to use interfaces instead. In production code you pass your production class for the interface, in a unit test you can create a "mock" class which implements the same interface but behaves in a way like needed in your test
When writing unit tests, you might consider using a mocking framework like Moq. It will save you a lot of time and make the test code smaller. You will find an introduction here: Unit Testing .NET Application with Moq Framework
Ideally you should design you application from the beginning in a way which has testability in mind. The "TDD" (test driven development) approach even goes one step further, here you write your tests before you write the actual code, but this approach is not used very often in my experience.
What would be a proper way to handle global "settings" in my sailsjs application? The user will want to change those settings via the web front of my app.
I imagine I could use a new model "GlobalSettings" with only one item, but I don't really know if it's a good "MVC" practice.
Since it is based on user input, it has to be stored in a database and therefore storing it in model seems like a right choice to me.
Having just 1 row/collection is completely ok in my opinion, especially in the no-SQL field. But for more reusability and scalability, you might want to consider to actually store each setting in invididual row, that might give you space to expand the usability of it in the future.
In my own opinion, I always find as a web app develops, you will start to realize there are more and more fields that you want the user to setup as their preference, as a good practice to relax the application.
For me I usually setup a meta_data model with name, value, criteria, and some other fields.
For example, when viewing your web page, 'Alice' may want a background color of black, 'Bob' may want a background color of green. Then you can let them modify or insert row into this meta_data collection. Then in your database, you will have
name value criteria
background_color black user_name='Alice'
background_color green user_name='Bob'
and it can be all kinds of values.
of course if you just have one value that can be changed by all of your users, it is probably a good idea to know who updated them. For this you would want to create a trigger, (if you are using a sql database)see trigger in mysql, so that every update on the table will trigger a function that stores what was changed and who changed it in another table
So yes, to answer your question, it is totally ok to have a model to store a value, and don't worry about only have one row, you will have more as you develop your app.
The config/globals.js file seems to be a good place to place a global configuration setting.
For convenience, Sails exposes a handful of global variables. By
default, your app's models, services, and the global sails object are
all available on the global scope; meaning you can refer to them by
name anywhere in your backend code (as long as Sails has been loaded).
Nothing in Sails core relies on these global variables - each and
every global exposed in Sails may be disabled in sails.config.globals
(conventionally configured in config/globals.js.)
Sailsjs.org Documentation - Globals
Alternatively you can use the sails.config object.
Custom Configuration
Sails recognizes many different settings, namespaced under different top level keys (e.g. sails.config.sockets and
sails.config.blueprints). However you can also use sails.config for
your own custom configuration (e.g.
sails.config.someProprietaryAPI.secret).
From the docs
There is also services which is global by default.
Overview
Services can be thought of as libraries which contain functions that you might want to use in many places of your application. For example,
you might have an EmailService which wraps some default email message
boilerplate code that you would want to use in many parts of your
application. The main benefit of using services in Sails is that they
are globalized--you don't have to use require() to access them.
It really depends on what kind of global you are wanting.
I’m curious to know how feasible it is to get away from the dependency onto the application’s internal structure when you create an automated test case. Or you may need to rewrite the test case when a developer modifies a part of the code for a bug fix, etc.
We could write several automated test cases based on the applications internal object structure, but lets assume that the object hierarchy changes after 6 months or so, how do we approach these kind of issues?
I can't speak for other testing tools but at least in QTP's case the testing tool introduces a level of abstraction over the application so that non-functional changes in the application often (but not always) have no effect on the way the testing tool identifies the object.
For example in QTP all web elements are considered to be direct children of the document so that changes in the DOM (such as additional tables) don't change the object's description.
In TestComplete, there are a couple of ways to make sure that the changed app structure does not break you tests.
You can set up the Aliases tree of the Name Mapping feature. In this case, if the app structure is changed, you need to modify the Aliases tree appropriately and your test will stay working without requirement to modify them.
You can use the Extended Find feature of the Name Mapping in order to ignore parts of the the actual object tree and search for a needed objects on deeper levels.
This is what I was forced to do after losing all my work twice due to changes on the DOM structure:
Every single time I need to work with an object, I use the Find function with the ID of the object, searching for the object on the Page object. This way, whenever the DOM gets updated, my tests still run smoothly.
The only thing that will break my tests is if the object's ID get changed, but that's not very probable to happen.
Here you can find some examples of the helper functions I use.
PhpUnit has a generator of skel based on an existing class.
But it's works once.
If later some new method are added (because a dev doesnt work with tdd ), the test file is incomplete.
Is there a tool to generate a skel for uncovered method ?
I don't know any, and I also don't see the need. That skeleton generator generates one test method per function it finds, but you cannot test all use cases of a slightly advanced function within only one test function.
Also, the name of the test function is generated - but better names can and should be created to describe the intended test case or behavior of the tested function. Like "testGetQuoteFromStockMarket" vs. "testGettingMicrosoftQuoteFromStockMarketShouldReturnQuoteObject" and "testGettingUmbrellaCorporationFromStockMarketShouldFailWithException".
Note that you cannot test the throwing of exceptions in combination with cases that do not throw exceptions.
So all in all there simply is no use case to create "one test method per method" at all, and if you add new methods, it is your task to manually add the appropriate number of new tests for that - the generated code coverage statistics will tell you how well you did, or which functions are untested.
AFAIK there is no built-in phpunit functionality to updated the auto-generated test code; that is typical of most code generators.
The good news is that each of the functions is added quite cleanly and independently. So what I would do is rename your existing unit test file to *.old, regenerate a fresh test file, then use meld (or the visual diff tool of your choice) to merge in the new functions.
Aside: automatic test generation is really only needed at the start of a new class anyway; the idea of exactly one unit test per function is more about generating nice coverage stats to please your boss; from the point of view of building good software, some functions will need multiple tests, and some functions (getters and setters come to mind) do not really need any, and sometimes multiple functions are best covered by a single unit test (getters and setters again come to mind).