I need to generate some test data for a Web and WPF application (both .NET Apps). So I had two possible solution in my mind.
Generate test data by using sql scripts. This approach has the issue that I need to validate the inserted data.
Use the insert API of my .NET code to generate test data. For this I am not sure if its oversized, but I would reuse the validation logic of my code.
Do you have any other suggestions? Maybe Microsoft tools for supporting those creation tasks?
Thanks!
Oliver
Don't use sql/db in your test. It is extra dependency that has nothing to do of your program logic. Normally, I would create an interface called IDataProvider and then provide different implementation i.e. DbDataProvider for normal execution or FakeDataProvdier for testing. With that IDataProvider interface, means that you can plug into whatever datasource your want for your testing e.g. TextFile, xml, json, objectbuilder using your own api etc.
Using insert API for generating test data is a better idea. But, why would you worry about performing validation to your test data ? Normally, a test should failed if the test data is wrong or you are not doing right functional test.
Related
We are in the process of finalizing a test framework, and are pretty impressed with Robot framework and STAF.
Unable to decide with the optimal approach for the below:
Want to be able to start tests from a server by selecting clients
Can we display all clients in the existing network?
Number of clients can increase/decrease over time
When we select a client we want to go fetch client properties dynamically
Is there a way to display dynamic properties on RIDE/STAX
Can we use any other framework and integrate with Robot? Staf/Stax?
User should be able to choose tests the client supports and build a config
Can we use RIDE or something similar to build test configurations/per client
Launch all clients in parallel, monitor and report results
Is there a way to launch and monitor results in parallel?
This is only a partial answer, based on my understanding of your question and lacking full knowledge of your situation.
Robot Framework can be started from a .cmd or .bat file. I don't know if the results can be sent somewhere else to be saved from there, but I'm fairly certain that yes, they can.
1.1. If you can get that list in Python or Java, then yes, you can do it in Robot Framework and pass variables and pass/fail results around the test suite. You might be able to use both at once, but I'm not sure yet.
Handled with the first item.
Robot + Python/Java can probably handle that.
3.1. I don't know, I use PyCharm as my Robot Framework IDE. It has an integrated console and allows for quick and easy managing of Python/RobotFramework files, as well as a lot of other languages, but I'd imagine that using Robot's Log to Console keyword, you could send the results directly to the console. So, yes.
3.2. Short answer: Not that I've ever heard of, but if you can run those with Java/Python and return the Pass/Fail results to Robot Framework, then yes.
Using multiple tags, the Robot Framework programmer can, at runtime, either run the test excluding particular tags or run the test including particular tags.
In theory: yes. Again, not something I've ever done, and honestly telling you how is beyond me, but I don't know a reason why you couldn't as long as you don't have any custom keywords that move the mouse cursor.
ASP.NET apps that I've developed (on ASP.NET 2.0) have typically been backed by a database; the great majority of the .NET code on the server loads data in the form of a DataSet or SqlDataReader and uses it to databind something like a DataGrid. The meaningful logic is either database dependent or user interface dependent.
In this context, how should I implement unit tests that would be run by a continuous integration server (probably CruiseControl.NET)? Should I set up a test database connection for it to use to test CRUD operations and more complex SPROCs, or should more logic be contained in the .NET code and not in a SPROC? This becomes more complex when there is structure in the database that the application expects to find (such as a Users table in something I'm writing for a CMS).
Also, what are the best ways to do unit-testing of user interfaces? I've found NUnitASP, which is now abandoned but mentions Selenium and Watir.
Take a look at the MVP pattern (Model View Presenter). This should allow you to isolate the behaviour of your system and unit test it properly.
Also, consider switching to MVC (I would go with Fubu over ASP.NET MVC). This will allow you to test controllers and have a more rails-like experience.
To automate, I use WatiN (like watir but for .NET). On top of it I use StoryTeller (Google "StoryTeller Jeremy Miller") to present what is happening in a more human readable way and to provide templates for QA to use.
I would highly recommend against any business logic in sprocs. Stay away from them. Look at the repository pattern to abstract away getting and setting data.
Hope this gets you started.
We are looking to build a cube in Microsft SQL server analysis services but would like to be able to use some of the automated testing infrastructure we have.
such as Cruise control for automated build, deployments and test.
I am looking for anyone that can give me any pointers on building tests against analysis services, and also any experience with adding these to a build pipeline.
Also if automation is not possible some manual test methods.
Recently I came upon BI.Quality project on codeplex and from what I can tell it's very easy to learn and to integrate into existing deployment process.
There is another framework named NBi. You've additional features compared to BI.Quality as to check the existence of a measure, dimension, attributes, the order of members, the count of members. Also when comparing two result sets it's often easier to spot the difference between them with NBi. The edition of the test-suites is also done in one single xml file validated by an XSD (better user-experience).
We're getting some errors, if we try to test something with the asp.net membership framework. It seems that it can't instantiate the asp.net membership environment and so, it can't access all the profiles, users and so on.
Has anybody seen a similar problem? Or is it working for someone?
Cheers,
Steve
If you are depending on external resources such as your database and the configuration files (used when using the ASP.NET membership) you aren't writing very effective unit tests. You will have to keep everything in sync including the data in the database. This because a maintenance nightmare.
If you want to test this way, I recommend setting up your configuration to have membership (you can grab this from your application). You then will also want to set up a test database to connect to. It needs to be populated with fake data, so you'll need scripts to make sure the data is consistent.
I would however recommend that you take the approach of doing more proper unit tests. When we refer to a "unit test" we mean testing a very small piece of code at a time. This code should not depend on anything else, so what you need to do is use interfaces and use fakes, stubs, or mocks so that your tests scope is enclosed to a single unit of code.
If you want to go this route I highly recommend reading Working Effectively with Legacy Code. There are also plenty of other books and resources which talk about how to find seams and decouple your code so you're able to test it. Once you get the hang of unit testing you'll be glad you looked into this.
The test framework is looking at the test project's web.config file which likely doesn't have the right configuration. You should really write interfaces around the authentication/membership providers and write some dummy implementations to test with.
Going on from Benrick's answer - I recommend you take a look at the ASP.NET MVC project - this has examples of the interfaces and wrappers you would need to have to properly unit test your code.
As the comments in the AccountController.cs file state:
The FormsAuthentication type is sealed and contains static members, so it is difficult to unit test code that calls its members. The interface and helper class below demonstrate how to create an abstract wrapper around such a type in order to make the AccountController code unit testable.
I'm wondering how people are currently resetting their data / cleaning up test remnants for their WatiN/Wartir tests?
For example, lets say there's a test to add a user into the system and the username has to be unique. Obviously the first run without any users should work fine, but the second run will fail without manual intervention.
There are a couple of strategies that you could do for this, I am assuming that you are using WatiN, with Nunit or VS Unit tests to run your tests.
Use transactions
An approach that is used when unit testing is that you "wrap" the whole test in a transaction and at the completion of the test roll the transaction back. In .net you can use System.Transactions for this.
Build a "stub page"
Build a page in your applicaiton that uses the existing business logic to delete your data. This page would need to be secured and ideally not even deployed in to production.
This is the approach that I would recommend.
Call a web service
Develop a web service, or call one directly from the app tier of the applicaiton to perform the delete. You will probably need to develop this as well.
Clean up directly
Build some classes in your test code to access the data and clean it up.
With any of these you will need to cleanup before and after you run your test, i.e. in the test setup and test cleanup methods. The reason to do it twice is that you should assume that your test has failed and not cleaned up properley.
Use Linq to Sql AFAIK if you are using Linq to sql, it works in-memory and wraps the whole update in a transaction for you automatically. If you simply don't call the SubmitChanges(); method then you should be fine, but I haven't tested this myself.
I have asked a developer to make a script that will reset database. After a suite of tests, I just call that script and start from clean database.
Mike - your question isn't unique for Watir/WatiN. It applies for any UI testing, so search around for similar solutions for Selenium, Windmill, and even headless integration tests (HtmlUnit, API tests, etc). I've answered this question a couple times personally on StackOverflow.
WatiN is for UI testing.
In order to test the scenario you are looking for, you can generate user id using the c# code that will make it unique (as against the way it is stored when you created the test).