I am working on a ASP.NET web app and I want to get the code coverage on the same application by the Coded UI automation test suite we currently have. Is it possible to get the code the code coverage with the DLLs that I have or do we necessarily need to use the instrumented build for it? And if instrumented build is a must how do we instrument the build in order to get the code coverage on it? Any pointer will be helpful.
Okay, So I guess there is not much info out there on the internet related to this (Actually there is a lot but non-compiled and un useful info). Here are the few links which we found out after searching :
Using Dynamic code coverage tool :- This tool works fine with newer versions This should help while using this tool.
Using VSInstr :- This guy has put a lot of efforts to document it completely. This works for older versions of .NET applications.
Hope this helps.
Related
I am currently working on replacing an outdated test system that we're using at the company I work at with Robot Framework. The current system is used for function testing embedded systems that we produce. We are using an in-house software for executing the test scripts, which is good consider that the operators that are running the tests doesn't have a broad knowledge in programming. As it is right now they just need to know the name of the product and then press "Start test" in the GUI, which is exactly what we want.
What I've learned and seen now after spending some days playing around in Robot Framework is that there's no built-in GUI in some sort for executing test cases. I haven't even found a third-party solution for this, except using software like Jenkins, but that is not a good solution when the test needs to run locally on the computer where the operator is working. Running the tests from the terminal directly is not a good solution either, just because it's not that "user-friendly".
I've read about the Robot Framework Public API, which is so well documented that I could pretty easy build a new "Test Executor", but if there exists software that does this already I'm glad to hear about it!
I know that RIDE has the functionality to run selected tests via a GUI, but what I've experienced is that it's pretty easy to make horrible mistakes in that software. Don't get me wrong, RIDE is a awesome tool for creating .robot scripts, but it is not great as a "Test Executor", especially not when we really need to keep a constant production flow to meet our deadlines.
How should I handle this? Do you build your own tools for executing tests, or is it that I've missed something?
We are in search of an automated testing tool for our project. As we are in testing department we prefer a tool which would have less programming in it. Please suggest some tools for us .Till now we are testing our application manually.
Our project is being developed in Java.
Is there any freeware tool that I could use or is it better to go for a paid tool?
Thanks in Advance.
Less programming? You'll need something like JUnit to write unit tests if you want to do serious regression testing, but unit tests require you to write some code
Here's a big list of open-source testing tools, some of them may offer what you want: http://java-source.net/open-source/testing-tools/junit
For example, T2 claims to be a random testing tool. As one, it is fully automatic, but one must keep in mind that the code coverage of random testing is in general very limited. It should be used as a complement to other testing methods. T2 checks for internal errors, run time exceptions, method specifications, and class invariant.
Not sure if you mean a CI tool or not, but we use Hudson at Zappos and it works pretty well.
http://hudson-ci.org/
..and there's also CruiseControl: http://cruisecontrol.sourceforge.net/
If you're not talking about CI, maybe you mean QA testing - in which case you should take a look at something like Selenium (for web apps):
http://seleniumhq.org/
If you're doing GUI testing? I'm not really familiar with that area, but I've heard about WinRunner and Rational:
http://en.wikipedia.org/wiki/HP_WinRunner
http://www-01.ibm.com/software/rational/offerings/quality/
..though neither are really free tools. Something like AutoIT might help you move widgets around, but it lacks the reporting parts:
http://www.autoitscript.com/autoit3/index.shtml
There could be two answer to you question:
Besides Selenium, though it has ample of advantages, I am reading about another tool which uses same API which Selenium use. The only changes in API I have seen so far is it reduces the complexity of functions thus making it more easier and simpler for user who is learning.
The tool is called 'Helium' and it has 50% (and more) less complex functions and code as Selenium has.
The only problem with this tool is it is paid tool for learning purpose and for implementing not-so-big scale project you can use it. But yeah after some time its gonna cost you.
I have implemented some code on Helium. Please let me know , if you face any issue initially or you are thinking to implement it.
Other being, you can use Selenium Builder(http://khyatisehgal.wordpress.com/2014/05/26/selenium-builder-exporting-and-execution/) which is an advanced form of Selenium IDE. It imports your command in different languages and does work more effectively and efficiently as Selenium IDE does(http://khyatisehgal.wordpress.com/2014/05/25/selenium-builder/) . So you can import scripts in Eclipse IDE and just execute them as is.
Please let me know , if you have any doubt in any of the tool.
As far I know this is not supported for asp.net projects. It's quite handy for testing static/shared functions.
Does anybody have any info if this will be available in future versions? Or if there are any alternative ways to test your functions in design mode?
This walkthrough article at MSDN says it does work at design time. I tried running a sample method in a sample console application in VS 2008 SP1 and it works a design time. But it runs application as you press enter first time in Immediate window. Subsequent time it doesn't run the application it just returns value.
You can't use if your code has compilation problems.
I think you should just look at the below tools. These might help you.
Snippet Compiler is great for what you are looking.
LINQPad is also good for testing out LINQ code.
What to do automatic nightly build of .NET web application and upload to the test server.
Or, if this tool is overkill, please suggest something else.
I have found this great tutorial: How to Hook Up a VS.NET 2005 Solution With CruiseControl.NET in a Few Minutes.
It's short but uses a real .Net project to illustrate the set up.
After that refine your configuration by reading each relevant section of CruiseControl.NET : Configuring the Server
And for a really nice and recent tutorial check Integrating MSBuild with CruiseControl.NET. Very good one!
The other answers are all great, but have you also looked at CI Factory? From the CI Factory site:
CI Factory is a Continuous Integration
system, not just a CI server. With
minimal configuration it creates a CI
system for you including much of your
development environment. It will
create a development tree, write your
build scripts, setup a super powered
version of CruiseControl.Net, import
it all into source control, and more.
It does all the work that everyone has
to do when creating a CI system. It
even has automated branch creation
baked in, so the next time you need to
create a branch for a release, CI
Factory will handle that for you.
CruiseControl.NET From Scratch
Its a great tool, we use it here todo hourly builds and manage multiple projects.
I used the instructions and example on the CruiseControl.Net website
Setting up from scratch Part 01.
What tools, preferably open source, are recommended for driving an automated test suite on a FLEX based web application? The same tool also having built in capabilities to drive Web Services would be nice.
Adobe distributes a test framework themselves: FlexUnit.
I heard of people using selenium as a free/open source testing tool. A quick google revealed a FLEX API for it. Not sure if it works or is still in development, but it may be worth a look.
http://sourceforge.net/projects/seleniumflexapi/
Are you looking to script code-level unit tests? If so, dpuint is the bomb: http://code.google.com/p/dpuint/ . This library makes it really easy to do automated testing on all sorts of asynchronous events, on either non-visual ActionScript objects or visual components. They also have a nice multi-page tutorial on the Google Code project page.
If you are looking for functional testing tools along the lines of automated record-and-playback simulating an end user using a Flex app, HP's QuickTest Pro is the Adobe-endorsed solution. It works great, but costs about $4,000 - $6,000 per seat.
Check out FlexMonkey. It does automated testing via FlexUnit tests.
Try looking at Melomel. It has Cucumber support baked right in and comes packaged with steps for most Halo and Spark components.
http://melomel.info
There's an automated test tool called RIATest that might fit the bill for you.
Unfortunately only for Windows, and not open source, but if it does the job it might be well worth the price ($399 at time of writing).
FunFX is an option for automating UI testing. I haven't used it extensively, but I've heard of some having success with it. Here is the article where I first learned about it.
I've been extensively using FunFX for several months now on a Flex 3 + Rails project. Not only is it open source, it's also written in Ruby, so integration with web services should be fairly easy. There are a few screencasts out there covering the basics.
The Flex code that your Flex app needs is contained in the SeleniumFlexAPI distribution .swc file, SeleniumFlexAPI.swc. Just include this file as a library when you compile your Flex app.
Sikuli is good tool which can be used to test flex/flash based web applications.
-It can automate anything on graphical user interface.
-It works on Windows, MAC OSX and Linux as well as iPhone and Android.
-Here is the Sikuli link
My preferred tool is Selenium Remote Control. There is a plug-in I discovered a few months ago:
http://code.google.com/p/flash-selenium/
This required 'hooks' to be written on the server side (ActionScript/Flex). Once they were added, I was able to do some browser testing using Selenium RC.
FunFX is great. We've used it extensively and have been very happy with it. The community is also active and very responsive, so that is a big plus for me.
The new version of the Selenium-Flex API (0.2.5) works great.