I am working on a ASP.NET web app and I want to get the code coverage on the same application by the Coded UI automation test suite we currently have. Is it possible to get the code the code coverage with the DLLs that I have or do we necessarily need to use the instrumented build for it? And if instrumented build is a must how do we instrument the build in order to get the code coverage on it? Any pointer will be helpful.
Okay, So I guess there is not much info out there on the internet related to this (Actually there is a lot but non-compiled and un useful info). Here are the few links which we found out after searching :
Using Dynamic code coverage tool :- This tool works fine with newer versions This should help while using this tool.
Using VSInstr :- This guy has put a lot of efforts to document it completely. This works for older versions of .NET applications.
Hope this helps.
We are working in a small team. We often had problems like developer1 did some changes in stored procedure or function and it affected work of developer2. Such issues are traced out by chance later. Please guide me how such issues can be stopped. Is there a free tool that we can run to test such issues?
Slowly introduce unit tests, focused integration tests and full system tests.
For all of those use a .net unit test framework to do it. It'll be what you do in the test what makes it be any of the above scenarios. Make sure to keep each of those 3 type of tests separately, as those will have a big difference on the speed it takes to execute them.
For the unit test framework I suggest NUnit but there are others, one that I've found interesting but never made the jump is xUnit.net.
For full system tests I suggest to run them in the unit test framework using WatiN. You could also go with Selenium RC.
We often had problems like developer1 did some changes in stored procedure or function and it affected work of developer2. Such issues are traced out by chance later.
For that specific type of scenario I strongly suggest focused integration tests. Full system tests might catch such scenario, but it will still left you to figure out why it broke.
Instead focus the test in the very specific db access code that makes the call to the procedure. By adding scenarios in there that reveal all the expectations developer2 had from said procedure when (s)he wrote the related .net code, regression issues with that integration code can be revealed very quickly and be dealt with very effectively. Also note that developer1 can easily run the focused integration tests that involve that procedure or area of the database many times / which is a lot more likely to happen than doing the same with full system tests.
You can do either automated unit testing using tools such as NUnit or automated black-box testing using tools such as Selenium. Note that both options (even with free tools) may need significant investment in terms of time and efforts. Typically, unit test cases are created by developers them selves while for automated black box testing, a separate team of QA is utilized - this is mostly because unit test cases are generally written in languages such as C#, VB.NET while automated black-box testing tools typically utilize scripting languages.
Hai Friends
I having the project in Vb i want to migrate that project in the vb.net.any tool available pls inform me.i have tried a lot.i have not installed the visual basic.with the help of remote server i am running that project.
Quite a few versions of Visual studio have a built-in Upgrade Wizard to help you with converting VB6 to Vb.Net code. I know that VS 2005 Pro has it but I'm not sure what other versions.
Here's an article about how to go about upgrading. And there's even a complete free e-book about it as can be found here.
Aside from the sources Ho1 mentions some of the biggest pitfalls are the lack of control arrays, printing and graphics. The printing can be partly mitigated by the use of Printer Compatibility. You can download the PCL as part of the Visual Basic Power Pack 3.0.
If you have room in your budget I would recommend ArtinSoft. www.artinsoft.com. They have a trial version that you can test out. They have been named Microsoft's preferred VB to .NET upgrade solution provider.
I have trialed artinsoft's upgrade companion.
I have also looked at vbmigration partner.
This is my, very limited, anecdotal experience.
On vbmigration partner they have some sample conversions of projects they found on planat sourcecode.
One is call ezdatabase.
If you run vbmigration partner's conversion it will crash if you click the connect/disconnect button more than twice.
On the vb6 version you can click this all day without crashing.
This project is small enough to put through the artinsoft trial of vbuc in its entirety, so i did that.
However after i converted it, there seemed to be a lot of compile errors.
It's not a fair comparison as obviously vb migration partner had lots of opportunity to perfect it before putting the converted code on their website. and yet it was easy to crash.
However I was also disappointed with artinsoft's tool as this was just a small (few hundred lines) crud application and yet there were a lot of compile errors.
Make of this what you will. I would like to hear of others' experiences.
EDIT : On the other hand if this is a true test of the relative capabilities of vb migration partner and artinsoft vbuc then vb migration arner is clearly the winner in this example
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I'm going to start coding some automated tests of our presentation soon. It seems that everyone recommends WatiN and Selenium. Which do you prefer for automated testing of ASP.NET web forms? Which of these products work better for you?
As a side note, I noticed that WatiN 2.0 has been in CTP since March 2008, is that something to be concerned about?
I'm currently working hard on a beta release of WatiN 2.0 somewhere in Q1 of 2009. It will be a major upgrade to the current CTP 2.0 versions and will basically give you the same functionality to automate FireFox and IE as version 1.3.0 offers for automating IE.
So no concerns there.
Jeroen van Menen
Lead dev WatiN
If you're looking to make a serious long-term investment in a framework that will continue to be improved and supported by the community, Selenium is probably your best bet. For example, I just came across this info on Matt Raible's blog:
As of Friday, Google has over 50 teams
running over 51K tests per day on
internal Selenium Farm. 96% of these
tests are handled by Selenium RC and
the Farm machines correctly. The other
4% are partly due to RC bugs, partly
to test errors, but isolating the
cause can be difficult. Selenium has
been adopted as the primary technology
for functional testing of web
applications within Google. That's the
good news.
I also went to one of the Selenium meetups recently and learned that Google is putting serious resources into improving Selenium and integrating it with WebDriver, which is an automated testing tool developed by Simon Stewart. One of the major advantages of WebDriver is that it controls the browser itself rather than running inside the browser as a Javascript application, which means that major stumbling blocks like the "same origin" problem will no longer be an issue.
We've tested both and decided to go with WaTiN. As others have pointed out, Selenium does have some nice features not found in WaTiN, but we ran into issues getting Selenium working and once we did it was definitely slower when running tests than WaTiN. If I remember correctly, the setup issues we ran into stemmed from the fact that Selenium had a separate app to control the actual browser where WaTiN did everything in process.
I've been trying 'em both out and here are my initial thoughts...
WatiN
The Good
Fast execution.
Script creation tools are independent projects; there are 2 that I know of: Wax (Excel based, hosted on CodePlex) and WatiN Test Record (hosted on SourceForge). Neither is as robust as Selenium IDE.
Very good IE support. Can attach and detach to/from running instances. Can access native window handles etc. (See script example below).
NuGet packaged, easy to get running in .NET, Visual Studio style environments and keep updated.
The Bad
Googling WatiN (watin xyz) often causes Google to recommend "watir xyz" instead. Not that much documentation out there.
What little there is (documentation), it is confusing; for example: at first blush it would appear that there is no native support for CSS selectors. Especially since there are extensions libraries like 'WatiNCssSelectorExtensions' and many blog articles about alternative techniques (such as injecting jQuery/sizzle into the page). On Stack Overflow, I found a comment by Jeroen van Menen which suggests that there is native support. At least the lead-developer spends time on Stack Overflow :)
No native XPath support.
No out-of-the-box remote execution/grid based execution.
Script Example (C#). You can't do this with Selenium (not that I know off, at least):
class IEManager
{
IE _ie = null;
object _lock = new object();
IE GetInstance(string UrlFragment)
{
lock (_lock)
{
if (_ie == null)
{
var instances = new IECollection(true); //Find all existing IE instances
var match = instances.FirstOrDefault(ie=>ie.Url.Contains(UrlFragment));
_ie = match ?? new IE();
if (match==null) //we created a new instance, so we should clean it up when done!
_ie.AutoClose = true;
}
}
return _ie;
}
}
Selenium
Slower than WatiN (especially since a new process has to be created).
Built-in CSS selectors/XPath support.
Selenium IDE is good (can't say great, but it’s the best in class!).
Feels more Java-ish than .NET-ish...but really, it's programming language agnostic; all commands are sent to an out-of-process 'Driver'. The driver is really a 'host' process for the browser instance. All communication must be serialised in/out across process boundaries, which might explain the speed issues relative to WatiN.
Decoupled processes - "Driver" and "Control" mean more robustness, more complexity, etc., but also easier to create grids/distributed test environments. Would have really liked it if the "distribution" mechanism (i.e. the communication between Driver & Control) were across WebSphere or other existing, robust, message queue manager.
Support chrome and other browsers out of the box.
Despite everything, I went with WatiN in the end; I mainly intend to write small screen-scraping applications and want to use LINQPad for development. Attaching to a remote IE instance (one that I did not spawn myself) is a big plus. I can fiddle around in an existing instance...then run a bit of script...then fiddle again etc. This is harder to do with Selenium, though I suppose "pauses" could be embedded in the script during which time I could fiddle directly with the browser.
The biggest difference is that Selenium has support for different browsers (not just IE or FF, see http://seleniumhq.org/about/platforms.html#browsers.
Also, Selenium has a remote control server (http://seleniumhq.org/projects/remote-control/), which means that you don't need to run the browser on the same machine the test code is running. You can therefore test your Web app. on different OS platforms.
In general I would recommend using Selenium. I've used WatiN few years ago, but I wasn't satisfied with its stability (it has probably improved by now). The biggest plus for Selenium for me is the fact that you can test the Web app. on different browsers.
Neither. Use Coypu. It wraps Selenium. Much more durable. https://github.com/featurist/coypu
Update
Ye Oliver you're right. Ok why's it better?
Personally I've found the Selenium driver for IE in particular to be very fragile - there's a number of 'standard' driver exceptions that I've time again found when driving Selenium for Unit Tests on ajax heavy websites.
Did I mention I want to write my scripts in c# as a Test Project ? Yes Acceptance Tests within a continous build deployment.
Well Coypu deals with the above. It's a wrapper for Selenium that allows test fixtures such as,
browser.Visit("file:///C:/users/adiel/localstuff.htm")
browser.Select("toyota").From("make");
browser.ClickButton("Search");
... which will spin up a (configurable brand of) browser and run the script. It works great with scoped regions and is VERY extendable.
There's more examples at GitHub and as Olvier below mentions, Adrian's video is excellent. I think it's the best way to drive browser based tests in the .Net world and tries to follow it's Ruby namesake capybara
I've used both, they both seem to work ok. My nod is for Selenium as it seemed to have better Ajax support. I believe WaTiN has matured though since last I used it so it should have the same thing.
The biggest thing would be which development environment do you like to be in? Selenium and Watin have recorders but Selenium is in the browser and watin is in visual studio. + and -'s to both of those.
Until now we are a pure Microsoft Shop for delivering solutions for the enterprise and went with WatiN. This may change in the future.
As a more recent source:
Microsoft printed in MSDN Magazine 12/2010 a BDD-Primer with the combination of SpecFlow with WatiN (cool BDD-Behavior Driven Development). Its author Brandon Satrom (msft Developer Evangelist) has also posted in December 2010 a Video Webcast teaching in detail 1:1 his above findings.
There is a Whitepaper from 04/2011 on Supporting ATDD/BDD with SpecLog, SpecFlow and Team Foundation Server (Acceptance Test Driven Development/Behavior Driven Development) from Christian Hassa, whose team built SpecFlow.
I use Watin, but haven't used Selenium. I can say I got up and running quickly on Watin and have had few to no problems. I can't think of anything I have wanted to do that I couldn't figure out with it. HTH
I generally use Selenium, mainly because I like the Selenium IDE plugin for FireFox for recording starting points for my tests.
I recommend WebAii since that's what I've had any success with and when using it my gripes were few. I never tried Selenium and I don't remember using WaTiN much, at least not to the point where I could get it to succesfully work. I don't know of any framework that deals with Windows dialogs gracefully, although WebAii has an interface for implementing your own dialog handlers.
I considered using both. I used the recorder for Selenium to build some tests in FF. I tried to do the same in Watin and found that the Watin Recorder (2.0.9.1228) is completely worthless for our sites. It appeared to be rendering the site in IE6 -- making our site effectively unusable for recording. We don't support IE6. I couldn't find any way to change the browser it is using. I only found one Watin Recorder out there. If there's more than one, or one that is kept up to date, please comment.
The Selenium Recorder IDE for Firefox is simple to use and ports tests to C#. It isn't great at this. I couldn't get porting test suites to work, despite reading a blog post or two that had workarounds. So there's a bit of manipulation of the generated code. Still, it works 90% and that's better than the alternative.
For my money/time, Selenium is superior just for the ease of building new tests. IE doesn't have any good developer toolbars that are anywhere near as good as Firebug, so I'm doing my development in Firefox to begin with, so having a good working recorder in Firefox is a huge bonus.
My conclusion here was a lot like that democracy quote by Churchill: Selenium is the worst form of automated UI testing. Except for all the other ones.
At the risk of going off on a tangent, I'd recommend Axe/WatiN. Axe allows tests to be written in Excel by 'Manual' Testers with no knowledge of the underlying test 'language'. It does need a 'Technician' to write the bespoke actions (IE. Today I had to do a slightly complex Table lookup & cross-reference) but once written the actions can be used in tests by the non-techy testers.
I also heard that the UK Government Gateway project (which I believe has 6K+ tests automated tests) recently ported all their tests from Axe/Winrunner to Axe/Watin within a week!! And many of the tests are pretty complex - I know as I worked on it a few years ago...
I'm looking at Selenium at the moment, as a potential Client uses it. But i do suggest a wee look at Axe as a layer above the 'work horse' tool.
If you have to access iframes, modal dialogs and cross domain iframes WatiN is a way to go. Selenium couldn't handle the iframes it was throwing commandtimeout exceptions. WatiN you could do lot more things especially if the website uses IE specific stuff like ShowModalDialog etc.. WatiN handles all of them very well. I could even do cross domain iframe access.
You will have to do both if you need to do IE and FF testing, but they are only going to work so well for presentation testing. They cant detect if one element is slightly off, just that the elements are present. I dont know of anything that can replace the human eye for UI / presentation testing, though you could do a few things to assist it (take screenshots of the pages at each step for users to review).
I am tasked with improving the performance of a particular page of the website that has an extremely high response time as reported by google analytics.
Doing a few google searches reveals a product that came with VS2003 called ACT (Application Center Test) that did load testing. This doesn't seem to be distributed any longer
I'd like to be able to get a baseline test of this page before I try to optimize it, so I can see what my changes are doing.
Profiling applications such as dotTrace from Jetbrains may play into it and I have already isolated some operations that are taking a while within the page using trace.
What are the best practices and tools surrounding performance and load testing? I'm mainly looking to be able to see results not how to accomplish them.
Here is an article showing how to profile using VSTS profiler.
If broken it is, fix it you should
Also apart from all the tools why not try enabling the "Health Monitoring" feature of asp.net.
It provides some good information for analysis. It emits out essential information related to process, memory, diskusage, counters etc. HM with VSTS loadtesting gives you a good platform for analysis.
Check out the below link..
How to configure HealthMonitoring?
Also, for reference to some checklist have a look at the following rules/tips from yahoo....
High performance website rules/tips
HttpWatch is also a good tool to for identifying specific performance issues.
HttpWatch - Link
Also have a look at some of the tips here..
10 ASP.NET Performance and Scalability secret
Take a look at the ANTS Profiler from Red Gate. I use a whole slew of the Red Gate products and am very satisfied!
There are a lot of different paths you can go down. Assuming a MS environment you can leverage some of the team system tools such as MS Team Tester to record tests and perform load testing against your site. These can be set to run as part of an automated build process.
A list of tools is located at: http://www.softwareqatest.com/qatweb1.html#LOAD
Now, you might start off simple. In this case install two firefox plugins: Firebug and YSlow for Firebug. These will give stats and point out issues such as page size, the number of requests made to get the page, etc. They will also make recommendations on some things to fix.
Further, you can use unit tests to execute a lot of the code behind to see what functions are hurting you.
You can do all sorts of testing if u have full MS dev system with TFS and Visual Studio Team Edition. Based on what I see here
I recently had a nice .Net bug which was running rampant. This tool sorta helped, but in your case, I could see it working nicely..
http://www.jetbrains.com/profiler/
Most of the time we've used WCAT from Microsoft. If your searches where bring up ACT then this is probably the tool you want to grab if you are looking for requests per second and the such. Mike Volodarsky has a good point pointing the way on how to grab this.
We use it quite a bit internally when it comes to testing our network infrastructure or new web application and it is incredibly flexible once you get going with it. And it seems every demo Microsoft has done for us with new web tech they seem to be busting out WCAT to show off the improvements.
It's command line driven so it's kinda old school, but if you want power and customization it can't be beat. Especially for free.
Now, we use DotTrace also on our own applications when trying to track down performance issues, and the RedGate tools are also nice. I'd definitely recommend a combination of the two of them. They both give you some pretty solid numbers to track down which part of your app is the slowdown and I can't imagine life without DotTrace.
Visual Studio Test Edition (2008 or 2010) comes with a very good load testing component for ASP.NET apps.
It allows you to get statistics for all the perfmon stats for a server (from basics like CPU and disk waits to garbage collection and SQL locks)
Create a load test for the page and run it, storing the stats in a database for the base line. Subsequent runs can be compared.