We recently had a web app that went out to site acceptance testing where they found severe performance problems related to request size (massive viewstate ASP.net).
We need to ammend our testing strategy to include performance testing, can anyone give us guidance on best practices please?
This is a very broad case to cover, but here are a few of the highlights of things that we do on a regular basis.
DO NOT just test on your network, get remote testing in. LAN connections are very fast, large pages and large load times can go by un-noticed. Ideally get to a place where it mimics the production location in regards to hardware and proximity/connection to the end user.
Use ANTS Profiler or similar tool to profile for expensive methods, and high memory usage.
Test with multiple users, to simulate load. Depending on the nature of the application also load test, either with multiple physical testers or with testing tools that allow you to simulate and script a load scenario.
Review the code to see if objects are retaining viewstate when they shouldn't need to.
I don't know a hard and fast set of "rules" but I find these are good starting points.
In addition to Mitchel's comments above I would recommend conducting load testing as part of your Continuos Integration (CI) process. Visual Studio Team Suite (Test Edition) contains a good load/stress test tool.
Related
I am quite new to running websites in general. I am familiar with statistical profilers for desktop applications, but unsure how to even begin profiling a website as there are a lot of additional potential bottlenecks and I'm not sure what profilers are available for websites.
I have looked around and seen useful suggestions in other questions, but I am not sure they are a very complete solution. The main suggestions are azure performance counters and suggestions from this answer.
Summarizing they are:
Use firebug to determine rendering time and loading time seperately so one can tell whether one has a rendering issue or a server issue.
If server side:
Test a small static page like a page with a single gif. If that is slow one has a CPU issue. Otherwise one is probably IO bound or has problems with database performance.
One can use performance counters to check server aspects such as:
memory
garbage collection
tcp/ip issues
bytes sent / recieved
requests requested, queued, rejected
request wait time, processing time
From my naive standpoint some things that seem to be missing from this list are the sort of profiling one has for a traditional desktop application, i.e. what the stack looked like what percentage of the time (i.e. what functions were we spending time in, and in what context). Another missing item is profiling the database performance, which seems like it may be different on azure than in a local environment especially if one starts dealing with scaling. Another is time spent on requests to third party services, though maybe that can be done with azure performance counters(?).
I apologize for the naive nature of this question. What tools and aspects am I missing here to profile an azure MVC asp.net website and what changes would you make to the above list?
There's a lot of aspects to profiling a site, in terms of database calls, business logic, rendering a view, and even client side performance (any jQuery that might run, for example).
StackOverflow's MiniProfiler is one of the easiest things to get going, just install a NuGet package, add some Javascript includes, and wrap whatever you want to test inside a using() block, and you'll see execution times (including LINQ-to-SQL and EF). You can even create steps if you want finer grained timings of individual calls.
The nice thing about MiniProfiler is you can enable/disable based on the environment, which makes it suitable for running inside Azure (as opposed to say, the Visual Studio Profiler).
You can also look at Azure Performance Counters, which will give you an idea of system resources, but isn't profiling in the sense that MiniProfiler is. It will however give you an idea of network latency and CPU and memory utilization.
Once you're satisfied there, you can use Chrome's Developer Tools to profile your application on the client side. It'll give you an idea of how well your Javascript is doing, including CSS selectors and rendering.
Also worth noting, Visual Studio has a really good Profiler in some higher editions that can give you deep insights into your code. Time spent in methods, call counts, etc.
Between these four methods, you should be able to find most bottlenecks, especially for a first pass.
I've read some of the topics on stress testing but what options are there for testing a site that uses membership to simulate many concurrent users logging/logged in? I guess one could use/create custom scripts but that would also proliferate the database.
How do some of the 'big boys' do it? BTW maybe stress testing isn't exactly what I'm referring too but like a test suite or anything that would be beneficial..
I can think:
Profiling
Application Suite testing. Does this exist for web? It seems difficult to do.
Stress testing.
Performance
Hardening/security
It has a steep(ish) learning curve but I use the free web stress tool from Microsoft called WCat.
I recall reading that this is used by the IIS Team to stress test new features - I guess you could call them 'big boys'? You need IIS to be able to run this of course.
I am looking for a simple web load-testing tool that has a GUI.
I need to run lots of small and simple tests (like hit page x 100 time and let me know how long it took).
I do not want to have to script every test as I would have to using WCAT or AB.
Also free would be nice.
If it matters I am using IIS7.
There are a number of services online that can do this type of testing for you as well. Of course, one of the downsides to this approach is that its harder to correlate the data from the service (which is what can be observed externally) with your own internal data about disk I/O, DB ops, etc. If you end up going this route I would suggest finding a vendor that will give you programmatic access to the raw test result data.
I found JMeter.
It is very advanced (way more than I need) but it has a nice GUI and has very good docs.
http://jmeter.apache.org/
I am on a project helping to analyze the load a VB.Net WinForms application can take. This app has been in production for several years and has many many products on it. We plan to add more products but see the client footprint rapidly increasing. This is contributing the degradation of performance on the system overall.
There is duscussion that moving the UI intensive portions of the app to ASP.Net it will reduce the client footprint and solve many of our issues.
My question which of the following will scale better in terms of performance and load?
- ASP.Net(VB) Web based architecture
- VB.NEt WinForms 3-tier architecture
Links to articles on the topic are also appreciated.
Additional Info
Client - Apparent issue is large memory footprint due to data caching (High cph utilization)
Middle Tier - web services that house BLC & DALC assemblies (Low utilization here)
Database - Multiple database that that serve data to the DALC via sprocs (Medium Utilization)
Deployment is not an issue, we have a very well developent methodology there.
Thanks in advance,
Freeon
Winforms will scale better than ASP.NET
B/c
a. when you use an ASP.NET client - (a Browser) you pay a price, html rendering - another price, Viewstate - a huge price.
about view state - it is a chunk of data that might grow more and more as long as you operate even on the same page.
You need to use special techniques in order to make a asp.net webform efficient (AJAX).
You don't have this on winform.
Anyway - a specific answer should be aware of your product functionality, architecture and design, so this is gust a general advice.
Not enough data...
In terms of a user interface a desktop application should out perform (by various measures) a web based one in all but the most trivial of cases - that's not to say that you can't produce a very decent and capable web application but even then Outlook Web Access is not Outlook on the desktop.
To further illustrate the point, look at the effort going into Silverlight and Adobe AIR which are attempts to provide desktop level capabilites with web level deployment.
So the question becomes one of asking what is it in the current desktop application that is causing the problem i.e. is it a deployment issue, a performance issue or something else?
If its deployment issue then that will suggest one set of solutions, if its a performance issue then things get a lot more interesting.
Either way, there is insufficient data to do anything other than generalise enthusiastically
With a web based application you can scale servers both up and out for both the web server itself and the database servers. I would think that a desktop based application would be somewhat limited in how much scaling can be done, along with the need to update each and every client installation when changes/bug fixes are done.
There are negatives about web based applications. They will live in a stateless enviroment, the UI maybe somewhat slower than a desktop installation. It is possible to create UI's that are very responsive using lots of Ajax/Javascript, but the development time for those RIA needs may be more than desktop development. Connectivity issues maybe be of a concern, along with user browsers and such.
Quick deployment though of updates is one huge benefit of a web based application. You only have to manage one installation rather than many.
Good luck with your project!
I think many developers know that uncomfortable feeling when users tell them that "The application is slow (again)."
In a complex web application there can be many possible reasons for a degradation in (perceived) performance: slow database response, bandwidth issues, bad caching etc. There certainly are issues which will never occur in a development or staging environment.
Now my question:
Is there a set of tools and/or methods which would provide a comprehensive "live" state on a IIS/ASP.NET/SQL Server production system in a visually way (not just performance counters):
Current HTTP requests (say the last n minutes)
Exceptions / timeouts
Bandwidth data
Number of open database connections / database calls
...
The primary goal is to see at a glance (or after looking closer) what problem is causing the performance problems.
I think the category of software you're looking for is ".net profiler" or ".net tracer". One such tool that you might consider is JetBrains' dotTrace. It gives you runtime stack traces and an array of counters that indicate possible bottlenecks.
Previously mentioned tools will certainly work. At our shop we needed finer information and built our own solution (long story: it was easier to code than to argue about tools and retrievable data).
I used LogParser to flip through the IIS logs and create output reports of those logs (e.g. result code breakdowns etc).
I used a combination of performance counters and WMI values to get the rest - you can read these using some pretty straightforward C# - this gives you full control that you can then dump to .csv etc for viewing/processing in excel or if you are updating a page as a control center.
I would probably also look at IIS.net as a great resource for IIS tools including debugging, security etc.
I followed urig's advice and found this software called SmartInspect.
Does anybody know this logging/monitoring tool? It seems to be a combination of real time console and developer library.
CLR 4.5 will have some new capabilities that will help you monitor ASP.NET performance live - without restarting your app. Basically you can re-JIT your code to include some monitoring-hooks in it, and then inspect time spent in classes/methods etc.
I'm sure dotTrace and other profiling tools will leverage this automatically, but it's worth checking out: C9 - Inside Re-JIT with David Broman