I think many developers know that uncomfortable feeling when users tell them that "The application is slow (again)."
In a complex web application there can be many possible reasons for a degradation in (perceived) performance: slow database response, bandwidth issues, bad caching etc. There certainly are issues which will never occur in a development or staging environment.
Now my question:
Is there a set of tools and/or methods which would provide a comprehensive "live" state on a IIS/ASP.NET/SQL Server production system in a visually way (not just performance counters):
Current HTTP requests (say the last n minutes)
Exceptions / timeouts
Bandwidth data
Number of open database connections / database calls
...
The primary goal is to see at a glance (or after looking closer) what problem is causing the performance problems.
I think the category of software you're looking for is ".net profiler" or ".net tracer". One such tool that you might consider is JetBrains' dotTrace. It gives you runtime stack traces and an array of counters that indicate possible bottlenecks.
Previously mentioned tools will certainly work. At our shop we needed finer information and built our own solution (long story: it was easier to code than to argue about tools and retrievable data).
I used LogParser to flip through the IIS logs and create output reports of those logs (e.g. result code breakdowns etc).
I used a combination of performance counters and WMI values to get the rest - you can read these using some pretty straightforward C# - this gives you full control that you can then dump to .csv etc for viewing/processing in excel or if you are updating a page as a control center.
I would probably also look at IIS.net as a great resource for IIS tools including debugging, security etc.
I followed urig's advice and found this software called SmartInspect.
Does anybody know this logging/monitoring tool? It seems to be a combination of real time console and developer library.
CLR 4.5 will have some new capabilities that will help you monitor ASP.NET performance live - without restarting your app. Basically you can re-JIT your code to include some monitoring-hooks in it, and then inspect time spent in classes/methods etc.
I'm sure dotTrace and other profiling tools will leverage this automatically, but it's worth checking out: C9 - Inside Re-JIT with David Broman
Related
I am quite new to running websites in general. I am familiar with statistical profilers for desktop applications, but unsure how to even begin profiling a website as there are a lot of additional potential bottlenecks and I'm not sure what profilers are available for websites.
I have looked around and seen useful suggestions in other questions, but I am not sure they are a very complete solution. The main suggestions are azure performance counters and suggestions from this answer.
Summarizing they are:
Use firebug to determine rendering time and loading time seperately so one can tell whether one has a rendering issue or a server issue.
If server side:
Test a small static page like a page with a single gif. If that is slow one has a CPU issue. Otherwise one is probably IO bound or has problems with database performance.
One can use performance counters to check server aspects such as:
memory
garbage collection
tcp/ip issues
bytes sent / recieved
requests requested, queued, rejected
request wait time, processing time
From my naive standpoint some things that seem to be missing from this list are the sort of profiling one has for a traditional desktop application, i.e. what the stack looked like what percentage of the time (i.e. what functions were we spending time in, and in what context). Another missing item is profiling the database performance, which seems like it may be different on azure than in a local environment especially if one starts dealing with scaling. Another is time spent on requests to third party services, though maybe that can be done with azure performance counters(?).
I apologize for the naive nature of this question. What tools and aspects am I missing here to profile an azure MVC asp.net website and what changes would you make to the above list?
There's a lot of aspects to profiling a site, in terms of database calls, business logic, rendering a view, and even client side performance (any jQuery that might run, for example).
StackOverflow's MiniProfiler is one of the easiest things to get going, just install a NuGet package, add some Javascript includes, and wrap whatever you want to test inside a using() block, and you'll see execution times (including LINQ-to-SQL and EF). You can even create steps if you want finer grained timings of individual calls.
The nice thing about MiniProfiler is you can enable/disable based on the environment, which makes it suitable for running inside Azure (as opposed to say, the Visual Studio Profiler).
You can also look at Azure Performance Counters, which will give you an idea of system resources, but isn't profiling in the sense that MiniProfiler is. It will however give you an idea of network latency and CPU and memory utilization.
Once you're satisfied there, you can use Chrome's Developer Tools to profile your application on the client side. It'll give you an idea of how well your Javascript is doing, including CSS selectors and rendering.
Also worth noting, Visual Studio has a really good Profiler in some higher editions that can give you deep insights into your code. Time spent in methods, call counts, etc.
Between these four methods, you should be able to find most bottlenecks, especially for a first pass.
My last project is a medium size asp.net web forms application. It is built using:
asp.net 3.5
ling to sql dbml --> sql server database (9 tables)
ext.net 1.6 (www.ext.net)
structuremap 2.5.3.0
This time I believed I did my best in terms of architectural design, code and data transfer optimizations. I followed all advice I could to work with the database efficiently through linq to sql and I built layers (model, repository, service, presentation) to separate concerns and lightweight the code in the aspx code behind files.
The problem is: I've installed the application in various web hosting servers with the same pitiful result: the application is struggling to work... pages are loading like in slow motion...
In the past I would say 'OK, I didn't do all I could to speed things up' but in this case I really tried to apply the best practices...
Is there anything else I can do about it? Or is it just asp.net for really small projects only?
thank you.
ASP.NET is fine for building large scale websites. As Brad mentioned, StackExchange sites are built using it, and StackOverflow is a very busy site indeed.
What you need to do first is measure performance; until you do that, you're just guessing at where the problem areas are.
So start with the browser - use a tool such as Firebug, or YSLOW, Google Chrome dev tools, whatever takes your fancy and run your site using the tool enabled. The tools can let you know how long things are taking to process eg requests, how long content is taking to download etc.
YSLOW will also give you some tips on anything it finds as being a bit slow e.g. you're making to many HTTP requests, you should consider minifying your CSS/JS files. You will get a general overview of how the site is performing and where problems could be.
To dig a bit deeper, use a tool like RedGate's ANTS Profiler, use the trial version and measure your website, and server side code, with that tool. There are other tools, though I'm not aware of any free ones.
My first question is that when its slow. Did you try your project in Local area network. Please check first there. If there slow then you need to improve little bit.
This slow performance depends on many things
such as large data load, multiple logic on one page etc.
Please let me know.
Thanks
Basit.
I have just started working over a website written in .net 2.0. Pages take long to load and response time is quite low, not sure where to start from in order to improve performance of the same.
Hardware is not a problem as there is enough memory and processor is also good enough.
Any Idea where should I start from and to improve the performance.
You can use a tool called Antz Profiler, and run your web application locally against this.
http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/. This will help you identify the methods that are expensive in the application. You can then break down the calls and try to identify whether the code is expensive or maybe DB calls.
Try and refactor and identify any code that you think could be improved, and then once this is done, move onto external calls.
If you have DB calls, then you can use SSMS to identify any issues in the query. When you run the queries, you can use 'Include Actual Execution Plan' to show you where the bottle-necks are.
Tutorial for SSMS query optimization : http://www.simple-talk.com/sql/sql-training/the-sql-server-query-optimizer/
Walkthrough for Antz: http://www.red-gate.com/products/dotnet-development/ants-performance-profiler/walkthrough
There is also a tips link here for general things to check for page speed: http://www.aspnet101.com/2010/03/50-tips-to-boost-asp-net-performance-part-i/
Also, you can use tools like YSlow in Firefox to check the http requests etc to see where you can reduce network calls.
You start by profiling the application to find where it spends most of its time.
When you find that, you come up with an idea of how to fix the top bottleneck.
Once you tried a fix, profile again to find out if you have made a difference - if not, rollback the change and try something else.
If successful, repeat the above process.
Popular profilers are dotTrace and ants profiler, both commercial products.
I have a slow asp.net program running. I would like to profile the production server to see what is going on, but I don't want to slow down the production server noticeably.
In general, is it standard practice to profile a production box or just local dev boxes? Also, what progams do you recommend to accomplish this?
I can recommend you to use "dynatrace Ajax edition 3" for client side profiling (it's free and easy tool) and "JetBrains dotTrace" for server side profiling. This tools does not slow down server as i know.
You can use Tracing and it is recommended to check these things on your local machine, but if you want to check something on server, you can enable tracing for short in your web.config.
ASP.NET tracing enables you to view diagnostic information about a single request for an ASP.NET page. ASP.NET tracing enables you to follow a page's execution path, display diagnostic information at run time, and debug your application. ASP.NET tracing can be integrated with system-level tracing to provide multiple levels of tracing output in distributed and multi-tier applications.
ASP.NET Tracing Overview
Tracing in ASP.NET
I guess the answer is really 'it depends'! I would start by considering whether the program runs slowly just on the production server, or whether it runs slowly on a development environment as well. I would also consider how closely I could get my development/test environment to match the production environment.
Once you've done that, consider whether there are any areas that could represent obvious bottlenecks that you might be able to eliminate. So, for example, is the ASP.NET application backed by some form of database? If it is, you can monitor the performance of the database separately and establish whether that is where the problem lies.
Next, try and be very specific about what you mean by 'slow performance'. Is it consistently slow (compared to what?), or just when you do specific actions. This may give you another clue as to where your problem lies, or at least what questions you should be asking.
Having answered lots of these questions, I'd then bust out ANTS Performance Profiler to try and profile what's going on. It has a fairly minimal overhead when profiling an application, and you should only really be running it for a fairly short time anyway, as you'll hopefully by this point have more specific questions you want to answer, or specific actions that you want to dig into.
Your best option is Prefix (http://www.prefix.io). It will let you see all of your SQL queries, logs, HTTP calls, and a lot more.
Another option is Glimpse or the Mini profiler.
I've had sporadic performance problems with my website for awhile now. 90% of the time the site is very fast. But occasionally it is just really, really slow. I mean like 5-10 seconds load time kind of slow. I thought I had narrowed it down to the server I was on so I migrated everything to a new dedicated server from a completely different web hosting company. But the problems continue.
I guess what I'm looking for is a good tool that'll help me track down the problem, because it's clearly not the hardware. I'd like to be able to log certain events in my ASP.NET code and have that same logger also track server performance/resources at the time. If I can then look back at the logs then I can see what exactly my website was doing at the time of extreme slowness.
Is there a .NET logging system that'll allow me to make calls into it with code while simultaneously tracking performance? What would you recommend?
Every intermittent performance problem I ever had turn out to be caused by something in the database.
You need to check out my blog post Unexplained-SQL-Server-Timeouts-and-Intermittent-Blocking. No, it's not caused by a heavy INSERT or UPDATE process like you would expect.
I would run a database trace for 1/2 a day. Yes, the trace has to be done on production because the problem doesn't usually happen in a low use environment.
Your trace log rows will have a "Duration" column showing how long an event took. You are looking at the long running ones, and the ones before them that might be holding up the long running ones. Once you find the pattern you need to figure out how things are working.
IIS 7.0 has built-in ETW tracing capability. ETW is the fastest and least overhead logging. It is built into Kernel. With respect to IIS it can log every call. The best part of ETW you can include everything in the system and get a holistic picture of the application and the sever. For example you can include , registry, file system, context switching and get call-stacks along with duration.
Here is the basic overview of ETW and specific to IIS and I also have few posts on ETW
I would start by monitoring ASP.NET related performance counters. You could even add your own counters to your application, if you wanted. Also, look to the number of w3wp.exe processes running at the time of the slow down vs normal. Look at their memory usage. Sounds to me like a memory leak that eventually results in a termination of the worker process, which of course fixes the problem, temporarily.
You don't provide specifics of what your application is doing in terms of the resources (database, networking, files) that it is using. In addition to the steps from the other posters, I would take a look at anything that is happening at "out-of-process" such as:
Databases connections
Files opened
Network shares accessed
...basically anything that is not happening in the ASP.NET process.
I would start off with the following list of items:
Turn on ASP.Net Health Monitoring to start getting some metrics & numbers.
Check the memory utilization on the server. Does re-cycling the IIS periodically remove this issue (memory leak??).
ELMAH is a good tool to start looking at the exceptions. Also, go though the logs your application might be generating.
Then, I would look for anti-virus software running at a particular time or some long running processes which might be slowing down the machine etc., a database backup schedule...
HTH.
Of course ultimately I just want to solve the intermittent slowness issues (and I'm not yet sure if I have). But in my initial question I was asking for a rather specific logger.
I never did find an answer for that so I wrote my own stopwatch threshold logging. It's not quite as detailed as my initial idea but it has the benefit of being very easy to apply globally to a web application.
From my experience performance related issues are almost always IO related and is rarely the CPU.
In order to get a gauge on where things are at without writing instrumentation code or installing software is to use Performance Monitor in Windows to see where the time is being spent.
Another quick way to get a sense of where problems might be is to run a small load test locally on your machine while a code profiler (like the one built into VS) is attached to the process to tell you where all the time is going. I usually find a few "quick wins" with that approach.