I am working on a .NET WebForms application and I have observed that whenever I build, after the build, the very first page load takes longer to load than usually. This happens even if I wait after building before I load a page. Is there a way to increase human workforce performance by changing IIS/.NET to initialize things on postbuild instead of first page load?
Yes you can, like this.
Quoting:
You can use Application Initialization Module which comes in-box with IIS 8.0, like this:
<applicationInitialization
doAppInitAfterRestart="true" >
<add initializationPage="/" />
</applicationInitialization>
This will send a request to the root of your app (initializationPage="/") every time your app starts automatically.
You can also configure the Start Mode for your application pool to Always Running which means every time IIS restarts, it will make sure to start your application pool immediately (this if from right click on your application pool then Advanced Settings).
Professional servers have hardly any latency, though it requires quite a bit of tweaking. Also, by default, applications recycle regularly on IIS (as well as when some kinds of exceptions occur, when some files are changed, or when some thresholds are reached). Professional web application hosting is anything but simple :) You might get help with that on Server Fault, perhaps.
Another option is to avoid mixing pre-compilation and JIT-compilation - if you only pre-compile, you don't need to do any compilation when the application is deployed, resulting in faster startup times. If you only deploy sources, the application domain doesn't need to be torn down when you make a change, which means that only the change needs to be recompiled, which is much faster.
And of course, ASP.NET Core is much, much faster in both scenarios - it can do the whole compilation in-memory, unlike the legacy system which uses csc to build multiple assemblies, save them to disk, load them from disk, merge them together, save that again, just to load it again and initialize.
Related
I have an asp.net mvc4 application that we deploy to about 1400 clients. Our current deployment process goes like this:
:: Publish the project files to a local directory
%msbuild% RPortal.csproj "/p:Platform=AnyCPU;Configuration=Release;ConfigurationName=Release;SolutionDir=%solutionDir%;PublishDestination=.\Deploy\Release" /t:PublishToFileSystem
And then we have some powershell scripts that push the new build to each of our clients, using other tools to help with the sync.
We are noticing some sever slowness with application boot up times (sometimes upwards of 5 minutes), and one of the approaches we've investigated to solve this would be the idea of ASP.Net Precompiling (http://msdn.microsoft.com/en-us/library/vstudio/bb398860%28v=vs.100%29.aspx)
In some experiments with the idea, it appears that calling aspnet_compile.exe on our published system does, in fact, create some new files. (A couple of dlls, and a few *.compiled files).
My questions are as follows:
How does this compilation differ from the primary compilation from the .cs source
files?
Does this compilation happen regardless, at first run, or only when run manually?
(related to 2), Does this compilation survive app pool restarts and server reboots?
With our current scenario, it seems that we are killing our servers trying to get 1400 applications to start up (yes, they all live on 1 webserver... that's not a situation I am in control of). The server will be humming along, with no particular problems, resonable resource consumption, and then, all of a sudden, our CPU will peg to 100% and stay there. The only factor we can tie it back to is that it happens when more than 5 (or so) of the 1400 apps are all booting up.
Our hope is that precompiling will front load most of the app start burden, but I clearly don't understand whats really going on here.
Any light you might shed would be most appreciated.
My main goal: make development faster by not having to wait as long each time I change the code of my site.
I'm developing an ASP.NET web application in Visual Studio 2010. While developing, I normally run the app with "Control+F5" (ie. start w/o debugging). This starts up the built-in ASP.NET Development Server. However, when I modify code and do this, I get the following:
-Press Control+F5
-The modified project(s) in my solution build
-The web-server is started (if it's not already)
-My browser opens & waits
At this point there's a 15-30 second delay before I see the first page. Reloading a page is instant, as well as modifying an .aspx page and reloading. But changing any code & recompiling causes another 15-30 second delay.
First, I'm trying to see where that time is spent (which is the point of this question). Looking at the log file, there's about 5 seconds until my global.asax.cs file runs. Then a wait of 10-15 seconds, then my Site.master.cs file runs. What is running in that time in between? What files does ASP.NET run and in what order?
Second, I can see that some of this time is spent with "csc.exe", which leads me to believe that the pages are being compiled-on-request. Can I precompile this code (again, using the built-in web server, not IIS) and will this be faster?
I'm open to other suggestions on how to make this faster. I want to minimize the time between modifying code & seeing changes on the site. There are multiple projects in this solution. One project uses the other as a reference. I'm on XP. I can use XP's IIS if that will make things faster. Any ideas?
Thnaks!
If you are curious about the internal processes and architecture of asp.net and IIS work then follow this link.. may be you will get your problem point on which you want to work.
low-level Look at the ASP.NET Architecture
and have look on this question somewhat similar to yours as Aristos comment on your question.
Slow Performance -- ASP .NET ASPNET_WP.EXE and CSC.EXE Running After Clicking Redirect Link
I deployed an ASP.NET web application last night and I when I woke up this morning it was very slow and would occasionally just throw a 'Service Unavailable' error.
I checked the Event Viewer and it was filled up with these errors:
An unhandled exception occurred and the process was terminated.
Exception: System.Runtime.Serialization.SerializationException
Message: Unable to find assembly 'MonoTorrent, Version=0.80.0.0, Culture=neutral, PublicKeyToken=null'
I'm puzzled as it was working perfectly when I deployed it (MonoTorrent is required to retrieve the number of seeders/leechers for a certain torrent off the tracker - this was working fine), but it's no longer working and whenever code that uses MonoTorrent gets involved, the worker process just crashes.
MonoTorrent.dll is in the /bin/ directory.
UPDATE 6/4/10: I compiled the MonoTorrent source code in with the rest of my web application, but it still crashes whenever it uses MonoTorrent. However, it now says that it is Unable to find assembly 'OpenPeer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null. Here, OpenPeer is the name of the web application's assembly.
This can happen in these circumstances:
The ASP.NET app creates a background thread, which throws an uncaught exception. It looks like ASP.NET catches the exception and wants to log it to the Event Log. To do this, it sends this exception from the Web app's app domain to its own app domain (the default one of the w3wp process). This needs a serialization/deserialization of the exception.
If the exception is a custom one (i.e. defined by the Web app), it cannot be deserialized in the main app domain of ASP.NET because the assembly defining the exception is typically in the Web app's bin directory, not where w3wp.exe is (c:\windows\system32\inetsrv). This causes a serialization exception and w3wp crashes.
There are possible ways to fix the issue (in a - very subjective - order of preference):
Copy the missing DLL in c:\windows\system32\inetsrv
Install the missing DLL in the GAC
Remove the cause of the exception (harder to do than to say, as we say in French)
Catch all exceptions from the background thread yourself and do the logging yourself.
Notes:
If WCF is used and the uncaught exception is FaultException, WCF swallows it and there is no crash
If the uncaught exception is in the thread of the Web request, there is a yellow screen of death, not this serialization exception
It really seems like a bug in ASP.NET
The above is actually a summary of my investigations of this issue yesterday and are only a theory. I tested fixes 1 and 4, as well as using FaultException.
Here are some things you can try..
1.) Flush ASP.Net Temp directory. Restart IIS and recycle Application pool.
2.) Make sure your web-application is running in FULL-TRUST if it really needs FULL-TRUST.
3.) Take the Assembly, try to use it in other asp.net application and run the test application on a seperate server. This might help you diagnose the problem. Also try to run the test asp.net app on the same server but in seperate application pool.
4.) Make sure the IIS website of your application is running under the user account with necessary security priviliges. Try running the application under Administratotr as user.
EDIT-1
5.) Also check if the assembly version is the same as mentioned in web.config. If there's a version mismatch then you can do AssemblyBinding Redirection in web.config.
6.) Also try registaering the Assembly in GAC and see if it loads properly.
EDIT-2
7.) Try reconfigring ASP.NET support on the server or maybe framework runtime re-setup may help. This may not be a sure-shot solution but looking at the problem condition we may want to try various solutions.
8.) Make sure you're not missing any critical update of your windows server platform.
I try to give you some ideas - what I do if I was on your position.
First of all I take a long look of the MonoTorrent.dll before some days that you make your question, and I look it again today. I found and the function that load the dll. My first opinion is that something have to do with the permissions.
I hope that you have access to the server - right ?
My first steps is that:
Ensure that your monotorrent.dll actuall have the right permissions to the bin directory, for Read, and execute by your asp.net app. Some times the copy of one dll, did not get the directory permissions buts carriage out his own permissions. To check if your dll have different permissions from the rest, just right click and see Properties | Security, then go to bin directory and do the same, and compare the Security permissions. If they are different then apply again the Directory permissions and make sure that the dll inherited by the directory.
My second step
Download the ProcessMonitor from sysinternals
http://technet.microsoft.com/en-us/sysinternals/bb896645.aspx
Run ProcessMonitor and try to recreate the error, stop it and analyse to see where and why the dll gets the denied permissions to run.
With the ProcessMonitor you can even see if there is any dll that can not found !
I have check the MonoTorrent dlls and I do not found anything unusual. He have kerner32.dll calls, and use unsafe code to run, ok nothing so special about.
So if you do that 2 steps and give me some feedback, maybe I can go further. (if not solve by you and what you find)
I would advice to setup Regular maintanence probably once in a week at sunday night etc for following,
Delete all temporary files
Delete all ASP.NET IIS temporary files
Restart Server
Problem is, ASP.NET web apps cause lot of temp files to be left in the disk, because of dynamic compilation of regex, seriliazation assemblies etc, such temp stuff never gets deleted, and more and more junk starts getting collected in temp locations, ASP.NET goes slower and slower, and a point comes in where disk as well as memory defragmentation reaches very high point, things start to fail.
No body likes to restart server once a week, but I remember we had no choice, in ASP.NET 1.1 we had stable system after restarting every day, in ASP.NET 2.0 onwards, we are good to have restarting scheduled at once a week.
I have found this problem and I have do all of thing as I can, such as clear temp file, restart server, delete and add reference and I also rebuild the solution. However I can't solving this problem. Finally I move my entity class (almost of them need to serialize) to new folder that I have added to the project and then this problem solved.
This method is work for me.
Try clearing the ASP.NET temp files. It's solved some odd issues before for me.
Otherwise, Fusion-logging may shed some light.
UPDATE: #Charlie - I'm not sure what to make of those logs...it looks like the failed log is from a different AppDomain. Notice the AppBase is set to "file:///c:/windows/system32/inetsrv/" and AppName is w3wp.exe.
I'm pretty sure the Event Viewer should show Application Id: LM/W3SVC/#/ROOT if it was the default AppDomain, too. At this point, all I've got is random guesses.
I notice you're running x64...does MonoTorrent perhaps require x86?
Have you double checked that the directory is an IIS application, and is configured for the correct version of ASP.NET?
Is there some other application that uses MonoTorrent on this server? Maybe a WCF service or something? I'm not sure where the Serialization is happening....
Try hooking the AssemblyResolve event and loading it manually.
Can you repro on a development machine? If not, maybe it's a borked FX install. Uninstall and reinstall.
Does restarting, recycling or stopping/starting the AppPool fix the issue temporarily, or cause the issue to appear?
You may want to type out your screenshot text too so you'll get some Google love....
Is the server timezone different than your timezone? I've had this issue when deploying resource files, the compile time was in the future so they would fail to load.
My guess that you have plenty of open but not closed connections. I mean the connections are not returned to the pool. It looks okay, when you start the application, but after some time there are only several sockets available in the pool and it goes slow. Another thing - non-closed connection may keep DLL in memory, not allowing to release the handler. Try to debug object destruction.
I know it's simple but I had this problem once and itwas because I had a Web Application project which contains
References
Folder and I just copied my files into a
Bin
folder, in any .net web application in the Project Properties windows, a Reference Path tab is available which by default should nothing be include on it. check this option and also Build tab in Project Properties window which Output path be as the same as bin\
I have an ASP.NET 3.5 website running under IIS7 on Windows 2008.
When I restart IIS (iisreset), then hit a page, the initial startup is really slow.
I see the following activity in Process Explorer:
w3wp.exe spawns, but shows 0% CPU
activity for about 60 seconds
Finally, w3wp.exe goes to 50% CPU for
about 5 seconds and then the page
loads.
I don't see any other processes using CPU during this time either. It basically just hangs.
What's going on during all that time? How can I track down what is taking all this time?
We had a similar problem and it turned out to be Windows timing out checking for the revocation of signing certificates. Check to see if your server is trying to call out somewhere (e.g. crl.microsoft.com). Perhaps you have a proxy setting incorrect? Or a firewall in the way? We ultimately determined we had enough control over the server and did not want to 'call home', so we simply disabled the check. You can do this with .NET 2.0 SP1 and later by adding the following to the machine.config.
<runtime> <generatePublisherEvidence enabled="false"/> </runtime>
I am not sure if you can just put this in your app.config/web.config.
IL is being converted into machine native code (Assembly) by the Just-In-Time compiler and you get to wait while all the magic happens.
When compiling the source code to
managed code, the compiler translates
the source into Microsoft intermediate
language (MSIL). This is a
CPU-independent set of instructions
that can efficiently be converted to
native code. Microsoft intermediate
language (MSIL) is a translation used
as the output of a number of
compilers. It is the input to a
just-in-time (JIT) compiler. The
Common Language Runtime includes a JIT
compiler for the conversion of MSIL to
native code.
Before Microsoft Intermediate Language
(MSIL) can be executed it, must be
converted by the .NET Framework
just-in-time (JIT) compiler to native
code. This is CPU-specific code that
runs on the same computer architecture
as the JIT compiler. Rather than using
time and memory to convert all of the
MSIL in a portable executable (PE)
file to native code. It converts the
MSIL as needed whilst executing, then
caches the resulting native code so
its accessible for any subsequent
calls.
source
Thats the compilation of asp.Net pages into intermediate language + JIT compilation - it only happens the first time the page is loaded. (See http://msdn.microsoft.com/en-us/library/ms366723.aspx)
If it really bothers you then you can stop it from happening by pre-compiling your site.
EDIT: Just re-read the question - 60 seconds is very long, and you would expect to see some processor activity during that time. Check the EventLog for errors / messages in the System and Application destinations. Also try creating a crash dump of the w3wp process during this 60 seconds - there is an chance you might recognise what its doing by looking at some of the call stacks.
If it takes exactly 60 seconds each time then its likely that its waiting for something to time out - 60 seconds is a nice round number. Make sure that it has proper connections to the domain controllers etc...
(If there are some IIS diagnostic tools that would do a better job then I'm afraid I'm not aware of them, this question might be more suited to ServerFault, the above is a much more developer-ish approach to troubleshooting :-p)
I found that there was a network delay making an initial connection from the front end web server to the database server.
The issue was peculiar to Windows 2008 and our specific network hardware.
The resolution was to disable the following on the web servers:
Chimney offload state
Receive window auto-tuning level
Greater than 60 seconds sounds fishy. Try running a test.html page to see how long that takes. That will isolate IIS7's role.
Then temporarily rename your web.config, global.asax and application folders and try a test.aspx page (very simple page). That will isolate ASP.NET.
If both of those are fast (i.e. about 10 seconds), then it's your application. But, if either are slow then not the application and something with the server itself.
This hat nothing to do with JIT compiling. The normal C# compiler compiles your code behind files (.aspx.cs) into intermediate language into an assembly at startup if this assembly dont exist or code files have changed. Your web site assembly is located in the "bin" folder of your web site.
In fact the JIT compiling occures after that, but this is very fast and won't take several minutes. JIT Compiling happens on every startup of an .net application and that won't take more than a view seconds.
You can avoid the copmpiling of your web site if you deploy the already compiled website assembly (YourWebsite.dll) into the bin folder. It is also possible to deploy only the aspx files and leave the code behind files (aspx.cs) files away.
I've just been battling a similar issue. For me it turned out to be that I had enabled internal logging for NLog. It added about 3 minutes to the startup time!
Original config
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
throwExceptions="false" throwConfigExceptions="false"
internalLogLevel="Debug"
internalLogFile="C:\Temp\NLog.Internal.txt">
Fixed Config
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
autoReload="true"
throwExceptions="false" throwConfigExceptions="false">
For Info I discovered this by using SysInternals' ProcMon.exe, filtering on the Process Name "w3wp.exe"
I have deployed an application written in ASP.NET 2.0 into production and it's experiencing some latency issues. Pages are taking about 4-5 seconds to load. GridView refreshing are taking around the same time to load.
The app runs fine on the develpment box. I did the following investigation on the server
Checked the available memory ... 80% used.
Cheched the processor ... 1%
Checked disk IO from perfmon, less than 15%
The server config is
Windows Server 2003 Sp2
Dual 2.0 GZH
2GB RAM
Running SQL Server 2005 and IIS only
Is there anything else I can troubleshoot? I also checked the event log for errors, it's clean.
EDITED ~ The only difference I just picked up is on the DEV box I am using IE7 and the clients are using IE6 - Could this be an issue?
UPDATE ~ I updated all clients to IE8 and noticed a 30% increase in the performance. I finally found out I left my debug=true in the web.config file. Setting that to flase got the app back to the stable performance... I still can't believe I did that.
First thing I would do is enable tracing. (see: https://web.archive.org/web/20210324184141/http://www.4guysfromrolla.com/webtech/081501-1.shtml)
then add tracing points to your page generation code to give you an idea of how long each part of the page build takes:
System.Diagnostics.Trace.Write(
"Starting Page init",
"TraceCheck");
//Init page
System.Diagnostics.Trace.Write(
"End Page init",
"TraceCheck");
System.Diagnostics.Trace.Write(
"Starting Data Fetch",
"TraceCheck");
//Get Data
System.Diagnostics.Trace.Write(
"End Data Fetch",
"TraceCheck");
etc
this way you can see exactly how long each stage is taking and then target that area.
Double check that you application is not running in debug mode. In your web.config file check that the debug attribute under system.web\compilation is set to false.
Besides making the application run slower and using more system memory you will also experience slow page loading since noting is cached when in debug mode.
Also check your page size. A developer friend of mine once loaded an entire table into viewstate. A 12 megabyte page will slip by when developing on your local machine, but becomes immediately noticeable in production.
Are you running against the same SQL Server as in your tests or a different one?
In order to find out where the time's coming from you could add some trace statements to your page load, and then load the page with tracing turned on. That might help point to the problem.
Also, what are the specs of your development box? The same?
Depending on the version of visual studio you have, Team Developer has a Performance Wizard you might want to investigate.
Also, if you use IE 8, it has a Profiler which will let you see how long the site takes to load in the browser itself. One of the first things to determine is whether the time is client side or server side.
If client side, start looking at what javascript you have and optimize / get rid of it.
If server side, you need to look at all of the performance counters (perfmon). For example, we had an app that crawled on the production servers due to a tremendous amount of JIT going on.
You also need to look at the communication between the web and database server. How long are queries taking? Are the boxes thrashing the disk drives? etc.