I have a simple MVC web app with 4 controllers that behaves fine on one Windows 2008R2 server, but on two new servers, one 2008R2 and one Windows 2012, takes an extremely long time to do the initial JIT compile after publishing.
The normal server takes about 10 seconds to load after a new version is published, the other servers take about 10 minutes for the first page, and several minutes to load the first access of each controller. After the initial compile the pages load normally.
The app targets .Net 4.5 Any CPU Debug mode. The server that works and one of the 'bad' servers have identical .Net 4.6.1 installed, the other has 4.5.2. One of the 'bad' servers has .Net Core 1.0 installed. In ProcMon csc.exe shows up under the w3wp.exe for the 10 minutes, but takes 0% CPU for most of that time. No errors in event viewer. FileMon doesn't show any access errors.
What can cause this extremely slow compile? What else is there to check for the cause of the problem?
Problem was caused by antivirus settings on the VMware host. Excluding the Temporary ASP.Net files folder from the antivirus scan solved the problem:
C:\Windows\Microsoft.NET\Framework64\v4.0.30319\Temporary ASP.NET Files
Related
We have 4 servers load balanced:
4 cores # 2.6Ghz (E5-2650 v2)
14GB RAM
Windows 2012 R2
High Performance power setting
IIS 8.5
ASP 5.3
EF 6.1
They each have a single application pool with one worker process and a single website. Each server has its own copy of the site (DLLs & views), running on a local disk. We are using IIS virtual directories to point to shares on a clustered file server for log files and common images etc (content only). The application pools are set to not shut down when idle (interval of 0) and we have also disabled the every-1740 minute recycle interval too.
We have New Relic's .NET agent installed on all servers, and looking through our slow transaction log, I can see that many requests are taking 15 seconds or so to complete. Looking into the trace, I can see a common call to System.Web.Compilation.AssemblyBuilder.Compile() and System.Web.Compilation.BuildManager.CompileWebFile().
As far as I know or understand, ASP would compile these views upon first request to them, and cache it (to the temporary ASP files in C:\Windows\Microsoft.Net) and then load from there for subsequent requests.
I'm confused how this is happening so often - when I visit these URLs, the TTFB is about 400ms, and due to constant load I can't see the websites "losing" their cache and needing to compile the views again. These pages are frequently hit - it's an e-commerce store and I can see that it happens often, and on our most popular pages: catalogue (category/brand/gender etc) listings and product details.
I've set the settings against each application pool to log an event when recycling, and there have been no events logged when I'm checking the WAS service in the event viewer. We also have New Relic server installed, and looking over the past 6 hours' worth of data, I can't see any dip in RAM usage on any of the servers - which would indicate the application pool recycling. This has really baffled me!
I'm thinking of moving towards pre-compiling our views as part of our release process - it makes sense really. But it feels like that is working around, or masking an issue which as far as I can see should not be happening. We build our site in Release mode and have <compilation debug="false" /> on all web.config files.
Can anyone think of any causes for this?
It is because of how JIT (Just-In-Time) compilation works.
When you build your application, it is converted into .NET Microsoft Intermediate Language (MSIL) or Intermediate Language (IL).
As your applications is accessed, Common Language Runtime (CLR) converts only executed IL parts of your code into native instructions.
Just-In-Time compilation process converts IL to native machine instructions and it is a part of CLR.
In a simplified terms when you run a .NET application and your program calls a method. JIT Compiler reads IL from metadata and compiles it into native instructions and run it. Next when your program calls the same method, CLR executes native CPU instructions directly. This process adds some overhead for the first method call. You can go with the other option of pre-compiling your application using NGEN, which is usually not recommended because you will loose some optimizations that only JIT can perform due to its awareness of underlying hardware platform. These two articles has more details
http://geekswithblogs.net/ilich/archive/2013/07/09/.net-compilation-part-1.-just-in-time-compiler.aspx and https://msdn.microsoft.com/en-us/library/ms366723.aspx
There are also other things you can try that might help you speed up your application. You can use IIS Application warm up module How to warm up an ASP.NET MVC application on IIS 7.5?, implement distributed caching etc to alleviate some of your application bottlenecks.
We have an ASP.NET 2.0 Web Application Pool running on .NET 3.5 and "all of sudden" we started to experience a weird session timeout which is kicking out the users from the system.
From the event viewer we can see a warning message saying that app pool is restarting due to an exception with no much detail on why. As you may guess session is In Proc and due to early mistakes in the application design using state server will require a lot of work and time which we do not have(I guess you never experienced that too) and also I am eager to find what is happening.
At some point we had decided to compile the site using: Aspnet_compiler -v /WebApplication1 this was done in the server itself. When we put both versions to run the compiled and the non-compiled and we noticed the issue occurs way more often in the compile version than the non-compiled version. Like every 5 min. in the compiled version vs 1 or 2 times a day in the non-compile.
What may be affecting the compiled version or forcing the app pool to restart?
There are a number of questions on stackoverflow relating to being 'unable to start debugging on the web server...' but I have variation of this problem: it's intermittent and it goes away if I restart VS2012. This would tend to indicate that some process or other is locking up another process, or a file is locked, etc, etc by devenv.exe (ie VS2012) and closing VS unlocks the file/process/whatever.
The Visual Studio solution contains a number of MVC4 websites that use IIS running on my 64-bit Windows 7 development computer. All the website, domain model and test projects in this VS solution are configured for .NET 4.5. MvcBuildViews is set to true in the CS project file.
I've tried recycling the IIS application pool for the start-up project, stopping and starting the IIS site, resetting IIS completely, to no avail. I have a suspicion the error message may actually be incorrect, but it's only a suspicion. Any ideas anyone?
I developed a project in ASP.Net MVC 3, my hosting is using iis7 (Win Web Serv 2008 R2), and the first request after the website sit's idle (during about 1-2 hours) is very slow.
I use VPS with 512Mb RAM. Can this be related with a too little RAM?
Can anyone help me with possible causes of such behaviour?
After a certain amount of inactivity IIS unloads the AppDomain. And then the first request loads the application once again which is slower. You could try to configure this period in the properties of IIS but there might also be other causes that an application unloads such as for example a certain threshold of CPU or memory usage is reached. Those thresholds are also configurable in IIS.
That's not something specific for ASP.NET MVC. It's true for all ASP.NET applications in general.
We had also this problem with ruby and passenger that takes the app out of memory after a while, but I found a nice application that fixed this issue for us without changing anything in the server configuration, the app is called wekkars, and you can find it here: http://www.wekkars.com
We have an ASP.NET application running on a webfarm. When we release a new version and copy it to the production servers, occasionally it happens that after a few hours the application reverts to a an earlier code base.
Have anyone else experienced something like this? Would sharing an application pool between two applications running different versions of the code make this happen?
Additional information:
3 x web servers running w2k3/iis6
ASP.NET 3.5
I've had this happen rarely on non-precompiled sites and the solution was to stop IIS and clear out the temporary cache at:
%WINDIR%\Microsoft.NET\Framework\v2.0.50727\Temporary ASP.NET Files
Is not a long-term solution, but will address the immediate problem. If not already, it is worth considering deploying the site pre-compiled.