getting:
aspnet_wp.exe (PID: 988) was recycled because memory consumption exceeded the 148 MB (60 percent of available RAM).
any suggestion of web.config etc optimization? did already all the debug/release compile stuff. but still it takes to fast to much memory. machine got 256mbs ram, on 512mb it runs smooth. want to squeeze it down as much as possible. within code it did also as much to keep memory low, but it kind of 50mb data only. this must be possible. or does the framework need so much?
You can try de-configuring all of the HttpModules that you don't use. Windows Authentication, for example, is a common one.
You might also take a look at any static variables that you're allocating.
Disabling ViewState on as many pages as possible might help a little.
Minimize or eliminate any extra DLLs that get loaded (from your bin directory or the GAC).
However, it's unlikely that you'll save much memory by hunting-and-pecking like that. If you're serious about chasing it down, you'll need a memory profiler tool, such as .NET Memory Profiler from SciTech. They have a free two week trial.
Well, it should fit, but it depends on what you're doing with it, and which version of IIS you're running.
Optimizing Memory Usage (IIS 6.0)
Servers running IIS 6.0 benefit from ample physical memory. Generally, the more memory that you add, the more the servers use and the better they perform. IIS 6.0 requires a minimum of 128 MB of memory; at least 256 MB is recommended. If you are running memory-intensive applications, your server might require much more memory to run optimally — more than the recommended 256 MB of memory.
IIS 7.0 however has somewhat larger requirements:
Minimum: 512 MB
Recommended: 2 GB or more
That article has a few other recommendations on optimisations you can carry out, I'd also recommend Tess Ferrandez's blog, especially her post ".NET Memory usage - A restaurant analogy" which explains memory allocation nicely, and her other posts on debugging memory usage - which start in a similar place to RickNZ's suggestions.
What are you putting in memory?
256mb is way below what a server would have, even if it was shared server you'd likely have more space to work with than that.
Are you caching?
What OS are you running?
Like RickNZ said trimming as much fat out as you can might save you some memory.
Honestly I think you simply have an unrealistic memory target.
Related
The screenshot below shows the change in cache related state of my zope instance over time (3 months so far).
We've increased the cache size several times over the period from 3000 all the way up to 6000000. With the exception of one recent blip, we have hit a ceiling of 30 Million (not sure what parameter that is) (see the 'by year' graph). This happened at a cache size of about 1000000, after which, changes to the cache size seemed to have no effect on the cached objects or memory usage of zope.
The zope/plone process moved from using about 500 MB of memory to using 3GB (we have 8GB on this server).
What I expected was that sliding the cache size upwards would allow zope to take advantage of more of the available server memory, but it is stuck at 3GB (out of a potential 8GB on the server).
Is there another setting that might be "capping" things at 3GB?
At a guess, your OS is limiting per-process memory size.
In a bash shell, check ulimit -v to see if a virtual memory limit is set. See man bash for the full list of options for that command.
See Limit memory usage for a single Linux process for more information on how to use ulimit.
I don't know what's going on with the memory of your server but you are doing this the wrong way: you simply can't have 6 million objects on memory, that's impossible: on a typical installation in Plone 4.x you should need somewhere in between 50 GB and 150 GB of memory for that.
to solve this we need more information: which Plone version are you using? how many objects do you have in your database? how many threads? what's the architecture of your server, 32 or 64-bit?
second, be sure to install the latest version of the munin.zope plugin to collect reliable information about your server (hat tip to Leo Rochael).
third, read this thread on the core developers list to understand how to calculate a more realistic number for your cache size (hat tip to Hanno Schlichting).
fourth, move the number up slowly and take time to monitor the results; check for total number of objects in memory and avoid memory swaps at any cost. You can stop increasing cache size if you see the number of objects is below your goal value. remember: you're never gonna have all the objects in memory, that's quite difficult because people tend to visit mostly only a subset of your content.
fifth, if you are in Plone 4.x test DateTime 3.0.3 (on a staging server before put it in production) this could decrease further your memory consumption by up to 40% (somebody told me it now works also in Plone 3.x, but I haven't check it my self).
sixth, share your result on the Plone setup list!
good night and good luck!
A 32-bit platform -- don't know if this is limited to Intel -- is limited to 3GB per process. That is because it can only address up to 4GB per process, and the bottom 1GB is used by the kernel. Of course PAE allows you to access up to 64GB, but there are certain per process limitations which you are running into here. You really cannot run a high-traffic plone site on a 32-bit platform anymore. Quite often the simplest solution is to upgrade your OS to the 64-bit version, because unless you have seriously ancient hardware, it should already be capable of running x86-64.
I have a website application running in it's own application pool on IIS 7.0. The application is an ASP.NET MVC 3 website.
I have noticed the memory usage for this applications corresponding w3wp IIS worker service is quite high ( 800 MB, with some fluctuation ).
I am trying to diagnose the problem and have tried the following:
I have disabled output page caching for the website at IIS level and then recycled the application pool. This causes the w3wp process to restart. The memory usage for this process then slowly creeps up to around 800 MB, it takes around 30 seconds to do so. There are no page requests being handled at this time. When I restart the website from IIS the memory size of the process does not alter.
I have tried running a debug copy of the application from VS 2010, there are no problems with memory usage.
Some ideas I have/questions are:
Is this problem related to the websites code? - Given that the memory rockets before any page requests have been sent/handled, I would assume this is NOT a code problem?
The application built in MVC has no handling of caching written into it.
The website uses real-time displaying of data, it uses ajax requests periodically, and is generally left 'open' for long periods of time.
Why does the memory usage rocket up after the application is recycled and no user requests are being sent? Is this because it is loading old cache information into it's memory from disk?
The application does NOT crash, I'm just concerned about the memory usage, it is not that big of a website...
Any ideas/help with getting to the bottom of this problem would be appreciated.
The best thing to do if you can afford to use a debugger is install the Windows Debugging Tools and use something like WinDbg and SOS.dll to figure out exactly what is it in memory.
once you've installed the tools then you can:
Launch Windbg.exe running elevated (as Administrator)
Use "File->Attach To Process" and choose w3wp.exe for the app you are trying to figure out. If you have many you can use Task Manager and add the command-line column to see the PID or use IIS Manager->Worker Processes to figure it out, and then choose that process in WinDBG.
run:
.loadby sos clr
!dumpheap -stat
At that point you should be able to see all types sorted by the most memory consumption so you can start with the ones at the bottom. (I would recommend exclude Strings, and Object since those are usually a side-effect and not the cause).
Use "!dumpheap -type type-here" to find the instances and use !gcroot for those to figure out why they are in memory, maybe due to a static field, or an event handler leaked, WCF channels not disposed, or things like that are common sources.
I just looked my server and my pools use 900-1000 MB Virtual size Memory, and 380 MB Working set. My sites run smooth with out problem for some years now, and I have checked the sites from all sides. My pool never recycles and the server runs until the next update continuously with 40% stable free physical memory.
If your memory is not continuously growing, then this memory is the code plus the data that you set as static, const, the string, and the possible cache, inside your application.
You can use process explorer to see the working and the virtual size memory.
You can also think to run a profile against your code to see if you have any "memory leak" or other issue. Find one from google: https://www.google.com/search?hl=en&q=asp.net+memory+profiler.
It probably doesn't apply here but thought I would throw it in for good measure. Recently I had a problem where my memory would go right up and max out when it really could of cleaned up 80% of it. Problem: It thought it about 2 more gig than it actually did so the GC was quite lazy. (It was due to a VM ware bug -windows was reporting 8 Gig but physically there was only 6.4). See blog.http://www.worthalook.net/2014/01/give-back-memory/
Something that might help: if you "rewrite" (open/save) the web.config , then your application will reset, you should monitor the memory usage from that point. If it keeps growing during usage, this could mean memory leak or insane caching. You might be able to identify which actions on your site lead to memory increase. During a long time the memory usage of an application should be stable.
My App pool is taking like 180mb to 220mb at any given time.
It sometimes goes down to 80mb but comes back to 180mb in few mins.
Is this behaviour normal? If the memory usage seems high, how can i reduce it?
We have like 500 employees of which at any given time atleast 200 employees will be working on that particular website.
I am using IIS 7.0, windows server 2008, Asp.net 3.5
Any help is greatly appreciated.
Abhi
It is totally dependent on your site. 180-220 mb is nothing. On 32bit windows you have to worry around 600mb. 64bit windows, it can be much higher.
Right click on your App Pool in IIS and choose Advance Settings... then scroll down and look for Private Memory Limit (KB) and Virtual Memory Limit (KB) near the very bottom. However like #BNL suggests your usage is nothing to really be concerned about.
Yes, the behavior you describe sounds normal. Garbage collection, among other things, can cause periodic fluctuations in memory use.
Unless your server is starting to page excessively, I wouldn't restrict the memory available to the AppPools. Internal ASP.NET features such as caching work better when they have plenty of memory available.
If you're concerned that the memory use is higher than it should be, then consider running your apps through a memory profiler, to find out how the memory is being used.
I'd like to describe strange issue I've noticed while analyzing my asp.net application in production and ask for some advice or opinion on the following matter.
Application usually runs with some 80-90 MB of memory footprint. This seems stable since no memory leaks have been detected so far - no slight increase in memory usage over time. Yet, problem occurs when application pool recycles (I'm using shared hosting and judging by logs it occurs either when app is idle for 20 mins or every ~30 hours - something like that). The issue is that used memory almost doubles for some period on recycle - it goes to some 160-170 MBs without any explanation. This is confusing, since it is common claim that recycling should purge the memory and all other resources - at least I get it that way. System holds this amount of memory for some 7-8 hours and then memory usage drops to it's usual level of 90-100 MB, again, with no apparent reason (at least not know to me). All the time, application seems to work well - no significant delays or troubles with site availability - to the users everything seems OK, no complaints so far. When looking at the memory consumption over time graph - it looks almost like a step function.
The important thing is that I haven't been able to reproduce this sort of behavior in my testing environment. Occasionally, I've been getting notes from provider administrators that my app is using more resources than allowed and this really bugs me.
So, what I would like to know - is there any possible scenario where application pool recycling does not release all memory resources ? Is there any advice or guideline what I should focus on ? I'm not an expert in this area, but I've been reading about things like overlapping recycling, serialization problems on recycle and couple more issues ... Any ideas ? Similar experience ?
Thanks
This post provides a pretty good overview of what happens when your site's app pool is recycled: http://blogs.msdn.com/b/tess/archive/2006/08/02/asp-net-case-study-lost-session-variables-and-appdomain-recycles.aspx
My speculation is that your memory usage is increasing due to the JIT-compilation that follows every recycle of the app pool. My guess is that your shared host has different configuration and environmental settings than your development server.
IMHO, if you're using ~100 megs of memory on a shared host, you are asking for trouble if it's a host like DiscountASP.NET or GoDaddy. If you care at all about that website, go get a VPS or some more configurable hosting where you can pay a premium for a higher memory limit.
I trying to track down the cause of an OutOfMemory for a website. This site has ~12,000 .aspx pages and the last time it crashed I captured a memory dump using adplus.
After some investigation I found a lot of heap fragmentation, there are around 100MB of Free blocks which can't be assigned. Digging deeper one of the Large Object Heaps is fragmented and the causes seems to be String interning as described [here][1]
Could this be caused by the number of pages in the site? As they are all compiled they sit in memory and by looking at the dump they are interned and PINNED which I think means they stick around for a while.
I would find this odd as there are many sites with more pages, but dynamic compilation could account for the growth in memory.
What other methods are there for finding the cause of the memory leak? I have tried to capture a dump using adplus in hang mode but this fails and the IIS worker process get recycled.
[1]: • Large Object Heap Fragmentation
Yep fragmentation is one of the reason because it will be more difficult to 'find' memory.
Some very known clues :
On a 32bit OS, your worker process can't exceed 2 GB in memory even if your system have more. An alternative is the /3GB switch (boot.ini file).
The likelihood of experiencing an OutOfMemoryException begins to increase dramatically when “Process\Virtual Bytes” is within 600 MB of the virtual address space limit (generally 2 GB), and secondly, tests have shown that “Process\Virtual Bytes” is often larger than “Process\Private Bytes” by no more than 600 MB
The CLR is allocating and handling the memory by blocks (64 MB from memory;)). It does not mean it use all this memory, it means one block will not be freed until one byte is used and it's how the memory become fragmented.
When the CLR can't allocate memory (new block needed), and it can't garbage collect or page some of the memory, OutOfMemoryException is going to happen in the best case or the server will be overloaded in the worst case.
About dynamic compilation, that's true but the most important is the <compilation debug="false" /> / <deployment retail=”true”/> switches. They turn on batch compilation which mean one DLL by directory (this is another point) and not one DLL per 'page' which cause virtual space fragmentation.
About one DLL by directory, even with batch compilation, more directories/sub-directories you have, more DLL you will have and more fragmented virtual address space you will have.
Personally, I use ANTS Memory Profiler to find memory issues.
There is a really good blog here by Tess Ferrandez that talks about memory issues and other types of debugging issues while developing ASP.NET applications.
One post in particular walks you through a tutorial to see if you have a leak. Its uses the nasty WinDBG, but she explains all the commands so you can get a clear picture of your used memory and it shows you exactly which objects are filling all the space.
I have used her site many times to diagnose memory leaks and other performance related problems.
I have also used tools like DebugDiag to help capture memory related issues and used its built in diagnostic tools to help find the problems.