I'm looking for a way for the application itself to monitor the amount of memory it is using, so I can record it in a log file every hour or so and keep an eye on the applications usage.
Its all hosted so we can make changes to the system to see what is going on so the solution will have to be from within the application code.
We may in future use the memory information to affect the caching policies.
Hmm, how detailed information do you need? If you just want the memory usage you can ask the GC. It knows. ;)
long bytes = GC.GetTotalMemory(false); // use 'false' to not wait for next collect
The variable 'bytes' will contain the number of bytes currently allocated in managed memory. I'm not sure whether the managed memory entails the entire process or just the AppDomain. You'll have to test this by running several AppDomains in one process and see if managed memory allocation is measured cross AppDomains. If they don't, then you can use this to measure total memory usage in an ASP.NET application.
If you want more specific information there's a diagnostics API for the CLR which you could interface with. There's also plenty of memory profilers out there, but if they'll work within an ASP.NET application I cannot say.
As an alternative, if you want more detailed information, you can read the performance counters using the System.Diagnostics.PerformanceCounter class. Here are some of the counters that you can plug into:
Request Bytes Out Total
Request Bytes In Total
Request Wait Time
Requests Executing
Requests/Sec
Errors Total
Related
Recently we experienced crashing on application (ASP.Net) with below issue since issue is resolved but a question arises in my mind; can we capture history of memory used by application? I want to add a job which will capture application memory usage after each half an hour.
Memory gates checking failed because the free memory (373817344 bytes)
is less than 5% of total memory. As a result, the service will not be
available for incoming requests. To resolve this, either reduce the
load on the machine or adjust the value of
minFreeMemoryPercentageToActivateService on the
serviceHostingEnvironment config element.
Edit: Question arise from link
We have a fairly popular site that has around 4 mil users a month. It is hosted on a Dedicated Box with 16 gb of Ram, 2 procc with 24 cores.
At any given time the CPU is always under 40% and the memory is under 12 GB but at the highest traffic we see a very poor performance. The site is very very slow. We have 2 app pools one for our main site and one for our forum. Only the site is being slow. We don't have any restrictions on cpu or memory per app pool.
I have looked at he Performance counters and I saw something very interesting. At our peek time for some reason Request are being queued. Overall context switching numbers are very high around 30 - 110 000 k.
As i understand high context switching is caused by locks. Can anyone give me an example code that would cause a high number of context switches.
I am not too concerned with the context switching, and i don't think the numbers are huge. You have a lot of threads running in IIS (since its a 24 core machine), and higher context switching numbers re expected. However, I am definitely concerned with the request queuing.
I would do several things and see how it affects your performance counters:
Your server CPU is evidently under-utilized, since you run below 40% all the time. You can try to set a higher value of "Threads per processor limit" in IIS until you get to a 50-60% utilization. An optimal value of threads per core by the books is 20, but it depends on the scenario, and you can experiment with higher or lower values. I would recommend trying setting a value >=30. Low CPU utilization can also be a sign of blocking IO operations.
Adjust the "Queue Length" settings in IIS properties. If you have configured the "Threads per processor limit" to be 20, then you should configure the Queue Length to be 20 x 24 cores = 480. Again, if the requests are getting Queued, that can be a sign that all your threads are blocked serving other requests or blocked waiting for an IO response.
Don't serve your static files from IIS. Move them to a CDN, amazon S3 or whatever else. This will significantly improve your server performance, because 1,000s of Server requests will go somewhere else! If you MUST serve the files from IIS, than configure IIS file compression. In addition use expire headers for your static content, so they get cached on the client, which will save a lot of bandwidth.
Use Async IO wherever possible (reading/writing from disk, db, network etc.) in your ASP.NET controllers, handlers etc. to make sure you are using your threads optimally. Blocking the available threads using blocking IO (which is done in 95% of the ASP.NET apps i have seen in my life) could easily cause the thread pool to be fully utilized under heavy load, and Queuing would occur.
Do a general optimization to prevent the number of requests that hit your server, and the processing time of single requests. This can include Minification and Bundling of your CSS/JS files, refactoring your Javascript to do less roundtrips to the server, refactoring your controller/handler methods to be faster etc. I have added links below to Google and Yahoo recommendations.
Disable ASP.NET debugging in IIS.
Google and Yahoo recommendations:
https://developers.google.com/speed/docs/insights/rules
https://developer.yahoo.com/performance/rules.html
If you follow all these advices, i am sure you will get some improvements!
We are using ASP.NET but sometimes our applications using very much resources of CPU or RAM. I want to restrict resources for every virtual directory/web site and get alert when they reached the alert level.
I don't know I can do this for IIS but I wonder is this possible for other web servers like apache?
In Java, you cannot have the Java Virtual Machine enforce CPU time limits for certain threads: the best you can do is set the thread priority for any threads that you create. If there is no other work to be done, then a thread will burn as many CPU cycles as it can get unless is it blocking on something (I/O, synchronization monitor, etc.).
Using JMX, you can get some information about threads to possibly detect some runaway-thread situation, but you can't directly control the amount of CPU Time allowed for the thread.
If you are willing to re-architect your dynamic-content and other services, you could write them in such a way that they can support a "unit of work" that is somehow less than the full request's worth of work, and then you can have your request-processing thread execute a unit of work and then sleep an appropriate amount of time. You can lessen your CPU usage by doing this, but you will also certainly slow-down response times measurably.
Is there anything wrong with a fully-utilized CPU if your users are happy? Perhaps the real solution to your problem is more or bigger hardware.
My Asp.net application uses
25-30% of the CPU on a test server which has 600 MB Ram on it.
I can see the asp_wb process taking that much percentage of CPU.
This is when I am testing using one user.
How many users can the server afford then without falling over?
Is there a relationship between the CPU Usage and number of user aka if there are 2 users my application will sky rocket to 60% of memory usage?
Or does/Should/How does the server handle this?
The asp.net is base on pools and not on users.
Some memory per user is going on user session, but I believe that you not hold huge amount of data on sessions (did you ?).
Now I suggest ti run process explorer from sysinternals, and check on w3wp.exe the working set and the virtual size of the memory for this. You can do that by open this 2 columns on Process Memory tab.
Then you see there how many memory asp.net needs for your application.
Second step if to check how you have configure your pool by open it. Maybe you have configure it to recycle too often, or to recycle when you have more than 125k working set memory, and your program have 200k working set memory. So you need to recalibrate some values.
Together with process explorer you can see how much memory your application need, and setup correctly the pool.
Of cource maybe there are other problems and other issues with the memory but asp.net is not eat memory for every user and you need to check where your memory is used - and the process explorer is a good tool for this job.
Hope this help.
Environment:
Windows 2003 Server (32 bit); IIS6, ASP.NET 2.0 (3.5); 4Gb Ram; 1 Worker Process
We have a situation where we have a very large System.XmlDocument is being loaded into memory, and then it heads into a complied XSL transform.
What is happening is when a web request comes in the server is sitting in an idle state with 2500Mb of available system memory.
As the XML DOM is populated, the available memory drops approx 500Mb at which point we get a System.OutOfMemoryException event. At this point the system should theoretically still have 2000Mb of available memory available to service the request (according to Perfmon).
The related questions I have are:
1) At what level in the stack is this out of memory limitation being met? OS? IIS? ASP.NET? worker process? Is this a per individual web request limit?
2) Is this limit configurable somewhere?
3) Why can’t this web request access the full available system memory?
1) I would guess at the worker process but this should be configurable within IIS to the limit of memory that a worker process can use. Another factor is what level of bits does your software use, e.g. 32 bit has a physical limit of 4 GB since this is the total address space.
2) Probably but don't forget that memory fragmentation may play a role in getting to out of memory faster than you think, e.g. if there is a memory request for a contiguous 1000 Mb piece of memory then this may not necessarily be found in the current memory.
3) Have you examined dump data to see what is in the memory when the exception gets thrown? If not, there are ways to get a snapshot of the memory to see what it looks like as this may give you more clues about what is going on.
You are running in a process. A process can only access 2 gigs of memory. This task is sharing memory with everything else running in this process, so this bit of code does not get the full 2 gig -- even if it is available.
There is a 3 gig switch on the os as well. I believe it is a registry setting. But you will have to search MSDN to find that info.
But realistically, you need to do this another way. Possibly by switching to a SAX style xml parser.
I'm sure there are some bright heads here that can answer your specific questions, but have you asked yourself if there is another way to do what you want? I specifically mean that you probably do not want to process a very large XML document, but you probably more specifically want to return something back to the client. Could you rewrite the code to avoid this XML document altogether, or perhaps not load it all into memory at the same time, and still produce the same end-result?
1) Dunno. Check your logs.
2) IIS limits memory divvied out to websites/application pools. Check your settings.
3) Servers are all about uptime; if an single app hogs all the resources everybody else suffers. Thats why enterprise apps like IIS limit memory to prevent runaways from taking down the entire server.