I have web application with updatepanels on it. The page has few gridviews on it. When I bind data to grids and do a updatepanel .update() the memory usage of browser will increase everytime.
When I do this ~60 times the memory usage will increase from ~100mb to ~750mb. After 1-2 hours of page use the memory usage will be like 1.5gb.
It seems that my website will never release memory and always keeps increasing.
I have a few scripts also on page. With IE memory profile heap testing it shows that the memory usage will stay the same and no new objects will appear to memory. Still the task manager shows that memory usage increases.
I use session state but that should be on server memory? I also use viewstate but it's size remains the same all the time. I don't use tracing. I tried to disable all caching from page. Is there something else I should give a try ?
Is there a way to see what is in memory? Is there a way to release all client side memory from server side code or from javarscipt/jquery?
What does browsers store each postback on asp.net?
There is a lot of memory profilers to server side but is there anything for client side except the developer tools on each browser?
I have been banging my head against the wall with this now over a week and no solutions can not be found so I will appreciate any tip on this nightmare..
Related
We have a problem on our website, seemingly at random (every day or so, up to once every 7-10 days) the website will become unresponsive.
We have two web servers on Azure, and we use Redis.
I've managed to run DotNetMemory and caught it when it crashes, and what I observe is under Event handlers leak two items seem to increase in count into the thousands before the website stops working. Those two items are CaliEventHandlerDelegateProxy and ArglessEventHandlerProxy. Once the site crashes, we get lots of Redis exceptions that it can't connect to the Redis server. According to Azure Portal, our Redis server load never goes above 10% in peak times and we're following all best practises.
I've spent a long time going through our website ensuring that there are no obvious memory leaks, and have patched a few cases that went under the radar. Anecdotally, these seem to of improved the website stability a little. Things we've checked:
All iDisposable objects are now wrapped in using blocks (we did this strictly before but we did find a few not disposed properly)
Event handlers are unsubscribed - there are very few in our code base
We use WebUserControls pretty heavily. Each one had the current master page passed in as a parameter. We've removed the dependency on this as we thought it could cause GC to not collect the page perhaps
Our latest issue is that when the web server runs fine, but then we run DotNetMemory and attach it to the w3wp.exe process it causes the CaliEventHandlerDelegateProxy and ArglessEventHandlerProxy event leaks to increase rapidly until the site crashes! So the crash is reproducible just by running DotNetMemory. Here is a screenshot of what we saw:
I'm at a loss now, I believe I've exhausted all possibilities of memory leaks in our code base, and our "solution" is to have the app pools recycle every several hours to be on the safe side.
We've even tried upgrading Redis to the Premium tier, and even upgraded all drives on the webservers to SSDs to see if it helps things which it doesn't appear to.
Can anyone shed any light on what might be causing these issues?
All iDisposable objects are now wrapped in using blocks (we did this
strictly before but we did find a few not disposed properly)
We can't say a lot about crash without any information about it, but I have some speculations about it.
I see 10 000 (!) not disposed objects handled by finalization queue. Let start with them, find all of them and add Dispose call in your app.
Also I would recommend to check how many system handles utilized by your application. There is an OS limit on number of handles and if they are exceeded no more file handles, network sockets, etc can be created. I recommend it especially since the number of not disposed objects.
Also if you have a timeout on accessing Redis get performance profiler and look why so. I recommend to get JetBrains dotTrace and use TIMELINE mode to get a profile of your app, it will show thread sleeping, threads contention and many many more information what will help you to find a problem root. You can use command line tool to obtain profile data, in order not to install GUI application on the server side.
it causes the CaliEventHandlerDelegateProxy and
ArglessEventHandlerProxy event leaks to increase rapidly
dotMemory doesn't change your application code and doesn't allocate any managed objects in profiled process. Microsoft Profiling API injects a dll (written in c++) into the profiling process, it's a part of dotMemory, named Profilng Core, playing the role of the "server" (where standalone dotMemory written in C# is a client). Profiling Core doing some work with gathered data before sending it to the client side, it requires some memory, which allocated, of course, in the address space of the profiling process but it doesn't affect managed memory.
Memory profiling may affect performance of your application. For example, profiling API disables concurrent GC when application is under profiling or memory allocation data collecting can significantly slow down your application.
Why do you thing that CaliEventHandlerDelegateProxy and ArglessEventHandlerProxy are allocated only under dotMemory profiling? Could you please describe how you explored this?
Event handlers are unsubscribed - there are very few in our code base
dotMemory reports an event handler as a leak means there is only one reference to it - from event source at there is no possibility to unsubscribe from this event. Check all these leaks, find yours at look at the code how it is happened. Anyway, there are only 110.3 KB retained by these objects, why do you decide your site crashed because of them?
I'm at a loss now, I believe I've exhausted all possibilities of memory leaks in our code base
Take several snapshots in a period of time when memory consumption is growing, open full comparison of some of these snapshots and look at all survived objects which should not survive and find why they survived. This is the only way to prove that your app doesn't have memory leak, looking the code doesn't prove it, sorry.
Hope if you perform all the activities I recommend you to do (performance profiling, full snapshots and snapshots comparison investigation, not only inspections view, checking why there are huge amount of not disposed objects) you will find and fix the root problem.
I need some information from you.I have used session.TimeOut=540 in application.Is that effects on my Application performance after some time.When number of users increases its getting very slow. response time nearly more that 2 minutes for a button click also.This is hosted in server in Application pool .I don't know about Application pool much.If Session Timeout is the problem i will remove it.Please suggest me the way to for more users.
Job Numbers,CustomerID,Tasks will come from one database.when the user click start Button then the data saved in another Database.I need this need to be faster for more Users
I think that you have some page(s) that make some work that takes time, or for some reason or a bug is keep open for more time than the usual.
This page is keep lock the session and hold the rest page from response because the session holds all the pages.
Now, together with the increase of the timeout this page is lock everything and here is you response time near to 2 minutes.
The solution is to locate the page that have the long running problem and fix it or make it faster by optimize the process, or if this page must keep the long time running, then disable the session for that one.
relative:
Web app blocked while processing another web app on sharing same session
What perfmon counters are useful for identifying ASP.NET bottlenecks?
Replacing ASP.Net's session entirely
Trying to make Web Method Asynchronous
Does ASP.NET Web Forms prevent a double click submission?
About server
Now from the other hand, if your server suffer from hardware, or bad setup then here is one other answer with points that you need to check to make it faster.
Find out where the time is spent
add the StopWatch in the method which you said "more that 2 minutes for a button click". you can find which statment spent the most time.
If it is a query on DB that cost time. Check your sql statement.
are you using "SELECT Count(*)" instead of "SELECT Count(Id)"? the * is always slower. also, don't try "SELECT * FROM...."
Use cache.
there are many ways to do cache. both in ASPX pages and your biz layer.
the OutputCache is the most easy way.
and also, cache the page (for example a blog post) on the first time when a user visit it.
Did you use memory paging?
be careful when doing paging on gridview or other list. If you just call DataSource=xxx and DataBind(), even with PagedDataSource, this is likely a memory paging. It cost a lot of performance. Please use stored procedures to do paging.
Check your server environment
where did you deploy the website? many ISP will limit brandwide and IIS connection count and also CPU time to your account.
if you have RD access to your server. you can watch CPU and memory usage to see if they are high when many user comes to your site. If the site is slow and neither CPU nor memory useage is high, it may be a network brandwide problem.
Here are some simple steps to narrow down the issue -
1) Get HTTPWatch (theres a free Basic version) available and check whats really taking time from an end user perspective. Look at number of requests, number of resources downloaded, and the payload. If there is nothing to worry move on to next
2) If its not client, then its usually the processing time on the server. Jump on to DB first - since this is quite easier to eliminate quickly. Look at how many DB calls are made (run profiler in staging or dev) and see if there are any long running queries, missing indexes or statistics, and note the IO. If all is well, move on
3) Check your app code. You could get on with VS.NET in build profiler or professional tools such as Ants. If code is fine then its your network or external calls that you make, check your network bandwidth. If you still cannot narrow down, check your environment/hardware
The best way to get to it is to apply load - You could use simple tools such as ab.exe (that comes as part of Apache Web server) to have concurrent hits on your server and run the App, DB profilers in the background to get to the issue.
Hope this helps!
Our users of a legacy web application written in ASP.net 2.0 are experiencing slow down after using the application over some time (after 2 hours or so).
For example, clicking a button performing some calculation and updating an ajax updatepanel takes several seconds while it is almost instant when they just have logged in.
If they log off and log back on of the web application, it gets back to the normal speed again, every time they do. The slow down is for a particular user and not global. It can fast for an user and slow for the user next to him.
I've looked at the IIS server and there is plenty of free memory in it and the CPU is almost always below 10%.
There are only a dozen users using this web application at time so scalability shouldn't be an issue.
The only think I can think of being affected by log offs, is session state.
I've already looked ViewState too and it is "only" 18kb at the slowest which shouldn't be the bottleneck, especially since they are on the same gigabit LAN and not remote. It is not any smaller when the applications runs fast.
Looking at session state in the code, it isn't used much and no large objects are stored.
According to my tests, it didn't grow bigger than 10 items and 500 bytes after a few minutes.
Even if there was a leak, the server should be able to cope with it considering the amounts of memory it has (4GB but only using 300MB).
I've cleared temporary files and it didn't seem to make any significant difference.
Do you think these slowdowns could come from something else than Session State, keeping in mind logging off and back on gets the speed back to normal?
If not, is there any tool you would suggest to profile sessionstate or a setting in IIS I should try to change?
what does the browser's memory use look like on the client machines when they experience the slowdown? It could be memory leaks in your javascript if you are using lots of ajax stuff and the browser's memory use continuously climbs.
Have you checked your database? (I assume you're consuming some data with an ajax app)
If you don't close the connection in the Finally block, you can wind up with zombie database connections that slow the app down. I don't know if this is the root cause, but probably worth looking at.
If this is an AJAX heavy page & the user is staying on the same page (i.e. lots of postbacks to itself) then it's possible that it's just ViewState growing to large. and the slow down you're experiencing is due to PageSize.
I would try and recreate the experience locally & monitor the page size/traffic with something like Firebug/Fiddler. If it is a ViewState/page size issue you could look at moving ViewState to the server side
See these for more info on it.
http://professionalaspnet.com/archive/2006/12/09/Move-the-ViewState-to-Session-and-eliminate-page-bloat.aspx
I have a web application that uses ASP.NET with "InProc" session handling. Normally, everything works fine, but a few hundred requests each day take significantly longer to run than normal. In the IIS logs, I can see that these pages (which usually require 2-5 seconds to run) are running for 20+ seconds.
I enabled Failed Request Tracing in Verbose mode, and found that the delay is happening in the AspNetSessionData section. In the example shown below, there was a 39-second gap between AspNetSessionDataBegin and AspNetSessionDataEnd.
I'm not sure what to do next. I can't find any reason for this delay, and I can't find any more logging features that could be enabled to tell me what's happening here. Does anyone know why this is happening, or have any suggestions for additional steps I can take to find the problem?
My app usually stores 1-5MB in session for each user, mostly cached data for searches. The server has plenty of available memory, and only runs about 50 users.
It could be caused by lock contention for the session state. Take a look at the last paragraph of MSDN's ASP.NET Session State Overview. See also K. Scott Allen's helpful post on this subject.
If a page is annotated with EnableSessionState="True" (or inherits the web.config default), then all requests for that page will acquire a write lock on the session state. All other requests that use session state -- even if they do not acquire a write lock -- are blocked until that request finishes.
If a page is annotated with EnableSessionState="ReadOnly", then the page will not acquire a write lock and so will not block other requests. (Though it may be blocked by another request holding the write lock.)
To eliminate this lock contention, you may want to implement your own [finer grained] locking around the HttpContext.Cache object or static WeakReferences. The latter is probably more efficient. (See pp. 118-122 of Ultra-Fast ASP.NET by Richard Kiessig.)
There is chance your are running up against the maximum amount of memory that Application Pool is allowed to consume, which causes a restart of the Application Pool (which would account for the delay you are seeing in accessing the session). The amount of memory on the server doesn't impact the amount of memory ASP.NET can use, this is controlled in the machine.config in the memoryLimit property and in IIS 6.0 later in IIS itself using the "Maximum memory used" property. Beyond that, have you considered alternatives to each user using 5 MB of session memory? This will not scale well at all and can cause a lot of issues while under load. Might caching be a more effective solution? Do the searches take so long that you need to do this, could the SQL/Database Setup be optimized to speed up your queries?
I have a update panel combined with gridview with sorting and paging.
I go into task manager to monitor the memory usage of the worker process (w3wp)
What I do is just click on the sort buttons rapidly.
With each click the memory of the process increases with about 2 mb
So I go from 30 mb memory usage to about 90. Then it stops at remains there, no memory is freed up. I am not using caching or session/application state.
What can be causing this, is there a setting in IIS to reduce the mem usage?
--
I also used .net profiler to examine my app memory usage: 4 mb, so what is the other 86 used for??? Even though it repots 4mb, in task manager it says 90 mb, so this leads me to believe that the rest is unamanaged memory which must be used by IIS in some way.
The .NET GC is non-deterministic. This means that it will run whenever it decides it should run. You can try calling GC.Collect() explicitly for example in the Page_Init event to see if the memory still increases but you have better remove it from the real app otherwise you are just preventing the GC from doing its work efficiently.
The issue is actually with the GridView, and not the UpdatePanel. Its records are stored in your ViewState, so that's being passed back and forth every single postback. Also, as you click on the sort buttons rapidly, you're generating multiple requests to sort the data. Depending on how you have your sort implemented, you could be duplicating the recordset for sorting with each click request.
There is no setting in IIS to "reduce memory usage" as it simply hosts your ASP.NET application. Your application needs to address its own memory concerns.
Sorting of a large amount of data can be a resource intensive process. I would say your best bet is to disable the sort button after it's been clicked and re-enable it once your data has been sorted.