In Chris Pruett's talk http://www.youtube.com/watch?v=7-62tRHLcHk
writing real time games for android, he meticulously avoided the GC from running by never losing a reference or creating a new allocation until a time of his choosing.
Should that design pattern be used in playN in general? I'd assume it's not possible if the library itself does not try to avoid allocation in the general case (memory pooling). I.e. if you receive new Touch.Events when implementing a Touch.Listener, if these and other objects aren't being constantly recycled, then attempts to use the API this way would be for naught... OR perhaps PlayN is close to being no-GC compliant and then it's still worth the developer putting in this effort.
Thanx :-)
Update: I posted the wrong Chris Pruett talk. Watch the bit here where he talks about the GC. http://www.youtube.com/watch?feature=player_detailpage&v=U4Bk5rmIpic#t=1748s
Most of Android devices have multi-core CPU now. It's not a real problem anymore I think.
However, you can check traces from GC in your device logcat. On my 3 years mono-core device, it's less than 5 ms when running my game.
If you're worry about memory management on Android, I recommend you this video: Google I/O 2011: Memory management for Android Apps
Related
We got a high traffic website which generates a lot of I/O. Within 10 minutes it has been reading over 10 gb of data (w3wp in question seen in task manager). For memory and application hangs I have been using WinDbg with success. But I don't know how I can find the object(s) / method(s) within a process which are responsible for the highest I/O.
Is this even possible?
Edit
The question is: Is there a way to profile I/O operations in a .NET assembly, say: list of threads sorted by highest disk I/O (or something similar that would help me where to look)
ANTS Performance Profiler
I have used this tool to great success - dealing with finding the specific instructions which are causing ~512GB of memory on a high-volume web farm getting chewed up within 5-10 minutes. Sounds like a very similar situation as yours.
Now, to be realistic - it's not going to magically solve your problem. It still requires a lot of setup, thorough analysis and detective work. But this tool definitely took the problem from "practically unsolvable" to "solvable within days".
Update:
As I mentioned in the comments (and Ben Emmett echoed), we can use ANTS to monitor memory, file system handles - pretty much any resource consumption and drill down the call stack to see the effects of specific routines.
I came up with this tool AppDynamics Lite which displays your application calls costs and performance in a visual way. It might help you to find out which functions are making the most costy IO operations.
Quoting;
Understand the health of your CLR with key metrics like response time, throughput, exception rate, and garbage collection time as well as key system resource like CPU, memory and disk I/O.
Worth giving a shot as it is trial/free for 30 days. Hope it helps.
Ps: I'm not affiliated with AppDynamics in any way.
You can use the (free) Windows Performance Toolkit from Windows 8 which does run also on Windows Vista and later. There you can turn on system wide profiling to see what was going on in all processes at once. No instrumentation necessary. Only one reboot is required to set an arcane registry key which is done by WPRUI.exe automatically.
With XPerf you could enable IO Init stack walking so that a call stack is taken for every IO which is started. The only issue is that the stacks will be broken for 64 bit processes which means that you will see only the first method above the BCL methods of your code because there is a Windows 7 bug in the stackwalking capabilities of the OS.
A workaround is to Ngen your assemblies or move to Server 2012 or switch to x86 for profiling to see deeper call stacks.
You will see all file IO and CPU activity even without any call stacks and the file names along how long the hard disc was used. That should give you good information which part of your app is causing the disc IO. From the partial call stacks you should be able to pinpoint your issue even without full stacks.
The tool will give you much more insight than any commercially available profiler at the expense that you need to learn how to use it. Since the call stacks do not end at your code or in user mode but in the kernel you can also determine if e.g. the virus scanner is causing significant IO delays. But you need to know how your processor does work. This toolset was originally aimed at kernel devs which explains why you see so many useless columns.
In the picture below you see file IO and CPU consumption stacked. When you select your high IO file in the disc IO graph it will highlight in the CPU consumption all related call stacks which were taken at the same time while the IO was active. This way you can diretly navigate from the IO to your potentially blocked threads.
The problem is with Memory management because I keep receiving “Out of Memory exception”.
Here are the scenarios where we face the problem:
Please note:
1. The site/application is developed in ASP.Net and uploaded on a server with the following specs:
- Windows Server 2008 (R2) Standard
- Intel Xeon L5520#2.27GHz 2.27GHz
- RAM = 8GB
- System Type = 64bit
The application is event management based web application where the requirements include saving huge amount of data in Sessions etc (mentioning this in case it is relevant)
The applications/site works fine until we:
Edit a file directly on the server
Update a file from repository
Copy/Paste a file (we don’t usually edit code using this technique)
Please note, all of the above hold true ONLY when the traffic to the site is high that is,
The issue/error “Out of Memory” is not produced when the traffic/visits is low
Details of:
System Properties > Advanced > Performance Settings > Advanced tab
Total paging file size for all drives: 16362 MB
In web.config
Is there any way we can debug this problem to the core and find out a solution. Can you please provide links/help where we can further investigate this problem?
Best regards,
Farrukh
Out of Memory Exceptions are common with applications that see periodic transaction surges while keeping larger volumes of data in memory. This problem does, however, depend on your application and architecture. Below are a few pointers:
Hardware - you have Xeon 5500 (Intel Nehalem chips). These are very good at handling memory. You should be good here.
OS - Windows Server 2008 R2 - As an OS this system will handle more than enough memory for you (you are good here, see link for capabilities: Memory Limits for Windows)
Physical Memory - Did you say you have 8 GB on the server? Note you app is allowing 16 GB. There is one issue. If your app requests more memory than physically available you will see your error. But this is not your only concern ...
CLR / GC limitations - Your application has a "paging file size" of 16+ GB. This is probably your issue.
GC is the heart of your problem for you. In terms of why, it is the same reason Java and the JVM have issues whenever an application exceeds 2-4 GB. That requires a look at the actual process of GC.
You have "old generation" and "young generation" Garbage Collection processes. As you app runs the CLR tries to keep your memory space organized. These processes force all threads to pause (phase changes) when GC mark and swap processes occur. The problem here is, depending on how your code is written and the amount of memory you keep around for long periods, you can run into memory issues.
Any time you press a runtime environment to exceed the 4 GB threshold you will see exponential increases in collection times. When you hit the "stop the world" pause (the old gen GC where everything gets cleaned up) the CLR has to go through the entire heap and de-allocate memory. Based on your app, 16 GB may give you issues even with more physical memory (Windows Server 2008 R2 - Enterprise or DataCenter can support 2 TB). Even if you feed it more physical memory you may see LONG collection times when your full GC hits.
Ideally I would do the following:
Get more physical memory (you never want to come withing 600MB of your total physical memory allocated to your application to avoid out of memory errors, but your buffer does depend on your load and the application's ability to handle it ... you may want a larger safety net to be safe).
Once you have the physical memory you need run GC logs while stressing the app. This will give you an idea where you see exponential degradation in performance and what level your app can support when considering Heap size (Memory). You may want to find a way to get your 16GB page down to a smaller size. I do know with .Net 4.0 Microsoft has made some solid improvements to the GC process, including allowing a background thread to maintain GC. This should give you the ability to support larger heaps (in theory) ... but nothing beats real tests on the app. Check out this link for more info:
Garbage Collection Performance (Asp.net 4.0) - Also, as I am limited on links. Navigate to the Fundamentals page for some great explanations on new GC features of ASP.Net 4.0
(http://msdn.microsoft.com/en-us/library/ee787088.aspx#concurrent_garbage_collection)
Hope this helps!
PS - Anyone out there on lesser hardware will need to be aware of the ASP.NET use of the GC thread. If you are running something in development like a Core Duo you have to consider that 50% of your compute power will go to GC optimization. This means that Hardware (number of cores) is important to consider. If you have more than you need this process should theoretically help performance. If you are constrained on cores either get better hardware or use an older version of ASP.Net or consider turning the feature off (if possible). Second, if latency is a concern, using "hyper-threading" does have an impact on performance as well. You always get better performance on "physical" cores ... but that will not be a concern for 99.9% of the applications out there.
2 GB by default. If the application is large address space aware (linked with /LARGEADDRESSAWARE), it gets 4 GB (see http://msdn.microsoft.com/en-us/library/aa366778.aspx)
They're still limited to 2 GB since many application depends on the top bit of pointers to be zero.
I have been developing a quite large application, and I uploaded it to my server some days ago. Now I have found out it has several memory leaks - Uh oh.
My server is running Windows Server 2008 on 1GB ram. When I have 0 people online, only 550-600mb is used. When one people comes online the memory starts skyrocketing, and if 3-4 people are online all 1GB ram is used.
The application is made in ASP.NET with AJAX. It has many updatepanels which runs every second and quite a lot of javascript. It uses 5-7 sessions at all times. I use LINQ to SQL as database communication.
I tried perfmon.exe on my server, and I found:
Gen 0 collections goes from 0% to
100% within minutes
Gen 1 collections
goes from 0% to 50% within 5 minutes
Gen 2 is very close to 0% at all
times
Total heap bytes goes up to
100% very fast
I also ran an analysis of my program with Visual Studio. 8% Of my total runtime is done in .ToList() methods, which properly is caused by LINQ to SQL.
My theories....
(1) Linq to SQL dataContext
This might be a crazy thing to do, but: In my data access layer I have a load of methods:
AddSomethingToDatabase();
AddSomethingElseToDatabase();
DeleteSomethingFromDatabase();
Each of these has the following initialization:
GameDataContext db = new GameDataContext();
Which means the above statement runes nearly every second or more.
(2) No objects implement IDisposable
I have to be honest: I have never worked with IDisposable. As far as I have read, this might be a problem.
Also, if this is the leak, which classes should implement it? I do not have any I/O work or others, only the DataContext.
(3) Loads of UpdatePanels and jQuery
I have some fear loads of updatepanels can give problems with performance, but I do not know how to check it.
So my question is: Any ideas on what the memory leak could be? Any ideas on how to find the memory leak? And any ideas on how to solve it?
I would love to hear from someone who has experience with the situation above!
Thanks,
Lars
I am not sure at all that there is a problem here. All the suggestions for memory leak troubleshooting seem to be just really bad advice when you have not yet established that you have a memory leak since your memory on the server is so low that this cannot be established.
So here is my 2 cents - some might not like it but as long as it could point you at the right direction, I do not mind the downvotes.
It seems that you have a very stringent memory requirement. 1GB of RAM for a Windows 2008 Server just gives about enough RAM to do its OS related job. This is way way below recommended RAM requirements for it which if I am not wrong is minimum 2 GB RAM. Overhead of just running a w3wp.exe and IIS would be around 200-300 MB. The fact that generation 2 is always is around 0% is the best evidence that all looks good and your server is probably being starved of the memory.
My suggestion is to give your server at least 2GB of RAM (4GB should be better) and then monitor the memory usage and see if it is going up. If so, post another question with your findings and we should be able to help.
You absolutely must ensure that IDisposable objects get Dispose called when you are done with them. The simplest way to do this is to use using:
using (GameDataContext db = new GameDataContext())
{
// code that uses 'db' goes in here
}
// Dispose called when 'using' scope ends
If you still have problems after doing this throughout, then profiling is needed, but fix this first since it's a no-brainer.
Your own objects usually only need to implement IDisposable if they encapsulate unmanaged resources for which you wish to guarantee deterministic release back to the OS, so that those resources - file handles, sockets, and so on - are not sitting around waiting for GC for an interval of time you cannot rely on.
I don't have an answer for your question 3), sorry.
I'd recommend you use a memory profiler. Redgate's ANTS is pretty superior; it can give you a breakdown of which objects are in memory at a given time.
I am not expert at this. But If you try ANTS Memory Profiler might help you figure out where the problem is.
Scitech memory profiler found our leaks and gives good advice.
Hi I wrote an application in flash AS3, and when I trace from flash the total memory usage of the total application is only about 9MB, But at the same time Task Manager Shows the memory usage as 110MB. Around 100MB difference.
Flash Trace Method System.totalMemory difference of the Trace from the Beginning of the application to end of the application.
The amount of memory used by the flash player isn't necessarily related to how much memory your application is using. The players memory usage depends on how much memory the os gives it and a number of other things, if you have plenty of free memory there's no reason not to have the flash player sit on some for when it's needed.
All in all, you only need to worry about the actual memory usage reported by System.totalMemory*
* But do note that it reports the memory used for all currently running flash apps
Internally Flash Player will garbage collect making System.totalMemory accurate for internal usage. But even when the memory is GC'd it isn't given right back to the system. In IE you can cause the browser to give the GC'd space back by minimizing the browser. So essentially the value you see in the Task Manager is a high water mark of memory usage. If you need that value to be lower then the only thing you can do is use less memory. For example, before loading / creating something new, wait until something else has been GC'd so that Flash Player doesn't allocate new memory for an instant. The challenge is in knowing when something has been actually GC'd. There isn't a good way to do that.
Can anyone recommend a way in which I can throttle an application based on the current disk usage or even CPU usage.
The application I am writing scans files on the hard disk and will be pretty hard disk intensive in itself.
Can anyone recommend a way in which I can either throttle down my application(or even pause it for that matter) when the disk usage is high(i.e. user himself is running very HDD or CPU intensive app)? Basically my application shouldn't hamper user's productivity. I know this is a pretty big research topic in itself. But I at least need some cues on how would I approach this.
Help in any form is highly appreciated. :)
Thanks.
Samrat.
Vista has added I/O Prioritization to Windows so if you're using that platform you can just let the O/S take care of it.
For other operating systems maybe finding the I/O latency, and if it is over some predefined threshold then sleep your disk scanner for a bit would work?
Take a look at this ("How can I programmatically limit my program’s CPU usage to below 70%?") and this ("Win32 Thread scheduling#The Larry Osterman answer")