I'm running Datastax Community Edition on Windows7 64 bit desktop PC with 8GB RAM. Running only a single node. Here allocated heap size is 1GB. While I tried to insert data(10 Milion) into a table through a Java application(Using casssandra java driver), its working fine. But when I tried to insert from an client-server program(It includes 2-3 thread initialization) then its blocking, raising a java.lang.OutOfMemoryError: Java heap space error. Noticeable point is, I checked the heap size, used memory, free memory after every insert transaction and there is enough space in heap. Also from the Ops-center web interface I checked the Heap size and it is never using full space ! I also tried to increase the heap size uncommenting #MAX_HEAP_SIZE="4G" from _cassandea_env.sh_ file but I got same resultQ1. What is causing this error arise?Q2. How to overcome thisThanks for any helpful suggestion
The log file history-
Ok. Its a mistake that I didn't free the cassandra connection after every transaction(In my case, insertion). After I did that now I can handle my desired transaction.
Related
We are running web application with 6GB heap assigned to it. But, after some time, it is giving out of memory.
The exception stack trace is given below.
Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2" java.lang.OutOfMemoryError: PermGen space
00:46:52,678 WARN ThreadPoolAsynchronousRunner:608 - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector#772df14c -- APPARENT DEADLOCK!!! Creating emergency threads for unassigned pending tasks!
00:46:52,682 WARN ThreadPoolAsynchronousRunner:624 - com.mchange.v2.async.ThreadPoolAsynchronousRunner$DeadlockDetector#772df14c -- APPARENT DEADLOCK!!! Complete Status:
Managed Threads: 3
Active Threads: 0
Active Tasks:
Pending Tasks:
com.mchange.v2.resourcepool.BasicResourcePool$AcquireTask#3e3e8a19
Pool thread stack traces:
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1,5,]
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0,5,]
Thread[com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2,5,]
Exception in thread "Task-Thread-for-com.mchange.v2.async.ThreadPerTaskAsynchronousRunner#6bbc0209" java.lang.OutOfMemoryError: PermGen space
Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#1" java.lang.OutOfMemoryError: PermGen space
Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#0" java.lang.OutOfMemoryError: PermGen space
Exception in thread "com.mchange.v2.async.ThreadPoolAsynchronousRunner$PoolThread-#2" java.lang.OutOfMemoryError: PermGen space
Is this the problem with C3p0 Connection pool?
According to the last few messages, you could be running out of PermGen memory. According to your post, the 6 GB are assigned to heap. If you have enough physical memory, double the memory assigned to PermGen. If the problem persists (and happens after more or less the same amount of time), revert the changes and consider analyzing heap through appropriate methods.
Use
-XX:PermSize=1024m -XX:MaxPermSize=1024m
for a 1-GB (1024 MB) allocation. You could need more.
Rather than guessing, my advice is to use a memory profiler to examine the contents of the heap when you run out of memory. This will give you a definitive answer as to what's taking all the memory, and would let you make informed choices as to what steps to take next.
You are getting permgen out of memory. This is more difficult to resolve than normal out of memory. Its not linked to usage of C3p0.
You need to do memory profiling and see whats taking up permgen space.
As a temporary measure , you can increase permgen space by using -XX:PermSize -XX:MaxPermSize
If you are using spring then one of common reasons for this error is improper usage of cglib library.
c3p0 and Tomcat's somewhat unusual and difficult implementation of class(un)loading under hot redeploy can lead to permgen memory issues. the next c3p0 pre-release has some fixes for this problem; they are implemented already, but for now you'd have to build from source via github, which is not necessarily easy.
in c3p0-0.9.5-pre4 and beyond, setting config param privilegeSpawnedThreads to true and contextClassLoaderSource to library should resolve the issue. i hope to release c3p0-0.9.5-pre4 within the next few days.
We're running into a strange problem. Our ASP.NET application is running on 64-bit Windows 2008/IIS7 machine with 16Gb of RAM. When w3wp.exe process reaches 4Gb (we track it simple via Task Manager on the server) - Out of Memory exeption is thrown even though there's a plenty of memory still available.
Is there a known issue were ASP.NET process is limited to 4Gb of memory on 64bit system (and using 64bit app pool)?
Is there any way to lift that limit?
It kind of sounds like you have an undisposed resource somewhere that ends up getting garbage collected eventually, but not quickly enough for your needs. Do you reuse any SQLConnection objects? Or MailClient objects? Or unmanaged Image objects?
As for the lower-than-expected memory limit, there are two types of memory use by a ASP.NET app. One is reserved memory and the other is actually used memory. I believe the task manager tracks actual memory use, but reserved memory probably also has a limit. To find out how much reserved memory your process is taking up, go to IIS7, click on the server (the top level, above app pools and sites folder), then click the Processes option and then click your app's process. It should show you CPU use, number of requests and memory usage (both reserved and actual).
I'm a developer in a large company that has some legacy code that requires a very large ammount of memory on export functions. To address this, ini_set('memory_limit', '4G'); is used.
The problem is that the script crashes with memory exaustion. If I set the limit to 2G, the script runs to the end. It doesn't even reaches 1GB peak memory usage.
Since the code is versioned and shared with the rest of the company I can't change the limit and changing it on my local install is cumbersome.
My question is: what can make a script crashes with 4GB limit but not 2GB?
PS: my setup is a virtualbox machine running Debian with nginx and php-fpm. The vm has 4GB RAM (although changing this doesn't seem to do any difference).
[update]
Created a new virtual machine with an 64 bits operation system and if I set the vm memory to 2GB it works. (If i use 4GB it doesn't).
Since i'm ok with 2GB, i'll close this issue.
It is a natural limitation: 2 or even 4 Gbs of address space are used for file mapping also which takes some memory pages.
The ultimate solution would be to use the 64-bit PHP interpreter (i.e., switch to 64-bit system, if possible).
Maybe you are on a 32bit system?
Well if your VM only has 4GB, then you probably should give it more memory.
On the 32 bit system 4GB is the limit of memory space. I guess that there can be some memory violations when PHP tries to get 4GB memory.
I have a small WCF service which is executed on an XP box with 256 megs of RAM running in VM.
When I make a request (with a request size of approximately 5mbs) to that service I always get the following message in the event log:
aspnet_wp.exe was recycled because memory consumption exceeded the 153 MB (60 percent of available RAM).
and the call fails with error 500.
I've tried to increase memory limit to 95% but it still takes up all the available memory and fails in the same manner.
It looks like something is wrong with my app (I do not reuse byte[] buffers and maybe something else) but I cannot find root cause of such memory overuse.
Profiling showed that all CLR objects that I have in memory together do not take up that much space.
Doing a dump analysis with windbg showed same situation - nothing that big in object heap.
How can I find out what is contributing to such memory overuse?
Is there any way to make a dump right before process is recycled (during peak mem usage)?
Tess Ferrandez's blog "If broken it is, fix it you should" has lots of hints, tips and recommendations for sorting out exactly this sort of problem.
Of particular use to you would be Lab 3: Memory, where she walks you through working out what has caused all the memory on your machine to disappear.
Could be a lot of things, hard to diagnose this one. Have you watched perfmon to see if the memory usage does peak on aspnet process or on the server itself? 256MB is pretty low, but it should still be able to handle it. Do you have a SWAP file on this machine? AT what point do you take the memory dump? Have you stepped though the code, and does it work on other machines? Perhaps it is getting stuck in a loop and leaking memory until it crashes?
On my machine (XP, 64) the ASP.net worker process (w3wp.exe) always launches with 5.5GB of Virtual Memory reserved. This happens regardless of the web application it's hosting (it can be anything, even an empty web page in aspx).
This big old chunk of virtual memory is reserved at the moment the process starts, so this isn't a gradual memory "leak" of some sort.
Some snooping around with windbg shows that the memory is question is Private, Reserved and RegionUsageIsVAD, which indicates it might be the work of someone calling VirtualAlloc. It also shows that the memory in question is allocated/reserved in 4 big chunks of 1GB each and a several smaller ones (1/4GB each).
So I guess I need to figure out who's calling VirtualAlloc and reserving all this memory. How do I do that?
Attaching a debugger to the process prior to the memory allocation is tricky, because w3wp.exe is a process launched by svchost.exe (that is, IIS/ASP.Net filter) and if I try to launch it myself in order to debug it it just closes down without all this profuse memory reservation. Also, the command line parameters are invalid if I resuse them (which makes sense because it's a pipe created by the calling process).
I can attach windbg it to the process after the fact (which is how I found the memory regions in question), but I'm not sure it's possible at that point to determine who allocated what.
David Wang answers this to a similar question:
[...] the ASP.Net performance developer tells me that:
The Reserved virtual memory is nothing to worry about. You can view
it as performance/caching prerequisite
of the CLR. And heavy load testing
shows that it is nothing to worry
about.
System.Windows.Forms - It's not pulled in by empty hello world ASPX
page. You can use Microsoft Debugging
Tools and "sx e ld
system.windows.forms" to identify what
is actually pulling it in at runtime.
Or you can ildasm to find the
dependency.
mscorlib - make sure it is GAC'd and NGen'd properly.
Virtual memory is just the address space allocated to the process. It has nothing to do with memory usage.
See:
Virtual Memory
Pushing the Limits of Windows: Virtual Memory
http://support.microsoft.com/kb/555223
Reserved memory is very different from allocated memory. Reserving memory just allocates address space. It doesn't commit any physical pages.
This address space is likely allocated by IIS for its heap. It will only commit pages when needed.
If you really want to launch w3wp.exe from windbg, you probably need to launch it with valid command-line arguments. You can use Process Explorer to determine what the command line for the current w3wp.exe process is. For instance, on my server, mine was:
c:\windows\system32\inetsrv\w3wp.exe -a \.\pipe\iisipmeca56ca2-3a28-452a-9ad3-9e3da7b7c765 -t 20 -ap "DefaultAppPool"
I'm not sure what the UID in there specifies, but it looks it's probably generated on the fly by the W3SVC service (which is what launched w3wp.exe) to name the pipe specified there. So you should definitely look at your command line before launching w3wp from windbg.