recently I met a issue on OpenCL.
If I create a large cl::Buffer( let's say It's 1920*1080*3 bytes large) with the flag CL_MEM_COPY_HOST_PTR. Whenever I do a enqueueWriteBuffer, it crashes with
"* ** stack smashing detected ***: bin/example1 terminated".
Weird enough. With CL_MEM_ALLOC_HOST_PTR. It gave me error -38( CL_INVALID_MEM_OBJECT) on enqueueWriteBuffer. Also with CL_MEM_USE_HOST_PTR, everything is fine.
But shouldn't all three CL_MEM_COPY/USE/ALLOC_HOST_PTR does the same job to create a working buffer? Why one works and other fails? What am I missing?
Edit:
HERE IS THE SOURCE CODE(example/hugeData.cpp)
https://github.com/marty1885/EasyCL
Related
I created a simple R Script that is run on a monthly basis by colleagues.
This script brings in a fairly chunky RDS file that has around 2.6M observations and 521 variables.
Against this file the following two commands are run:
Latest$MFU <- substr(Latest$SUB_BUSINESS_UNIT_CODE, 1, 2)
Latest$LENGTH <- str_length(Latest$POLICYHOLDER_COMPANY_NAME_LAST_NAME)
This script has run perfectly for the last three years, but today, for some reason, it is now failing for all three people tasked to run it and has indeed fallen over for myself too.
The error message received is
Error: cannot allocate vector of size 10.0 Mb
At first I assumed that their computers were running out of memory, or they were not using 64Bit R, or some other reason such as not restarting their computers, etc.
It turns out though that they have plenty of memory available, have restarted their computers, are using 64 Bit R in R Studio and all are using different versions of R Studio/R.
I tried running the process myself, my computer has 32GB of Ram and 768GB of Hard Drive space free. I am getting the same error message.......
So, must be a corrupt source file I figure. Try last months file which all ran just fine last month for everyone and same error.
Maybe just try stringr package instead then, move around the problem that way. Nope, no dice, exact same error message.
I have to admit I'm stumped. I have tried gc(), tried previous versions of the file, tried cutting the file in half and running it that way, it just flat out refuses to run.
Anyone know of an alternative to stringr/base R commands to get the length of a character string as a new variable and to get a substring as a new variable?
What about rm(list=ls()) before running, and memory.limit(size = 16265*4) (or another big number) ?
I am running into an error on a big job in R. I running it as an R script. I keep getting the error that Error in chol.default(F.mat) :
the leading minor of order 1 is not positive definite.
I normally run my job in a qsub but that only gives me an error output but I can't poke around. I then tried running my job locally but my 4gb Macbook was completely overwhelmed.
Now I am trying using screen name and running it on a screen with options(error=recover). Now I am running into the same error as above but I don't know how to access the data frames. I get recover called non-interactively; frames dumped, use debugger() to view but then I get put into my bash line and I don't know how to open up the data frame.
Any ideas?
This is a bit awkward since (1) it's more or less remote debugging and (2) I don't actually ever try to debug non-interactively myself, but: it seems that
options(error=function() dump.frames(to.file=TRUE)) might be worth trying?
After your frames dump to a file (last.dump.rda in the working directory,by default), you should be able to run load("last.dump.rda"); debugger(last.dump) to get back to the debugging environment.
Two caveats:
I haven't actually tested this, just read & interpreted ?dump.frames;
I strongly recommend that you test this with short test runs, either running your original code on a small subset of your data or setting a mini-test script something like
options(error=function() dump.frames(to.file=TRUE))
Sys.sleep(60)
stop("testing error exit")
I am trying to use the coreNLP package. I ran the following commands and encounter the GC overhead limit exceeded error.
library(rJava)
downloadCoreNLP()
initCoreNLP()
Error is like this :
Loading classifier from edu/stanford/nlp/models/ner/english.conll.4class.distsim.crf.ser.gz ... Error in rJava::.jnew("edu.stanford.nlp.pipeline.StanfordCoreNLP", basename(path)) :
java.lang.OutOfMemoryError: GC overhead limit exceeded
Error during wrapup: cannot open the connection
I don't know much of Java, can someone help me with this?
I found a more general solution: increase the heap space for rJava, as described here:
Cause: The default heap size for libraries that rely on rJava is 512MB. It is relatively easy to exceed this maximum size.
Solution: Increase the JVM heap size in rJava's options support:
options(java.parameters = "-Xmx4096m")
Note that this step must be performed prior to loading any packages.
Then I ran:
initCoreNLP(mem = "4g")
...and the entire CoreNLP loaded and ran successfully.
#indi I ran into the same problem (see R's coreNLP::initCoreNLP() throws java.lang.OutOfMemoryError) but was able to come up with a more repeatable solution than simply rebooting.
The full syntax for the init command is
initCoreNLP(libLoc, parameterFile, mem = "4g", annotators)
Increasing mem did not help me, but I realized that you and I were both getting stuck with one of the classifiers in the ner annotator (named entity recognition). Since all I needed was parts-of-speech tagging, I replaced the init command with the following:
initCoreNLP(mem = "8g", annotators = c("tokenize", "ssplit", "pos"))
This caused the init command to execute in a flash and with no memory problems. BTW, I increased mem to 8g just because I have that much RAM. I'm sure I could have left it at the default 4g and it would have been fine.
I don't know if you need the ner annotator. If not, then explicitly list the annotators argument. Here is a list of the possible values: http://stanfordnlp.github.io/CoreNLP/annotators.html. Just pick the ones you absolutely need to get your job done. If you do need ner, then again figure out the minimal set of annotators you need and specify those.
So there you (and hopefully others) go!
Tried the following, but in vain -
options(java.parameters = "-Xmx1000m") - to increase the heap size
gc() - which will cause a garbage collection to take place automatically
Ultimately got solved on its own after restarting my machine!
When I run a large R scripts (works nicely as expected, basically produces a correct PDF at the end of the script (base plotting plus beeswarm, last line of script is dev.off()), I notice that the PDF is finished after ~3 seconds and can even be opened in other applications, long before the console output (merely few integer values and echo of code ~400 lines) is finished (~20 seconds). There are no errors reported. In between, the echo stops and does nothing for seconds.
I work with R Studio V0.97.551, R version 3.0.1, on Win-7.
gc() or close and restart R did not help, and the data structures used are not big anyway (5 dataframes with up to 60 obs and 64 numeric or short character variables). The available memory should be sufficient (according to task manager, around 4 GB throughout), but CPU is busy during that time.
I agree this is not reproducible for other people w/o the script, which is however too large to post, but maybe someone has experienced the same problem or even an explanation or suggestion what to check? Thanks in advance!
EDIT:
I run exactly the same code directly in R 3.0.1 (w/o RStudio), and the problem was gone, suggesting the problem is related to RStudio. I added the tag RStudio, but I am not sure if I am now supposed to move this question somewhere else?
Recently I came across similar problem--running from RStudio becomes very slow, even when it is executing something as simple as example('plot'). After searching around, this post pointed me to the right place that eventually led to a workaround: resetting RStudio by renaming the "RStudio-Desktop Directory". The exact way to do so depends upon the OS you are using, and you could find the detail instruction here. I just tried it, and it works.
I'm wondering if anyone else has ever encountered this problem. I'm writing a fairly small amount of data to a csv file. It's about 30 lines, 50 times.
I'm using a for loop to write data to the file.
It seems "finicky" sometimes the operation completes successfully, and other times it stops after the first ten times (300 lines) other times 3, or 5... by telling me
"cannot open connection".
I imagine it is some type of timeout. Is there a way to tell R to "slow down" when writing tables?
Before you ask: there's just too much code to provide an example here.
Code would help, despite your objections. R has a fixed-size connection pool and I suspect you are running out of connection.
So make sure you follow the three-step of
open connection (and check for error as a bonus)
write using the connection
close the connection
I can't reproduce it on a R 2.11.1 32bit on a Windows 7 64bit. For these kind of things, please provide more info on your system (see e.g. ?R.version, ?Sys.info )
Memory is a lot faster than disk access. 1500 lines are pretty much manageable in the memory and can be written to file in one time. If it's different sets of data, add an extra factor variable indicating the set (set1 to set50). All your data is easily manageable in one dataframe and you avoid having to access the disk many times.
In case it really is for 50 files, this code illustrates the valuable advice of Dirk :
for(i in 1:50){
...
ff <- file("C:/Mydir/Myfile.txt",open="at")
write.table(myData,file=ff)
close(ff)
}
See also the help: ?file
EDIT : you should use open="at" instead of open="wt". "at" is appending mode. "wt" is writing mode. append=T is the same as open="at".