I am using the mediation package to run a mediation analysis.
But when I call the mediate function, R eats up all RAM, then it eats up all swap, and then RStudio suddenly closes.
The same happens if I call Rscript from the terminal, and the terminal suddenly closes.
Here is measures for RAM and swap. Notice the peaks:
This happens both with the R 4.0.5 version that comes with the Fedora repository (which uses Open BLAS), both with a custom R 4.1.0 version I compiled from source against Intel MKL.
What is the cause of this, and how I can debug?
Here is my current sessionInfo():
> sessionInfo()
R version 4.1.0 (2021-05-18)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Fedora 34 (Workstation Edition)
Matrix products: default
BLAS/LAPACK: /opt/intel/oneapi/mkl/2021.2.0/lib/intel64/libmkl_gf_lp64.so.1
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C LC_TIME=en_GB.UTF-8
[4] LC_COLLATE=en_US.UTF-8 LC_MONETARY=en_GB.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_GB.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_GB.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_4.1.0 tools_4.1.0
If the application needs all RAM and all virtual RAM you definitely hit the hard limit of your computer. If the application now requests more RAM from the OS, that request has to fail. How do you expect the application to deal with it?
As we do not know why more RAM was requested (it could be a memory leak, it could be that the algorithm just requires that much) we cannot distinguish bug or feature, but we can try things around that.
One quite cheap thing to try is to increase virtual memory size. Execution will get slower (disk access is measured in ms as opposed to RAM access times measured in ns), but maybe you find a value that the application can complete the task with.
If you cannot find any limit on virtual memory and the application always crashes due to the out-of-memory situation, you are likely facing a bug.
In any case you may want to talk to the vendor about this perception.
Related
I am trying to run the some unconstrained optimization on a large problem on
a super computer, so I am trying to use the ucminf (although this problem
also works if I use the ucminf option in optimx). When I run any simple
optimization, I get the following message:
Error in .Call("mfopt", fnstr, grstr, rho, PACKAGE = "ucminf") :
"mfopt" not available for .Call() for package "ucminf"
For simplicity, here is a simple bit of test code that gives me the error.
library(ucminf)
test<-function(x){
(x-3)^2}
ucminf(0,test)
All of this works fine on my personal computer, but does not work on the
super computer. I have tried this on R/3.5.0 and R/3.3.0 on the super
computer and they both give me the same error.
Here is my Session info.
R version 3.5.0 (2018-04-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS release 6.9 (Final)
Matrix products: default
BLAS: /usr/public/R/3.5.0.gnu/lib64/R/lib/libRblas.so
LAPACK: /usr/public/R/3.5.0.gnu/lib64/R/lib/libRlapack.so
locale:
[1] LC_CTYPE=en_US.iso885915 LC_NUMERIC=C
[3] LC_TIME=en_US.iso885915 LC_COLLATE=en_US.iso885915
[5] LC_MONETARY=en_US.iso885915 LC_MESSAGES=en_US.iso885915
[7] LC_PAPER=en_US.iso885915 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.iso885915 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] ucminf_1.0-5 numDeriv_2016.8-1
loaded via a namespace (and not attached):
[1] compiler_3.5.0
I have uninstalled and reinstalled the package using Rinstall('ucminf'). Does anyone have experience with this and how to fix it?
I have a shiny server and actually I have been able to upload and use "online" some simple apps, and also the shiny and rmardown test works properly.
But then, when I try to use my own application (which works locally, on my computer), I get the following error message (I looked in log -> var/log/shiny-server):
su: ignore --preserve-environment, it's mutually exclusive to --login.
Loading required package: viridisLite
Loading required package: heatmaply
======================
Welcome to heatmaply version 0.14.1
Type citation('heatmaply') for how to cite the package.
Type ?heatmaply for the main documentation.
The github page is: https://github.com/talgalili/heatmaply/
Please submit your suggestions and bug-reports at: https://github.com/talgalili/heatmaply/issues
Or contact: <tal.galili#gmail.com>
======================
Listening on http://127.0.0.1:43894
Execution halted
The session info:
R version 3.5.0 (2018-04-23)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)
Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=es_ES.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=es_ES.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=es_ES.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=es_ES.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.5.0
I reinstaled both packages, and I also try to call them in R (sudo -i R to use R in console and then library(viridisLite) and library(heatmaply)) and they both works without problem.
When I try to use my App, server gets disconected and I have to restart it (My app starts to charge but when I reach to log issue, stops working)
Maybe the R version has to do with my issue, but I tried to change it unsuccesfully.
Best rewards,
Daniel.
I have installated the last shiny-server version and also went to wired connection -> IPv4-settings and delete the additional DNS server. It works!
I have a headless R server which serves the recent rstudio-server.
I need to pass it some objects via svSocket.
Because the svSocket uses TCL/TK message queue, which needs a working X11 session I know I need to embed things in a virtual X11 environment.
I log into the server via ssh and in the command line I put the following commands:
sudo Xvfb :0 &
export DISPLAY=":0"
/usr/lib/rstudio-server/bin/rserver &
R
Then I log into the rstudio-server via its web interface and type these commands:
> library(svSocket)
> startSocketServer()
[1] TRUE
then, on this R session I put the following
> library(svSocket)
> con<-socketConnection(host='localhost', port=8888)
> evalServer(con,'2+2')
I expect to get "4" result, but instead the R hangs, never returning any prompt.
If I replace the rstudio-server with the regular R, everything works correctly.
What is so special in the way it treats rsession that this example breaks? How to fix it?
SessionInfo():
R version 3.2.3 (2015-12-10)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 16.04 LTS
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=pl_PL.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=pl_PL.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=pl_PL.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=pl_PL.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
You need to mention the port number when you start the socket connection
> library(svSocket)
> startSocketServer(port=8888)
It should work after this.
When I load a map on R's widget (through ggmap) and run the program directly though R's console, the map gets properly maximized when I maximize the R's window.
The same doesn't happen when I run R through Qt. I am using RInside.
Through Qt when I run R, the widget gets shown indeed (with the map on it), but when I maximize the R window, the map does NOT get maximized. It remains the same sized!
It doesn't happen in any particular case. It happens all the time I run R though Qt, and never when I run R through R's console.
What hardware/software information should be presented here?
> sessionInfo()
R version 2.15.1 (2012-06-22)
Platform: x86_64-unknown-linux-gnu (64-bit)
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=C LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
>
Using Qt version 4.7.0
> cat /etc/issue && uname -a
Welcome to openSUSE 11.4 "Celadon" - Kernel \r (\l).
Linux linux-trra 2.6.37.1-1.2-desktop #1 SMP PREEMPT 2011-02-21 10:34:10 +0100 x86_64 x86_64 x86_64 GNU/Linux
The simplest R program which causes this:
R.parseEvalQ ("library (ggmap); library (raster);");
qtToR ["currentFileName"] = currentFileName;
R.parseEvalQ ("load (file = currentFileName); print (ggmap (mapImageData));");
Could it be possible that X11 server is not properly installed on my system? Or is this a know problem with print?
Is there any alternative way to get this running properly through ggmap?
![enter image description here][2]
A temporary solution to the problem of map not getting maximized is to set the initial dimensions of X11 server.
X11 (width = 11, height = 11);
This shows a nearly maximized window by default, and the map also gets shown maximized.
I have a rather large chunk of code that breaks sometimes, as far as I can see randomly, with the error message:
This application has requested the Runtime to terminate it in an
unusual way. Please contact the application's support team for more
information.
Some research showed me that this seems to be a Windows/C runtime message when abort is called (see for example this link). This drives me nuts: Since it is not an error thrown by R I have no idea where to look. Does anyone have any clue where R or maybe data.table (if that's possible that a package calls the abort function in C runtime [??]) calls the abort function?
Here are some further information:
The problem is independent from the machine: I tried it on two different machines, it crashed on both of them sometimes.
The problem is independent on the R version: I tried it with 2.13.1, 2.13.2, and 2.14.0.
Both machines run Windows 7 (64 bit).
The problem seems to be related to the size of my data.tables. When I artificially reduce the size of the bigger data.table, the code runs like a charm. Interestingly, however, one machine has far more RAM than the other (16 GB compared to 6 GB). This additional RAM, however, doesn't really help, at least it seems so.
The problem is not reproducible and breaks at different sections in my code. I noticed this because my code is run in a Sweave document, so I can open the .tex file after R has crashed and it always stops at a different position. However, it always seems to be when a data.table operation is called (that doesn't mean a lot though because my code relies heavily on data.table). However, even when I don't call Sweave, but just run the code, it breaks sometimes as well. So it does not seem to be related to Sweave.
It has nothing to do with the editor I'm using. I use RStudio, but reproduced this behavior by running the code in a plain R command window.
That's basically all possible explanations I've come up with. So it would be great if anyone has any hints on where this error could come from or what else I could check.
PS: I won't be at my machine the next couple of days, so I hope you forgive me when I don't give feedback immediately. Nevertheless, I wanted to post this question before Xmas, otherwise I couldn't enjoy it with my beloved R suffering and I'm sitting at home, not trying to cure it...
UPDATE
I looked further into the issue and after a while, I got a rather minimal example with data.table that breaks my R session. If this issue is fixed and it solves the crash of R as described here (note that this is a big if because the example I posted on the data.table list just breaks my R session and does not end it with the error message I described here), I will write an answer here and accept it.
Suddenly, same error came up from rvest package in the lines:
raw_HTML %>% html_nodes(xpath=HTML_table_xpath)
Switching to R 3.3.1 64bit fixed the problem (which remains with 32 bit R for now). That may be a workaround for some. In my case, rJava package requires 32 bit R :(
In case it helps anyone:
> sessionInfo()
R version 3.3.1 (2016-06-21)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 7 x64 (build 7601) Service Pack 1
locale:
[1] LC_COLLATE=English_United States.1252 LC_CTYPE=English_United States.1252 LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C LC_TIME=English_United States.1252
attached base packages:
[1] parallel stats graphics grDevices utils datasets methods base
other attached packages:
[1] rvest_0.3.2 xml2_1.0.0 xts_0.9-7 zoo_1.7-13 doParallel_1.0.10 iterators_1.0.8
[7] foreach_1.4.3 plyr_1.8.4 jsonlite_0.9.22 futile.logger_1.4.1
loaded via a namespace (and not attached):
[1] Rcpp_0.12.5 lattice_0.20-33 codetools_0.2-14 XML_3.98-1.4 R6_2.1.2 grid_3.3.1
[7] futile.options_1.0.0 magrittr_1.5 mail_1.0 httr_1.2.0 stringi_1.1.1 curl_0.9.7
[13] lambda.r_1.1.7 tools_3.3.1 stringr_1.0.0 selectr_0.2-3
OK, this was basically a hard to detect issue with data.table that should be fixed with the version 1.7.8. For more information, see the NEWS file.