I have a cluster with one master and 3 compute nodes. computes have restricted access to the internet.
I installed Julia and Ipopt on the master node. then copied the entire installation to the compute nodes. Ipopt is added and built on the master.
On the master:
/share/apps/Julia/julia-903644385b/
Pkg.dir(): "/home/user1/.julia/v0.6"
On the compute:
partition1/apps/Julia/julia-903644385b/
Pkg.dir(): "/partition1/apps/Julia/julia-903644385b/share/julia/site/v0.6"
base=/partition1
export JULIADIR=$base/apps/Julia/julia-903644385b/
export JULIA_HOME=$JULIADIR/bin
export JULIA_PKGDIR="/partition1/apps/Julia/julia-903644385b/share/julia/site/v0.6"
export LD_LIBRARY_PATH=$JULIADIR/lib/julia:$JULIA_PKGDIR/v0.6/Ipopt/deps/usr/lib/:$LD_LIBRARY_PATH
export PATH=$JULIADIR/bin:$PATH
running a sample Ipopt script is resulting in an error:
ERROR: LoadError: LoadError: Unable to load
libipopt (/share/apps/Julia/julia-903644385b/share/julia/site/v0.6/Ipopt/deps/usr/lib/libipopt.so)
Julia on compute node is trying to pickup Ipopt from the master node. Is this matter of clearing any cache or a missing env variable?
Thanks
On the compute node, locate the file Ipopt/deps/deps.jl, and adapt the path where to find the libipopt.so:
# Load dependencies
#checked_lib libipopt "/full/path/to/v0.6/Ipopt/deps/usr/lib/libipopt.so"
Related
Guys, I'm having problems deploying my streamlit application, I include the RCurl package in the packages.txt list, however the deploy is interrupted with the following message:
unable to locate package RCurl
Another interesting thing:
The first time I deployed, RCurl was not included, and my application ran normally.
However when I performed a second deployment of the same application, when executing a function dependent on this package it returned "there is no package called RCurl"
Importantly, this same function was performed on the first deploy.
From then on I tried both ways:
listing the RCurl in packages.txt, and
not listing RCurl in packages.txt
in the first form the deploy is completed but when executing the function the following message is returned "there is no package called RCurl"
and when trying the second way the deploy is interrupted with the following message: 'Unable to locate package RCurl'
Can anyone give me a hint of a possible solution?
Thanks!
I have installed R on my windows server 2019 and installed Rcpp package, then compressed its installation to a zip file (R-4.0.2.zip).
In a C++ application on another machine, I decompress this R-4.0.2.zip, set R_HOME and PATH to point to the extracted directory and /bin/x64, respectively. I then load R.dll to run a simple R script.
I have run this application on my machine using the extracted R-4.0.2.zip and also another machine with no pre-installation of R where I have access to its graphical user interface. However, when I run it on another machine (server), which I don't have access to its graphical interface, the R suicides itself with a message in a messagebox (that I can only see with a windows debugger from its dump).
Fatal error: unable to initialize the JIT
user32!SoftModalMessageBox+0x228b [d:\xx\windows\core\ntuser\client\msgbox.c # 1235]
user32!MessageBoxWorker+0x2ec [d:\xx\windows\core\ntuser\client\msgbox.c # 782]
user32!MessageBoxTimeoutW+0xd5 [d:\xx\windows\core\ntuser\client\msgbox.c # 439]
user32!MessageBoxTimeoutA+0x102 [d:\xx\windows\core\ntuser\client\msgbox.c # 483]
user32!MessageBoxExA+0x10 [d:\xx\windows\core\ntuser\client\msgbox.c # 369]
user32!MessageBoxA+0x4e [d:\xx\windows\core\ntuser\client\msgbox.c # 342]
R!R_Suicide+0x39
R!setup_Rmainloop+0x954
Any idea what could be the potential problem for this? do I need to install a specific package such as Rserve, any specific firewall rule? [the CPP application and R script run on the same machine though]
I'm building a R package for binary classification and I'm using opencpu to host it. Currently I've saved the h5 file as .RData file(serialized), which is then loaded in the environment using the .onLoad() function in R. This enables the R script to use the environment variable to load keras model using keras::unserialized_model().
I've tried directly using keras::load_model_hdf5() in the code, but after building and deploying on opencpu, when I try to hit the prediction API, I get error
ioerror: unable to open file (unable to open file: name = '/home/modelfile_26feb.h5', errno = 13, error message = 'permission denied', flags = 0, o_flags = 0)
I have changed permission for the file(777) and even the groups but still getting the error.
I even tried putting the file in inst/extdata folder so that it gets in the package but still same error.
Can anyone help on this, or suggest some alternative to load the h5 model directly?
Which OS does OpenCPU run on? Why does it try to write in /home/, this is very unusual? The best solution is to adapt your code to write in getwd() or tempdir(). Even better is to store data in a local database or redis server and let R read it from there, so you don't need disk access at all.
If you run on Ubuntu Server, reading from /home/ is not permitted by default. If you want to allow this, you need to add apparmor rules, see section 3.5 of the server manual.
Some relevant topics from the opencpu mailing list:
write in home dir: https://groups.google.com/d/msg/opencpu/5vRvgSKY-qE/4xMzZCGJBAAJ
keras in opencpu: https://groups.google.com/d/msg/opencpu/HhRzFVVFdaA/n5Nu1sxyFgAJ
write tmp folder: https://groups.google.com/d/msg/opencpu/Y1tYhaQUzwU/ubSEd_CDCgAJ
This is the error message:
> h2o.init()
Error in dirname(path) : path too long
In addition: There were 12 warnings (use warnings() to see them)
This is one of the warning messages (the others are similar):
> warnings()
Warning messages:
1: In normalizePath(path.expand(path), winslash, mustWork) :
path[1]="\\FILE-EM1-06/USERDATA2$/john134/My Documents/./../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../../..": The filename or extension is too long
Any idea how to work around this error?
Thanks
It seems that Windows path string is limited to (maybe) 256 length. Usually, setting a the path setwd(shorterExistingWorkDir) works and should address your issue.
I struggled with this issue quite a bit, including upgrading.
Most folks are assuming that you've literally just set an incredibly long path. I don't think this is the case (it wasn't for me, at least). It's that the PATH may be set on a network drive or other device where the underlying mapped paths are more complicated.
A related thread is here on the H2O forum:
Main issue is the user had a Windows drive that did not conform to the norm, i.e., "C://", etc. Instead, the user had a network drive
(DTCHYB-AZPX015/). This caused issues in the search for a config
file as there was no "root" (In this case, "root" is reaching your Win
drive). Since there was no "root", the path to search kept expanding
until it caused R to error out with the above exception.
The fix is to NOT search for a config when h2o.init() is called. Rather, only search for a config if a user asks to do so. My proposal
is to add a new field to h2o.init()called ignore_config. This
field will be set to TRUE by default.
When calling h2o.init() the R environment signal the launching of h2o application (actually a web server) in the backend which was installed when you install H2O package into R. The local runtime environment uses the full path of the location where H2O jar file is located. Because the packages is installed deep inside the nested folders in your file system it cross the valid limit of OS path 256 character length and fails to launch the backend H2O server and you see this error. In your case you are using external path so adds up more characters in the path to make the problem worse..
For example the h2o.jar is located in my OSX machine as below:
/Library/Frameworks/R.framework/Resources/library/h2o <-- H2O package Path
/Library/Frameworks/R.framework/Resources/library/h2o/java/h2o.jar <-- Jar Path
As you are using Windows, what you need is to find ways to reduce this path to OS limit and it will work.
The other solution is to run h2o.jar separately and then just use R to connect to H2O cluster. The steps are as below:
Download H2O 3.10.4.2 and unzip to a folder close to root so you do not hit 265 char limit again. Also install 3.10.4.2 R Package. (Try to keep the same version)
Run H2O > java -jar h2o.jar
From RStudio console try > h2o.init()
So if there is already H2O cluster running the h2o.init() will connect to a running H2O cluster instead to start one and you will by pass above problem.
If you hit any problem write here and we will help you.
I have created the R script environment (used this command to create it "create script environment RSCRIPT command '/usr/local/R/bin/Rscript --vanilla --slave'") and tried running the one R script but it fails with the below error message.
ERROR: RS 10 S 332659 R 31A004F LO:Script stderr: external script vfork child: No such file or directory
Is it because of the below line which i am using in the script ?
mydata <- read.csv(file=file("stdin"), header=TRUE)
if (nrow(mydata) > 0){
I am not sure what is it expecting.
I have one more questions to ask.
1) do we need to install the R package on our unix box ? if not then the kognitio package has it
I suspect the problem here is that you have not installed the R environment on ALL the database nodes in your system - it must be installed on every DB node involved in processing (as explained in chapter 10 of the Kognitio Guide which you can download from http://www.kognitio.com/forums/viewtopic.php?t=3) or you will see errors like "external script vfork child: No such file or directory".
You would normally use a remote deployment tool (e.g. HP's RDP) to ensure the installation was identical on all DB nodes. Alternatively, you can leverage the Kognitio wxsync tool to synchronise files across nodes.
Section 10.6 of the Kognitio Guide also explains how to constrain which DB nodes are involved in processing - this is appropriate if your script environment should not run on all nodes for some reason (e.g. it has an expensive per-node/per-core licence). That does not seem appropriate for using R though.