I have an rscript that is on my local desktop but I want to run this in a directory that is on a remote password protected sever. R is already installed in this computing cluster but what do I use in my terminal to execute the R script?
Thanks!
I want to login to a remote server via ssh and start R afterwards.
ssh -X user#166.167.0.1 '/usr/bin/R'
Fatal error: you must specify '--save', '--no-save' or '--vanilla'
Which tells me that start of R failed.
How can I solve it ?
According to this Q&A in this github
It seems like the source of the problem is not with /usr/bin/R parameters, but with the parameters of the RServe itself.
This is a message from Rserve itself, and in the documentation. eg:
You can start Rserve with Rserve(args="--no-save")
Check here under Launching Rserve in the link below
https://www.rforge.net/Rserve/doc.html
More info from Rserve docs:
Launching Rserve Rserve comes now as an R package, so one way to start Rserve is from within R, just type
library(Rserve)
Rserve()
That command knows how to find Rserve, how to setup the environment
and how to start it, regardless of your platform.
Note: depending on how you are running R, it may require additional
parameters such as --no-save (R will tell you). In that case you have
to pass that parameter in the args argument such as
Rserve(args="--no-save")
However, Rserve is a stand-alone program, so it can be started
directly as well. If you installed Rserve from a source package (on
unix), type:
R CMD Rserve
Depending on the purpose and libraries on your computer you may want
to add --gui-none or --no-save (see R documentation). Rserve passes
any parameters to the underlying R engine except for Rserver-related
parameters (see command line arguments). After initialization Rserve
daemonizes itself to work as a server. It can be shut down gracefully
(i.e. it will wait until all existing connections are finished) if it
recieves shutdown command from an authorized connection. It terminates
not so gracefully if it receives usual termination signals.
If no config file is supplied, Rserve accepts no remote connections,
requires no authentication and file transfer is enabled. For more
details about how to configure Rserve, see below.
Answering the question in the note:
You should configure Rserve on the remote host.
The configuration process can be done once, using Rserve configuration files.
In order to login to the remote host, you can use the following command:
ssh -X user#166.167.0.1
Start Rserve with --no-save parameter:
R CMD Rserve(args="--no-save")
Here is a discussion how to setup Rserve as a service.
I am trying to run a shell script to execute a binary on a remote linux box. Both the binary and the shell script are on my local window machine. Is there any way through which i can run the binary to the remote machine directly from windows through command line tools like PLINK?
I don't want to put the binary and the script to all the remote linux boxes which
i want them to run on,Instead I want to run the shell script which will intern invoke the binary and do the desirable functions directly through my local machine.
You can run the shell script remotely, just by piping it through ssh:
cat my_script.sh | ssh -T my_server
(Or whatever the windows/plink equivalent is.)
However, you can't run the binary remotely through a pipe, the file will have to exist on the remote server. You can do this by pushing the file from your windows machine to a known location on the remote server, and then editing your script to expect the file to exist in that location:
scp my_binary my_server:/tmp
cat my_script.sh | ssh -T my_server
And then just have your script run that binary:
/tmp/my_binary
Or you can write the script so that it pulls the binary file from a central location where you're hosting it:
wget -O /tmp/my_binary http://my_fileserver/my_binary
/tmp/my_binary
Note, if the shell script doesn't do anything else besides invoke the binary, then you don't need it. You can just fire the commands directly through ssh:
ssh -T my_server "cd /tmp && wget http://my_fileserver/my_binary && ./my_binary"
You will have to copy the binary to the remote Linux box before it can be executed. However, you could have a script on the windows machine that uses sftp to transfer the binary program to a temporary directory under /tmp before running it, so there is no manual setup required.
I would like to run all the R script (script1.R, script2.R, ...) stored in a directory (~/Sims). Moreover I would like that each script run in a separate terminal. The os I'm using is OS X 10.9.5.
I used a bash script with the following commands:
#!/bin/bash
FILES=~/Sims/*.R
for f in $FILES
do
xterm -e bash -c "R --vanilla < $f; exec bash" &
done
I would like to find an alternative for xterm (given that under os x require to install the package X11 and on some machine I can't install it) that it is part of os x (like the Terminal app)
I would like to not exit from the R environment at the end of the R script
This will mimic your xterm configuration but use new Terminal.app sessions instead:
for f in *.R
do
osascript -e "tell app \"Terminal\" to do script \"R --vanilla < /FULL/PATH/TO/${f}\""
done
As far as keeping the R session alive, I'm not sure that's possible.
Working in R 2.14.1, on Windows 7
Using the package parallel in R, I'm trying to take advantage of cores outside of my local machine available on my network, where all remote hosts I am connecting to are identical Windows machines.
The basic form of the commands are as such to make the connection.
library(parallel)
#assume 8 cores per machine
cl<-makePSOCKcluster(c(rep("localhost", 8), rep("otherhost", 8)))
Of course, trying to debug these things can be pretty tricky, but here is where I'm at with it.
If I specify the manual = TRUE flag as below
cl<-makePSOCKcluster(c(rep("localhost", 8), rep("otherhost", 8)), manual=TRUE)
there are no problems connecting to the remote host, and running a parallel process. The computers have identical setups to the one that I am working on. Yet, when this manual flag is not set, the connection command hangs.
This seems to indicate to me that since the manual flag bypasses ssh to make the connection to the host, that ssh is the problem when manual=FALSE.
It is not guaranteed at the moment that the remote computers have ssh on them. The question is, given that I have all the pertinent windows login information for my remote hosts, and that I cannot change the settings on the remote computers, how would I connect to cores on remote machines with the package parallel in R without specifying manual = true?
Alternatively, if ssh must be installed for this to happen, let's assume all computers have ssh on them. How would I connect to cores on the remote machines without circumventing ssh?
If you need any more information please let me know, I appreciate the time.
UPDATE 1
8-26-14
Thanks to Steve Weston for his insights. I will provide an update with the exact tools and setup I use to get my system working when it's up and running.
Feel free to comment or post if you have anything else to add as to what may be the best route to go in remote connecting to a windows machine from a windows machine via makePSOCKcluster, where the manual flag is set to FALSE.
When creating a PSOCK cluster with manual=FALSE, the only way to start a worker on a remote machine is with "ssh", "rsh", or something command-line compatible, such as "plink" from PuTTY. The reason is that makePSOCKcluster starts the remote workers using the "system" function to execute commands of the form:
ssh -l user otherhost '/usr/lib/R/bin/Rscript' -e 'parallel:::.slaveRSOCK()' MASTER=myhost PORT=10187 OUT=/dev/null TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
You can confirm this by looking at the source code for the newPSOCKnode function in the file snowSOCK.R from the parallel package.
For this to work, the ssh-compatible command must be available on the local machine and a corresponding ssh daemon must be running on each of the remote machines, otherwise makePSOCKcluster will simply hang. I've found that installing a good, working ssh daemon is the difficult part on Windows.
Unfortunately, manual=TRUE is generally the easiest way to create a PSOCK cluster on multiple Windows machines.
Helle everyone, I had the same problem and I managed to solve it. It is June 2018 when I'm writing this answer, my OS is windows 10 and the R version is 3.2.2. It is surprising to see this problem still exists after 4 years. I hope it can be fixed in the following release.
Before you move on, please make sure you can access the server in cmd using ssh. I didn't put any password in my code because I have the private key, you don't need to do that and you will see the reason later.
Fixing The problem
File directory
Since the function makePSOCKcluster works when manually start the workers, my first trying is to let manual=TRUE, and see what's the output. Here is my result:
machineAddresses <-list(list(host='192.168.1.220',user='jeff'))
cl <- makePSOCKcluster(spec,manual = F)
> Manually start worker on 192.168.1.220 with
"C:/PROGRA~1/R/R-32~1.2/bin/x64/Rscript" -e
"parallel:::.slaveRSOCK()" MASTER=DESKTOP-U5JA32O PORT=11756
OUT=/dev/null TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
Ok, Here is the first problem. The Rscript location is incorrect(The location of Rscript in the server). Generally, it locates in C:\Program Files. In my server is C:\Program Files\R\R-3.2.2\bin. So we need to correct them by adding more option to tell this stupid code where the Rscript is:
machineAddresses <-list(list(host='192.168.1.220',
user='jeff',rscript="C:/Program Files/R/R-3.3.2/bin/Rscript"))
CMD problem
Once you fix the directory problem, you will find that the code still hangs forever. Then we need to check if we can manually access the server in R, my code is:
system("ssh jeff#192.168.1.220")
> GetConsoleMode on STD_INPUT_HANDLE failed with 6
I honestly don't know what does this error mean, but we just need to fix that. Inspired by #Steve Weston, I decide to use PuTTY, so I install it, and change my code to:
machineAddresses <-list(list(host='192.168.1.220',user='jeff',rscript="C:/Program Files/R/R-3.3.2/bin/Rscript",rshcmd="plink -pw qwer"))
The option -pw means the password. Because I'm a newbie to PuTTY, I don't know how to let the private key automatically work in PuTTY. Therefore, I use the easiest way to deal with that: put your password! The above code is equivalent to the following in cmd:
plink -pw qwer jeff#192.168.1.220 Rscript -e parallel:::.slaveRSOCK() MASTER=DESKTOP-U5JA32O PORT=11063 OUT=/dev/null TIMEOUT=2592000 METHODS=TRUE XDR=TRUE
And this is exactly what we will do if we manually create the workers. For those who are new like me, you need to add the PuTTY directory in PATH in your environmental variables to run plink. Here are my final codes:
machineAddresses <-list(list(host='192.168.1.220',user='jeff',rscript="C:/Program Files/R/R-3.3.2/bin/Rscript",rshcmd="plink -pw qwer"))
cl <- makePSOCKcluster(machineAddresses,manual = F)
I run it with no problem at all. In summary, the function makePSOCKcluster makes two mistakes:
Assuming a wrong R directory in the server(At least it should assume the same directory as my local computer, but it didn't! I don't know where that strange directory comes from)
Using ssh command to start the connection, which does not work in R. It works well in cmd, but not in R. I don't know the reason.
If you are still not able to use makePSOCKcluster, here is one trick: Try to connect to the server in R using system function first. It can give you some error code, that may instruct you where the problem is. Here is my debugging code:
system("plink -pw qwer jeff#192.168.1.220 Rscript -e parallel:::.slaveRSOCK() MASTER=DESKTOP-U5JA32O PORT=11063 OUT=/dev/null TIMEOUT=2592000 METHODS=TRUE XDR=TRUE")