I am trying to move data files created in R (3.3, Windows 64) to a remote Linux server. Something along the lines of
mydata <- data.frame(a = c(1,2), b = c(3,4))
write.csv(mydata, "data.csv")
system("scp data.csv [username#ip.address:~/]")
However, this returns
Warning message: command ... had status 127
which I believe has to do with scp's password prompt. R does not know how to respond to the prompt, so it aborts. I've also tried using shell(), but that returns
'scp' is not recognized as an internal or external command, operable program or batch file.```
So I guess my question is: Is there a way to get R's system() function to respond to prompts, or else is there another way to scp from R?
Additional information:
The remote IP address changes every day so I cannot use a passphrase-free SSH key
I cannot easily install tools like expect or sshpass to add to a bash script that would run from R
There is an scp function in the Rcurl package, but it only handles retrieval from remote to local. I'm trying to deposit to remote.
Related
I am writing an R code on a Linux system using RStudio. At some point in the code, I need to use a system call to a command that will download a few thousand of files from the lines of a text file:
down.command <- paste0("parallel --gnu -a links.txt wget")
system(down.command)
However, this command takes a little while to run (a couple of hours), and the R prompt stays locked while the command runs. I would like to keep using R while the command runs on the background.
I tried to use nohup like this:
down.command <- paste0("nohup parallel --gnu -a links.txt wget > ~/down.log 2>&1")
system(down.command)
but the R prompt still gets "locked" waiting for the end of the command.
Is there any way to circumvent this? Is there a way to submit system commands from R and keep them running on the background?
Using ‘processx’, here’s how to create a new process that redirects both stdout and stderr to the same file:
args = c('--gnu', '-a', 'links.txt', 'wget')
p = processx::process$new('parallel', args, stdout = '~/down.log', stderr = '2>&1')
This launches the process and resumes the execution of the R script. You can then interact with the running process via the p name. Notably you can signal to it, you can query its status (e.g. is_alive()), and you can synchronously wait for its completion (optionally with a timeout after which to kill it):
p$wait()
result = p$get_exit_status()
Based on the comment by #KonradRudolph, I became aware of the processx R package that very smartly deals with system process submissions from within R.
All I had to do was:
library(processx)
down.command <- c("parallel","--gnu", "-a", "links.txt", "wget", ">", "~/down.log", "2>&1")
processx::process$new("nohup", down.comm, cleanup=FALSE)
As simple as that, and very effective.
I'm trying to use the R.matlab package but frankly I'm not able to start the Matlab server.
I'm on OS X 10.11.5 and use Matlab R2015b.
Within R I get
> Matlab()
[1] "Matlab: The MATLAB host is 'localhost' and communication goes via port 9999. Objects are passed via the local file system (remote=FALSE). The connection to the MATLAB server is closed (not opened)."
> Matlab$startServer()
[1] 0
sh: matlab: command not found
But within a terminal I can indeed launch Matlab with the matlab command.
Also within a Terminal
matlab -nodesktop -nosplash -r MatlabServer
results in
Undefined function or variable 'MatlabServer'.
>>
I managed to fix it to run matlab from terminal. You just to go to your user profile ~/usr/your_nickame and edit file .bash_profile in any editor (and save!) and add
alias matlab=“/Applications/MATLAB_R2015a.app/bin/matlab”
That way terminal knows what you mean by matlab and where to find it and execute it. And it will make the alias permanent. That is option if you are starting the server from terminal
If you want to start it from R directly, I managed to make it work by setting what matlab command runs:
options(matlab="/Applications/MATLAB_R2015a.app/bin/matlab")
Then the usual works:
require(R.matlab)
Matlab$startServer()
matlab <- Matlab()
isOpen <- open(matlab)
< M A T L A B (R) >
Copyright 1984-2015 The MathWorks, Inc.
R2015a (8.5.0.197613) 64-bit (maci64)
February 12, 2015
To get started, type one of these: helpwin, helpdesk, or demo.
For product information, visit www.mathworks.com.
Running MatlabServer v3.5.9-9000
MATLAB v7.x or higher detected.
Saving with option -V6.
Added InputStreamByteWrapper to dynamic Java CLASSPATH.
----------------------
MATLAB server started!
----------------------
MATLAB working directory: /Users/air/Desktop/Dissertation/myPack/finalPack
Trying to open server socket (port 9999)...Error using MatlabServer (line 130)
Java exception occurred:
java.net.BindException: Address already in use
at java.net.PlainSocketImpl.socketBind(Native Method)
at
java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:376)
at java.net.ServerSocket.bind(ServerSocket.java:376)
at java.net.ServerSocket.<init>(ServerSocket.java:237)
at java.net.ServerSocket.<init>(ServerSocket.java:128)
>>
>
I have a CSV file and I want to work on it I've tried to read it by using this code
d = read.table( pipe( 'ssh don#140.184.134.189 "cat cluster.csv"' ), header = T )
But I get no result and get this message:
"error in read table"
Without asking about my password.
Also, how do you run an R script fes.r that is located on the same server?
You can first try this, continuing along the lines you are on:
> d <- read.table(pipe('ssh -l don 140.184.134.189 "cat cluster.csv"'))
don#140.184.134.189 password: # type password here
If you don't get prompted for a password, then there is likely a configuration problem with your ssh. Please note that ssh has to be installed and in your $PATH (meaning R can invoke it from anywhere it is running).
If this option doesn't work, then you can try using scp from the RCurl package.
Try the following:
x = scp("140.184.134.189", "cluster.csv", "PASSPHRASE", user="don")
Here you should replace "PASSPHRASE" with the password of your local SSH key.
One other thing to check is whether "cluster.csv" is really the correct path to your file on the remote server. But it seems that you are not even getting this far, so fix the ssh problem first.
Hat tip to this Stack Overflow post for inspiration.
You could take a different approach and install Rstudio server on your remote linux machine.
Rstudio server
You can avoid the password problem by setting up an ssh key pair, and adding your public key to the ~/.ssh/authorized_keys file on the server.
You can see how to run an R script from command line here: Run R script from command line
I can't start Rserve in debug mode.
I wrote these commands in R:
library(Rserve)
Rserve(debug=T, args="RS-enable-control", quote=T, port = 6311)
library(RSclient)
c=RSconnect(host = "localhost", port = 6311)
RSeval(c, "xx<-12")
RSeval(c, "2+6")
RSeval(c, "xx")
RSclose(c)
install.packages("fpc")
I placed the Rserve_d.exe in the same directory where the R.dll file is located. But when I launch it and I launch Tableau with the Rserve connection I can't see anything in the debug console, just these few lines.
Rserve 1.7-3 () (C)Copyright 2002-2013 Simon Urbanek
$Id$
Loading config file Rserv.cfg
Failed to find config file Rserv.cfg
Rserve: Ok, ready to answer queries.
-create_server(port = 6311, socket = <NULL>, mode = 0, flags = 0x4000)
INFO: adding server 000000000030AEE0 (total 1 servers)
I tried another solution by the command Rserve(TRUE) in R, but I can't see the transactions between R and Tableau neither in the Rstudio console.
I wanted then to print the output of the variable in R from the R-script function, by print(.arg1). But nothing appears in the R console
but when I run print in the R console it works fine.
According to this article*, RServe should be run with the following command to enable debugging:
R CMD Rserve_d
An alternative is to use the ‘write.csv’ command within the calculated field that calls an R script, as suggested by this FAQ document from Tableau
Starting Rserve_d.exe from command line works. Most likely you have multiple instances of Rserve running and Tableau is sending requests to one that is not Rserve_d running in the command line.
Did you try killing all Rserve processes and then starting Rserve_d from command line?
If you don't want to run from the command line you can try starting Rserve in process from RStudio by typing run.Rserve() then using print() statements in your Tableau calculated fields for things you want to print.
In the R bin directory, you have two executables Rserve for normal execution and Rserve.dbg for debug execution. Use
R CMD Rserve.dbg
My OS is CENTOS7 and I am using the R installation from anaconda. If your RServe debug executable has a different name you should be using that.
I am running a code for a client. Is there a R function / command to connect the computer to the available wireless and type in the passcode automatically?
Probably not.
However, you can run any OS command with system, so if you have a shell command foo which does what you want, you can just do system("foo") in R.
Most connections to the "outside world" with R are handled through the OS version of system-level 'libcurl' package. There is a package for R called RCurl. The authentication system is described in the variaous pages of:
help(package="RCurl") # brings up the index page.
help(getURL, package="RCurl") # has examples of authentication to a server with R code.
You should also read:
?connections