Capture and process output from external commands from R - r

Take this simple python script for example:
#!/usr/bin/env python3
# /tmp/xxx.py
import time
for i in range(1000):
print(i)
time.sleep(1)
It continuously outputs numbers. I can call it from R like this:
system2("/tmp/xxx.py", stdin=?)
where stdin can be set to NULL, "", TRUE or a filename. But what I am looking for is a way to process these numbers in realtime. For example, whenever a number is printed from this python script, I want to multiply the number by Pi and then print it to the console. Is there a way to do this in R?

Not an expert, but I got something working.
First of all, I used the following /tmp/xxx.R executable Rscript instead of your python script as I found out python was buffering its output (not printing one line at a time) which makes it hard to test:
#!/usr/bin/env Rscript
for (i in 1:5) {
cat(i, "\n")
Sys.sleep(1)
}
Then the R code:
system('mkfifo /tmp/xxx.fifo')
f <- fifo("/tmp/xxx.fifo", 'r')
p <- pipe('/tmp/xxx.R > /tmp/xxx.fifo; echo OVER > /tmp/xxx.fifo', 'w')
while(TRUE) {
line <- readLines(f, n = 1)
if (length(line) > 0) {
if (line == "OVER") break
cat(pi * as.numeric(line), "\n")
}
Sys.sleep(0.1)
}
close(f)
close(p)
some of which was inspired from this: https://stackoverflow.com/a/5561188/1201032.
Hope it answers your question.

Related

Recursion error in R (Fibonacci sequence)

So I am trying to learn R on my own and am just working through the online tutorial. I am trying to code a recursive function that prints the first n terms of the Fibonacci sequence and can't get the code to run without the error:
Error in if (nterms <= 0) { : missing value where TRUE/FALSE needed
My code does ask me for input before entering the if else statement either which I think is odd as well. Below is my code any help is appreciated.
#Define the fibonacci sequence
recurse_fibonacci <- function(n) {
# Define the initial two values of the sequence
if (n <= 1){
return(n)
} else {
# define the rest of the terms of the sequence using recursion
return(recurse_fibonacci(n-1) + recurse_fibonacci(n-2))
}
}
#Take input from the user
nterms = as.integer(readline(prompt="How many terms? "))
# check to see if the number of terms entered is valid
if(nterms <= 0) {
print("please enter a positive integer")
} else {
# This part actually calculates and displays the first n terms of the sequence
print("Fibonacci Sequence: ")
for(i in 0:(nterms - 1)){
print(recurse_fibonacci(i))
}
}
This is a problem of readline in non-interactive mode. readline does not wait for a keypress and immediately executes the next instruction. The solution below is the solution posted in this other SO post.
I post below a complete answer, with the Fibonnaci numbers function a bit modified.
recurse_fibonacci <- function(n) {
# Define the initial two values of the sequence
if (n <= 1){
n
} else{
# define the rest of the terms of the sequence using recursion
Recall(n - 1) + Recall(n - 2)
}
}
#Take input from the user
cat("How many terms?\n")
repeat{
nterms <- scan("stdin", what = character(), n = 1)
if(nchar(nterms) > 0) break
}
nterms <- as.integer(nterms)
# check to see if the number of terms entered is valid
if(nterms <= 0) {
print("please enter a positive integer")
} else {
# This part actually calculates and displays the first n terms of the sequence
print("Fibonacci Sequence: ")
for(i in 0:(nterms - 1)){
print(recurse_fibonacci(i))
}
}
This code is the contents of file fib.R. Running in a Ubuntu 20.04 terminal gives
rui#rui:~$ Rscript fib.R
How many terms?
8
Read 1 item
[1] "Fibonacci Sequence: "
[1] 0
[1] 1
[1] 1
[1] 2
[1] 3
[1] 5
[1] 8
[1] 13
rui#rui:~$
To make it work with Rscript replace
nterms = as.integer(readline(prompt="How many terms? "))
with
cat ("How many terms?")
nterms = as.integer (readLines ("stdin", n = 1))
Then you can run it as Rscript fib.R, assuming that the code is in the file fib.R in the current working directory.
Otherwise, execute it with source ("fib.R") within an R shell.
Rscript does not operate in interactive mode and does not expect any input from the terminal. Check what interactive () returns in both the cases. Rscript will return FALSE as it is non-interactive, but the same function when run within an R shell (with source ()) it will be true.
?readline mentions that it cannot be used in non-interactive mode. Whereas readLines explicitely connect to stdin.
The code works fine but you shouldn't enter it into the terminal as is. My suggestion: put the code into a script file (ending .R) and source it (get help about it with ?source but it's actually pretty straightforward).
In R-Studio you can simply hit the source button.

calling a R function from python code with passing arguments

rtest = function(input ,output) {
a <- input
b <- output
outpath <- a+b
print(a+b)
return(outpath)
}
I have just return this R code for as a function for getting sum of two numbers. I tried to run this function from my python code using subprocess by passing 2 numbers as arguments. But it does not return sum value as return output. Do you know any method for implement this in python3 by passing function arguments.
my python code using subprocess is:
args=['3','10'] # (i tried to pass aruments like this)
command="Rscript"
path2script = '/...path/rtest.R'
cmd = [command, path2script] +args
x = subprocess.check_output(cmd, universal_newlines=True)
print(x)
but x return ' ' null value
This could be easily done by rpy2 library in python.
import rpy2.robjects as ro
path="specify/path to/ R file"
def function1(input,output):
r=ro.r
r.source(path+"rtest.R")
p=r.rtest(input,output)
return p
a=function1(12,12) # calling the function with passing arguments
Thanks.

how do you get the tail -50 of the remote file in R

I need to be able to login to a system and get to get the tail -50 of the file. I have teh following set of code but this gets the whole file. Since the file is so huge, I need to run the script every 10 minutes and get the last 50 lines of the file. This is what I have:
library(RCurl)
for(k in 1:2) {
day1<-day-k
day1<-format(day1, "%Y%m%d")
day_vector<-c(day_vector, day1)
}
servers<-c("ServerA", "ServerB","ServerC")
for (i in 1:length(servers)) {
for(j in 1:length(day_vector)) {
ser_day<-paste(servers[i], day_vector[j], sep=".")
url<-paste0(servers[i], ".example.net/opt/files/appl_log_")
url<-paste(url, ser_day, sep="")
url<-paste0("sftp://", url)
tryCatch({
x<-getURL(url, userpwd="account1:passwd1", connecttimeout=60)
writeLines(x, paste0(servers[i], ".txt"))
},error = function(e)
{
print("unable to get the file")
})
Is there any way to get the tail of the file?

RCurl: Display progress meter in Rgui

Using R.exe or Rterm.exe, this gives an excellent progress meter.
page=getURL(url="ftp.wcc.nrcs.usda.gov", noprogress=FALSE)
In Rgui I am limited to:
page=getURL(url="ftp.wcc.nrcs.usda.gov",
noprogress=FALSE, progressfunction=function(down,up) print(down))
which gives a very limited set of download information.
Is there a way to improve this?
I start doubting that with standard R commands it is possible to reprint overwriting the current line, which is what RCurl does in non-GUI mode.
I am glad to tell that I was wrong. At least for a single line, \r can do the trick. In fact:
conc=function(){
cat(" abcd")
cat(" ABCD", '\n')
}
conc()
# abcd ABCD
But:
over=function(){
cat(" abcd")
cat("\r ABCD", "\n")
}
over()
# ABCD
That given, I wrote this progressDown function, which can monitor download status rewriting always on the same same line:
library(RCurl) # Don't forget
### Callback function for curlPerform
progressDown=function(down, up, pcur, width){
total=as.numeric(down[1]) # Total size as passed from curlPerform
cur=as.numeric(down[2]) # Current size as passed from curlPerform
x=cur/total
px= round(100 * x)
## if(!is.nan(x) && px>60) return(pcur) # Just to debug at 60%
if(!is.nan(x) && px!=pcur){
x= round(width * x)
sc=rev(which(total> c(1024^0, 1024^1, 1024^2, 1024^3)))[1]-1
lb=c('B', 'KB', 'MB', 'GB')[sc+1]
cat(paste(c(
"\r |", rep.int(".", x), rep.int(" ", width - x),
sprintf("| %g%s of %g%s %3d%%",round(cur/1024^sc, 2), lb, round(total/1024^sc, 2), lb, px)),
collapse = ""))
flush.console() # if the outptut is buffered, it will go immediately to console
return(px)
}
return(pcur)
}
Now we can use the callback with curlPerform
curlProgress=function(url, fname){
f = CFILE(fname, mode="wb")
width= getOption("width") - 25 # you can make here your line shorter/longer
pcur=0
ret=curlPerform(url=url, writedata=f#ref, noprogress=FALSE,
progressfunction=function(down,up) pcur<<-progressDown(down, up, pcur, width),
followlocation=T)
close(f)
cat('\n Download', names(ret), '- Ret', ret, '\n') # is success?
}
Running it with a small sample binary:
curlProgress("http://www.nirsoft.net/utils/websitesniffer-x64.zip", "test.zip")
the intermediate output at 60% is (no # protection):
|................................. | 133.74KB of 222.75KB 60%
where KB, will be adjusted to B, KB, MB, GB, based on total size.
Final output with success status, is:
|.......................................................| 222.61KB of 222.75KB 100%
Download OK - Ret 0
Note, the output line width is relative to R width option (which controls the maximum number of columns on a line) and can be customised changing the curlProgress line:
width= getOption("width") - 25
This is enough for my needs and solves my own question.
Here's a simple example using txtProgressBar. Basically, just do a HEAD request first to get the file size of the file you want to retrieve, then setup a txtProgressBar with that as its max size. Then you use the progressfunction argument to curlPerform to call setTxtProgressBar. It all works very nicely (unless there is no "content-length" header, in which case this code works by just not printing a progress bar).
url <- 'http://stackoverflow.com/questions/21731548/rcurl-display-progress-meter-in-rgui'
h <- basicTextGatherer()
curlPerform(url=url, customrequest='HEAD',
header=1L, nobody=1L, headerfunction=h$update)
if(grepl('Transfer-Encoding: chunked', h$value())) {
size <- 1
} else {
size <- as.numeric(strsplit(strsplit(h$value(),'\r\nContent-Type')[[1]][1],
'Content-Length: ')[[1]][2])
}
bar <- txtProgressBar(0, size)
h2 <- basicTextGatherer()
get <- curlPerform(url=url, noprogress=0L,
writefunction=h2$update,
progressfunction=function(down,up)
setTxtProgressBar(bar, down[2]))
h2$value() # return contents of page
The output is just a bunch of ====== across the console.
What about:
curlProgress=function(url, fname){
f = CFILE(fname, mode="wb")
prev=0
ret=curlPerform(url=url, writedata=f#ref, noprogress=FALSE,
progressfunction=function(a,b){
x=round(100*as.numeric(a[2])/as.numeric(a[1]))
if(!is.nan(x) && x!=prev &&round(x/10)==x/10) prev<<-x else x='.'
cat(x)
}, followlocation=T)
close(f)
cat(' Download', names(ret), '- Ret', ret, '\n')
}
?
It prints dots or percent download divisible by 10 and breaks line on 50%.
And with a small 223 KB file:
curlProgress("http://www.nirsoft.net/utils/websitesniffer-x64.zip", "test.zip")
it sounds like this:
................10...............20................30...............40...............50
..............................70...............80...............90...............100... Download OK - Ret 0
I start doubting that with standard R commands it is possible to reprint overwriting the current line, which is what RCurl does in non-GUI mode.

Output of R script is hidden in the console

z <- 5
count <- 0
while(z > 0 && z < 10){
X=rbinom(1,1, 0.5)
if(X == 1)
{
z <- z+1
}
else if(X == 0)
{
z <- z-1
}
count <- count+1
}
print(count)
Hi, this is my R script. I was wondering why when I type in:
source ('filename.R')
,there is no output in the console. But when I run another R script:
x <- 1:10
print(x)
it prints to the console.
I'm using Rx64 3.0.2. Thank you.
Try using: ?source
# This will echo all input and not truncate 150+ character lines..
source("filename.R", echo=TRUE,max.deparse.length=10000, continue.echo = getOption("continue"))
To amplify Prasanna's answer, here's the help file info:
echo logical; if TRUE, each expression is printed after parsing,
before evaluation.
print.eval logical; if TRUE, the result of eval(i) is printed for
each expression i; defaults to the value of echo.
Since the default value is echo=FALSE, you see nothing. This is a good default, since most of the time source is used to load functions rather than execute scripts, and people generally :-) don't want the function source splattered all over the console.

Resources