I‘m writing a simple function that reads multiple files from the user, then it prints into the terminal (print) file name and yes or no next to it, if it was valid/invalid. I would like to use x/✓symbols.
The problem however, since the check symbol is not in ascii, R cmd is throwing a warning. I tried several approaches (charToRaw, intToUtf8, symbol(“\326”) yet non is working with simple print in terminal.
As an example:
Df <- data.frame(file = myfiles, status = “x”)
Df$status[1]= “✓”
print (Df)
Any idea? Thanks
The cli package offers a way to print these with cli_alert_success and cli_alert_danger. Presumably, you have some more complicated check as to if the file was valid. Store that as a boolean instead of the explicit character.
Df <- data.frame(file = myfiles, isValid = myValidityCheckFx(myfiles))
purrr::walk2(
Df$file, Df$isValid,
~if(.y) cli::cli_alert_success(.x) else cli::cli_alert_danger(.x)
)
Related
Is it possible to get R to write a plot in bitmap format (e.g. PNG) to standard output? If so, how?
Specifically I would like to run Rscript myscript.R | other_prog_that_reads_a_png_from_stdin. I realise it's possible to create a temporary file and use that, but it's inconvenient as there will potentially be many copies of this pipeline running at the same time, necessitating schemes for choosing unique filenames and removing them afterwards.
I have so far tried setting outf <- file("stdout") and then running either bitmap(file=outf, ...) or png(filename=outf, ...), but both complain ('file' must be a non-empty character string and invalid 'filename' argument, respectively), which is in line with the official documentation for these functions.
Since I was able to persuade R's read.table() function to read from standard input, I'm hoping there's a way. I wasn't able to find anything relevant here on SO by searching for [r] stdout plot, or any of the variations with stdout replaced by "standard output" (with or without double quotes), and/or plot replaced by png.
Thanks!
Unfortunately the {grDevices} (and, by implication, {ggplot2}) seems to fundamentally not support this.
The obvious approach to work around this is: let a graphics device write to a temporary file, and then read that temporary file back into the R session and write it to stdout.
But this fails because, on the one hand, the data cannot be read into a string: character strings in R do not support embedded null characters (if you try you’ll get an error such as “nul character not allowed”). On the other hand, readBin and writeBin fail because writeBin categorically refuses to write to any device that’s hooked up to stdout, which is in text mode (ignoring the fact that, on POSIX system, the two are identical).
This can only be circumvented in incredibly hacky ways, e.g. by opening a binary pipe to a command such as cat:
dev_stdout = function (underlying_device = png, ...) {
filename = tempfile()
underlying_device(filename, ...)
filename
}
dev_stdout_off = function (filename) {
dev.off()
on.exit(unlink(filename))
fake_stdout = pipe('cat', 'wb')
on.exit(close(fake_stdout), add = TRUE)
writeBin(readBin(filename, 'raw', file.info(filename)$size), fake_stdout)
}
To use it:
tmp_dev = dev_stdout()
contour(volcano)
dev_stdout_off(tmp_dev)
On systems where /dev/stdout exists (which are most but not all POSIX systems), the dev_stdout_off function can be simplified slightly by removing the command redirection:
dev_stdout_off = function (filename) {
dev.off()
on.exit(unlink(filename))
fake_stdout = file('/dev/stdout', 'wb')
on.exit(close(fake_stdout), add = TRUE)
writeBin(readBin(filename, 'raw', file.info(filename)$size), fake_stdout)
}
This might not be a complete answer, but it's the best I've got: can you open a connection using the stdout() command? I know that png() will change the output device to a file connection, but that's not what you want, so it might work to simply substitute png by stdout. I don't know enough about standard outputs to test this theory, however.
The help page suggests that this connection might be text-only. In that case, a solution might be to generate a random string to use as a filename, and pass the name of the file through stdout so that the next step in your pipeline knows where to find your file.
I am looking to create a custom function in R that will allow the user to call the function and then it will produce an auto complete pipeline ready for them to edit, this way they can jump into quickly customizing the standard pipeline instead of copy pasting from an old script or retyping. How can I go about setting up this sort of autocomplete:
#pseudo code what I type---
seq.Date(1,2,by = "days") %>%
pblapply(function(x){
read.fst(list.files(as.character(x), as.data.table = T) %>%
group_by(x) %>%
count()
}) %>% rbindlist()
#how can I write a function so that when I call that function, it ouputs an autocomplete
#of the above so that I can go ahead and start just customizing the code? Something like this
my_autocomplete_function = function(x) {
print(
"
seq.Date(as.Date(Sys.Date()),as.Date(Sys.Date()+1),by = 'days') %>%
pbapply::pblapply(function(x){
fst::read.fst(list.files(as.character(x), as.data.table = T)) %>%
#begin filtering and grouping by below custom
group_by()
}) %>% rbindlist()
")
}
#I can just print the function and copy paste the text from the output in my console to my script
my_autocomplete_function()
#but I rather it just autocomplete and appear in the script if possible?
Putting text into the command line will probably be a function of the interface you are using to run R - is it plain R, Rstudio, etc?
One possibility might be to use the clipr package and put the code into the clipboard, then prompt the user to hit their "paste" button to get it on the command line. For example this function which creates a little code string:
> writecode = function(x){
code = paste0("print(",x,")")
clipr::write_clip(code)
message("Code ready to paste")}
Use it like this:
> writecode("foo")
Code ready to paste
Then when I hit Ctrl-V to paste, I see this:
> print(foo)
I can then edit that line. Any string will do:
> writecode("bar")
Code ready to paste
[ctrl-V]
> print(bar)
Its one extra key for your user to press, but having a chunk of code appear on the command line with no prompting might be quite surprising for a user.
I spend my day reading the source code for utils auto completion. The linebuffer only contains the code for ... one line, so that can't do fully what your looking for here, but it is quite customizable. Maybe the rstudio source code for autocompletion has more answers. Here is an example to write and activate your own custom auto completer. It is not well suited for packages as any new pacakge could overwrite the settings.
Below a
#load this function into custom.completer setting to activate
rc.options("custom.completer" = function(x) {
#perhaps debug it, it will kill your rsession sometimes
#browser()
###default completion###
##activating custom deactivates anything else
#however you can run utils auto completer also like this
#rstudio auto completation is not entirely the same as utils
f = rc.getOption("custom.completer")
rc.options("custom.completer" = NULL)
#function running base auto complete.
#It will dump suggestion into mutable .CompletionEnv$comps
utils:::.completeToken() #inspect this function to learn more about completion
rc.options("custom.completer" = f)
###your custom part###
##pull this environment holding all input/output needed
.CompletionEnv = utils:::.CompletionEnv
#perhaps read the 'token'. Also 'linebuffer', 'start' & 'end' are also useful
token = .CompletionEnv$token
#generate a new completion or multiple...
your_comps = paste0(token,c("$with_tomato_sauce","$with_apple_sauce"))
#append your suggestions to the vanilla suggestions/completions
.CompletionEnv$comps = c(your_comps,.CompletionEnv$comps)
print(.CompletionEnv$comps)
#no return used
NULL
})
xxx<tab>
NB! currently rstudio IDE will backtick quote any suggestion which is not generated by them. I want to raise an issue on that someday.
Bonus info: A .DollarNames.mys3class-method can be very useful and works with both rstudio and utils out of the box e.g.
#' #export
.DollarNames.mys3class = function(x, pattern = "") {
#x is the .CompletionEnv
c("apple","tomato")
}
My R workflow is usually such that I have a file open into which I type R commands, and I’d like to execute those commands in a separately opened R shell.
The easiest way of doing this is to say source('the-file.r') inside R. However, this always reloads the whole file which may take considerable time if big amounts of data are processed. It also requires me to specify the filename again.
Ideally, I’d like to source only a specific line (or lines) from the file (I’m working on a terminal where copy&paste doesn’t work).
source doesn’t seem to offer this functionality. Is there another way of achieving this?
Here's another way with just R:
source2 <- function(file, start, end, ...) {
file.lines <- scan(file, what=character(), skip=start-1, nlines=end-start+1, sep='\n')
file.lines.collapsed <- paste(file.lines, collapse='\n')
source(textConnection(file.lines.collapsed), ...)
}
Using the right tool for the job …
As discussed in the comments, the real solution is to use an IDE that allows sourcing specific parts of a file. There are many existing solutions:
For Vim, there’s Nvim-R.
For Emacs, there’s ESS.
And of course there’s the excellent stand-alone RStudio IDE.
As a special point of note, all of the above solutions work both locally and on a server (accessed via an SSH connection, say). R can even be run on an HPC cluster — it can still communicate with the IDEs if set up properly.
… or … not.
If, for whatever reason, none of the solutions above work, here’s a small module[gist] that can do the job. I generally don’t recommend using it, though.1
#' (Re-)source parts of a file
#'
#' \code{rs} loads, parses and executes parts of a file as if entered into the R
#' console directly (but without implicit echoing).
#'
#' #param filename character string of the filename to read from. If missing,
#' use the last-read filename.
#' #param from first line to parse.
#' #param to last line to parse.
#' #return the value of the last evaluated expression in the source file.
#'
#' #details If both \code{from} and \code{to} are missing, the default is to
#' read the whole file.
rs = local({
last_file = NULL
function (filename, from, to = if (missing(from)) -1 else from) {
if (missing(filename)) filename = last_file
stopifnot(! is.null(filename))
stopifnot(is.character(filename))
force(to)
if (missing(from)) from = 1
source_lines = scan(filename, what = character(), sep = '\n',
skip = from - 1, n = to - from + 1,
encoding = 'UTF-8', quiet = TRUE)
result = withVisible(eval.parent(parse(text = source_lines)))
last_file <<- filename # Only save filename once successfully sourced.
if (result$visible) result$value else invisible(result$value)
}
})
Usage example:
# Source the whole file:
rs('some_file.r')
# Re-soure everything (same file):
rs()
# Re-source just the fifth line:
rs(from = 5)
# Re-source lines 5–10
rs(from = 5, to = 10)
# Re-source everything up until line 7:
rs(to = 7)
1 Funny story: I recently found myself on a cluster with a messed-up configuration that made it impossible to install the required software, but desperately needing to debug an R workflow due to a looming deadline. I literally had no choice but to copy and paste lines of R code into the console manually. This is a situation in which the above might come in handy. And yes, that actually happened.
My R workflow is usually such that I have a file open into which I type R commands, and I’d like to execute those commands in a separately opened R shell.
The easiest way of doing this is to say source('the-file.r') inside R. However, this always reloads the whole file which may take considerable time if big amounts of data are processed. It also requires me to specify the filename again.
Ideally, I’d like to source only a specific line (or lines) from the file (I’m working on a terminal where copy&paste doesn’t work).
source doesn’t seem to offer this functionality. Is there another way of achieving this?
Here's another way with just R:
source2 <- function(file, start, end, ...) {
file.lines <- scan(file, what=character(), skip=start-1, nlines=end-start+1, sep='\n')
file.lines.collapsed <- paste(file.lines, collapse='\n')
source(textConnection(file.lines.collapsed), ...)
}
Using the right tool for the job …
As discussed in the comments, the real solution is to use an IDE that allows sourcing specific parts of a file. There are many existing solutions:
For Vim, there’s Nvim-R.
For Emacs, there’s ESS.
And of course there’s the excellent stand-alone RStudio IDE.
As a special point of note, all of the above solutions work both locally and on a server (accessed via an SSH connection, say). R can even be run on an HPC cluster — it can still communicate with the IDEs if set up properly.
… or … not.
If, for whatever reason, none of the solutions above work, here’s a small module[gist] that can do the job. I generally don’t recommend using it, though.1
#' (Re-)source parts of a file
#'
#' \code{rs} loads, parses and executes parts of a file as if entered into the R
#' console directly (but without implicit echoing).
#'
#' #param filename character string of the filename to read from. If missing,
#' use the last-read filename.
#' #param from first line to parse.
#' #param to last line to parse.
#' #return the value of the last evaluated expression in the source file.
#'
#' #details If both \code{from} and \code{to} are missing, the default is to
#' read the whole file.
rs = local({
last_file = NULL
function (filename, from, to = if (missing(from)) -1 else from) {
if (missing(filename)) filename = last_file
stopifnot(! is.null(filename))
stopifnot(is.character(filename))
force(to)
if (missing(from)) from = 1
source_lines = scan(filename, what = character(), sep = '\n',
skip = from - 1, n = to - from + 1,
encoding = 'UTF-8', quiet = TRUE)
result = withVisible(eval.parent(parse(text = source_lines)))
last_file <<- filename # Only save filename once successfully sourced.
if (result$visible) result$value else invisible(result$value)
}
})
Usage example:
# Source the whole file:
rs('some_file.r')
# Re-soure everything (same file):
rs()
# Re-source just the fifth line:
rs(from = 5)
# Re-source lines 5–10
rs(from = 5, to = 10)
# Re-source everything up until line 7:
rs(to = 7)
1 Funny story: I recently found myself on a cluster with a messed-up configuration that made it impossible to install the required software, but desperately needing to debug an R workflow due to a looming deadline. I literally had no choice but to copy and paste lines of R code into the console manually. This is a situation in which the above might come in handy. And yes, that actually happened.
I am trying to run a script that has lots of comments to explain each table, statistical test and graph. I am using RStudio IDE as follows
source(filename, echo=T)
That ensures that the script outputs everything to the console. If I run the following sequence it will send all the output to a txt file and then switch off the output diversion
sink("filenameIwantforoutput.txt")
source(filename, echo=T)
sink()
Alas, I am finding that a lot of my comments are not being outputted. Instead I get
"...but only if we had had an exclusively b .... [TRUNCATED]".
Once before I learned where to preserve the output but that was a few months ago and now I cannot remember. Can you?
Set the max.deparse.length= argument to source. You probably need something greater than the default of 150. For example:
source(filename, echo=TRUE, max.deparse.length=1e3)
And note the last paragraph in the Details section of ?source reads:
If ‘echo’ is true and a deparsed
expression exceeds
‘max.deparse.length’, that many
characters are output followed by ‘
.... [TRUNCATED] ’.
You can make this behavior the default by overriding the source() function in your .Rprofile.
This seems like a reasonable case for overriding a function because in theory the change should only affect the screen output. We could contrive an example where this is not the case, e.g., can capture the screen output and use as a variable like capture.output(source("somefile.R")) but it seems unlikely. Changing a function in a way that the return value is changed will likely come back to bite you or whoever you share your code with (e.g., if you change a default of a function's na.rm argument).
.source_modified <- source
formals(.source_modified)$max.deparse.length <- Inf
# Use 'nms' because we don't want to do it for all because some were already
# unlocked. Thus if we lock them again, then we are changing the previous
# state.
# (https://stackoverflow.com/a/62563023/1376404)
rlang::env_binding_unlock(env = baseenv(), nms = "source")
assign(x = "source", value = .source_modified, envir = baseenv())
rlang::env_binding_lock(env = baseenv(), nms = "source")
rm(.source_modified)
An alternative is to create your own 'alias'-like function. I use the following in my .Rprofile:
s <- source
formals(s)$max.deparse.length <- Inf
formals(s)$echo <- TRUE