I have written a function, which takes some time to run (due to a 1000+ loop on a huge dataset in combination with forecasting model testing).
To have any idea on the status, while the function is called, I use the message command inside the for-loop in the function. The problem is that all the messages are shown in the console after the function is finished, instead of showing immediately. So it doesn't help me :)
I tried to find a solution on Stackoverflow, but didn't found one. I looked for instance on the question "showing a status message in R". All answers and example codes in that topic still give me only text in the console after a function is processed instead of immediately.
How to solve this? Is there maybe a setting in R which prevents immediate printing of message text in the console?
note: examples I tried below, which give the same results as my function; showing text after processing the function.
example1 (Joshua Ulrich):
for(i in 1:10) {
Sys.sleep(0.2)
# Dirk says using cat() like this is naughty ;-)
#cat(i,"\r")
# So you can use message() like this, thanks to Sharpie's
# comment to use appendLF=FALSE.
message(i,"\r",appendLF=FALSE)
flush.console()
}
example2 (Tyler):
test.message <- function() {
for (i in 1:9){
cat(i)
Sys.sleep(1)
cat("\b")
}
}
edit: the first example does work ('flush console' was the problem)...but when I tested it, I commented out flush console for some reason :S
test.message <- function() {
for (i in 1:9){
cat(paste(as.character(i),'\n'))
flush.console()
Sys.sleep(1)
}
}
which is similar to the recommendation by fotNelton.
Edit: ttmaccer is most likely right. I've just tested on a Ubuntu server and the code works without flushing the console.
I seem to think this maybe a windows specific problem. On linux or running R in a cygwin shell the flush.console() may not be needed.
You may be interested in using one of the progress bar functions (winProgressBar, tkProgressBar, or txtProgressBar). The win version only works on windows, but the win and tk versions have the advantage that they do not clutter your output, but rather open another small window and display the progress there.
The progress through a loop can be shown with the progress bar, but other detailed information can be updated and shown with the label argument.
Related
Whenever I get an error in my code, I have to start looking on which line it is. Having only 500 lines or so, this is doable, but I can imagine the efficiency boost I would get if there'd be a keyboard shortcut that automatically moves my cursor to the error.
Anyone that found such a shortcut?
I am working in R script.
You could set error option to browser or recover (as suggested by #Ben Bolker).
With browser:
options(error=browser)
test <- function() {
1+1
stop('custom error')
}
#some code
cat('OK \n')
test()
cat('After error \n')
With recover:
And to get back to default:
options(error=NULL)
Have you tried R Studio? It has "Debug" tool
https://support.rstudio.com/hc/en-us/articles/200713843?version=1.3.1093&mode=desktop
Also in console you can see the warning with the wrong line. Sorry if I did not understand your question.
I am quite new to R and am trying to access some information on the internet, but am having problems with connections that don't seem to be closing. I would really appreciate it if someone here could give me some advice...
Originally I wanted to use the WebChem package, which theoretically delivers everything I want, but when some of the output data is missing from the webpage, WebChem doesn't return any data from that page. To get around this, I have taken most of the code from the package but altered it slightly to fit my needs. This worked fine, for about the first 150 usages, but now, although I have changed nothing, when I use the command read_html, I get the warning message " closing unused connection 4 (http:....." Although this is only a warning message, read_html doesn't return anything after this warning is generated.
I have written a simplified code, given below. This has the same problem
Closing R completely (or even rebooting my PC) doesn't seem to make a difference - the warning message now appears the second time I use the code. I can run the querys one at a time, outside of the loop with no problems, but as soon as I try to use the loop, the error occurs again on the 2nd iteration.
I have tried to vectorise the code, and again it returned the same error message.
I tried showConnections(all=TRUE), but only got connections 0-2 for stdin, stdout, stderr.
I have tried searching for ways to close the html connection, but I can't define the url as a con, and close(qurl) and close(ttt) also don't work. (Return errors of no applicable method for 'close' applied to an object of class "character and no applicable method for 'close' applied to an object of class "c('xml_document', 'xml_node')", repectively)
Does anybody know a way to close these connections so that they don't break my routine? Any suggestions would be very welcome. Thanks!
PS: I am using R version 3.3.0 with RStudio Version 0.99.902.
CasNrs <- c("630-08-0","463-49-0","194-59-2","86-74-8","148-79-8")
tit = character()
for (i in 1:length(CasNrs)){
CurrCasNr <- as.character(CasNrs[i])
baseurl <- 'http://chem.sis.nlm.nih.gov/chemidplus/rn/'
qurl <- paste0(baseurl, CurrCasNr, '?DT_START_ROW=0&DT_ROWS_PER_PAGE=50')
ttt <- try(read_html(qurl), silent = TRUE)
tit[i] <- xml_text(xml_find_all(ttt, "//head/title"))
}
After researching the topic I came up with the following solution:
url <- "https://website_example.com"
url = url(url, "rb")
html <- read_html(url)
close(url)
# + Whatever you wanna do with the html since it's already saved!
I haven't found a good answer for this problem. The best work-around that I came up with is to include the function below, with Secs = 3 or 4. I still don't know why the problem occurs or how to stop it without building in a large delay.
CatchupPause <- function(Secs){
Sys.sleep(Secs) #pause to let connection work
closeAllConnections()
gc()
}
I found this post as I was running into the same problems when I tried to scrape multiple datasets in the same script. The script would get progressively slower and I feel it was due to the connections. Here is a simple loop that closes out all of the connections.
for (i in seq_along(df$URLs)){function(i)
closeAllConnections(i)
}
I am trying to understand the way the YourCast R package works and make it work with my data.
For example, if a function produces errors, I
get the source code of that function using YourCast:::bad.fn
add outputs of critical
values at critical stages
use reassignInPackage(name="original.fn", package="YourCast", value="my.fn")
Once I find the cause of the error, I fix it in the function and reassign it in the package.
However, for some strange reason this does not work for non-hidden functions.
For example:
install.packages("YourCast")
Library(YourCast)
YourCast:::check.depvar
This will print the hidden function check.depvar. One line if (all(ix == 1:3)) will produce an error message if any of the x is missing.
Thus, I change the whole function to the following and replace the original formula:
mzuba.check.depvar <- function(formula)
{
return (grepl("log[(]",as.character(formula)[2]))
}
reassignInPackage("check.depvar",
pkgName="YourCast",
mzuba.check.depvar)
rm(mzuba.check.depvar)
Now YourCast:::check.depvar will print my version of that function, and everything is fine.
However
YourCast::yourcast or YourCast:::yourcast or simply yourcast will print the non-hidden function yourcast. Suppose I want to change that function as well.
reassignInPackage(name="yourcast",
pkgName="YourCast",
value=test)
Now, YourCast::yourcast and YourCast:::yourcast will print the new, modified version but yourcast still gives the old version!
That might not a problem if I could simply call YourCast::yourcast instead of yourcast, but that produces some kind of error that I can't trace back because suddenly R-Studio does not print error messages at all anymore!, although it still does something if it is capable to:
> Uagh! do something!
> 1 + 1
[1] 2
> Why no error msg?
>
Restarting the R-session will solve the error-msg problem, though.
So my question is: How do I reassign non-hidden functions in packages?
Furthermore (this would faciliate testing a lot), is there a way to make all hidden functions available without using the ::: operator? I.e., How to export all functions from a package?
I know that the sink() function can be used to divert R output into a file, e.g.
sink('sink-closing.txt')
cat('Hello world!')
sink()
Is there a simple command to close all outstanding sinks?
Below, I elaborate on my question.
Suppose that my R-script opens a sink() in an R-script, but there is an error in the R-script which occurs before the script closes the sink(). I may run the R-script multiple times, trying to fix the error. Finally, I want to close all the sinks and print to the console. How do I do so?
Finally, in the interest of concreteness, I provide a MWE to illustrate the problem I face.
First, I write an R-script sink-closing.R which has an error in it.
sink('sink-closing.txt')
foo <- function() {
cat(sprintf('Hello world! My name is %s\n',
a.variable.that.does.not.exist))
}
foo()
sink()
Next, I source the R-script multiple times, say 3 times by mistake as I try to find and fix the bug.
> source('~/Dropbox/cookbook/r-cookbook/sink-closing.R')
Error in sprintf("Hello world! My name is %s\n", a.variable.that.does.not.exist) :
object 'a.variable.that.does.not.exist' not found
Now, suppose that I am debugging the R-script and want to print to the console. I can call sink() multiple times to close the earlier sinks. If I call it 3 times, then I can finally print to the console as before. But how do I know how many sinks I need to close?
closeAllConnections() # .........................
I'm getting upvotes for this as time goes along but Simon.S.A and others are better.
You can use sink.number() to tell you how many diversions are already set and then call sink that many times. Putting it into a function you could have this
sink.reset <- function(){
for(i in seq_len(sink.number())){
sink(NULL)
}
}
Based on #mnel's comment:
sinkall <- function() {
i <- sink.number()
while (i > 0) {
sink()
i <- i - 1
}
}
Should close all open sinks.
You may also encounter this problem when dealing with devices and plots, where the number of open devices isn't reported anywhere. For a more general case you could use this:
stopWhenError <- function(FUN) {
tryCatch({
while(TRUE) {
FUN()
}
}, warning = function(w) {
print("All finished!")
}, error = function(e) {
print("All finished!")
})
}
stopWhenError(sink) # for sink.
stopWhenError(dev.off) # close all open plotting devices.
EDIT:
sink throws a warning not an error so I've modified the code so that it won't run forever, whoops!
The most common time I experience this is when an error occurs preventing a sink from closing. For example, the following will leave an open sink after execution.
sink("output.txt")
my_function_that_will_error()
sink()
This can be avoided using on.exit(sink()). This will close the sink "when the current function exits (either naturally or as the result of an error)" (documentation here).
But you do have to change the order:
sink("output.txt")
on.exit(sink())
my_function_that_might_error()
So we create the sink, tell R to close it when it exits, and then execute the code that might error. This will close the sink regardless of whether the code errors or not.
I am wondering if there is a function to clear the console in R and, in particular, RStudio I am looking for a function that I can type into the console, and not a keyboard shortcut.
Someone has already provided such a function in this StackExchange post from 2010. Unfortunately, this depends on the RCom package and will not run on Mac OS X.
cat("\014")
is the code to send CTRL+L to the console, and therefore will clear the screen.
Far better than just sending a whole lot of returns.
If you are using the default R console, the key combination Option + Command + L will clear the console.
You may define the following function
clc <- function() cat(rep("\n", 50))
which you can then call as clc().
shell("cls") if on Windows,
shell("clear") if on Linux or Mac.
(shell() passes a command (or any string) to the host terminal.)
cat("\f") may be easier to remember than cat("\014").
It works fine for me on Windows 10.
In Ubuntu-Gnome, simply pressing CTRL+L should clear the screen.
This also seems to also work well in Windows 10 and 7 and Mac OS X Sierra.
Here's a function:
clear <- function() cat(c("\033[2J","\033[0;0H"))
then you can simply call it, as you call any other R function, clear().
If you prefer to simply type clear (instead of having to type clear(), i.e. with the parentheses), then you can do
clear_fun <- function() cat(c("\033[2J","\033[0;0H"));
makeActiveBinding("clear", clear_fun, baseenv())
I developed an R package that will do this, borrowing from the suggestions above. The package is called called mise, as in "mise en place." You can install and run it using
install.packages("mise")
library(mise)
mise()
Note that mise() also deletes all variables and functions and closes all figures by default. To just clear the console, use mise(vars = FALSE, figs = FALSE).
In linux use system("clear") to clear the screen.
If you are using the default R console CTRL + L
RStudio - CTRL + L
cat("\014") . This will work. no worries
You can combine the following two commands
cat("\014");
cat(rep("\n", 50))
Another option for RStudio is rstudioapi::sendToConsole("\014"). This will work even if output is diverted.
sink("out.txt")
cat("\014") # Console not cleared
rstudioapi::sendToConsole("\014") # Console cleared
sink()
I know this question is very old, but I found myself visiting it many times looking for a totally different answer:
n = 20
for (i in 0:n) {
cat(100*i/n, "% \r")
flush.console()
Sys.sleep(0.01) #do something slow
}
flush.console() will kind of "clear the console in r and studio", maybe not in OP's terms but still.
This code will act like a progress bar in the console. For each iteration, the percentage is incremented and then erased on the next iteration.
Note that this won't work without \r or with an \n, for some reason.