I have the following code:
urls <- c(
"xxxxx",
"http://stat.ethz.ch/R-manual/R-devel/library/base/html/connections.html",
"http://en.wikipedia.org/wiki/Xz"
)
readUrl <- function(url) {
out <- tryCatch(
readLines(con=url, warn=FALSE),
error=function(e) {
message(paste("URL does not seem to exist:", url))
message(e)
return(NA)
},
finally=message(paste("Processed URL:", url))
)
return(out)
}
y <- lapply(urls, readUrl)
When I run it, I get:
URL does not seem to exist: xxxxx
cannot open the connectionProcessed URL: xxxxx
Processed URL: http://stat.ethz.ch/R-manual/R-devel/library/base/html/connections.html
Processed URL: http://en.wikipedia.org/wiki/Xz
Warning message:
In file(con, "r") : cannot open file 'xxxxx': No such file or directory
but I expected:
URL does not seem to exist: xxxxx
cannot open the connectionProcessed URL: xxxxx
Warning message:
In file(con, "r") : cannot open file 'xxxxx': No such file or directory
Processed URL: http://stat.ethz.ch/R-manual/R-devel/library/base/html/connections.html
Processed URL: http://en.wikipedia.org/wiki/Xz
So, why do I get:
Warning message:
In file(con, "r") : cannot open file 'xxxxx': No such file or directory
The call to readLines issues warnings. You can suppress warnings with suppressWarnings(). Try this:
readUrl <- function(url) {
out <- tryCatch(
suppressWarnings(readLines(con=url, warn=FALSE)),
error=function(e) {
message(paste("URL does not seem to exist:", url))
message(e)
return(NA)
},
finally=message(paste("Processed URL:", url))
)
return(out)
}
y <- lapply(urls, readUrl)
The screen output:
URL does not seem to exist: xxxxx
cannot open the connectionProcessed URL: xxxxx
Processed URL: http://stat.ethz.ch/R-manual/R-devel/library/base/html/connections.html
Processed URL: http://en.wikipedia.org/wiki/Xz
Alternatively, you can use options(warn=1) to display warnings as they occur. Try this:
readUrl <- function(url) {
op <- options("warn")
on.exit(options(op))
options(warn=1)
out <- tryCatch(
readLines(con=url, warn=FALSE),
error=function(e) {
message(paste("URL does not seem to exist:", url))
message(e)
return(NA)
},
finally=message(paste("Processed URL:", url))
)
return(out)
}
y <- lapply(urls, readUrl)
The output:
Warning in file(con, "r") :
cannot open file 'xxxxx': No such file or directory
URL does not seem to exist: xxxxx
cannot open the connectionProcessed URL: xxxxx
Processed URL: http://stat.ethz.ch/R-manual/R-devel/library/base/html/connections.html
Processed URL: http://en.wikipedia.org/wiki/Xz
To display all warnings, simply run this:
options(warn=1)
Here's how it works:
Set warn = -1 (or any negative integer), to ignore all warnings
Set warn = 0 (the default), to display 10 or fewer warnings
Set warn = 1 (as above), to display all warnings
Set warn = 2 (or any positive integer greater than 2) to convert all warnings into errors
Further explanation available by running ?options:
integer value to set the handling of warning messages. If warn is negative all warnings are ignored. If warn is zero (the default) warnings are stored until the top–level function returns. If 10 or fewer warnings were signalled they will be printed otherwise a message saying how many were signalled. An object called last.warning is created and can be printed through the function warnings. If warn is one, warnings are printed as they occur. If warn is two (or larger, coercible to integer), all warnings are turned into errors.
Related
I have been using readRDS(gzcon(url("my dropbox links")))for a long time to load my saved .rds field from Dropbox without any issues. But ever wonder why readRDS("my dropbox links") does not do the same thing. I got an error like
Error in gzfile(file, "rb") : cannot open the connection
It seems like a fair, simple question, but I couldn't figure it myself. Many thanks in advance~
another alternative would be to first download the file and then read the RDS:
download.file("https:www.somesite.com/somefile.rds",
"data.rds",
method = "curl")
The readRDS - function checks if the argument is a connection and but doesn't create URLS itself:
> readRDS
function (file, refhook = NULL)
{
if (is.character(file)) {
con <- gzfile(file, "rb")
on.exit(close(con))
}
else if (inherits(file, "connection"))
con <- if (inherits(file, "url"))
gzcon(file)
else file
else stop("bad 'file' argument")
.Internal(unserializeFromConn(con, refhook))
}
<bytecode: 0x5648012c7c50>
<environment: namespace:base>
therefore the url - function is needed.
> link<-"https://www.google.com"
> inherits(link,"connection")
[1] FALSE
> link2<-url("https://www.google.com")
> inherits(link2,"connection")
[1] TRUE
i'm trying to run a API request for a number of parameters with the lapply function in R.
However, when i run this function, i get the error " Error in file(con, "r") : cannot open the connection"
Google suggests using setInternet2(TRUE) to fix this issue, however, i get the error: Error: 'setInternet2' is defunct.
See help("Defunct"
localisedDestinationNameForGivenLang <- function (LocationId) {
gaiaURL <- paste0("https://URL/",LocationId, "?geoVersion=rwg&lcid=", "1036",
"&cid=geo&apk=explorer")
print(LocationId)
localisation <- fromJSON(gaiaURL)
}
lapply(uniqueLocationId, localisedDestinationNameForGivenLang)
Can someone suggest a fix please?
Here's a sample of how you could identify which sites are throwing errors while still getting response from the ones that don't:
urls = c("http://citibikenyc.com/stations/test", "http://citibikenyc.com/stations/json")
grab_data <- function(url) {
out <- tryCatch(
{fromJSON(url)},
error=function(x) {
message(paste(url, x))
error_msg = paste(url, "threw an error")
return(error_msg)
})
return(out)
}
result <- lapply(urls, grab_data)
result will be a list that contains API response for urls that work, and error msg for those that don't.
I want to read a zip file from the web, my code is as followed
temp<-tempfile()
download<-download.file("http://depts.washington.edu/control/LARRY/TE/IDVs/idv1.zip",temp)
data<-read.table(unz(temp,"r.dat"),head=FALSE)
unlink(temp)
But it shows an error
Error in open.connection(file, "rt") : cannot open the connection
In addition: Warning message:
In open.connection(file, "rt") :
cannot locate file 'r.dat' in zip file 'C:\Users\CHENGF~2\AppData\Local\Temp\RtmpgtJShr\file361c5d0a55eb'
I don't know why it can't read the data, hope someone can help me!
this works for all the idv files except the idv1 which appears to be corrupted. You would need to unzip idv1.zip using another tool and read it in...
readrdat <- function(n) {
fname <- paste0("idv",n)
zipname <- paste0(fname,".zip")
weblink <- paste0("http://depts.washington.edu/control/LARRY/TE/IDVs/",zipname)
download.file(weblink,zipname)
data <- read.table(unz(zipname,paste0(fname,"/r.dat")),header=FALSE)
unlink(zipname)
return(data)
} #readrdat
lsdata <- lapply(1:15, function(n) {
tryCatch(readrdat(n), error=function(e) NULL)
})
lapply(lsdata, is.null)
I heard about exception handling first time in python two days ago and consequently I want to apply here in R. I had a look at a number of questions post either here in stack overflow or some other online Q&As but I am still really confused in using it.
I would really appreciate if someone can answer it with this simple example so later on I can apply it to my questions.
For example I have 3 data files with file names shown below; and the first file is a 0 bytes empty file. What I can do to continue run the loop for all files and the number extracted from the empty file can be expressed as NA?
> output_names_hdf5_list[1:5]
[1] "simulation-results fL=0.1,fks=1,fno=0.1,fnc=0.1,fr=0.1,fs=0.1.hdf5"
[2] "simulation-results fL=0.1,fks=1,fno=0.1,fnc=0.1,fr=0.1,fs=1.05.hdf5"
[3] "simulation-results fL=0.1,fks=1,fno=0.1,fnc=0.1,fr=0.1,fs=2.hdf5"
for (i in 1:5){
channelflow_outlet[,i]=h5read(paste(outputdir, output_names_hdf5_list[i], sep=""),"Channel")$Qc_out[460,][2:100]
}
With try function I can manage to run the program without stuck in an error message but when I replace the argunment with channelflow_outlet[,i]= h5read(....) inside try function, it just returns error.
for (i in 1:5){
try(h5read(paste(outputdir, output_names_hdf5_list[i], sep=""),"Channel")$Qc_out[460,][2:100])
}
Without error handling, it will have a error message like this.
> h5read(paste(outputdir, output_names_hdf5_list[1], sep=""),"Channel")$Qc_out[460,][2:100]
HDF5: unable to open file
Error in h5checktypeOrOpenLoc(file, readonly = TRUE) :
Error in h5checktypeOrOpenLoc(). File 'D:/Data/Mleonard/pytopkapi.staged.makefile/RunModel/Output/3x6-729-04072014/simulation-results fL=0.1,fks=1,fno=0.1,fnc=0.1,fr=0.1,fs=0.1.hdf5' is not a valid HDF5 file.
>
I hope my code helps. For those messages in the code, you can delete them if you want. They are here purely to help you see where it shows warning or error.
setwd("D:/Dropbox/Test/"); outputdir = "D:/Dropbox/Test/"
output_names_hdf5_list=c("simulation-results fL=0.1,fks=1,fno=1.05,fnc=1.05,fr=1.05,fs=1.05.hdf5",
"simulation-results fL=0.1,fks=1,fno=1.05,fnc=2,fr=1.05,fs=1.05.hdf5",
"simulation-results fL=0.1,fks=1,fno=2,fnc=1.05,fr=0.1,fs=1.05.hdf5",
"simulation-results fL=0.1,fks=1,fno=2,fnc=1.05,fr=2,fs=2.hdf5",
"simulation-results fL=0.5,fks=1,fno=2,fnc=2,fr=0.1,fs=1.05.hdf5")
channelflow_outlet = matrix(NA, nrow=100, ncol=5)
hdf5_list_reading_tool= function(output_names_hdf5_list) {
out = tryCatch(
{
message("This is the 'try' part")
h5read(paste(outputdir, output_names_hdf5_list, sep=""),"Channel")$Qc_out[460,][2:100]
},
error=function(cond) {
message("Here's the original error message:")
message(cond)
return(rep(NA,100))
},
warning=function(cond) {
message("Here's the original warning message:")
message(cond)
return(rep(NA,100))
},
finally={
message(paste("Processed URL:", output_names_hdf5_list))
message("Some other message at the end")
}
)
return(out)
}
channelflow_outlet=sapply(output_names_hdf5_list, hdf5_list_reading_tool)
Problem
It seems that within a function, when you evaluate an expression that yields an error more than once, you get the warning restarting interrupted promise evaluation. For instance:
foo <- function() stop("Foo error")
bar <- function(x) {
try(x)
x
}
bar(foo())
yields
Error in foo() : Foo error
Error in foo() : Foo error
In addition: Warning message:
In bar(foo()) : restarting interrupted promise evaluation
How to avoid this warning and deal with it properly?
Background
Especially with operations like writing to a database, you might encounter locking errors that require you to retry your operation a few times. Hence I'm creating a wrapper around tryCatch that re-evaluates an expression up to n times until successful:
tryAgain <- function(expr, n = 3) {
success <- T
for (i in 1:n) {
res <- tryCatch(expr,
error = function(e) {
print(sprintf("Log error to file: %s", conditionMessage(e)))
success <<- F
e
}
)
if (success) break
}
res
}
However, I'm getting loads of restarting interrupted promise evaluation messages:
> tryAgain(foo())
[1] "Log error to file: Foo error"
[1] "Log error to file: Foo error"
[1] "Log error to file: Foo error"
<simpleError in foo(): Foo error>
Warning messages:
1: In doTryCatch(return(expr), name, parentenv, handler) :
restarting interrupted promise evaluation
2: In doTryCatch(return(expr), name, parentenv, handler) :
restarting interrupted promise evaluation
Ideally I want to avoid these messages altogether rather than just muffle them, since I might also want to handle genuine warnings coming from expr.
You can also try this without silent=TRUE if you want each error message to show. In neither case will you get the message about promises:
foo <- function() stop("Foo error")
bar <- function(x) {
try(eval.parent(substitute(x)), silent = TRUE)
x
}
bar(foo())