I'm using R 3.1.2 with RStudio 0.98 on Windows 7 32 bits.
I want to download some weather forecasts files of the GFS model, to be found on an open ftp server, e.g.:
ftp://nomads.ncdc.noaa.gov/GFS/Grid4/201412/20141221/gfs_4_20141221_0000_000.grb2
The internet connection is done through a proxy (.Renviron is properly configured), and I'm basically using the donwload.file function for this purpose.
url <- file.path("ftp://nomads.ncdc.noaa.gov/GFS/Grid4/201412/20141221/gfs_4_20141221_0000_000.grb2")
download.file(url, destfile="temp.grb2", mode="wb")
Where I get the following error message:
trying URL 'ftp://nomads.ncdc.noaa.gov/GFS/Grid4/201412/20141221/gfs_4_20141221_0000_000.grb2'
using Synchronous WinInet calls
Error in download.file(url, destfile = "temp.grb2", mode = "wb", :
cannot open URL 'ftp://nomads.ncdc.noaa.gov/GFS/Grid4/201412/20141221/gfs_4_20141221_0000_000.grb2'
In addition: Warning message:
In download.file(url, destfile = "temp.grb2", mode = "wb", :
InternetOpenUrl failed: 'Operation timed out'
This message appears exactly 30 seconds after running those lines, and no issues appear when downloading a smaller file, such as 'ftp://nomads.ncdc.noaa.gov/GFS/Grid4/201412/20141221/gfs_4_20141221_0000_000.inv', so I assume it's a matter of timeout configuration.
Setting:
options(timeout=240) doesn't seem to work.
With another computer, using R 3.0.2 with RStudio 0.98 on Windows 8 64 bits, and without using proxy connection, it works perfect.
Any suggestions?
Related
I have a problem with obtaining data from specific website - when trying to download raw website data with R 3.6.3 using following example code:
website_raw <- readLines("https://tge.pl/gaz-rdn?dateShow=09-02-2022")
The result I got is:
Error in file(con, "r") : cannot open the connection In addition: Warning message: In file(con, "r") : InternetOpenUrl failed: 'the connection with the server was reset'
readLines() method used to work fine on this website but from one week on it fails. I've tried also download.file() method: at the beginning the result was the same (error, connection reset) but after setting options(download.file.method = "libcurl"), website file starts to download but then it suddenly stops with information:
trying URL 'https://tge.pl/gaz-rdn?dateShow=09-02-2022'
Error in download.file("https://tge.pl/gaz-rdn?dateShow=09-02-2022", "test.html") :
cannot open URL 'https://tge.pl/gaz-rdn?dateShow=09-02-2022'
In addition: Warning message:
In download.file("https://tge.pl/gaz-rdn?dateShow=09-02-2022", "test.html") :
URL 'https://tge.pl/gaz-rdn?dateShow=09-02-2022': status was 'Failure when receiving data from the peer'
I've tried also disabling Use Internet Explorer library/proxy for HTTP in Rstudio Global Options but it didn't help. Another solution that I've tested was read_html() from rvest package - getting following error:
Error in open.connection(x, "rb") : Send failure: Connection was reset
Downloading data from other websites works fine though, with all considered methods.
Is there any way I can download data from this website with R?
Any kind of help or suggestion will be highly appreciated
I am trying to use RMarkdown to knit my report to a pdf. I am on my work computer (without administrative privileges and behind a firewall), and chose TinyTeX as a LaTeX distribution because I was hoping it would still work without having to involve the IT department (at my place of work it takes ages to get elevated privileges). I had no problems installing the tinytex R package, but I run into errors when using the package to install TinyTeX. The code with errors is below:
>tinytex::install_tinytex()
trying URL 'http://mirror.ctan.org/systems/texlive/tlnet/install-tl.zip'
trying URL 'http://mirror.ctan.org/systems/texlive/tlnet/install-tl.zip'
Content type 'application/zip' length 22541272 bytes (21.5 MB)
downloaded 21.5 MB
trying URL 'https://yihui.org/gh/tinytex/tools/pkgs-custom.txt'
trying URL 'https://yihui.org/gh/tinytex/tools/pkgs-custom.txt'
Content type 'text/plain; charset=utf-8' length 551 bytes
downloaded 551 bytes
trying URL 'https://yihui.org/gh/tinytex/tools/tinytex.profile'
trying URL 'https://yihui.org/gh/tinytex/tools/tinytex.profile'
Content type 'text/plain; charset=utf-8' length 295 bytes
downloaded 295 bytes
Then at this point I get a dialog box that pops up and states: "Next you may see two error dialog boxes about the missing luatex.dll, and an error message like "Use of uninitialized value in bitwise or (|)..." in the end. These messages can be ignored." When I click 'Okay', I don't see any other dialog boxes but I get the following code:
Starting to install TinyTeX to C:\Users\*****\AppData\Roaming/TinyTeX. It will take a few minutes.
Automated TeX Live installation using profile: ../tinytex.profile
cannot contact mirror.ctan.org, returning a backbone server!
Loading http://www.ctan.org/tex-archive/systems/texlive/tlnet/tlpkg/texlive.tlpdb
C:\Users\*****\AppData\Local\Temp\RtmpgHbZh0\install-tl-20200214\install-tl: TLPDB::from_file could not initialize from: http://www.ctan.org/tex-archive/systems/texlive/tlnet/tlpkg/texlive.tlpdb
C:\Users\*****\AppData\Local\Temp\RtmpgHbZh0\install-tl-20200214\install-tl: Maybe the repository setting should be changed.
C:\Users\*****\AppData\Local\Temp\RtmpgHbZh0\install-tl-20200214\install-tl: More info: https://tug.org/texlive/acquire.html
TinyTeX installed to C:\Users\*****\AppData\Roaming/TinyTeX
Please quit and reopen your R session and IDE (if you are using one, such as RStudio or Emacs) and check if tinytex:::is_tinytex() is TRUE.
Warning messages:
1: In download.file(url, output, ..., method = method) :
URL 'http://mirror.ctan.org/systems/texlive/tlnet/install-tl.zip': status was 'Couldn't connect to server'
2: In download.file(url, output, ..., method = method) :
URL 'https://yihui.org/gh/tinytex/tools/pkgs-custom.txt': status was 'Couldn't connect to server'
3: In download.file(url, output, ..., method = method) :
URL 'https://yihui.org/gh/tinytex/tools/tinytex.profile': status was 'Couldn't connect to server'
4: In file.remove("TinyTeX/install-tl.log") :
cannot remove file 'TinyTeX/install-tl.log', reason 'No such file or directory'
After this, I try tinytex:::is_tinytex() and I get FALSE.
I am using R version 3.6.2, RStudio version 1.2.1335 and I'm on a Windows 10 x64.
I'm not familiar enough with R, RMarkdown or TinyTeX to understand what is going wrong and how to fix it. Perhaps it is because of no admin rights or the firewall...Any help is appreciated!
This might be a very trivial issue with R-related Ubuntu settings, but I'm pretty new to it and have no clue to solve this issue.
I'm trying to load a file from URL on a linux Ubuntu machine into R (R-Studio) with different strategies:
load(url("http://www.yaj.ch/data/test.rda"))
read.csv(url("http://www.yaj.ch/data/test.csv"))
readstata13::read.dta13("http://www.yaj.ch/data/test.dta")
data.table::fread("http://www.yaj.ch/data/test.csv")
but they all fail with similar error messages:
Error in load(url("http://www.yaj.ch/data/test.rda")) :
cannot open the connection to 'http://www.yaj.ch/data/test.rda'
In addition: Warning message:
In load(url("http://www.yaj.ch/data/test.rda")) :
cannot open URL 'http://www.yaj.ch/data/test.rda': HTTP status was '403 Forbidden'
that of fread is somewhat different:
trying URL 'http://www.yaj.ch/data/test.csv'
Error in download.file(input, tmpFile, method = method, mode = "wb", quiet =
!showProgress) :
cannot open URL 'http://www.yaj.ch/data/test.csv'
In addition: Warning message:
In download.file(input, tmpFile, method = method, mode = "wb", quiet =
!showProgress) :
cannot open URL 'http://www.yaj.ch/data/test.csv': HTTP status was '403 Forbidden'
This is what I got when I was mobile:
Error in load(url("http://www.yaj.ch/data/test.rda")) :
cannot open the connection to 'http://www.yaj.ch/data/test.rda'
Calls: <Anonymous> ... withCallingHandlers -> withVisible -> eval -> eval -> load
Execution halted
When I Shift-click on the link in the script, the data opens in the browser, so the connection seems to be okay.
On my Windows machine everything works fine. So I guessed the issue is related to Ubuntu, as if R has connection problems or wrong access rights. But I'm not sure, since strangely enough e.g. this one opens in Ubuntu:
load(url("http://www.rdatamining.com/data/graph.rdata?attredirects=0&d=1"))
This solution wasn't helpful to me, and Google won't tell me anything either.
Is there any help?
I want to download the soft file of the GPL6480 platform in R.
I've used this command :
gset<-getGEO("GPL6480")
but I faced with this problem:
curl: (6) Could not resolve host: www.ncbi.nlm.nih.gov
File stored at:
/tmp/RtmpbHgZqQ/GPL6480.soft
Error in file(fname, "r") : cannot open the connection
In addition: Warning messages:
1: In download.file(myurl, destfile, mode = mode, quiet = TRUE, method = getOption("download.file.method.GEOquery")) :
download had nonzero exit status
2: In file(fname, "r") :
cannot open file '/tmp/RtmpbHgZqQ/GPL6480.soft': No such file or directory
I googled for this problem but I can't find the answer. acctauly i don't know that what is the problem exactly.you should know that my connection to net is ok.
I'm using R version 3.2.0 on Ubuntu 14.04.2 trusty
tnx mansoor
file<-tempfile(fileext=".csv")
download.file(url="ftp://pubftp.spp.org/Markets/DA/LMP_By_SETTLEMENT_LOC/2014/03/28/DA-LMP-SL-201403280100.csv",destfile=file,mode="wb")
This works in R proper (I'm not sure what to call it). However in RStudio it hangs for several minutes and then I get the following
trying URL 'ftp://pubftp.spp.org/Markets/RTBM/LMP_By_SETTLEMENT_LOC/2014/03/25/11/RTBM-LMP-SL-201403251015.csv'
using Synchronous WinInet calls
Error in download.file(url = "ftp://pubftp.spp.org/Markets/RTBM/LMP_By_SETTLEMENT_LOC/2014/03/25/11/RTBM-LMP-SL-201403251015.csv", :
cannot open URL 'ftp://pubftp.spp.org/Markets/RTBM/LMP_By_SETTLEMENT_LOC/2014/03/25/11/RTBM-LMP-SL-201403251015.csv'
In addition: Warning message:
In download.file(url = "ftp://pubftp.spp.org/Markets/RTBM/LMP_By_SETTLEMENT_LOC/2014/03/25/11/RTBM-LMP-SL-201403251015.csv", :
InternetOpenUrl failed: ''
It is a small file so it shouldn't time out but I really don't know what the problem is.
I found two solutions.
1) Go to Tools > Global Options > Packages, and unselect "Use Internet Explorer library/proxy for HTTP".
2) This worked for another user, but not for me: setInternet2(use=FALSE)
(https://support.rstudio.com/hc/communities/public/questions/200656136-Issue-With-RStudio-and-GEOquery)
Note: when in RGUI I entered setInternet2(use=TRUE), then tried the download, it gave the "using Synchronous WinInet calls" messages and hung; but then Windows Firewall popped up, and when I allowed RGUI through it, the download began.
I have the same problem when I download a file in RStudio when using
> source("http://www.statmethods.net/RiA/wmc.txt")
Error in file(filename, "r", encoding = encoding) :
cannot open the connection
In addition: Warning message:
In file(filename, "r", encoding = encoding) :
InternetOpenUrl failed: '安全频道支持出错'
Then I try
> options(download.file.method="libcurl", url.method="libcurl")
> source("http://www.statmethods.net/RiA/wmc.txt")
> wmc
It worked
I had a similar issue using R's download.file in a for loop in RStudio. It would download the url for the first several links and then I'd get "InternetOpenUrl failed: 'The operation timed out'" for all subsequent downloads. I tried the the suggestion by sssheridan to unselect the Internet Explorer option in R global options which did not work. I also tried setInternet2(use=T) but this is no longer available in R.
What worked for me was to remove the cache by including cacheOK = F as an argument in download.file. I think this is because I had previously hit the links that were timing out.
Go to Tools > Global Option > Packages > uncheck "Use secure download method for HTTP".