OS- Windows 7
R version 3.0.3
What I typed on R console :
if(!file.exists("data")){dir.create("data")}
fileUrl <- "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.xlsx?accessType=DOWNLOAD"
download.file(fileUrl,destfile="./data/cameras.xlsx",method="curl")
dateDownloaded <- date()
What I got on R console
Warning messages:
1: running command 'curl "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.xlsx?accessType=DOWNLOAD" -o "./data/cameras.xlsx"' had status 127
2: In download.file(fileUrl, destfile = "./data/cameras.xlsx", method = "curl") :
download had nonzero exit status
How do I make it right ?
Your code works for me. You probably don't have cURL installed on your system. See ?download.file:
For methods "wget", "curl" and "lynx" a system call is made to the tool given by method, and the respective program must be installed on your system and be in the search path for executables.
Install cURL and make sure it is in your path.
After removing the method="curl" parameter, it works well on Windows.
Related
I am writing an R package and one of its functions download and unzips a file from a link (it is not exported to the user, though):
download_f <- function(download_dir) {
utils::download.file(
url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php",
destfile = file.path(download_dir, "fines.rar"),
mode = 'wb',
method = 'libcurl'
)
utils::unzip(
zipfile = file.path(download_dir, "fines.rar"),
exdir = file.path(download_dir)
)
}
This function works fine with me when I run it within some other function to compile an example in a vignette.
However, with R CMD check in github action, it fails consistently on ubuntu 16.04, release and devel. It [says][1]:
Error: Error: processing vignette 'IBAMA.Rmd' failed with diagnostics:
cannot open URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php'
--- failed re-building ‘IBAMA.Rmd’
SUMMARY: processing the following file failed:
‘IBAMA.Rmd’
Error: Error: Vignette re-building failed.
Execution halted
Error: Error in proc$get_built_file() : Build process failed
Calls: <Anonymous> ... build_package -> with_envvar -> force -> <Anonymous>
Execution halted
Error: Process completed with exit code 1.
When I run devtools::check() it never finishes running it, staying in "creating vignettes" forever. I don't know if these problems are related though because there are other vignettes on the package.
I pass the R CMD checks with mac os and windows. I've tried switching the "mode" and "method" arguments on utils::download.file, but to no avail.
Any suggestions?
[1]: https://github.com/datazoompuc/datazoom.amazonia/pull/16/checks?check_run_id=2026865974
The download fails because libcurl tries to verify the webservers certificate, but can't.
I can reproduce this on my system:
trying URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php'
Error in utils::download.file(url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php", :
cannot open URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php'
In addition: Warning message:
In utils::download.file(url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php", :
URL 'https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php': status was 'SSL peer certificate or SSH remote key was not OK'
The server does not allow you to download from http but redirects to https, so the only thing to do now is to tell libcurl to not check the certificate and accept what it is getting.
You can do this by specifying the argument -k to curl
download_f <- function(download_dir) {
utils::download.file(
url = "https://servicos.ibama.gov.br/ctf/publico/areasembargadas/downloadListaAreasEmbargadas.php",
destfile = file.path(download_dir, "fines.rar"),
mode = 'wb',
method = 'curl',
extra = '-k'
)
utils::unzip(
zipfile = file.path(download_dir, "fines.rar"),
exdir = file.path(download_dir)
)
}
This also produces some download progress bar, you can silence this by setting extra to -k -s
This now opens you up to a Machine In The Middle Attack. (You possibly already are attacked this way, there is no way to check without verifying the current certificate with someone you know at the other side)
So you could implement an extra check, e.g. check the sha256sum of the downloaded file and see if it matches what you expect to receive before proceeding.
myfile <- system.file("fines.rar")
hash <- sha256(file(myfile))
OS: Win 7 64 bit
RStudio Version 1.1.463
As per Getting and Cleaning Data course, I attempted to download a csv file with method = curl:
fileUrl <- "https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD"
download.file(fileUrl, destfile = "./cameras.csv", method = "curl")
Error in download.file(fileUrl, destfile = "./cameras.csv", method =
"curl") : 'curl' call had nonzero exit status
However, method = libcurl resulted a successful download:
download.file(fileUrl, destfile = "./cameras.csv", method = "libcurl")
trying URL
'https://data.baltimorecity.gov/api/views/dz54-2aru/rows.csv?accessType=DOWNLOAD'
downloaded 9443 bytes
changing from *http***s** to http produced exactly the same results for curl and libcurl, respectively.
Is there anyway to make this download work via method = curl as per the course?
Thanks
As you can see from ?download.file:
For methods "wget" and "curl" a system call is made to the tool given
by method, and the respective program must be installed on your system
and be in the search path for executables. They will block all other
activity on the R process until they complete: this may make a GUI
unresponsive.
Therefore, you should install curlfirst.
See this How do I install and use curl on Windows? to learn how.
Best!
I believe there were a few issues here:
Followed the steps in the link quoted by #JonnyCrunch
a) Reinstalled Git for windows;
b) added C:\Program Files\Git\mingw64\bin\ to the 'PATH' variable;
c) Disabled Use Internet Explorer library/proxy for HTTP in RStudio in: Tools > Options > Packages
d) Attempted steps in 'e)' below and added data.baltimorecity.gov
website to exclusions as per Kaspersky anti-virus' prompt;
e) Then in RStudio:
options(download.file.method = "curl")
download.file(fileUrl, destfile="./data/cameras.csv")
Success!
Thank you
In this tutorial, there is a command pymol.dccm(cij, pdb, type="launch"). But I was told
> pymol.dccm(cij, pdb, type="launch")
Error in pymol.dccm(cij, pdb, type = "launch") :
Launching external program failed
make sure 'C:/python27/PyMOL/pymol.exe' is in your search path
In addition: Warning message:
running command 'C:/python27/PyMOL/pymol.exe -cq' had status 127
I already have pymol installed on my PC. Can I ask how to add another search path to R?
Now I think pymol is a sub-package in bio3d. But I already installed bio3d and other commands can work (e.g. pdb <- read.pdb()). But why the pymol command could not work?
I tried
> .libPaths("path/to/pymol2/")
> .libPaths("path/to/pymol2/PyMOL")
> .libPaths("path/to/pymol2/PyMOL/PyMOLWin.exe")
> pymol.dccm(cij, pdb, type="launch")
Error in pymol.dccm(cij, pdb, type = "launch") :
Launching external program failed
make sure 'C:/python27/PyMOL/pymol.exe' is in your search path
In addition: Warning message:
running command 'C:/python27/PyMOL/pymol.exe -cq' had status 127
> PyMOLWin.dccm(cij, pdb, type="launch")
Error: could not find function "PyMOLWin.dccm"
So the .libPaths did not return error. But pymol.dccm and PyMOLWin.dccm did not work.
I also tried to install pymol package in R
> install.packages("pymol")
Warning in install.packages :
package ‘pymol’ is not available (for R version 3.2.2)
There's a mistake in the tutorial command itself. The correct syntax for dccm is
pymol(cij, pdb, type="launch",exefile="C:/Program Files/pymol")
where exefile = file path to the ‘PYMOL’ program on your system (i.e. how is ‘PYMOL’ invoked). If NULL, use OS-dependent default path to the program.
Try the following code, it worked perfectly for me:
pymol(cm, pdb.open, type="launch", exefile="%userprofile%/PyMOL/PyMOLWin.exe")
.libPaths("path/to/package/library") probably does what you need.
.libPaths gets/sets the library trees within which packages are looked for.
Set the path to the parent directory of the directory with the package name rather than the package directory itself.
I recently updated R to the latest release: 3.4.0. R is installed on a network location H:/. Now something weird is happening when I try to install a local binary package:
filename <- paste0("R:/path/independeR_", versions, ".zip")
install.packages(filename,
repos = NULL, type = "source",
lib = gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1]))
Both H:/ and R:/ are network locations. In .libPaths() the default location is in the H:/ location, but it shows with the entire network adress. In the call to install.packages I substituted this.
The code above fails, with the following output:
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as an internal or external command,
operable program or batch file.
Warning in install.packages :
running command '"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"' had status 1
Warning in install.packages :
installation of package ‘R:/path/independeR_0.1.8.zip’ had non-zero exit status
There is two things that surprise me here. The directorynames are all jumbled up (DOCU~UZL instead of Documents etc), but for some reason Command promt seems fine with that. The more interesting thing is the following:
When I try to put the command "//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/bin/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip" directly into Command Prompt, the output is very similar:
C:\Users\jdubbeldam>"//networkpath/home/JDUB~PN6/DOCU~UZL/R/R-34~TN4.0/b
in/x64/R" CMD INSTALL -l "H:\Documents\R\R-3.4.0\library" "R:/path/independeR_0.1.8.zip"
'\\networkpath\home\JDUB~PN6\DOCU~UZL\R\R-34~TN4.0' is not recognized as
an internal or external command,
operable program or batch file.
For some reason CMD seems to cut off the path to the command halfway through. I guess that this is because the command is too long. When I try the same command, but with H:/Documents/R/R-3.4.0/bin/x64/R, the installation goes just fine.
I would like to be able to automatically install this package from a script, so I would like a solution to this problem from within R. Is there a way to get R to use the shorter H:/Documents/R/R-3.4.0/bin/x64/R?
I was having problems with updating packages, while searching I found that there is a reported bug affecting getting the timestamps from files in Windows 10:
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17261
Starting in R 3.4.0, file.info sometimes returns for mtime,
atime and ctime for directories. It seems to have something to do
with sharing. This affects functions that use file.mtime like
update.packages.
Reproduce:
Try file.info() with a random directory. If it returns a legitimate
file time, open a windows explorer window and navigate to the
directory, then run the call again and it will return s.
In some cases it returns s even if the directory is not open (or
in the path of an open explorer window), but this is not consistent.
With a comment of Tomas Kalibera:
Thank you for the report, this is a known bug that has been fixed
recently in R-devel
I found that when I closed the file explorer windows the timestamps worked correctly and didn't show up as NA's.
This may be a lead for why your code isn't working.
I managed to make the following work. However, I think it is extremely ugly, and would still like to see if it is possible to do it someway else.
filename <- paste0("R:/path/independeR_", versions, ".zip")
cmd <- file.path(gsub("//networkpath/home/[[:alnum:]]*/", "H:/",
gsub("//networkpath/home/[[:alnum:]]*~[[:alnum:]]*/",
"H:/", R.home())), "bin/x64/R")
libname <- gsub("\\\\\\\\networkpath/home/[[:alpha:]]*/",
"H:/", .libPaths()[1])
call <- paste(paste0('"', cmd, '"'),
"CMD",
"INSTALL",
"-l",
paste0('"', libname, '"'),
"--no-lock",
paste0('"', filename, '"'),
sep = " ")
system(call)
I am new to the R programming language and am having basic issues with it. I want to untar a file, but it has not been able to work for me.
Here is the code that I enter:
untar("CD_data.tar", exdir="data")
It then returns the following error message:
/bin/sh: /usr/bin/gnutar: No such file or directory
Warning message:
In untar("CD_data.tar", exdir = "data") :
‘/usr/bin/gnutar -xf 'CD_data.tar' -C 'data'’ returned error code 127
Please help! Thanks!
R on OS X 10.9 (Mavericks) seems to set a wrong TAR environment variable.
You can fix this by adding the following to your .Rprofile (or executing it manually):
Sys.setenv(TAR = '/usr/bin/tar')
Alternatively, you can provide the tar path as an argument when calling untar.
My 2 cents is that you are using a mac and have not installed tar. You are getting value 127 because the command is not found within your $PATH and it's not a built-in command (which is usually the case if you were in unix...
In other words you need to install tar.
Or run it in linux.