I try to download an excel file using download.file().
If I go directly to the link using the browser, I can download the file without problems.
However, using download.file does only download a broken file with Excel error: "The file you are trying to open is in a different format than specified by the file extension."
Here is my code:
url <- "http://obieebr.banrep.gov.co/analytics/saw.dll?Download&Format=excel2007&Extension=.xlsx&BypassCache=true&path=%2Fshared%2fSeries%20Estad%c3%adsticas%2F1.%20Tasa%20Interbancaria%20%28TIB%29%2F1.1.TIB_Serie%20hist%C3%B3rica%20IQY&lang=es&NQUser=publico&NQPassword=publico&SyncOperation=1"
download.file(url, destfile = paste0(base_dir, "test.xls"), mode = "wb", method="libcurl")
Any ideas how to download this file?
Many thanks for your help!
Try this, it works for me:
download.file(url,destfile = "./second.xlsx",mode = "wb")
The file you are trying to download is simply not an excel file. Actually what you obtain is an html file (try to change the file extension to '.html', then open in your browser). So your code is not the problem.
Related
Am trying to download a file from a url. If I manually download and unzip it works fine, however, using download.file the zip is corrupt. I wonder if it is something to do with manually specifying the file name. Why is this neccesary when using the download.file function? Is it not possible to simply specify a url and folder and it download the file as it is named on the server?
##using R
##download url
url <- 'https://www.eea.europa.eu/data-and-maps/data/eea-reference-grids-2/gis-files/denmark-shapefile/at_download/file'
download.file(url, destfile ='Denmark_shapefile.zip')
unzip(zipfile = 'Denmark_shapefile.zip', exdir = '.')
Unzip fails as the file is corrupt
destfile ='Denmark_shapefile.zip' is only specifying the name the downloaded file should have. You can write anything you want in there.
Your code works for me exactly as it is, but you could try using this download statement instead of yours which is specified for writing and is binary safe.
download.file(url, destfile ='Denmark_shapefile.zip', mode='wb', cacheOK=FALSE)
One way to do is,
url <- 'https://www.eea.europa.eu/data-and-maps/data/eea-reference-grids-2/gis-
files/denmark-shapefile/at_download/file'
Your working directory is were files are downloaded
library(stringr)
fold = str_replace_all(getwd(), '/', '\\\\')
A random file name is assigned to zip file.
fil = tempfile(pattern = "file", tmpdir = fold, fileext = '.zip')
download.file(url, fil, mode = 'wb')
I am downloading a large xlsx file as a part of a function. This file is removed with file.remove() in linux and mac but I have permission denied in windows machines. Below is the code for my function.
download.file(
'http://mirtarbase.mbc.nctu.edu.tw/cache/download/7.0/miRTarBase_MTI.xlsx',
'miRTarBase.xlsx', mode = "wb")
readxl::read_excel('miRTarBase.xlsx') -> miRTarBase
write.csv(miRTarBase, 'miRTarBase.csv')
read.csv('miRTarBase.csv', row.names = 1) -> miRTarBase
file.remove("miRTarBase.xlsx")
I get the following error message in my console
Warning message:
In file.remove("miRTarBase.xlsx") :
cannot remove file 'miRTarBase.xlsx', reason 'Permission denied'.
Again this warning only appears in windows.
Furthermore, after checking the properties of the file itself the 'Read-only' attribute is unchecked.
Following this, the following code works perfectly fine so I do not think the issue is with the folder either.
file.remove("miRTarBase.csv")
I believe the issue lies in how .xlsx files are treated in windows.
When I try to delete the .xlsx file while Rstudio is still running I get a File in use warning message. After closing the R session the .xlsx file can be deleted with no hassle.
This has confused me because I am not used to working with windows. Has anyone had this issue before? Would appreciate any help that can be given. Many thanks.
Have you tried saving as a temporary file in windows?
tmp <- tempfile()
download.file(
'http://mirtarbase.mbc.nctu.edu.tw/cache/download/7.0/miRTarBase_MTI.xlsx', tmp, mode = "wb")
readxl::read_excel(tmp) -> miRTarBase
write.csv(miRTarBase, 'miRTarBase.csv')
read.csv('miRTarBase.csv', row.names = 1) -> miRTarBase
file.remove(tmp)
I am trying to download a ncdf file using rCurl. Can anyone provide any advice on why this is not working?
require(RCurl)
require(ncdf4)
url <- "https://oceandata.sci.gsfc.nasa.gov/MODIS-Aqua/Mapped/Seasonal_Climatology/4km/sst/"
filename <-"A20021722014263.L3m_SCSU_NSST_sst_4km.nc"
download.file(paste0(url, filename),destfile = paste0("~/Desktop/", filename), method="curl")
setwd("~/Desktop/")
files<-dir(pattern="*.nc")
f<-nc_open(files[1])
Error in R_nc4_open: NetCDF: Unknown file format
Error in nc_open(files[1]) :
Error in nc_open trying to open file A20021722014263.L3m_SCSU_NSST_sst_4km.nc
It appears that the file downloaded is an error file in XML format? If you open it in Notepad, you'll see it contains stuff like
Sorry, an error has occurred. Use the back button to return to the previous page or go to the Ocean Color Home Page
Are you sure that the filename you're wanting to download actually exists in that URL?
I am trying to download an excel zip file into a R Project folder that I have started. I'm new to R so I'm a little puzzled at the error message that I'm receiving.
The file is an excel file and so first I created a variable for the file:
excel2file="http://op.nsf.data/dataFiles/Housing2013EXCEL.zip"
Then I used the coding:
download.file(excel2file, destfile= "~/Home/Documents/Data")
I receive this error message:
Error in download.file(excel2file, destfile = "~/Home/Documents/Data") :
cannot open destfile '~/Home/Documents/Data', reason 'Permission denied'
I tried looking at other examples of permission denied and I think it may be my destination file but I am not sure why or the steps to trouble shoot it.
destfile should be a filename, not a directory. For example:
download.file(excel2file, destfile= "~/Home/Documents/Data/Housing2013EXCEL.zip")
Also, that URL doesn't seem to be valid, but that's a different (non-R) problem.
Make sure you don't have the file already open in the destfile, as the Permission denied error also comes from being unable to overwrite an open file.
This tripped me up for a while.
You could use the package readr and its function read_csv. It allows you to download the file, but it has to be a downloaded files such as a CSV file, something like this worked for me
library(readr)
newz <- read_csv("https://www.stats.govt.nz/assets/Uploads/Annual-enterprise-survey/Annual-enterprise-survey-2020-financial-year-provisional/Download-data/annual-enterprise-survey-2020-financial-year-provisional-csv.csv")
View(newz)
about the "Permission Denied" I still doesn`t solved that problem
You can also use basename with paste, which would be useful if downloading a bunch of files.
For example:
(excel2file="http://op.nsf.data/dataFiles/Housing2013EXCEL.zip")
(file_name <- basename(excel2file))
download.file(excel2file, destfile= paste("~/Home/Documents/Data",file_name, sep="/"))
add a name in destfile like /downloadedfile.csv"
I have an url, and I want to download the file via R, I notice that download.file would be helpful, but my problem seems different:
url <- "http://journal.gucas.ac.cn/CN/article/downloadArticleFile.do?attachType=PDF&id=11771"
destfile <- "myfile.pdf"
download.file(url, destfile)
It doesn't work! I notice that if my url is in the form of xxx.pdf, then the code above is no problem, otherwise the file that is downloaded is corrupt.
Does anyone know how to solve this problem?
Setting the mode might be required to treat the file as binary data while saving it. If I leave that argument out, I get a blank file, but this way works for me:
url <- "http://journal.gucas.ac.cn/CN/article/downloadArticleFile.do?
attachType=PDF&id=11771"
destfile <- "myfile.pdf"
download.file(url, destfile, mode="wb")
I am trying to download an nc file with R. It downloads well but I get this error when trying to open it:
Error in R_nc4_open: NetCDF: Unknown file format Error in
nc_open("SM_D2010323_Map_SATSSS_data_1day.nc") : Error in nc_open
trying to open file SM_D2010323_Map_SATSSS_data_1day.nc
(return_on_error= FALSE )
url <- "https://www.star.nesdis.noaa.gov/data/socd1/coastwatch/products/miras/nc/SM_D2010323_Map_SATSSS_data_1day.nc"
destfile <- "***/SM_D2010323_Map_SATSSS_data_1day.nc"
download.file(url, destfile)
nc_data <- nc_open('SM_D2010323_Map_SATSSS_data_1day.nc')
But when I use the same URL on my web browser, I can open the file without any problems with R.