I'm trying to download the zip file from this url:
url1 <- http://www.clinicaltrials.gov/ct2/results?cond=%22acne%22&studyxml=true
Here's my code:
tempZip <- tempfile()
download.file(url1, tempZip)
And here's the error I get:
Warning message:
In download.file(url1, tempZip) :
downloaded length 817445 != reported length 200
Any ideas?
EDIT: OK, after seeing agstudy's reply below, I found that the file was indeed being downloaded (it also appears to be the correct file size). Now the problem is when I try to unzip the file - it days the file is corrupted.
Maciej, I agree that it would be better to use a link with a .zip extension, however, there's no way to get that from this website.
OK, I figured out what was wrong. Because this url does not specifically have ".zip" at the end, the download.file function does not know to use a binary download. This code fixes the problem:
url1 <- http://www.clinicaltrials.gov/ct2/results?cond=%22acne%22&studyxml=true
tempZip <- tempfile()
download.file(url1, tempZip, mode="wb")
If you don't specify the mode argument, the downloaded zip file will be corrupt.
You don't have direct link to the file. R try to download webpage not file. Use link which end with '.zip'.
Maybe useful be using XML or RCurl package to scrape links to datasets from this webpage.
Related
See the following code (Windows 10, R 3.6.3)
download.file(
url = "https://www6.ohiosos.gov/ords/f?p=VOTERFTP:DOWNLOAD::FILE:NO:2:P2_PRODUCT_NUMBER:363",
destfile = paste0("SWVF_1_22_", format(as.Date(Sys.Date()), "%Y%m%d"), ".txt.gz")
)
Once I try to unzip this with 7zip, this simply says "Data error."
The funniest thing is, this code used to work before, going back to late Jan 2019.
In addition, if you go to the original url, it downloads fine and it is also unzipped properly via either 7zip or WinZip.
Downloaded files either way have the same file size, which is why I never suspected corruption of files for an entire year and more. I've tried changing destfile argument, as well as using tempfile(). Neither works.
This style of download.file works fine for zip files. Just this .txt.gz files.
I have absolutely no idea what is going on, and whether my files can be recovered at all. Any advice?
I'm trying to download a long list of podcasts, but when I use the download.file command in R it corrupts the audio file into a bunch of crackling noises.
Could any of you recommend a dedicated audio-downloading package, or recommend a download.file method that would be better suited for download audio. I went through the ones listed in the help file, but none worked. ("auto", "internal", "wininet", "libcurl", "wget" and "curl")
The downloading portion of the code looks similar to this:
url <- "http://play.podtrac.com/npr-510289/npr.mc.tritondigital.com/NPR_510289/media/anon.npr-mp3/npr/pmoney/2016/06/20160603_pmoney_podcast.mp3?
orgId=1&d=1121&p=510289&story=480606726&t=podcast&e=480606726&siteplayer=true&dl=1"
download.file(url = url, destfile = "test.mp3")
I attempted different audio files from different sites and had similar results.
Edit: In response to the question by VC.One, this a url to the initial section of the Hex code. I added in more than the couple of lines he requested because the first section looked like file information which may or may not be relevant:
Try mode = "wb" in download.file(). I had the same issue you mentioned and this solved it for me.
I am trying to download an Excel file (xls) from the Australia Bureau of Statistics in r with the following code. However, everytime I try to run the line with the read_excel command my session crashes.
library(readxl)
target <- 'http://www.ausstats.abs.gov.au/ausstats/meisubs.nsf/LatestTimeSeries/6202001/$FILE/6202001.xls'
path <- paste0(getwd(),"/","6202001.xls")
download.file(target, destfile = path)
#read_excel(path = path) << problem line
I think it might have something to do with the excel file pop-up when you go to put the link into a browser and download it that way, but I'm not sure!
Do I need to change the file before I go to read it at all?
Any help would be great.
Download the file in binary mode (default for download.file is ASCII mode) with the mode argument set to wb:
download.file(myurl, mydestfile, mode="wb")
Environment:
Windows 7 OS
RStudio Version 0.99.491
I have been programming in R for about 4 months via the Coursera Data Science curriculum, but I have NEVER been successful in using the unzip function.
I've looked at the forums for hours for potential solutions, syntax problems, undefined arguments, etc., but to no avail. I eventually unzip the contents manually and proceed with the assignment, but I am tired of not knowing why it is not working.
Here are a few examples of the error:
fileName <- "StormData.zip"
unzip(fileName, exdir = mainDir,subDir)
Warning message: In unzip(fileName, exdir = mainDir, subDir) : error
1 in extracting from zip file
unzip(fileName)
Warning message: In unzip(fileName) : error 1 in extracting from zip
file
unzip(fileName, "stormdata.csv")
Warning message: In unzip(fileName, "stormdata.csv") : error 1 in
extracting from zip file
unzip(fileName, "stormdata.csv", list = TRUE)
Error in unzip(fileName, "stormdata.csv", list = TRUE) : zip file
'StormData.zip' cannot be opened
Any suggestions would be greatly appreciated.
I was getting the same error.
I changed the path --
from :
uzp <- "C:\\Users\\Sharvari\\Downloads\\rprog%2Fdata%2Fspecdata"
to
uzp <- "C:\\Users\\Sharvari\\Downloads\\rprog%2Fdata%2Fspecdata.zip"
and it works fine!
setwd("C:\\Users\\Sharvari\\Downloads")
uzp <- "C:\\Users\\Sharvari\\Downloads\\rprog%2Fdata%2Fspecdata.zip"
unzip(uzp, exdir = "C:\\Users\\Sharvari\\Desktop\\specdata")
I too was getting that error 1 message when trying to unzipping a zip file. Glitch in my case was the conflict between working directory and zip file path.
My case was:
My working directory was like "C:/Users/SCOTT/Desktop/Training"
While my zip file was located in "C:/Users/SCOTT/Desktop/Training/house_consumption_data"
When I was trying to execute this:
unzip("house_data.zip")
Possibly your file is in a different folder.
I have had the same problem trying to download and unzip the same file, for the same course. And I have had problems with unzip in the past and was determined to solve it this time too.
Eventually the extension of the file turned out to be csv.bz2. And than this Extract bz2 file in R post solved my problem.
After downloading the file I was able to read it directly with
stormdata <- read.csv("stormdata.zip")
without using unzip.
This error seems to appear whenever openXLS is unable to open the specified file.
It could be a wrong name, wrong directory or the file might be encrypted or password protected
change your zip file format this error will appear while the zip format problems occur, look at your zip file it should be "rar" change it to "zip". the function works only for "zip" format files.
I faced the same issue. Make sure that, you specify the correct name of the file(get it from the properties of .zip file) in the following code.
file = read.table(unzip("file_name.csv.zip"), sep = ",", header = TRUE)
In my case, Was just mentioning file_name.zip and R was throwing the error.
Also, there are two functions for unzipping files in R
1) unz - to extract single element from zip file/s
2) unzip - to extract all the present elements from the .zip file
I usually prefer unzip.
If you will use unz in the above code, R will throw error again.
I encountered the same error using install_course_zip' with a zip file. I followed all the instructions for the command faithfully but kept getting errors relating to the 'exdir'. I moved the zip file to various directories without success.
I finally used getwd() to get the working directory and then placed the zip file in that directory. I then was able to use the zip file name without having to use any folder structure and this worked. I still have no idea why R would not accept a different directory.
I had list of files to be unzipped and processed; I was facing same error
"error 1 in extracting from zip file"
used full directory and set working directory code worked
files <- list.files(path="C:\\Users\\Tejas naik\\Documents\\land", pattern=".zip$")
out_dir<- "C:\\Users\\Tejas naik\\Documents\\input"
setwd("C:\\Users\\Tejas naik\\Documents\\land")
for (i in files) {
#nzip(paste(out_dir,i), exdir=out_dir)
unzip(i,exdir=out_dir)
}
This error was happening bit differently in my case . As there was no zip file ,the issue was file was open in excel so this error was poping up .
It's crucial to give the full name (including the path) of the zip-file to the unzip function.
So instead of file.zip, it should be C:\user\name\file.zip.
In case you're using the list.files function, one should set the full.names option to TRUE.
For me the error is fixed after I add \ backslash character to the filepath.
Example:
from
unzip("abc\aaa.zip")
to
unzip("abc\\aaa.zip")
I am trying to open a shapefile in R, but I am getting the following error message:
Error in getinfo.shape(filen) : Error opening SHP file
I have checked other responses and most problems seem to have been solved by ensuring that the .dbf and .shx files are in the same folder. I have them all in the same folder (along with some other extensions too) but I still get the error message. I work on a mac.
This is my code:
getinfo.shape("/Users/Suz/Desktop/DWH satellite maps/20100517_Composite.shp")
I have tried it without the .shp extension, and with other commands, such as readShapePoints etc. Nothing has worked so far. Please help, I am new to R and making maps, and after extensive Googling and forum-reading I am still stuck.
I just had the same problem. Often come other files with your SHP-file. If they are missing the file cant be loaded.
So search if there are any other file ext. with "20100517_Composite" at the source you got your file.
Cant comment yet but i wanted the ppl to save time, if this is the problem.
You could try getinfo.shape(file.choose()) to select the file via a pop up window. If this works then it is probably an issue with your input string.
Note: I'm on linux, but I think file.choose() should work for mac.
After having the same issue, I did some digging and found a nice thread [here].1
Turns out that after checking the list.files() command and found that my files were not there, and
even though I had included the file path in my original code, it still produced the error shown in the question. I then moved all the files into the working directory and then the command below worked.
readShapeSpatial()
Also simply changing the wd would work as well.
setwd("directory_path")
I figured I'd put this here as #jbaums suggested because it would have saved me some time in solving this issue.
I had the same problem until i removed the .shp extension.
So instead of
readShapeSpatial("/Users/Suz/Desktop/DWH satellite maps/20100517_Composite.shp")
go with
readShapeSpatial("/Users/Suz/Desktop/DWH satellite maps/20100517_Composite")
If you have all the files in the working directory it should work like a charm.
The easy way to read a shapefile in R is
either (to get a Spatial*) object
library(raster)
x <- shapefile("/Users/Suz/Desktop/DWH satellite maps/20100517_Composite.shp")
or (to get a sf object)
library(sf)
st_read("/Users/Suz/Desktop/DWH satellite maps/20100517_Composite.shp")
(but do not use the deprecated (incomplete and obsolete) function readShapeSpatial)
In action:
library(raster)
library(sf)
f <- system.file("external/lux.shp", package="raster")
s1 <- shapefile(f)
s2 <- st_read(f)
If this does not work, you need to check if your file exists:
file.exists(f)
To get a list of shapefiles in a directory, you can do
path <- "c:/temp" # change with your directory name
ff <- list.files(path, pattern='\\.shp$', full.names=TRUE)
This is still a problem. I solve it more directly calling the shapefile using file.choose() and manually selecting the file. Hope this helps for anyone.
library (rgdal)
a = readOGR (file.choose()) #then selecting the shape file manually