I tried to open an shp file on my Mac, using this code:
library(tidyverse)
library(sf)
library(rgeos)
sf_trees_raw <- readr::read_csv('https://raw.githubusercontent.com/rfordatascience/tidytuesday/master/data/2020/2020-01-28/sf_trees.csv')
temp_shapefile <- tempfile()
download.file("https://www2.census.gov/geo/tiger/TIGER2017//ROADS/tl_2017_06075_roads.zip", temp_shapefile)
sf_roads <- unzip(temp_shapefile, "tl_2017_06075_roads.shp") %>%
read_sf()
But I receive this error message:
Error: Cannot open "/Users/name/Documents/Playground/Trees_SF/tl_2017_06075_roads.shp"; The source could be corrupt or not supported. See `st_drivers()` for a list of supported formats.
I tried other shp files, and I receive the same error message:
map <- read_sf("per_admbnda_adm1_2018.shp")
Error: Cannot open "/Users/name/Documents/Playground/Trees_SF/per_admbnda_adm1_2018.shp"; The source could be corrupt or not supported. See `st_drivers()` for a list of supported formats.
I tried copying the shx and dbf files but it doesn't fix the problem.
Any help will be greatly appreciated.
Try unzipping the whole download first, then reading the shapefile file. Although for sf you only need to point at the .shp file, all the other ones (.cpg, .prj, .shx, etc) need to be unzipped and in the same directory.
temp_shapefile <- tempfile()
download.file("https://www2.census.gov/geo/tiger/TIGER2017//ROADS/tl_2017_06075_roads.zip", temp_shapefile)
unzip(temp_shapefile)
sf_roads <- read_sf('tl_2017_06075_roads.shp')
Related
I am using one of the files in here: http://orca.science.oregonstate.edu/1080.by.2160.monthly.hdf.vgpm.m.chl.m.sst.php:
untar(tarfile = "http://orca.science.oregonstate.edu/data/1x2/monthly/vgpm.r2018.m.chl.m.sst/hdf/vgpm.m.2010.tar", exdir = "./foo")
I get error: ar.exe: Error opening archive: Failed to open 'http://orca.science.oregonstate.edu/data/1x2/monthly/vgpm.r2018.m.chl.m.sst/hdf/vgpm.m.2010.tar'
so I manually had to download the file and untar it ( that is why cant provide a reproducible example here). Inside there are files of .hdf format:
I also was not able to read them:
library(ncdf4)
ncin <- nc_open(".\\vgpm.m.2010\\vgpm.2010001.hdf")
ncin
lon <- ncvar_get(ncin,"fakeDim0")
head(lon)
lat <- ncvar_get(ncin,"fakeDim1")
head(lat)
fillvalue <- ncatt_get(ncin,"npp","_FillValue")
Can you please help to explain why i cant utar the file and why .hdf files have no fill value?
You should be able to untar the file once you have downloaded it. Download the file first to your working directory, then untar from your working directory: untar("vgpm.m.2002.tar", exdir = "mydir"). Your issue is likely with the connection. There can be many reasons for that which are specific to your computer's settings. You'll need to troubleshoot that separately.
Once you untar the directory, the contents inside are not .hdf files. They are compressed .hdf files (thus why their file names end in .gz). You'll need to decompress:
library(R.utils)
gunzip("mydir/vgpm.2002335.hdf.gz", remove = FALSE)
Once you actually have the .hdf file, you need to open it and then read it. You are correct to use ncdf4 because it accommodates multiple .hdf file formats. Some of the older formats would need different packages or software.
To open and read it, you need two different functions, nc_open() and ncvar_get():
library(ncdf4)
dat <- nc_open("mydir/vgpm.2002335.hdf", write = TRUE)
ncvar_get(dat)
Note that these functions will NOT work if you have not completed the pre-requisite set-up explained in detail in the documentation. For example:
Both the netCDF library and the HDF5 library must already be installed on your machine for this R interface to the library to work.
I also tried to rasterize it: it also works great:
library(raster)
x <- raster("~.\\vgpm.2010001.hdf")
extent(x) <- extent(-180, 180, -90, 90)
crs(x) <- "+proj=longlat +datum=WGS84"
NAvalue(x) <- -9999
#plot(x)
f1<- as.data.frame(x, xy=TRUE)
I am attempting to load a .Rdata file without having to download the actual file to my computer.
Here is what I have so far:
primate_URL <- "https://ftp.ncbi.nlm.nih.gov/geo/series/GSE118nnn/GSE118546/suppl/GSE118546_macaque_fovea_all_10X_Jan2018.Rdata.gz"
con <- gzcon(url(primate_URL))
load(con)
When I run the script, it returns this error:
Error in load(con) :
the input does not start with a magic number compatible with loading from a connection
Any tips on what I might be doing wrong?
Try following. I tried it with a small file from the same source of same type.
primate_URL <- "https://ftp.ncbi.nlm.nih.gov/geo/series/GSE118nnn/GSE118992/suppl/GSE118992_supplementaryData.Rdata.gz"
download.file(primate_URL, "GSE118992_supplementaryData.Rdata.gz")
file.x <- gzfile("GSE118992_supplementaryData.Rdata.gz", open = "r")
file.x.data <- readLines(file.x, encoding="UTF-8")
Hi I am trying to read and plot on a custom shape file in R which is not a map.
This is the code I use and the error I get in return:
library(rgdal)
mySHP<- choose.files()
myFile<- readOGR(mySHP)
Error in ogrListLayers(dsn = dsn) : Cannot open data source
If your file is a shapefile, you need to specify the dsn which is the directory where is saved the shapefile and layer which is the name of the shapefile without the extension. You cannot really do it with choose.files. At least not that simply.
myFile <- readOGR(dsn='path.to.folder', layer='nameOfShapefile')
In an effort to make reproducible posts at SO, I am trying to download data files to a temporary location and from there to load them into R. I am mostly using code from JD Longs answer in this SO Post. The downloading and unzipping works all fine, but I am unable to load the file from the temporary directory. This is the code I am using:
library(maptools)
tmpdir <- tempdir()
url <- 'http://epp.eurostat.ec.europa.eu/cache/GISCO/geodatafiles/NUTS_2010_03M_SH.zip'
file <- basename(url)
download.file(url, file)
unzip(file, exdir = tmpdir )
## I guess the error is somewhere in the next two lines
shapeFile <- paste(tmpdir,"/Shape/data/NUTS_RG_03M_2010")
EU <- readShapeSpatial(shapeFile)
# --> Error in getinfo.shape(fn) : Error opening SHP file
I have been looking into the man files for tempdir() without success. Settinge the working directory to the temporary location didn't work either. I probably miss something very basic here. Do you have any hints how to get around this?
shapeFile <- paste(tmpdir,"/Shape/data/NUTS_RG_03M_2010", sep="")
As default, paste use a space as separator, which cause your path to be wrong.
Of course, the alternative, as of R 2.15.0, would be paste0:
shapefile <- paste0(tmpdir,"/Shape/data/NUTS_RG_03M_2010")
I have a very large zip file and i am trying to read it into R without unzipping it like so:
temp <- tempfile("Sales", fileext=c("zip"))
data <- read.table(unz(temp, "Sales.dat"), nrows=10, header=T, quote="\"", sep=",")
Error in open.connection(file, "rt") : cannot open the connection
In addition: Warning message:
In open.connection(file, "rt") :
cannot open zip file 'C:\Users\xxx\AppData\Local\Temp\RtmpyAM9jH\Sales13041760345azip'
If your zip file is called Sales.zip and contains only a file called Sales.dat, I think you can simply do the following (assuming the file is in your working directory):
data <- read.table(unz("Sales.zip", "Sales.dat"), nrows=10, header=T, quote="\"", sep=",")
The methods of the readr package also support compressed files if the file suffix indicates the nature of the file, that is files ending in .gz, .bz2, .xz, or .zip will be automatically uncompressed.
require(readr)
myData <- read_csv("foo.txt.gz")
No need to use unz, as now read.table can handle the zipped file directly:
data <- read.table("Sales.zip", nrows=10, header=T, quote="\"", sep=",")
See this post
This should work just fine if the file is sales.csv.
data <- readr::read_csv(unzip("Sales.zip", "Sales.csv"))
To check the filename without extracting the file. This works
unzip("sales.zip", list = TRUE)
If you have zcat installed on your system (which is the case for linux, macos, and cygwin) you could also use:
zipfile<-"test.zip"
myData <- read.delim(pipe(paste("zcat", zipfile)))
This solution also has the advantage that no temporary files are created.
The gzfile function along with read_csv and read.table can read compressed files.
library(readr)
df = read_csv(gzfile("file.csv.gz"))
library(data.table)
df = read.table(gzfile("file.csv.gz"))
read_csv from the readr package can read compressed files even without using gzfile function.
library(readr)
df = read_csv("file.csv.gz")
read_csv is recommended because it is faster than read.table
In this expression you lost a dot
temp <- tempfile("Sales", fileext=c("zip"))
It should be:
temp <- tempfile("Sales", fileext=c(".zip"))
For remote-based zipped files
samhsa2015 <- fread("curl https://www.opr.princeton.edu/workshops/Downloads/2020Jan_LatentClassAnalysisPratt_samhsa_2015F.zip | funzip")
answer from here: https://stackoverflow.com/a/37824192/12387385)