Related
I have a single simple raster in EPSG:7532 that I am trying to project to EPSG:4326 but is failing
The source data is a Lidar point clould that I am able to process using the lidR package. The data source is in the link below
https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/WI_BrownRusk_2020_B20/WI_Brown_2_2020/LAZ/USGS_LPC_WI_BrownRusk_2020_B20_02531702.laz
l1 = readLAS("USGS_LPC_WI_BrownRusk_2020_B20_02531702.laz")
> l1
class : LAS (v1.4 format 6)
memory : 2.5 Gb
extent : 25349, 26849, 170258, 171758 (xmin, xmax, ymin, ymax)
coord. ref. : NAD83(2011) / WISCRS Brown (m) + NAVD88 height - Geoid18 (m)
area : 2.25 km²
points : 35.57 million points
density : 15.79 points/m²
density : 12.89 pulses/m²
convert to a spatRaster:
dsm <- rasterize_canopy(l1, res = 1.0, pitfree(c(0,2,5,10,15), c(0, 1.5)))
> dsm
class : SpatRaster
dimensions : 1500, 1501, 1 (nrow, ncol, nlyr)
resolution : 1, 1 (x, y)
extent : 25349, 26850, 170258, 171758 (xmin, xmax, ymin, ymax)
coord. ref. : NAD83(2011) / WISCRS Brown (m) (EPSG:7532)
source : memory
name : Z
min value : 185.836
max value : 333.709
The point of failure is the attempt to project to geographic format:
dsm_test <- terra::project(dsm, "EPSG:4326", method="bilinear")
> dsm_test <- terra::project(dsm, "EPSG:4326", method="bilinear")
Error: [project] cannot get output boundaries
In addition: Warning messages:
1: In x#ptr$warp(SpatRaster$new(), y, method, mask, FALSE, opt) :
GDAL Error 1: PROJ: vgridshift: could not find required grid(s).
2: In x#ptr$warp(SpatRaster$new(), y, method, mask, FALSE, opt) :
GDAL Error 1: PROJ: pipeline: Pipeline: Bad step definition: proj=vgridshift (File not found or invalid)
3: In x#ptr$warp(SpatRaster$new(), y, method, mask, FALSE, opt) :
GDAL Error 1: Too many points (961 out of 961) failed to transform, unable to compute output bounds.
A similar topic here, but seems different.
https://stackoverflow.com/questions/72404897/what-is-causing-this-raster-reprojection-error
This issue is not resulting from reprojection from EPSG:7532 to EPSG:4326 per se, but seems rather connected to the fact, that your SpatRaster object created via rasterize_canopy() comes with a vertical datum, apparently causing problems downstream:
VERTCRS["NAVD88 height - Geoid18 (m)",
VDATUM["North American Vertical Datum 1988"],
CS[vertical,1],
AXIS["up",up,
LENGTHUNIT["meter",1]],
GEOIDMODEL["GEOID18"],
ID["EPSG",5703]]]
The quick & dirty solution would be to simply override crs definition by EPSG:7532 and dropping references in Z dimension, although this does not feel 100 % right. On the other hand, I'm not sure how terra handles vertical crs information and if it is possible to keep this information at all.
library(lidR)
library(terra)
#> terra 1.6.49
l1 = readLAS("USGS_LPC_WI_BrownRusk_2020_B20_02531702.laz")
#> Warning: There are 53206 points flagged 'withheld'.
dsm <- rasterize_canopy(l1, res = 1.0, pitfree(c(0,2,5,10,15), c(0, 1.5)))
crs(dsm) <- "epsg:7532"
dsm_4326 <- project(dsm, "epsg:4326", method="bilinear")
dsm_4326
#> class : SpatRaster
#> dimensions : 1235, 1727, 1 (nrow, ncol, nlyr)
#> resolution : 1.093727e-05, 1.093727e-05 (x, y)
#> extent : -88.0786, -88.05972, 44.49092, 44.50443 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326)
#> source(s) : memory
#> name : Z
#> min value : 185.8360
#> max value : 295.1357
From my personal point of view, it would be better to have both crs (xy, z) listed in the SpatRaster summary with reproject() being able to address z-transformations separately (or have e.g. terra::transform()), e.g. when you wanted to just transform your heights but still keep EPSG:7532 as your xy reference system.
coord. ref. xy: NAD83(2011) / WISCRS Brown (m) (EPSG:7532)
coord. ref. z: NAVD88 height - Geoid18 (m) (EPSG:5703)
I am struggling to open a NetCDF file and convert it into a raster using R. The
data is supposed to be on a regular grid of 25 km by 25 km. It contains sea
ice concentration in the Arctic.
library(terra)
#> terra 1.5.21
library(ncdf4)
file <- "~/Downloads/data_sat_Phil_changt_grid/SIC_SMMR_month_2015.nc"
I am getting a warning about the extent not found.
r <- rast(file)
#> Error in R_nc4_open: No such file or directory
#> Warning: [rast] GDAL did not find an extent. Cells not equally spaced?
We can see that there is a problem with the coordinates/extent.
r
#> class : SpatRaster
#> dimensions : 448, 304, 12 (nrow, ncol, nlyr)
#> resolution : 0.003289474, 0.002232143 (x, y)
#> extent : 0, 1, 0, 1 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> source : SIC_SMMR_month_2015.nc:sic
#> varname : sic
#> names : sic_1, sic_2, sic_3, sic_4, sic_5, sic_6, ...
I can open the nc file with nc_open() and I see that the coordinates are present.
nc <- nc_open(file)
names(nc$var)
#> [1] "lat" "lon" "sic"
lat <- ncvar_get(nc, "lat")
lon <- ncvar_get(nc, "lon")
dim(lat)
#> [1] 304 448
dim(lon)
#> [1] 304 448
dim(r)
#> [1] 448 304 12
Is it possible to assemble this data (the SIC values and the coordinates) to create a SpatRaster?
The nc file can be downloaded here: https://easyupload.io/pfth0s
Created on 2022-05-20 by the reprex package (v2.0.1)
The data are gridded, but the file does not specify the coordinates, nor the coordinate reference system. The file specifies the lon/lat values associated with the cells, but does not help us much, as these are clearly not on a regular grid. That is easy to see from plot(r)
NAflag(r) = -9999
plot(r,1)
And also from
p = cbind(as.vector(lon), as.vector(lat))
plot(p, cex=.1, xlab="lon", ylab="lat")
So what you need to find out, is which coordinate reference system (crs) is used, clearly some kind of polar crs. And what the extent of the data set is.
From the website you point to, I take it we can use:
crs(r) = "+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +k=1 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs"
ext(r) = c(-3850000, 3750000, -5350000, 5850000)
r
#class : SpatRaster
#dimensions : 448, 304, 12 (nrow, ncol, nlyr)
#resolution : 25000, 25000 (x, y)
#extent : -3850000, 3750000, -5350000, 5850000 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs
#source : SIC_SMMR_month_2015.nc:sic
#varname : sic
#names : sic_1, sic_2, sic_3, sic_4, sic_5, sic_6, ...
The results look good:
g = geodata::gadm("Greenland", level=0, path=".")
gg = project(g, crs(r))
plot(r,1)
lines(gg)
But this is of course not a good way to do such things; the ncdf file should have contained all the metadata required.
I need to project longitude/latitude coordinates in the terra package, but I don't believe it is working correctly, as I am trying to extract data from a raster with this projection, but the data is not being extracted correctly.
Here's my lon/lat points and the code I am using to try to project them.
latlon_df <- structure(list(Lon = c(-103.289, -96.6735, -96.9041, -96.76864,
-102.4694, -96.6814, -97.7504, -99.6754, -96.4802, -103.0007,
-96.8897, -101.8539, -103.9717, -101.253, -99.1134, -96.5849,
-98.0301, -99.9537, -99.4601, -99.7122, -103.8278, -98.931, -102.1081,
-101.7162, -100.115, -101.3448, -100.7805, -103.5606, -96.5302,
-99.4156, -103.281, -100.0063, -97.9928, -100.7208, -98.5289,
-96.762, -96.9218, -97.1024, -103.3793, -101.0841, -102.6745,
-96.9188, -97.5154, -100.7435, -98.6938), Lat = c(45.5194, 44.3099,
43.0526, 44.3252, 45.5183, 43.7316, 45.6796, 45.4406, 44.7154,
44.0006, 43.7687, 43.9599, 43.4737, 44.9875, 45.0292, 44.0867,
45.5735, 44.9895, 44.5256, 43.5938, 43.7343, 45.7163, 45.9189,
43.1672, 45.6716, 45.9154, 45.7963, 44.6783, 44.5073, 43.7982,
43.3784, 44.2912, 43.3841, 43.2002, 44.8579, 43.5048, 43.5033,
45.1055, 44.4245, 45.4167, 44.5643, 44.304, 45.2932, 43.5601,
43.7321)), class = "data.frame", row.names = c(NA, -45L))
latlons <- terra::vect(latlon_df,geom=c('Lon','Lat'),crs="+proj=longlat")
lcc <- terra::project(latlons,"+proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs")
var_df <- terra::extract(grib_data,lcc)[,-1]
The raster data (grib_data) I am using comes from here (it is way too big for me to put on here). https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2
I am not sure what I am doing wrong here, as I have used this method previously, and it seemed to work fine. Any help would be wonderful.
EDIT: The specific problem I am having is that I am not getting any different values for each lon/lat pair. The value for each variable is different, but all the values for the stations (different lon/lats are the same).
Why do you think it has to do with the projection? Either way, it appears to work for me.
url <- "https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2"
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
url <- paste0(url, ".idx")
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
library(terra)
r <- rast("hrrr.t00z.wrfsubhf00.grib2")
r
#class : SpatRaster
#dimensions : 1059, 1799, 49 (nrow, ncol, nlyr)
#resolution : 3000, 3000 (x, y)
#extent : -2699020, 2697980, -1588806, 1588194 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs
#source : hrrr.t00z.wrfsubhf00.grib2
#names : 0[-] ~here", 0[-] ~tops", 0[-] ~here", 0[-] ~here", 0[-] ~face", 1000[~ound", ...
You can check of the points overlap with the raster data
plot(r, 1)
points(lcc)
And extract. It takes very long with grib files, but it does appear to work
e <- extract(r, lcc)
head(e[,c(1,6,9)])
# ID 0[-] SFC="Ground or water surface" 0[-] SFC="Ground or water surface".1
#1 1 85100 11.775471
#2 2 54400 11.087971
#3 3 79300 9.900471
#4 4 49200 10.712971
#5 5 70800 9.212971
#6 6 56600 11.400471
Make sure you have the current (CRAN) version, or perhaps the development version that you can install like this:
install.packages('terra', repos='https://rspatial.r-universe.dev')
You can speed things up a lot by doing a single read from disk (by adding zero in this example)
e <- extract(r+0, lcc)
That is not always possible and I need to do some optimization behind the scences.
I am not sure why but I cant read shapefiles at all in r!
On a MacOS Mojave,
I have tried:
afg <- readOGR("afg.shp")
afg <- readOGR(dsn = ".", layer = "afg")
The shapefiles are from this website: http://www.diva-gis.org/gdata, Afghanistan roads.
This same error keeps occuring:
Error in ogrInfo(dsn = dsn, layer = layer, encoding = encoding, use_iconv = use_iconv, :
Cannot open layer
However, this code works:
countries <- readOGR("countries.geojson")
Can someone figure out why?
I'm on Mojave and this works fine:
httr::GET(
url = "http://biogeo.ucdavis.edu/data/diva/rds/AFG_rds.zip",
httr::write_disk("AFG_rds.zip"),
httr::progress()
) -> res
fils <- unzip("AFG_rds.zip", exdir = "afg-roads")
fils
## [1] "afg-roads/AFG_roads.dbf" "afg-roads/AFG_roads.prj"
## [3] "afg-roads/AFG_roads.shp" "afg-roads/AFG_roads.shx"
rgdal::readOGR("afg-roads/AFG_roads.shp")
## OGR data source with driver: ESRI Shapefile
## Source: "/Users/bob/Development/afg-roads/AFG_roads.shp", layer: "AFG_roads"
## with 1409 features
## It has 5 fields
## class : SpatialLinesDataFrame
## features : 1409
## extent : 60.6075, 74.67168, 29.38582, 38.48983 (xmin, xmax, ymin, ymax)
## coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0
## variables : 5
## names : MED_DESCRI, RTT_DESCRI, F_CODE_DES, ISO, ISOCOUNTRY
## min values : Unknown, Primary Route, Road, AFG, AFGHANISTAN
## max values : Without Median, Unknown, Trail, AFG, AFGHANISTAN
I faced the same problem too. Don't try renaming the file. After unzipping, you will get a number of files, one of which will be a shapefile. In my case, all those files had the same name but different extensions. Then just do what you were already doing:
afg <- readOGR(dsn = ".", layer = "AFG_roads")
Hopefully this will work.
Through attempting to get an intersect result from a single point and a single polygon I have found what I believe can only be a bug in the R raster package intersect function.
I have 1 polygon and 1 point, and use intersect as follows:
intersect(a_point, a_polygon)
Where a_point contains an id attribute. This fails with the error:
Error in j[, 2] : incorrect number of dimensions
However, if I reverse the arguments and do:
intersect(a_polygon, a_point)
It works fine, but doesn't return the id from the point shape file as part of the result which I require. This is expected behaviour, so fine but I need it to work the other way around.
To rule out there being some peculiarity with my polygon or point data, I created a single polygon and single point spatial object and tested the same hypothesis, and the same result occurred as above with these 'raw' objects.
The following is the code for generating these two 'fake' objects for completeness and so that it can be reproduced:
test_list_x = list(530124, 530125) #For when I use 2 points
test_list_y = list(176949, 176950) #For when I use 2 points
data_frame_object = data.frame(530124, 176950)
names(data_frame_object) = c("Longitude", "Latitude")
coordinates(data_frame_object)=~Longitude+Latitude
proj4string(data_frame_object)=CRS("+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +datum=OSGB36 +units=m +no_defs +ellps=airy +towgs84=446.448,-125.157,542.060,0.1502,0.2470,0.8421,-20.4894")
fake_point_shape_object=SpatialPointsDataFrame(data_frame_object, data.frame(id=1:length(data_frame_object)))
coords = matrix( nrow=5, ncol=2)
coords[1,1] = 530106.8
coords[1,2] = 176953.3
coords[2,1] = 530127.5
coords[2,2] = 176953.3
coords[3,1] = 530127.5
coords[3,2] = 176933.3
coords[4,1] = 530106.8
coords[4,2] = 176933.3
coords[5,1] = 530106.8
coords[5,2] = 176953.3
my_fake_polygon = Polygon(coords)
polygon_list = list(my_fake_polygon)
polygon_set <- lapply(seq_along(polygon_list), function(i) Polygons(list(polygon_list[[i]]), i ))
new_polygons <- SpatialPolygons(polygon_set)
new_polygons#proj4string = CRS("+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +datum=OSGB36 +units=m +no_defs +ellps=airy +towgs84=446.448,-125.157,542.060,0.1502,0.2470,0.8421,-20.4894")
df <- data.frame("1")
names(df) = "id"
my_fake_polygon <- SpatialPolygonsDataFrame(new_polygons,df)
Now here's the thing, if I create 2 points next to each other (so they are both within the polygon) instead of just one, it works fine, no error. Suggesting there is a bug associated with intersection between 1 point and 1 polygon, WHEN the point carries an attribute to be returned in the intersection process.
You might ask why do you actually need to have the attribute returned if there is just one point, this is because it is an iterative process in which it may not be just one point, it could be none or many.
I would appreciate somebody explaining this error or confirming my findings.
Here are your example data in a more concise way.
library(raster)
pnt <- SpatialPoints(cbind(530124, 176950))
pol <- spPolygons(matrix(c(530106.8, 530127.5, 530127.5, 530106.8, 530106.8, 176953.3, 176953.3, 176933.3, 176933.3, 176953.3), ncol=2))
Now illustrate the problem.
intersect(pol, pnt)
#class : SpatialPolygons
#features : 1
#extent : 530106.8, 530127.5, 176933.3, 176953.3 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
# this fails
intersect(pnt, pol)
#Loading required namespace: rgeos
#Error in j[, 2] : incorrect number of dimensions
# but it works with two points!
intersect(bind(pnt, pnt), pol)
#class : SpatialPoints
#features : 2
#extent : 530124, 530124, 176950, 176950 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
This was another drop=TRUE bug caused by the R default of "dropping" matrices to vectors when a single row is selected. This was fixed in raster version 2.6-11 (not on CRAN yet).
Sorry i can't answer your intersect bug question, but it might be simpler for now to use sp::over to return polygon attributes to points
# dummy polygon
xym <- as.matrix(data.frame(x=c(16.48438,17.49512,24.74609,22.59277,16.48438),
y=c(59.73633,55.12207,55.03418,61.14258,59.73633)))
# make into SpatialPolygon
p = Polygon(xym)
ps = Polygons(list(p),1)
sps = SpatialPolygons(list(ps))
# Promote to SPDF and give an attribute
SPDF = SpatialPolygonsDataFrame(sps, data.frame(N = "hello", row.names = 1))
# make 2 points, one inside the polygon and one outside
p <- data.frame(x=c(16,18),y=c(58,58))
coordinates(p) <- ~x + y
# plot to check
plot(sps)
plot(p,add=T)
# perform the over, returns a named vector for every point in the SpatialPoints
res <- unname(over(p,SPDF))
# promote points to SpatialPointsDataFrame and put in new polygon attribute
data <- data.frame(ID=row.names(p),pol=res)
sp <- SpatialPointsDataFrame(p, data)