Related
I need to project longitude/latitude coordinates in the terra package, but I don't believe it is working correctly, as I am trying to extract data from a raster with this projection, but the data is not being extracted correctly.
Here's my lon/lat points and the code I am using to try to project them.
latlon_df <- structure(list(Lon = c(-103.289, -96.6735, -96.9041, -96.76864,
-102.4694, -96.6814, -97.7504, -99.6754, -96.4802, -103.0007,
-96.8897, -101.8539, -103.9717, -101.253, -99.1134, -96.5849,
-98.0301, -99.9537, -99.4601, -99.7122, -103.8278, -98.931, -102.1081,
-101.7162, -100.115, -101.3448, -100.7805, -103.5606, -96.5302,
-99.4156, -103.281, -100.0063, -97.9928, -100.7208, -98.5289,
-96.762, -96.9218, -97.1024, -103.3793, -101.0841, -102.6745,
-96.9188, -97.5154, -100.7435, -98.6938), Lat = c(45.5194, 44.3099,
43.0526, 44.3252, 45.5183, 43.7316, 45.6796, 45.4406, 44.7154,
44.0006, 43.7687, 43.9599, 43.4737, 44.9875, 45.0292, 44.0867,
45.5735, 44.9895, 44.5256, 43.5938, 43.7343, 45.7163, 45.9189,
43.1672, 45.6716, 45.9154, 45.7963, 44.6783, 44.5073, 43.7982,
43.3784, 44.2912, 43.3841, 43.2002, 44.8579, 43.5048, 43.5033,
45.1055, 44.4245, 45.4167, 44.5643, 44.304, 45.2932, 43.5601,
43.7321)), class = "data.frame", row.names = c(NA, -45L))
latlons <- terra::vect(latlon_df,geom=c('Lon','Lat'),crs="+proj=longlat")
lcc <- terra::project(latlons,"+proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs")
var_df <- terra::extract(grib_data,lcc)[,-1]
The raster data (grib_data) I am using comes from here (it is way too big for me to put on here). https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2
I am not sure what I am doing wrong here, as I have used this method previously, and it seemed to work fine. Any help would be wonderful.
EDIT: The specific problem I am having is that I am not getting any different values for each lon/lat pair. The value for each variable is different, but all the values for the stations (different lon/lats are the same).
Why do you think it has to do with the projection? Either way, it appears to work for me.
url <- "https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2"
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
url <- paste0(url, ".idx")
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
library(terra)
r <- rast("hrrr.t00z.wrfsubhf00.grib2")
r
#class : SpatRaster
#dimensions : 1059, 1799, 49 (nrow, ncol, nlyr)
#resolution : 3000, 3000 (x, y)
#extent : -2699020, 2697980, -1588806, 1588194 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs
#source : hrrr.t00z.wrfsubhf00.grib2
#names : 0[-] ~here", 0[-] ~tops", 0[-] ~here", 0[-] ~here", 0[-] ~face", 1000[~ound", ...
You can check of the points overlap with the raster data
plot(r, 1)
points(lcc)
And extract. It takes very long with grib files, but it does appear to work
e <- extract(r, lcc)
head(e[,c(1,6,9)])
# ID 0[-] SFC="Ground or water surface" 0[-] SFC="Ground or water surface".1
#1 1 85100 11.775471
#2 2 54400 11.087971
#3 3 79300 9.900471
#4 4 49200 10.712971
#5 5 70800 9.212971
#6 6 56600 11.400471
Make sure you have the current (CRAN) version, or perhaps the development version that you can install like this:
install.packages('terra', repos='https://rspatial.r-universe.dev')
You can speed things up a lot by doing a single read from disk (by adding zero in this example)
e <- extract(r+0, lcc)
That is not always possible and I need to do some optimization behind the scences.
I am trying to convert a csv to an sf spatial data file, however I'm getting errors that I cant' figure out.
Example:
library(tidyverse)
library(sf)
#> Linking to GEOS 3.8.0, GDAL 3.0.4, PROJ 6.3.1
point_df <- tibble::tribble(
~city_name, ~longitude, ~latitude,
"Akron", -81.5190053, 41.0814447,
"Albany", -73.7562317, 42.6525793,
"Schenectady", -73.9395687, 42.8142432,
"Albuquerque", -106.650422, 35.0843859,
"Allentown", -75.4714098, 40.6022939,
"Bethlehem", -75.3704579, 40.6259316,
"Atlanta", -84.3879824, 33.7489954,
"Augusta", -82.0105148, 33.4734978,
"Austin", -97.7430608, 30.267153,
"Bakersfield", -119.0187125, 35.3732921
)
point_sf <- st_as_sf(point_df, coords = c("longitude", "latitude"))
point_sf <- st_set_crs(point_sf, 4326)
st_transform(point_sf, 102003)
#> Warning in CPL_crs_from_input(x): GDAL Error 1: PROJ: proj_create_from_database:
#> crs not found
#> Error in CPL_transform(x, crs, aoi, pipeline, reverse): crs not found: is it missing?
Any help would be greatly appreciated.
EDIT
I found a kludgy solution which I adapted from this github page, but I am stil looking for a more systematic solution if possible. https://github.com/r-spatial/sf/issues/1419
The solution here is to convert the sf object into sp then change back to sf.
reProject <- function (sf, proj_in = "+init=epsg:4326",
proj_out = "+proj=aea +lat_1=20 +lat_2=60 +lat_0=40 +lon_0=-96 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs") {
require(sp)
data_sp <- as(sf, "Spatial")
proj4string(data_sp) <- CRS(proj_in)
sf_out <- st_as_sf(spTransform(data_sp, CRS(proj_out)))
}
dat_out <- reProject(point_sf)
It appears something was expected to happen with the following line of code. But that something is not happening.
point_sf <- st_as_sf(point_df, coords = c("longitude", "latitude"))
While this line of code creates the simple feature geometric point objects, this code does not create the simple feature geometry column (sfc) object. And since there is no sfc object, the next line of code does not work.
point_sf <- st_set_crs(point_sf, 4326)
In this other line of code, the function, st_set_crs(), retrieves a coordinate reference system from sf or sfc objects. But neither the sf or the sfc objects currently exist.
Therefore, the sfc object must be first created before using the function: st_set_crs().
It really helps to follow the following steps whenever doing these types of simple feature projects.
x.sfg <- st_multipoint(c(lon,lat), dim = "XY") # create sf geometry from lon/lat
x.sfc <- st_sfc(x.sfg, crs = 4326) # create sfc from geometry
x.sf <- st_sf(df, x.sfc) # create sf object from sfc
First convert the log and lat to vectors, then create the matrix, and then create the simple feature objects in the correct progression.
lon <- c(-81.5190053, -73.7562317, -73.9395687, -106.650422, -75.4714098, -75.3704579, -84.3879824, -82.0105148, -97.7430608, -119.0187125)
lat <- c(41.0814447, 42.6525793, 42.8142432, 35.0843859, 40.6022939, 40.6259316, 33.7489954, 33.4734978, 30.267153, 35.3732921)
m <- matrix(data = c(lon, lat), nrow = 10, ncol = 2, byrow = FALSE)
m.sfg <- st_multipoint(m, dim = "XY")
m.sfc <- st_sfc(m.sfg, crs = 4326)
m.sf <- st_sf(df, m.sfc)
head(m.sf, 3)
Then create a base plot of the continental US, and then plot the simple feature object onto the base map.
plot(US_48, axes = TRUE)
plot(m.sf, add= TRUE, pch = 19, col = "red")
The link shown above with the question does not seem to have anything related to this question. The answer shown here does not convert the sf object into sp then change back to sf.
The plot is shown at link:
I'm trying to use the wrld_simple data from the maptools package to plot a Cartogram that involves some European countries based on their Population in 2005:
countries = c('Austria', 'Belgium', 'Bulgaria', 'Cyprus', 'Czech Republic', 'Denmark', 'Estonia', 'Finland', 'France', 'Germany', 'Greece', 'Hungary', 'Iceland', 'Ireland', 'Italy', 'Latvia', 'Lithuania', 'Luxembourg', 'Malta', 'Netherlands', 'Norway', 'Poland', 'Portugal', 'Slovenia', 'Spain', 'Sweden', 'United Kingdom')
Using the maptools package, I got the wrld_simpl data, and retrieved the data for those particular countries:
data(wrld_simpl)
mapsimple = wrld_simpl[wrld_simpl$NAME %in% countries]
plot(mapsimple)
So far so good, but when I plot it out:
cartogram= cartogram(mapping1, "POP2005", itermax = 7)
Please use cartogram_cont() instead of cartogram().
Error: Using an unprojected map. This function does not give correct centroids and distances for longitude/latitude data:
Use "st_transform()" to transform coordinates to another projection.
I try using cartogram_cont:
> cartogram= cartogram_cont(mapping1, "POP2005", itermax = 7)
Error: Using an unprojected map. This function does not give correct centroids and distances for longitude/latitude data:
Use "st_transform()" to transform coordinates to another projection.
I tried transforming it to sf and then doing the st_transform(), but to no avail:
sfno = st_as_sf(mapping1)
sfnoproj = st_transform(sfno, coords = c("lon", "lat"), crs = "+proj=longlat +datum=WGS84", agr = "constant")
cartogram= cartogram_cont(sfnoproj, "POP2005", itermax = 7)
Error: Using an unprojected map. This function does not give correct centroids and distances for longitude/latitude data:
Use "st_transform()" to transform coordinates to another projection.
I'm assuming I don't have the correct values for the projection right. How would I know to set it better? If I were to use other values besides 'POP2005', how do I go about it? What does the 'itermax' in cartogram mean (I don't quite understand the given definition)
Any help would be greatly appreciated as I'm quite new to this. Thank you!!!!
Some inconsistencies in your sample data (mapsimple v. mapping1 and a missing comma), so updated below:
library(maptools)
library(sf)
library(ggplot2)
countries = c('Austria', 'Belgium', 'Bulgaria', 'Cyprus', 'Czech Republic', 'Denmark', 'Estonia', 'Finland', 'France', 'Germany', 'Greece', 'Hungary', 'Iceland', 'Ireland', 'Italy', 'Latvia', 'Lithuania', 'Luxembourg', 'Malta', 'Netherlands', 'Norway', 'Poland', 'Portugal', 'Slovenia', 'Spain', 'Sweden', 'United Kingdom')
data(wrld_simpl)
mapsimple = wrld_simpl[wrld_simpl$NAME %in% countries,]
As the error indicated, your data was unprojected. It uses latitude/longitude, which doesn't work for cartogram_cont().
sfno <- st_as_sf(mapsimple)
st_crs(sfno)
Coordinate Reference System:
EPSG: 4326
proj4string: "+proj=longlat +datum=WGS84 +no_defs"
Your st_transform() was reprojecting sfno as the same thing: EPSG 4326:
sfnoproj <- st_transform(sfno, coords = c("lon", "lat"),
crs = "+proj=longlat +datum=WGS84",
agr = "constant")
st_crs(sfnoproj)
Coordinate Reference System:
EPSG: 4326
proj4string: "+proj=longlat +datum=WGS84 +no_defs"
Instead, choose a projected coordinate system for Europe, like 23038.
sfproj <- st_transform(sfno, crs = 23038)
st_crs(sfproj)
Coordinate Reference System:
EPSG: 23038
proj4string: "+proj=utm +zone=38 +ellps=intl +towgs84=-87,-98,-121,0,0,0,0 +units=m +no_defs"
Then, we can make and plot the cartogram:
cartogram <- cartogram_cont(sfproj, "POP2005", itermax = 7)
ggplot() +
geom_sf(data = cartogram, aes(fill = POP2005))
I am trying to plot data in R using mapView for the grid in the Pacific that crosses the longitude 180deg. The native crs is "+proj=merc +lon_0=150 +lat_ts=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0".
The example of equivalent data is:
test.coords<-as.data.frame(cbind(c(runif(15,-180,-130),runif(5,160,180)),runif(20,40,60)))
test.sp <- SpatialPointsDataFrame(coords = cbind(test.coords$V1,test.coords$V2), data = test.coords,
proj4string = CRS("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0"))
test.sp <- spTransform(test.sp,
"+proj=merc +lon_0=150 +lat_ts=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0")
plot(test.sp)
mapview(test.sp)
When using plot function, the points are centered, while using mapView they get split between two sides of the map (linked image). Can the view using mapView be centered on a different longitude?
Using native crs for this map does not help. I get an error.
mapview(plot, native.crs=TRUE)
Warning message:
sf layer is not long-lat data
Thank you
mapView image Pacific
Ok, so this is a semi-optimal solution to your question for several reasons:
this will only work for point data
it may not scale well for large point collections
projecting to another CRS may not work anymore (though that is not really related, as the issue at hand is about visualising)
For sets of lines or polygons, we would need another approach, but I am currently not aware of a sp/sf native solution to round-tripping coordinates of an object.
library(sp)
library(sf)
#> Linking to GEOS 3.6.1, GDAL 2.2.3, PROJ 4.9.3
library(mapview)
test.coords<-as.data.frame(cbind(c(runif(15,-180,-130),runif(5,160,180)),runif(20,40,60)))
test.sp <- SpatialPointsDataFrame(coords = cbind(test.coords$V1,test.coords$V2), data = test.coords,
proj4string = CRS("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0"))
test_sf = st_as_sf(test.sp)
shift = function(x) {
geom = st_geometry(x)
st_geometry(x) = st_sfc(
lapply(seq_along(geom), function(i) {
geom[[i]][1] = ifelse(geom[[i]][1] < 0, geom[[i]][1] + 360, geom[[i]][1])
return(geom[[i]])
})
, crs = st_crs(geom)
)
return(x)
}
mapview(shift(test_sf)) +
mapview(test_sf, col.regions = "orange")
Created on 2019-12-11 by the reprex package (v0.3.0)
I chose to use sf rather than sp for this because mapview uses sf internally and was hoping to find a general approach to this problem which could potentially be integrated.
Here is a workaround for polygons - it produces some warnings, but overall does the job.
# transform grid to standard latitude and longitude
# original grid in crs "+proj=merc +lon_0=150 +lat_ts=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0"
grid<-spTransform(grid,"+init=epsg:4326")
summary(coordinates(grid))
# V1 V2
# Min. :-179.8 Min. :37.17
# 1st Qu.:-168.8 1st Qu.:51.70
# Median :-154.3 Median :55.02
# Mean :-118.0 Mean :54.65
# 3rd Qu.:-131.4 3rd Qu.:58.31
# Max. : 179.8 Max. :65.56
out <- lapply(1:length(grid), function(i) grid[i, ])
for (i in 1:length(grid)) {
cds <- slot(slot(slot(out[[i]], "polygons")[[1]], "Polygons")[[1]], "coords")
cds_polygon <- Polygons(list(Polygon(cds)), ID="out.x")
out.x <- SpatialPolygons(list(cds_polygon))
proj4string(out.x) <- CRS("+init=epsg:4326")
if (cds[1,1]>0) { # depending to which side one want to move polygons that are on both sides of International Date Line
out.y <- elide(out.x, shift=c(-360,0))
} else {
out.y<-out.x}
if(i==1){grid.shifted<-out.y} else {
grid.shifted<-raster::union(grid.shifted,out.y)
}
}
# add data in the original grid
grid.shifted <- SpatialPolygonsDataFrame(grid.shifted, grid#data, match.ID = F)
mapview(grid.shifted)
Through attempting to get an intersect result from a single point and a single polygon I have found what I believe can only be a bug in the R raster package intersect function.
I have 1 polygon and 1 point, and use intersect as follows:
intersect(a_point, a_polygon)
Where a_point contains an id attribute. This fails with the error:
Error in j[, 2] : incorrect number of dimensions
However, if I reverse the arguments and do:
intersect(a_polygon, a_point)
It works fine, but doesn't return the id from the point shape file as part of the result which I require. This is expected behaviour, so fine but I need it to work the other way around.
To rule out there being some peculiarity with my polygon or point data, I created a single polygon and single point spatial object and tested the same hypothesis, and the same result occurred as above with these 'raw' objects.
The following is the code for generating these two 'fake' objects for completeness and so that it can be reproduced:
test_list_x = list(530124, 530125) #For when I use 2 points
test_list_y = list(176949, 176950) #For when I use 2 points
data_frame_object = data.frame(530124, 176950)
names(data_frame_object) = c("Longitude", "Latitude")
coordinates(data_frame_object)=~Longitude+Latitude
proj4string(data_frame_object)=CRS("+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +datum=OSGB36 +units=m +no_defs +ellps=airy +towgs84=446.448,-125.157,542.060,0.1502,0.2470,0.8421,-20.4894")
fake_point_shape_object=SpatialPointsDataFrame(data_frame_object, data.frame(id=1:length(data_frame_object)))
coords = matrix( nrow=5, ncol=2)
coords[1,1] = 530106.8
coords[1,2] = 176953.3
coords[2,1] = 530127.5
coords[2,2] = 176953.3
coords[3,1] = 530127.5
coords[3,2] = 176933.3
coords[4,1] = 530106.8
coords[4,2] = 176933.3
coords[5,1] = 530106.8
coords[5,2] = 176953.3
my_fake_polygon = Polygon(coords)
polygon_list = list(my_fake_polygon)
polygon_set <- lapply(seq_along(polygon_list), function(i) Polygons(list(polygon_list[[i]]), i ))
new_polygons <- SpatialPolygons(polygon_set)
new_polygons#proj4string = CRS("+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +datum=OSGB36 +units=m +no_defs +ellps=airy +towgs84=446.448,-125.157,542.060,0.1502,0.2470,0.8421,-20.4894")
df <- data.frame("1")
names(df) = "id"
my_fake_polygon <- SpatialPolygonsDataFrame(new_polygons,df)
Now here's the thing, if I create 2 points next to each other (so they are both within the polygon) instead of just one, it works fine, no error. Suggesting there is a bug associated with intersection between 1 point and 1 polygon, WHEN the point carries an attribute to be returned in the intersection process.
You might ask why do you actually need to have the attribute returned if there is just one point, this is because it is an iterative process in which it may not be just one point, it could be none or many.
I would appreciate somebody explaining this error or confirming my findings.
Here are your example data in a more concise way.
library(raster)
pnt <- SpatialPoints(cbind(530124, 176950))
pol <- spPolygons(matrix(c(530106.8, 530127.5, 530127.5, 530106.8, 530106.8, 176953.3, 176953.3, 176933.3, 176933.3, 176953.3), ncol=2))
Now illustrate the problem.
intersect(pol, pnt)
#class : SpatialPolygons
#features : 1
#extent : 530106.8, 530127.5, 176933.3, 176953.3 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
# this fails
intersect(pnt, pol)
#Loading required namespace: rgeos
#Error in j[, 2] : incorrect number of dimensions
# but it works with two points!
intersect(bind(pnt, pnt), pol)
#class : SpatialPoints
#features : 2
#extent : 530124, 530124, 176950, 176950 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
This was another drop=TRUE bug caused by the R default of "dropping" matrices to vectors when a single row is selected. This was fixed in raster version 2.6-11 (not on CRAN yet).
Sorry i can't answer your intersect bug question, but it might be simpler for now to use sp::over to return polygon attributes to points
# dummy polygon
xym <- as.matrix(data.frame(x=c(16.48438,17.49512,24.74609,22.59277,16.48438),
y=c(59.73633,55.12207,55.03418,61.14258,59.73633)))
# make into SpatialPolygon
p = Polygon(xym)
ps = Polygons(list(p),1)
sps = SpatialPolygons(list(ps))
# Promote to SPDF and give an attribute
SPDF = SpatialPolygonsDataFrame(sps, data.frame(N = "hello", row.names = 1))
# make 2 points, one inside the polygon and one outside
p <- data.frame(x=c(16,18),y=c(58,58))
coordinates(p) <- ~x + y
# plot to check
plot(sps)
plot(p,add=T)
# perform the over, returns a named vector for every point in the SpatialPoints
res <- unname(over(p,SPDF))
# promote points to SpatialPointsDataFrame and put in new polygon attribute
data <- data.frame(ID=row.names(p),pol=res)
sp <- SpatialPointsDataFrame(p, data)