I have a .csv file of 9,000+ UTM coordinates that I would like to convert into decimal degrees and I am having a bit of trouble. I have searched through several of the posts that have been posted here and elsewhere and I can't seem find a solution that transforms my set of UTM's into usable and accurate lat/long's.
I essentially have two questions: 1) does anyone see any issues with my code; and 2) is anyone familiar with forgoing transformation of UTM's into lat/long's and just using UTM's in the Rgooglemaps package?
Here are some examples of my code and data:
Data:
>head(utm)
-Northing Easting
1 4236576 615805
2 4236576 615805
3 4236576 615805
4 4236576 615805
5 4236576 615805
6 4236576 615805
Code thus far:
utm <- read.csv(file="utm.csv", header=TRUE, sep=",")
library(rgdal)
utm <- utm[complete.cases(utm),]
utm1 <- data.frame(x=utm$Northing,y=utm$Easting)
coordinates(utm1) <- ~x+y
class(utm1)
proj4string(utm1) <- CRS("+proj=utm +zone=10 +datum=WGS84 +units=m +ellps=WGS84")
utm2 <- spTransform(utm1,CRS("+proj=longlat +datum=WGS84"))
Results
> head(utm2)
SpatialPoints:
x y
[1,] -91.08516 4.727323
[2,] -91.08516 4.727323
[3,] -91.08516 4.727323
[4,] -91.08516 4.727323
[5,] -91.08516 4.727323
[6,] -91.08516 4.727323
Coordinate Reference System (CRS) arguments: +proj=longlat +datum=WGS84 +ellps=WGS84
+towgs84=0,0,0
So, I am getting some output, but I am not getting sensible output. Is there something I am missing here? Also, for what its worth, I was planning on using the "Rgooglemaps" package for creating some heat maps and kernel density plots.
I'm using the following code to convert from UTM to Lat/Long. It's working for the London area
wgs84 = "+init=epsg:4326"
bng = '+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000
+ellps=airy +datum=OSGB36 +units=m +no_defs'
ConvertCoordinates <- function(easting,northing) {
out = cbind(easting,northing)
mask = !is.na(easting)
sp <- sp::spTransform(sp::SpatialPoints(list(easting[mask],northing[mask]),proj4string=sp::CRS(bng)),sp::CRS(wgs84))
out[mask,]=sp#coords
out
}
I think you just need to swap X and Y--Northing as Y, Easting as X. The original code worked for me with this change.
Related
I try to apply a gridded transformation of coordinates in sf, but it does not work. Is there something else to do apart from calling sf_proj_network(TRUE)? Here is the code I used:
pt_27572 <- data.frame(X=55824.4970,Y=2394454.2120) # set coordinates of one point in EPSG:27572
pt_2154 <- data.frame(X=107242.8310,Y=6832277.1820) # known accurate coordinates of the same point in EPSG:2154
ptSf_27572 <- st_as_sf(pt_27572, coords = c("X", "Y"), crs = 27572) # build sf object
ptSf_2154 <- st_as_sf(pt_2154, coords = c("X", "Y"), crs = 2154) # build sf object
sf_proj_network(TRUE) # allow search for online datum grids in the PROJ CDN
[1] "https://cdn.proj.org"
sf_proj_pipelines("EPSG:27572", "EPSG:2154") # grids (fr_ign_gr3df97a.tif) seem to be found for accurate transformation from EPSG:27572 to EPSG:2154
Candidate coordinate operations found: 3
Strict containment: FALSE
Axis order auth compl: FALSE
Source: EPSG:27572
Target: EPSG:2154
Best instantiable operation has accuracy: 1 m
Description: Inverse of Lambert zone II + NTF (Paris) to NTF (1) + NTF to
RGF93 (1) + Lambert-93
Definition: +proj=pipeline +step +inv +proj=lcc +lat_1=46.8 +lat_0=46.8
+lon_0=0 +k_0=0.99987742 +x_0=600000 +y_0=2200000
+ellps=clrk80ign +pm=paris +step +proj=push +v_3
+step +proj=cart +ellps=clrk80ign +step
+proj=xyzgridshift +grids=fr_ign_gr3df97a.tif
+grid_ref=output_crs +ellps=GRS80 +step +inv
+proj=cart +ellps=GRS80 +step +proj=pop +v_3 +step
+proj=lcc +lat_0=46.5 +lon_0=3 +lat_1=49 +lat_2=44
+x_0=700000 +y_0=6600000 +ellps=GRS80
ptSf_2154_grid <- st_transform(ptSf_27572,crs=2154) # apply transformation
st_distance(ptSf_2154_grid,ptSf_2154) # incorrect (ungridded) transformation, the distance should be zero. 3.777 m is the known error for the ungridded transformation.
Units: [m]
[,1]
[1,] 3.777346
Thanks for your help.
See https://github.com/r-spatial/sf/issues/1815. The underlying reason is a PROJ regression for versions 7.2.1-8.1.1 fixed by https://github.com/OSGeo/PROJ/pull/2884.
I need to project longitude/latitude coordinates in the terra package, but I don't believe it is working correctly, as I am trying to extract data from a raster with this projection, but the data is not being extracted correctly.
Here's my lon/lat points and the code I am using to try to project them.
latlon_df <- structure(list(Lon = c(-103.289, -96.6735, -96.9041, -96.76864,
-102.4694, -96.6814, -97.7504, -99.6754, -96.4802, -103.0007,
-96.8897, -101.8539, -103.9717, -101.253, -99.1134, -96.5849,
-98.0301, -99.9537, -99.4601, -99.7122, -103.8278, -98.931, -102.1081,
-101.7162, -100.115, -101.3448, -100.7805, -103.5606, -96.5302,
-99.4156, -103.281, -100.0063, -97.9928, -100.7208, -98.5289,
-96.762, -96.9218, -97.1024, -103.3793, -101.0841, -102.6745,
-96.9188, -97.5154, -100.7435, -98.6938), Lat = c(45.5194, 44.3099,
43.0526, 44.3252, 45.5183, 43.7316, 45.6796, 45.4406, 44.7154,
44.0006, 43.7687, 43.9599, 43.4737, 44.9875, 45.0292, 44.0867,
45.5735, 44.9895, 44.5256, 43.5938, 43.7343, 45.7163, 45.9189,
43.1672, 45.6716, 45.9154, 45.7963, 44.6783, 44.5073, 43.7982,
43.3784, 44.2912, 43.3841, 43.2002, 44.8579, 43.5048, 43.5033,
45.1055, 44.4245, 45.4167, 44.5643, 44.304, 45.2932, 43.5601,
43.7321)), class = "data.frame", row.names = c(NA, -45L))
latlons <- terra::vect(latlon_df,geom=c('Lon','Lat'),crs="+proj=longlat")
lcc <- terra::project(latlons,"+proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs")
var_df <- terra::extract(grib_data,lcc)[,-1]
The raster data (grib_data) I am using comes from here (it is way too big for me to put on here). https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2
I am not sure what I am doing wrong here, as I have used this method previously, and it seemed to work fine. Any help would be wonderful.
EDIT: The specific problem I am having is that I am not getting any different values for each lon/lat pair. The value for each variable is different, but all the values for the stations (different lon/lats are the same).
Why do you think it has to do with the projection? Either way, it appears to work for me.
url <- "https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2"
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
url <- paste0(url, ".idx")
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
library(terra)
r <- rast("hrrr.t00z.wrfsubhf00.grib2")
r
#class : SpatRaster
#dimensions : 1059, 1799, 49 (nrow, ncol, nlyr)
#resolution : 3000, 3000 (x, y)
#extent : -2699020, 2697980, -1588806, 1588194 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs
#source : hrrr.t00z.wrfsubhf00.grib2
#names : 0[-] ~here", 0[-] ~tops", 0[-] ~here", 0[-] ~here", 0[-] ~face", 1000[~ound", ...
You can check of the points overlap with the raster data
plot(r, 1)
points(lcc)
And extract. It takes very long with grib files, but it does appear to work
e <- extract(r, lcc)
head(e[,c(1,6,9)])
# ID 0[-] SFC="Ground or water surface" 0[-] SFC="Ground or water surface".1
#1 1 85100 11.775471
#2 2 54400 11.087971
#3 3 79300 9.900471
#4 4 49200 10.712971
#5 5 70800 9.212971
#6 6 56600 11.400471
Make sure you have the current (CRAN) version, or perhaps the development version that you can install like this:
install.packages('terra', repos='https://rspatial.r-universe.dev')
You can speed things up a lot by doing a single read from disk (by adding zero in this example)
e <- extract(r+0, lcc)
That is not always possible and I need to do some optimization behind the scences.
I would like to mask my background map using a shapefile representing a regional boundary. To do this I have read a spatial raster into Rstudio using read_osm
library(sp)
library(tmaptools)
HB_map <- spData::nz %>%
filter(Name=="Hawke's Bay") %>%
tmaptools::read_osm(type = "stamen-terrain")
I have then imported my shapefile
libary(sf)
Regional_boundary <- sf::st_read("regional_boundary.shp")
sf::st_crs(Regional_boundary)= 2193
Regional_boundary_sf_poly <- sf::st_transform(Regional_boundary, 27200) %>%
sf::st_cast(to="POLYGON")
I have a number of GIS data sets so I re-project my raster so it is in the same projection as my GIS data (i'm not sure if this bit is right)
test_map <- projectRaster(HB_map, crs ="+init=epsg:27200")
I then check the data projections are consistent
crs(Regional_boundary_sf_poly)
[1] "+proj=nzmg +lat_0=-41 +lon_0=173 +x_0=2510000 +y_0=6023150 +ellps=intl +towgs84=59.47,-5.04,187.44,0.47,-0.1,1.024,-4.5993 +units=m +no_defs"
crs(test_map)
+init=epsg:27200 +proj=nzmg +lat_0=-41 +lon_0=173 +x_0=2510000 +y_0=6023150 +datum=nzgd49
+units=m +no_defs +ellps=intl +towgs84=59.47,-5.04,187.44,0.47,-0.1,1.024,-4.5993
Now I apply my mask:
Library(raster)
test_map_mask <- raster::mask(test,Regional_boundary_sf_poly,inverse=FALSE,updatevalue=NA, updadateNA=FALSE)
And check the results
tmap::qtm(test_map_mask)
It all seems to work except the map colours no longer look like the original "stamen-terrain" but are instead different shades of orange". How do adjust the settings to make the map look like the original but with the mask?
Thanks for you help.
Kind regards,
Simon
NEW ANSWER
Try to get the tiles with cartography::getTiles, it seems to work. Note that you should have sf v0.9-0 and cartography v2.4-0 installed, both recently updated on CRAN. A complete list of tiles serves available on getTileshere.
library(sf)
library(cartography)
library(dplyr)
nz <- spData::nz %>% st_transform(27200)
#Get tile
HB_map <- cartography::getTiles(nz, type = "Stamen.Terrain")
class(HB_map)
#> [1] "RasterBrick"
#> attr(,"package")
#> [1] "raster"
st_crs(HB_map)$proj4string
#> [1] "+proj=nzmg +lat_0=-41 +lon_0=173 +x_0=2510000 +y_0=6023150 +ellps=intl +towgs84=59.47,-5.04,187.44,0.47,-0.1,1.024,-4.5993 +units=m +no_defs "
st_crs(nz)$proj4string
#> [1] "+proj=nzmg +lat_0=-41 +lon_0=173 +x_0=2510000 +y_0=6023150 +ellps=intl +towgs84=59.47,-5.04,187.44,0.47,-0.1,1.024,-4.5993 +units=m +no_defs "
library(raster)
test_map_mask <-
raster::mask(
HB_map,
nz,
)
#tmap
tmap::qtm(test_map_mask)
#> Warning: replacing previous import 'sf::st_make_valid' by
#> 'lwgeom::st_make_valid' when loading 'tmap'
#> Numeric values of test_map_mask interpreted as RGB values with max.value = 255. Run tm_shape(test_map_mask) + tm_raster() to visualize the data.
Created on 2020-04-04 by the reprex package (v0.3.0)
OLD ANSWER
You may want to check getPngLayer and pngLayer from cartography, basically with this you create a png file as a raster masked with a sf and plot it.
If your problem is the plotting, pngLayer Is basically a wrapper of raster::plotRGB.
Extra ball is that you can get also Stamen Terrain using getTiles , also on cartography, but make sure you install the latest version (released today on CRAN).
Reprex:
library(sf)
library(cartography)
library(dplyr)
HB_map <- spData::nz %>% getTiles(type = "Stamen.Terrain")
class(HB_map)
#> [1] "RasterBrick"
#> attr(,"package")
#> [1] "raster"
Regional_boundary <- spData::nz
st_crs(HB_map)$proj4string
#> [1] "+proj=tmerc +lat_0=0 +lon_0=173 +k=0.9996 +x_0=1600000 +y_0=10000000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs "
st_crs(Regional_boundary)$proj4string
#> [1] "+proj=tmerc +lat_0=0 +lon_0=173 +k=0.9996 +x_0=1600000 +y_0=10000000 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs "
library(raster)
test_map_mask <-
raster::mask(
HB_map,
Regional_boundary,
)
#> Warning: st_crs<- : replacing crs does not reproject data; use st_transform for
#> that
par(mar=c(0,0,0,0))
tilesLayer(test_map_mask)
plot(st_geometry(Regional_boundary), border = "red", lwd=2, add = TRUE)
Created on 2020-04-04 by the reprex package (v0.3.0)
Vignette: https://dieghernan.github.io/cartographyvignette/
Source code: https://github.com/riatelab/cartography/blob/master/R/tilesLayer.R
Thanks to the answer by dieghrnan I can now get some example data, and the below now runs fine for me
library(sf)
library(cartography)
library(raster)
library(spData)
nz <- spData::nz
HB <- cartography::getTiles(nz, type = "Stamen.Terrain")
HB_mask <- raster::mask(HB, nz)
plotRGB(HB_mask)
If you want a projection:
test <- projectRaster(HB_mask, crs ="+init=epsg:27200")
# or perhaps
# test <- projectRaster(HB_mask, method="ngb", crs ="+init=epsg:27200")
plotRGB(test)
old answer:
At what step do you loose the colors? my guess is at projectRaster? I cannot get rJava to work so I cannot run this, but I would try
colortable(test_map_mask) <- colortable(HB_map)
tmap::qtm(test_map_mask)
I am trying to plot data in R using mapView for the grid in the Pacific that crosses the longitude 180deg. The native crs is "+proj=merc +lon_0=150 +lat_ts=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0".
The example of equivalent data is:
test.coords<-as.data.frame(cbind(c(runif(15,-180,-130),runif(5,160,180)),runif(20,40,60)))
test.sp <- SpatialPointsDataFrame(coords = cbind(test.coords$V1,test.coords$V2), data = test.coords,
proj4string = CRS("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0"))
test.sp <- spTransform(test.sp,
"+proj=merc +lon_0=150 +lat_ts=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0")
plot(test.sp)
mapview(test.sp)
When using plot function, the points are centered, while using mapView they get split between two sides of the map (linked image). Can the view using mapView be centered on a different longitude?
Using native crs for this map does not help. I get an error.
mapview(plot, native.crs=TRUE)
Warning message:
sf layer is not long-lat data
Thank you
mapView image Pacific
Ok, so this is a semi-optimal solution to your question for several reasons:
this will only work for point data
it may not scale well for large point collections
projecting to another CRS may not work anymore (though that is not really related, as the issue at hand is about visualising)
For sets of lines or polygons, we would need another approach, but I am currently not aware of a sp/sf native solution to round-tripping coordinates of an object.
library(sp)
library(sf)
#> Linking to GEOS 3.6.1, GDAL 2.2.3, PROJ 4.9.3
library(mapview)
test.coords<-as.data.frame(cbind(c(runif(15,-180,-130),runif(5,160,180)),runif(20,40,60)))
test.sp <- SpatialPointsDataFrame(coords = cbind(test.coords$V1,test.coords$V2), data = test.coords,
proj4string = CRS("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0"))
test_sf = st_as_sf(test.sp)
shift = function(x) {
geom = st_geometry(x)
st_geometry(x) = st_sfc(
lapply(seq_along(geom), function(i) {
geom[[i]][1] = ifelse(geom[[i]][1] < 0, geom[[i]][1] + 360, geom[[i]][1])
return(geom[[i]])
})
, crs = st_crs(geom)
)
return(x)
}
mapview(shift(test_sf)) +
mapview(test_sf, col.regions = "orange")
Created on 2019-12-11 by the reprex package (v0.3.0)
I chose to use sf rather than sp for this because mapview uses sf internally and was hoping to find a general approach to this problem which could potentially be integrated.
Here is a workaround for polygons - it produces some warnings, but overall does the job.
# transform grid to standard latitude and longitude
# original grid in crs "+proj=merc +lon_0=150 +lat_ts=0 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0"
grid<-spTransform(grid,"+init=epsg:4326")
summary(coordinates(grid))
# V1 V2
# Min. :-179.8 Min. :37.17
# 1st Qu.:-168.8 1st Qu.:51.70
# Median :-154.3 Median :55.02
# Mean :-118.0 Mean :54.65
# 3rd Qu.:-131.4 3rd Qu.:58.31
# Max. : 179.8 Max. :65.56
out <- lapply(1:length(grid), function(i) grid[i, ])
for (i in 1:length(grid)) {
cds <- slot(slot(slot(out[[i]], "polygons")[[1]], "Polygons")[[1]], "coords")
cds_polygon <- Polygons(list(Polygon(cds)), ID="out.x")
out.x <- SpatialPolygons(list(cds_polygon))
proj4string(out.x) <- CRS("+init=epsg:4326")
if (cds[1,1]>0) { # depending to which side one want to move polygons that are on both sides of International Date Line
out.y <- elide(out.x, shift=c(-360,0))
} else {
out.y<-out.x}
if(i==1){grid.shifted<-out.y} else {
grid.shifted<-raster::union(grid.shifted,out.y)
}
}
# add data in the original grid
grid.shifted <- SpatialPolygonsDataFrame(grid.shifted, grid#data, match.ID = F)
mapview(grid.shifted)
I have a bunch of NZ Map Grid coordinates, which I want convert to lat/long. Based on this question, here is what I tried.
library(sp)
options(digits = 11) # to display to greater d.p.
Attempt 1:
proj4string <- "+proj=nzmg +lat_0=-41.0 +lon_0=173.0 +x_0=2510000.0
+y_0=6023150.0 +ellps=intl +units=m"
p <- proj4::project(c(2373200, 5718800), proj = proj4string, inverse=T)
Attempt 2
dat <- data.frame(id = c(1), x = c(2373200) , y = c(5718800))
sp::coordinates(dat) = ~x+y
sp::proj4string(dat) = CRS('+init=epsg:27200')
data_wgs84 <- spTransform(dat, CRS('+init=epsg:4326'))
print(data_wgs84)
If I run my coordinates through the linz coordinate conversion tool I get a slightly different result, which is the "true" result.
Results:
171.30179199 -43.72743909 # attempt 1 - ~200m off linz
171.30190004, -43.72577765 # attempt 2 - a few meters off linz
171.30189464, -43.72576664 # linz
Based on Mike T's answer I should be using a "distortion grid transformation method" and he links to a "nzgd2kgrid0005.gsb grid shift file".
My Question: Is it possible to do this conversion using R without downloading additional files (nzgd2kgrid0005.gsb)? I want to share my code with others without them having to download any additional files.
Any advice much appreciated.
Turns out it is pretty simple, if you have the rgdal package installed, the required nzgd2kgrid0005.gsb file is included and you don't need to download anything extra.
You just need to use the full PROJ.4 string as outlined in Mike T's answer.
dat <- data.frame(id = c(1), x = c(2373200) , y = c(5718800))
sp::coordinates(dat) = ~x+y
proj4string <- "+proj=nzmg +lat_0=-41 +lon_0=173 +x_0=2510000 +y_0=6023150
+ellps=intl +datum=nzgd49 +units=m +towgs84=59.47,-5.04,187.44,0.47,-0.1,1.024,-4.5993
+nadgrids=nzgd2kgrid0005.gsb +no_defs"
sp::proj4string(dat) = sp::CRS(proj4string)
data_wgs84 <- sp::spTransform(dat, sp::CRS('+init=epsg:4326'))
as.data.frame(data_wgs84)
id x y
1 171.3018946 -43.72576664
Which is the same as the output from the LINZ coordinate conversion tool. Hopefully this saves someone else a bit of time.