I need to project longitude/latitude coordinates in the terra package, but I don't believe it is working correctly, as I am trying to extract data from a raster with this projection, but the data is not being extracted correctly.
Here's my lon/lat points and the code I am using to try to project them.
latlon_df <- structure(list(Lon = c(-103.289, -96.6735, -96.9041, -96.76864,
-102.4694, -96.6814, -97.7504, -99.6754, -96.4802, -103.0007,
-96.8897, -101.8539, -103.9717, -101.253, -99.1134, -96.5849,
-98.0301, -99.9537, -99.4601, -99.7122, -103.8278, -98.931, -102.1081,
-101.7162, -100.115, -101.3448, -100.7805, -103.5606, -96.5302,
-99.4156, -103.281, -100.0063, -97.9928, -100.7208, -98.5289,
-96.762, -96.9218, -97.1024, -103.3793, -101.0841, -102.6745,
-96.9188, -97.5154, -100.7435, -98.6938), Lat = c(45.5194, 44.3099,
43.0526, 44.3252, 45.5183, 43.7316, 45.6796, 45.4406, 44.7154,
44.0006, 43.7687, 43.9599, 43.4737, 44.9875, 45.0292, 44.0867,
45.5735, 44.9895, 44.5256, 43.5938, 43.7343, 45.7163, 45.9189,
43.1672, 45.6716, 45.9154, 45.7963, 44.6783, 44.5073, 43.7982,
43.3784, 44.2912, 43.3841, 43.2002, 44.8579, 43.5048, 43.5033,
45.1055, 44.4245, 45.4167, 44.5643, 44.304, 45.2932, 43.5601,
43.7321)), class = "data.frame", row.names = c(NA, -45L))
latlons <- terra::vect(latlon_df,geom=c('Lon','Lat'),crs="+proj=longlat")
lcc <- terra::project(latlons,"+proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs")
var_df <- terra::extract(grib_data,lcc)[,-1]
The raster data (grib_data) I am using comes from here (it is way too big for me to put on here). https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2
I am not sure what I am doing wrong here, as I have used this method previously, and it seemed to work fine. Any help would be wonderful.
EDIT: The specific problem I am having is that I am not getting any different values for each lon/lat pair. The value for each variable is different, but all the values for the stations (different lon/lats are the same).
Why do you think it has to do with the projection? Either way, it appears to work for me.
url <- "https://nomads.ncep.noaa.gov/pub/data/nccf/com/hrrr/prod/hrrr.20210612/conus/hrrr.t00z.wrfsubhf00.grib2"
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
url <- paste0(url, ".idx")
if (!file.exist(basename(url))) download.file(url, basename(url), mode="wb")
library(terra)
r <- rast("hrrr.t00z.wrfsubhf00.grib2")
r
#class : SpatRaster
#dimensions : 1059, 1799, 49 (nrow, ncol, nlyr)
#resolution : 3000, 3000 (x, y)
#extent : -2699020, 2697980, -1588806, 1588194 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=lcc +lat_0=38.5 +lon_0=262.5 +lat_1=38.5 +lat_2=38.5 +x_0=0 +y_0=0 +R=6371229 +units=m +no_defs
#source : hrrr.t00z.wrfsubhf00.grib2
#names : 0[-] ~here", 0[-] ~tops", 0[-] ~here", 0[-] ~here", 0[-] ~face", 1000[~ound", ...
You can check of the points overlap with the raster data
plot(r, 1)
points(lcc)
And extract. It takes very long with grib files, but it does appear to work
e <- extract(r, lcc)
head(e[,c(1,6,9)])
# ID 0[-] SFC="Ground or water surface" 0[-] SFC="Ground or water surface".1
#1 1 85100 11.775471
#2 2 54400 11.087971
#3 3 79300 9.900471
#4 4 49200 10.712971
#5 5 70800 9.212971
#6 6 56600 11.400471
Make sure you have the current (CRAN) version, or perhaps the development version that you can install like this:
install.packages('terra', repos='https://rspatial.r-universe.dev')
You can speed things up a lot by doing a single read from disk (by adding zero in this example)
e <- extract(r+0, lcc)
That is not always possible and I need to do some optimization behind the scences.
Related
Similar to this question:
I would like to know how to do the reverse and save an .img raster image into a USGS DEM format.
Based on GDAL docs, it seems like it would be possible but when I run rgdal::getGDALDriverNames() in R I get the following:
name long_name create copy isRaster
139 USGSDEM USGS Optional ASCII DEM (and CDED) FALSE TRUE TRUE
which seems to imply that it won't create these files?
I was hoping to do something like:
library(raster)
# read
img <- raster("Raster_100ft_2022_10_18.img")
# convert to DEM
writeRaster(img, 'test.dem')
But raster doesn't seem to recognize that output format.
Is there some other method to save as USGS DEM files?
Thanks
For me it works with terra. If that's proper "USGSDEM" file, that's another question. From gdal reference it should save the file as well: https://gdal.org/drivers/raster/usgsdem.html
f <- system.file("ex/elev.tif", package="terra")
r <- terra::rast(f)
terra::writeRaster(r, filename = "test.dem", filetype = "USGSDEM", overwrite = TRUE)
raster::raster("test.dem")
#> class : RasterLayer
#> dimensions : 90, 95, 8550 (nrow, ncol, ncell)
#> resolution : 0.008333333, 0.008333333 (x, y)
#> extent : 5.741667, 6.533333, 49.44167, 50.19167 (xmin, xmax, ymin, ymax)
#> crs : +proj=longlat +datum=WGS84 +no_defs
#> source : test.dem
#> names : elevation
#> values : 141, 547 (min, max)
Created on 2022-10-20 with reprex v2.0.2
I am struggling to open a NetCDF file and convert it into a raster using R. The
data is supposed to be on a regular grid of 25 km by 25 km. It contains sea
ice concentration in the Arctic.
library(terra)
#> terra 1.5.21
library(ncdf4)
file <- "~/Downloads/data_sat_Phil_changt_grid/SIC_SMMR_month_2015.nc"
I am getting a warning about the extent not found.
r <- rast(file)
#> Error in R_nc4_open: No such file or directory
#> Warning: [rast] GDAL did not find an extent. Cells not equally spaced?
We can see that there is a problem with the coordinates/extent.
r
#> class : SpatRaster
#> dimensions : 448, 304, 12 (nrow, ncol, nlyr)
#> resolution : 0.003289474, 0.002232143 (x, y)
#> extent : 0, 1, 0, 1 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> source : SIC_SMMR_month_2015.nc:sic
#> varname : sic
#> names : sic_1, sic_2, sic_3, sic_4, sic_5, sic_6, ...
I can open the nc file with nc_open() and I see that the coordinates are present.
nc <- nc_open(file)
names(nc$var)
#> [1] "lat" "lon" "sic"
lat <- ncvar_get(nc, "lat")
lon <- ncvar_get(nc, "lon")
dim(lat)
#> [1] 304 448
dim(lon)
#> [1] 304 448
dim(r)
#> [1] 448 304 12
Is it possible to assemble this data (the SIC values and the coordinates) to create a SpatRaster?
The nc file can be downloaded here: https://easyupload.io/pfth0s
Created on 2022-05-20 by the reprex package (v2.0.1)
The data are gridded, but the file does not specify the coordinates, nor the coordinate reference system. The file specifies the lon/lat values associated with the cells, but does not help us much, as these are clearly not on a regular grid. That is easy to see from plot(r)
NAflag(r) = -9999
plot(r,1)
And also from
p = cbind(as.vector(lon), as.vector(lat))
plot(p, cex=.1, xlab="lon", ylab="lat")
So what you need to find out, is which coordinate reference system (crs) is used, clearly some kind of polar crs. And what the extent of the data set is.
From the website you point to, I take it we can use:
crs(r) = "+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +k=1 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs"
ext(r) = c(-3850000, 3750000, -5350000, 5850000)
r
#class : SpatRaster
#dimensions : 448, 304, 12 (nrow, ncol, nlyr)
#resolution : 25000, 25000 (x, y)
#extent : -3850000, 3750000, -5350000, 5850000 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs
#source : SIC_SMMR_month_2015.nc:sic
#varname : sic
#names : sic_1, sic_2, sic_3, sic_4, sic_5, sic_6, ...
The results look good:
g = geodata::gadm("Greenland", level=0, path=".")
gg = project(g, crs(r))
plot(r,1)
lines(gg)
But this is of course not a good way to do such things; the ncdf file should have contained all the metadata required.
Through attempting to get an intersect result from a single point and a single polygon I have found what I believe can only be a bug in the R raster package intersect function.
I have 1 polygon and 1 point, and use intersect as follows:
intersect(a_point, a_polygon)
Where a_point contains an id attribute. This fails with the error:
Error in j[, 2] : incorrect number of dimensions
However, if I reverse the arguments and do:
intersect(a_polygon, a_point)
It works fine, but doesn't return the id from the point shape file as part of the result which I require. This is expected behaviour, so fine but I need it to work the other way around.
To rule out there being some peculiarity with my polygon or point data, I created a single polygon and single point spatial object and tested the same hypothesis, and the same result occurred as above with these 'raw' objects.
The following is the code for generating these two 'fake' objects for completeness and so that it can be reproduced:
test_list_x = list(530124, 530125) #For when I use 2 points
test_list_y = list(176949, 176950) #For when I use 2 points
data_frame_object = data.frame(530124, 176950)
names(data_frame_object) = c("Longitude", "Latitude")
coordinates(data_frame_object)=~Longitude+Latitude
proj4string(data_frame_object)=CRS("+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +datum=OSGB36 +units=m +no_defs +ellps=airy +towgs84=446.448,-125.157,542.060,0.1502,0.2470,0.8421,-20.4894")
fake_point_shape_object=SpatialPointsDataFrame(data_frame_object, data.frame(id=1:length(data_frame_object)))
coords = matrix( nrow=5, ncol=2)
coords[1,1] = 530106.8
coords[1,2] = 176953.3
coords[2,1] = 530127.5
coords[2,2] = 176953.3
coords[3,1] = 530127.5
coords[3,2] = 176933.3
coords[4,1] = 530106.8
coords[4,2] = 176933.3
coords[5,1] = 530106.8
coords[5,2] = 176953.3
my_fake_polygon = Polygon(coords)
polygon_list = list(my_fake_polygon)
polygon_set <- lapply(seq_along(polygon_list), function(i) Polygons(list(polygon_list[[i]]), i ))
new_polygons <- SpatialPolygons(polygon_set)
new_polygons#proj4string = CRS("+proj=tmerc +lat_0=49 +lon_0=-2 +k=0.9996012717 +x_0=400000 +y_0=-100000 +datum=OSGB36 +units=m +no_defs +ellps=airy +towgs84=446.448,-125.157,542.060,0.1502,0.2470,0.8421,-20.4894")
df <- data.frame("1")
names(df) = "id"
my_fake_polygon <- SpatialPolygonsDataFrame(new_polygons,df)
Now here's the thing, if I create 2 points next to each other (so they are both within the polygon) instead of just one, it works fine, no error. Suggesting there is a bug associated with intersection between 1 point and 1 polygon, WHEN the point carries an attribute to be returned in the intersection process.
You might ask why do you actually need to have the attribute returned if there is just one point, this is because it is an iterative process in which it may not be just one point, it could be none or many.
I would appreciate somebody explaining this error or confirming my findings.
Here are your example data in a more concise way.
library(raster)
pnt <- SpatialPoints(cbind(530124, 176950))
pol <- spPolygons(matrix(c(530106.8, 530127.5, 530127.5, 530106.8, 530106.8, 176953.3, 176953.3, 176933.3, 176933.3, 176953.3), ncol=2))
Now illustrate the problem.
intersect(pol, pnt)
#class : SpatialPolygons
#features : 1
#extent : 530106.8, 530127.5, 176933.3, 176953.3 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
# this fails
intersect(pnt, pol)
#Loading required namespace: rgeos
#Error in j[, 2] : incorrect number of dimensions
# but it works with two points!
intersect(bind(pnt, pnt), pol)
#class : SpatialPoints
#features : 2
#extent : 530124, 530124, 176950, 176950 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
This was another drop=TRUE bug caused by the R default of "dropping" matrices to vectors when a single row is selected. This was fixed in raster version 2.6-11 (not on CRAN yet).
Sorry i can't answer your intersect bug question, but it might be simpler for now to use sp::over to return polygon attributes to points
# dummy polygon
xym <- as.matrix(data.frame(x=c(16.48438,17.49512,24.74609,22.59277,16.48438),
y=c(59.73633,55.12207,55.03418,61.14258,59.73633)))
# make into SpatialPolygon
p = Polygon(xym)
ps = Polygons(list(p),1)
sps = SpatialPolygons(list(ps))
# Promote to SPDF and give an attribute
SPDF = SpatialPolygonsDataFrame(sps, data.frame(N = "hello", row.names = 1))
# make 2 points, one inside the polygon and one outside
p <- data.frame(x=c(16,18),y=c(58,58))
coordinates(p) <- ~x + y
# plot to check
plot(sps)
plot(p,add=T)
# perform the over, returns a named vector for every point in the SpatialPoints
res <- unname(over(p,SPDF))
# promote points to SpatialPointsDataFrame and put in new polygon attribute
data <- data.frame(ID=row.names(p),pol=res)
sp <- SpatialPointsDataFrame(p, data)
I have been trying to use the 'PhenologyRaster' function of the greenbrown package to model the growing season of my study area. However, everytime I run the function, I get empty outputs (e.g. the SOS.2016 layer will show as NA). My question is the following: am I having issues because I am running the function on one single year of data, or because Landsat time series are somewhat irregular (i.e. frequency of ~30 scenes per year)?
I am using the following piece of code to run the PhenologyRatser function:
PhenoTest = PhenologyRaster(landsat2016,start=c(2016,1,3),end=c(2016,12,20),freq=24,approach="Deriv",min.mean=-0.5,tsgf='TSGFspline',interpolate=TRUE)
The function is applied on a raster stack with the following characteristics:
class : RasterBrick
dimensions : 526, 591, 310866, 18 (nrow, ncol, ncell, nlayers)
resolution : 30, 30 (x, y)
extent : 604965, 622695, 4208175, 4223955 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=utm +zone=10 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
data source : in memory
names : X2016.01.03, X2016.01.19, X2016.02.04, X2016.03.07, X2016.03.23, X2016.04.24, X2016.05.10, X2016.05.26, X2016.06.27, X2016.07.13, X2016.07.29, X2016.08.14, X2016.08.30, X2016.09.15, X2016.10.01, ...
min values : -0.1964, NA, -0.5382, NA, -0.4696, -0.2197, -0.2803, -0.4274, -0.4827, -0.2631, -0.5256, -0.4856, -0.5631, -0.3204, -0.5512, ...
max values : 0.1714, NA, 0.2425, NA, 0.2061, 0.5173, 0.4583, 0.2470, 0.3629, 0.5165, 0.2981, 0.2802, 1.6199, 0.5016, 0.3007, ...
I also had the same issue. What I did was that I created a dummy series for three years and then the data were successfully run.
b.1 <- brick(r.1, r.2, r.3, r.4, r.5, r.6, r.7, r.8, r.9, r.10, r.11, r.12)
b.2 <- stack(b.2, b.2, b.2)
pheno.test <- PhenologyRaster(b.2, start=c(2016,1), freq=12, approach="White",
tsgf="TSGFspline", interpolate=T)
I'm an absolutely new one in R (it's my second day), it's a little bit difficult for me. So, sorry for this question, but I really need help.
I've already read
R - Plotting netcdf climate data
Plot NetCDF variable-grid data file using ggplot2: "Vector is too large" error
but still don't understand.
I've got to plot NMF2 from this set of data in geo coordinates https://drive.google.com/file/d/0B6IqnlmRMSpcNFBXWWlha1JUUzQ/view?usp=sharing
The easiest way to do it was:
>library(raster)
>varRaster<-raster("F18-SSUSI_EDR-NIGHT-DISK_DD.20150107_SN.26920-00_DF.NC", varname="NMF2_DISK")
>cols <- rev(rainbow(255))
>plot(varRaster, col=cols)
Now I've got a plot, but the grid is not a geo one. So there are two questions:
What I have to do to get the correct grid?
How is it possible to add the world map layer?
Thank you in advance for your help.
UPGRADE
Taking a better look to my data I found out that I have to change missval to NA and also I need to transpose. The best variant I've found is this tutorial and everything is almost all right but the data is now strained all ower the map...And it has to be just a wide track of sattelites scan.
I can't put here the image, because my reputation is very low:(
library(raster)
library(ncdf)
data = open.ncdf("F18-SSUSI_EDR-NIGHT-DISK_DD.20150107_SN.26920-00_DF.NC")
lon=get.var.ncdf(data,"PIERCEPOINT_NIGHT_LONGITUDE")
lat=get.var.ncdf(data,"PIERCEPOINT_NIGHT_LATITUDE")
lon<-lon-180
dim(lat)
nmf2=get.var.ncdf( ex.nc, "NMF2_DISK")
nmf2[nmf2 == -1e+30] <- NA
dim(nmf2)
nmf2_un=get.var.ncdf( ex.nc, "NMF2_DISK_UNCERTAINTY")
nmf2_un[nmf2_un == -1e+30] <- NA
nmf2_1 <- raster(t(nmf2)[ncol(nmf2):1, ])
nmf2_un_1 <- raster(t(nmf2_un)[ncol(nmf2_un):1, ])
w <- brick(nmf2_1, nmf2_un_1)
projection(w) <- CRS("+init=epsg:4326")
extent(w) <- c(min(lon), max(lon), min(lat), max(lat))
plot(w[[1]])
library(maptools)
data(wrld_simpl)
plot(wrld_simpl, add = TRUE)
I would be very glad if somebody can tell me what's wrong!
UPDATE 2
Tried to use raster only. Still have a mistake with projection
library(raster)
inputfile <- "F18-SSUSI_EDR-NIGHT-DISK_DD.20150107_SN.26920-00_DF.NC"
lat <- raster(inputfile, varname="PIERCEPOINT_NIGHT_LATITUDE")
lon <- raster(inputfile, varname="PIERCEPOINT_NIGHT_LONGITUDE")
plat <- rasterToPoints(lat)
plon <- rasterToPoints(lon)
lonlat <- cbind(plon[,3], plat[,3])
lonlat <- SpatialPoints(lonlat, proj4string = CRS("+proj=longlat +datum=WGS84"))
extent(lonlat)
#class : Extent
#xmin : 0.008961686
#xmax : 359.983
#ymin : -84.95161
#ymax : 89.68419
pr <- raster(inputfile, varname="NMF2_DISK")
extent(pr) <- extent(lonlat)
pr
#class : RasterLayer
#dimensions : 408, 13, 5304 (nrow, ncol, ncell)
#resolution : 27.69031, 0.4280289 (x, y)
#extent : 0.008961686, 359.983, -84.95161, 89.68419 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
#data source : C:\Users\Svetlana\Science\GUVI\R\SSUSI\F18-SSUSI_EDR-NIGHT- DISK_DD.20150107_SN.26920-00_DF.NC
#names : NMF2_DISK
#zvar : NMF2_DISK
r <- projectRaster(pr, crs=CRS("+proj=longlat +datum=WGS84"))
#Error in projectRaster(pr, crs = CRS("+proj=longlat +datum=WGS84")) :
# input projection is NA
What's wrong?
And the other question is how to work with missval while using raster? I mean that in spite of using NA there is 8 000 000 for missed values of data. What I have to do with this?