rast function error with MOD09GA hdf images in R - r

I have some MODIS images downloaded. When I try to create a SpatRaster from the hdf files, using terra rast function, it works perfectly for "MOD09A1" but it doesn´t work for "MOD09GA".
terra::rast("C:/Users/User/AppData/Local/Temp/_modis/MOD09GA.A2011025.h08v06.006.2015216102410.hdf")
Error: [rast] number of rows and/or columns do not match
What is the problem? Is there any other function I could use? Thanks!

The problem is that the file has subdatasets with different resolutions.
To get the file your are using
# remotes::install_github("rspatial/luna")
aoi <- c(-106, -105, 26, 27)
f <- luna::getModis("MOD09GA", "2011-01-25", "2011-01-26", aoi, download=TRUE, path=".", user="*", password="*")
f
[1] "./MOD09GA.A2011025.h08v06.006.2015216102410.hdf"
To see the subdatasets:
library(terra)
describe_sds(f)
id name desc nrow ncol nlyr
1 HDF4_EOS:EOS_GRID:...MODIS_Grid_1km_2D:num_observations_1km [1200x1200] num_observations_1km MODIS_Grid_1km_2D (8-bit integer) 1200 1200 1
(...)
12 HDF4_EOS:EOS_GRID:...MODIS_Grid_500m_2D:sur_refl_b01_1 [2400x2400] sur_refl_b01_1 MODIS_Grid_500m_2D (16-bit integer) 2400 2400 1
(...)
So you need to access the different subdatasets seperately, like this
b1 <- rast(f, 12)
b2 <- rast(f, 13)
b1
# class : SpatRaster
# dimensions : 2400, 2400, 1 (nrow, ncol, nlyr)
# resolution : 463.3127, 463.3127 (x, y)
# extent : -11119505, -10007555, 2223901, 3335852 (xmin, xmax, ymin, ymax)
# coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +R=6371007.181 +units=m +no_defs
# data source : MOD09GA.A2011025.h08v06.006.2015216102410.hdf:MODIS_Grid_500m_2D:sur_refl_b01_1
# names : sur_refl_b01_1
Perhaps followed by
bb <-c(b1, b2)
Or create a SpatDataSet like this
x <- sds(f, 12:22)
In the development version of terra you can then proceed and do
y <- collapse(x)
or something like the below to get a particular set of subdatasets (that have the same spatial resolution)
r <- rast(f, 12:18)
r
#class : SpatRaster
#dimensions : 2400, 2400, 7 (nrow, ncol, nlyr)
#resolution : 463.3127, 463.3127 (x, y)
#extent : -11119505, -10007555, 2223901, 3335852 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +R=6371007.181 +units=m +no_defs
#source(s) : MOD09GA.A2011025.h08v06.006.2015216102410.hdf:MODIS_Grid_500m_2D:sur_refl_b01_1
MOD09GA.A2011025.h08v06.006.2015216102410.hdf:MODIS_Grid_500m_2D:sur_refl_b02_1
MOD09GA.A2011025.h08v06.006.2015216102410.hdf:MODIS_Grid_500m_2D:sur_refl_b03_1
... and 4 more source(s)
#names : sur_refl_b01_1, sur_refl_b02_1, sur_refl_b03_1, sur_refl_b04_1, sur_refl_b05_1, sur_refl_b06_1, ...

Related

"Error: [mask] cannot create dataset" when trying to mask a SpatRaster with a SpatVector with Terra

Simply trying to use a vector (.shp) to mask a SpatRaster using terra::mask; get the following error
>Error: \[mask\] cannot create dataset
LCC84 <- rast("C:/Users_forest_VLCE2_1984.tif")
vec <- vect("C:/Users/Land_Management_Units.shp")
vec_proj <- project(vec, LCC84)
LCC84_masked <- terra::mask(LCC84, vec_proj)
Error: [mask] cannot create dataset
vec
#class : SpatVector
#geometry : polygons
#dimensions : 1, 8 (geometries, attributes)
#extent : -117.3165, -115.1691, 50.70613, 52.27127 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat NAD83 (EPSG:4269)
LCC84
#class : SpatRaster
#dimensions : 128340, 193936, 1 (nrow, ncol, nlyr)
#resolution : 30, 30 (x, y)
#extent : -2660911, 3157169, -851351.9, 2998848 (xmin, xmax, ymin, ymax)
#coord. ref.: Lambert_Conformal_Conic_2SP
#source : CA_forest_VLCE2_1984.tif
#name : CA_forest_VLCE2_1984
crs(LCC84, proj=TRUE)
[1] "+proj=lcc +lat_0=49 +lon_0=-95 +lat_1=49 +lat_2=77 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs"
You can use the following code
library(terra)
library(sf)
#Read the data
LCC84 <- rast("C:/Users_forest_VLCE2_1984.tif")
vec <- st_read("C:/Users/Land_Management_Units.shp")
#Convert the crs of shapefile
vec_proj <- sf::st_transform(vec, crs(LCC84))
#Masking the raster using the shapefile
LCC84_masked <- terra::mask(LCC84, vec_proj)
It works for me with the data you provided
library(terra)
#terra 1.6.51
v <- vect("Extent_BNP_Extact.shp")
r <- rast("CA_forest_VLCE2_1984.tif")
pv <- project(v, r)
z <- crop(r, pv, mask=T)
r
#class : SpatRaster
#dimensions : 128340, 193936, 1 (nrow, ncol, nlyr)
#resolution : 30, 30 (x, y)
#extent : -2660911, 3157169, -851351.9, 2998848 (xmin, xmax, ymin, ymax)
#coord. ref. : Lambert_Conformal_Conic_2SP
#source : CA_forest_VLCE2_1984.tif
#name : CA_forest_VLCE2_1984
v
# class : SpatVector
# geometry : polygons
# dimensions : 1, 2 (geometries, attributes)
# extent : 474065.5, 635666.6, 5613645, 5798288 (xmin, xmax, ymin, ymax)
# source : Extent_BNP_Extact.shp
# coord. ref. : NAD83 / UTM zone 11N (EPSG:26911)
# names : Shape_Leng Shape_Area
# type : <num> <num>
# values : 6.744e+05 2.828e+10
plot(z)
First cropping seems logical here because the entire dataset has ~25 billion cells. I also tried
mask(r, pv)
It took a while to run, but it worked. If it does not for you, my guess would be that you may not have sufficient disk-space in your temp folder. See terraOptions() for the location of the temp folder.
Also, you do the equivalent of
pv <- project(v, "EPSG:4269")
But that makes no sense, as your raster data do not have the EPSG:4269 coordinate reference system (lon/lat NAD83).

what is "extent" in raster::stack?

x <- raster::stack('D:/MODIS/MCD19A2.A2008355.h24v06.006.2018043191140.hdf')
band1 <- x[[1]]
band1
#class : RasterLayer
#band : 1 (of 16 bands)
#dimensions : 1200, 1200, 1440000 (nrow, ncol, ncell)
#resolution : 926.6254, 926.6254 (x, y)
#extent : 6671703, 7783654, 2223901, 3335852 (xmin, xmax, ymin, ymax)
#crs : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +R=6371007.181 +units=m +no_defs
#source : MCD19A2.A2008355.h24v06.006.2018043191140.hdf
#names : Optical_Depth_047_1
Does the extent means the MODIS data contained in hdf4 files?
"extent" refers to the spatial extent of the data. It shows the minimum and maximum x and y coordinates of the data.

R language, unreasonable (?) raster resolutions

Sorry for the very stupid question, but I'm really stuck here... I need to create a Digital Elevation Model for my study area. For this, I downloaded an SRTM (1 arc-seg resolution, freely available from the net) image, which comprises a region wider than my area of interest. The original raster has these characteristics:
class : RasterLayer
dimensions : 3601, 3601, 12967201 (nrow, ncol, ncell)
resolution : 0.0002777778, 0.0002777778 (x, y)
extent : -45.00014, -43.99986, -22.00014, -20.99986 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : s22_w045_1arc_v3.tif
names : s22_w045_1arc_v3
values : -32768, 32767 (min, max)
I need to (1) increase the resolution (initially of 30.75662 * 28.68392 m) to 1 * 1 m (that is, I really do not care about the exactitude of the elevations) and (2) crop a squared area of 2000 * 2000 m centered at a given coordinate. So, the first step I'm following is to re-project to UTM:
projection(r) <- "+proj=utm +zone=23 +datum=WGS84"
But the resolution units do not change after that:
class : RasterLayer
dimensions : 3601, 3601, 12967201 (nrow, ncol, ncell)
resolution : 0.0002777778, 0.0002777778 (x, y)
extent : -45.00014, -43.99986, -22.00014, -20.99986 (xmin, xmax, ymin, ymax)
crs : +proj=utm +zone=23 +datum=WGS84 +units=m +no_defs
source : s22_w045_1arc_v3.tif
names : s22_w045_1arc_v3
values : -32768, 32767 (min, max)
If I try to set the resolutions in meters manually, then generates an empty raster. Can anybody be so kind as to throw some light on me here?
You are changing (i.e. overwriting) the CRS, not projecting the raster. Usually, it is recommended to create a template raster with the CRS and the resolution you need and reproject the raster using this template.
See here an example, I am switching to terra for the analysis since it is a newer and faster package, but I would show also how to convert it back to raster format:
library(raster)
#> Loading required package: sp
# Faking your data
r <- raster(
nrows = 3601, ncols = 3601,
ext = extent(c(-45.00014, -43.99986, -22.00014, -20.99986))
)
values(r) <- seq(-32768, 32767, length.out = ncell(r))
r
#> class : RasterLayer
#> dimensions : 3601, 3601, 12967201 (nrow, ncol, ncell)
#> resolution : 0.0002777784, 0.0002777784 (x, y)
#> extent : -45.00014, -43.99986, -22.00014, -20.99986 (xmin, xmax, ymin, ymax)
#> crs : +proj=longlat +datum=WGS84 +no_defs
#> source : memory
#> names : layer
#> values : -32768, 32767 (min, max)
plot(r)
# End of faking data
# Change to terra, much faster
library(terra)
#> terra 1.5.21
r_terra <- terra::rast(r)
template <- terra::project(r_terra, "+proj=utm +zone=23 +datum=WGS84")
# Change to the desired res
res(template) <- c(2000, 2000)
# Reproject
r_terra_reproj <- terra::project(r_terra, template)
r_terra_reproj
#> class : SpatRaster
#> dimensions : 56, 52, 1 (nrow, ncol, nlyr)
#> resolution : 2000, 2000 (x, y)
#> extent : 499985.4, 603985.4, -2433195, -2321195 (xmin, xmax, ymin, ymax)
#> coord. ref. : +proj=utm +zone=23 +datum=WGS84 +units=m +no_defs
#> source : memory
#> name : layer
#> min value : -32371.95
#> max value : 32182.81
terra::plot(r_terra_reproj)
# Back to RasterLayer
r_reproj <- raster(r_terra_reproj)
r_reproj
#> class : RasterLayer
#> dimensions : 56, 52, 2912 (nrow, ncol, ncell)
#> resolution : 2000, 2000 (x, y)
#> extent : 499985.4, 603985.4, -2433195, -2321195 (xmin, xmax, ymin, ymax)
#> crs : +proj=utm +zone=23 +datum=WGS84 +units=m +no_defs
#> source : memory
#> names : layer
#> values : -32371.95, 32182.81 (min, max)
Created on 2022-06-10 by the reprex package (v2.0.1)

Error in mosaic of rasters from different extent using terra package in R?

I have a folder that contains around 191 GeoTIFF files (each file is a different DEM (elevation) tile of a much larger area). I want to merge all the tiles into one raster file. I am using the terra package and was successfully able to load each raster and aggregate them from 2 metre resolution to 30 metre resolution. However, when running the mosaic function to merge them all, I run into an error (see error message below). I have been able to run the mosaic function on a smaller subset of just three tiles, but when I scale up to all the files, this becomes an issue.
By calling the summary of the rasters (see below), the aggregation does slightly change the extent - could this be the issue? resample might be an option, but each individual raster has a different extent and I'm not exactly sure how to implement this fix.
Not sure a sample data set would help since I know the functions work. I am running this code on a high-performance cluster so it's not very efficient to run small batches of code.
library(terra)
files <- as.list(list.files("./DEM_tiles", full.names = TRUE))
raster.list <- lapply(files, rast)
for(i in 1:length(raster.list)){
raster.list[[i]] <- aggregate(raster.list[[i]], fact = 15)
}
raster.mosaic <- do.call(mosaic, raster.list)
> Error: [mosaic] internal error: extents do not match ()
> Execution halted
Below is an example of what two of the tiles look like:
### Before Aggregation
[[1]]
class : SpatRaster
dimensions : 25000, 25000, 1 (nrow, ncol, nlyr)
resolution : 2, 2 (x, y)
extent : -1800000, -1750000, -6e+05, -550000 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs
source : 35_23_1_1_2m_v3.0_reg_dem.tif
name : 35_23_1_1_2m_v3.0_reg_dem
[[2]]
class : SpatRaster
dimensions : 25000, 25000, 1 (nrow, ncol, nlyr)
resolution : 2, 2 (x, y)
extent : -1800000, -1750000, -550000, -5e+05 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs
source : 35_23_1_2_2m_v3.0_reg_dem.tif
name : 35_23_1_2_2m_v3.0_reg_dem
### After Aggregation
[[1]]
class : SpatRaster
dimensions : 1667, 1667, 1 (nrow, ncol, nlyr)
resolution : 30, 30 (x, y)
extent : -1800000, -1749990, -600010, -550000 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs
source : memory
name : 35_23_1_1_2m_v3.0_reg_dem
min value : -15.62178
max value : 233.6489
[[2]]
class : SpatRaster
dimensions : 1667, 1667, 1 (nrow, ncol, nlyr)
resolution : 30, 30 (x, y)
extent : -1800000, -1749990, -550010, -5e+05 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs
source : memory
name : 35_23_1_2_2m_v3.0_reg_dem
min value : -15.27713
max value : 243.0772
The error was because the rasters were not aligned, and due to a bug. I now get
library(terra)
#terra version 1.2.1
crs <- "+proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m"
r1 <- rast(nrow=1667, ncol=1667, ext=c(-1800000, -1749990, -600010, -550000), crs=crs)
r2 <- rast(nrow=1667, ncol=1667, ext=c(-1800000, -1749990, -550010, -5e+05), crs=crs)
values(r1) <- 1:ncell(r1)
values(r2) <- 1:ncell(r2)
m <- mosaic(r1, r2)
#Warning message:
#[mosaic] rasters did not align and were resampled
m
#class : SpatRaster
#dimensions : 3334, 1667, 1 (nrow, ncol, nlyr)
#resolution : 30, 30 (x, y)
#extent : -1800000, -1749990, -600010, -499990 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs
#source : memory
#name : lyr.1
#min value : 1
#max value : 2778889
Also
#Elia's suggestion is a good work-around:
r1 <- writeRaster(r1, "test1.tif", overwrite=TRUE)
r2 <- writeRaster(r2, "test2.tif", overwrite=TRUE)
v <- vrt(c("test1.tif", "test2.tif"), "test.vrt", overwrite=TRUE)
v
#class : SpatRaster
#dimensions : 3334, 1667, 1 (nrow, ncol, nlyr)
#resolution : 30, 30 (x, y)
#extent : -1800000, -1749990, -600020, -5e+05 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=stere +lat_0=90 +lat_ts=70 +lon_0=-45 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs
#source : test.vrt
#name : test
#min value : 1
#max value : 2778889

Reproject MODIS data using R (results in NAs or no spatial extent)

I am using GLASS albedo data stored here for pre-2000 (AVHRR) data and here for post-2000 data (MODIS). My end goal is to create a raster stack of each month that contains white sky albedo data from 1982-2015. The problem I have run into is that the MODIS and AVHRR data are in different spatial reference systems and I can't seem to reproject them to be in the same system.
I convert from hdf to tif using R like this:
fileavhrr <- ".../GLASS02B05.V04.A1990161.2018062.hdf"
filemodis<-".../GLASS02B06.V04.A2013169.2017128.hdf"
gdal_translate(get_subdatasets(filemodis)[10], dst_dataset =
".../modis.tif")
gdal_translate(get_subdatasets(fileavhrr)[8], projwin = c(-180,90,180,50), dst_dataset = ".../avhrr.tif") #ideally I'd only like data north of 50 degrees
avhrr<- raster(".../avhrr.tif")
#class : RasterLayer
#dimensions : 800, 7200, 5760000 (nrow, ncol, ncell)
#resolution : 0.05, 0.05 (x, y)
#extent : -180, 180, 50, 90 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
#values : -32768, 32767 (min, max)
modis<- raster(".../modis.tif")
#class : RasterLayer
#dimensions : 3600, 7200, 25920000 (nrow, ncol, ncell)
#resolution : 154.4376, 308.8751 (x, y)
#extent : -20015109, -18903159, 8895604, 10007555 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181
+b=6371007.181 +units=m +no_defs
#values : -32768, 32767 (min, max)
Here are things I have tried:
1.) Use the MODIS Reprojection Tool. For whatever reason, this tool seems to think the subdatasets of the MODIS .hdf files are only one tile (the upper left most tile, tile 0,0) and not the global dataset. My understanding is that the MODIS data are global (not in tiles?), so I do not know why the MRT is doing this.
2.) Use the raster package in R.
projectedMODIS <- projectRaster(modis,avhrr,method="bilinear")
This returns a raster with values that are all NA:
class : RasterLayer
dimensions : 800, 7200, 5760000 (nrow,> ncol, ncell)
resolution : 0.05, 0.05 (x, y)
extent : -180, 180,> 50, 90 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
values : NA, NA (min, max)
3.) Use the gdalUtils package in R:
gdalwarp(srcfile=get_subdatasets(filemodis)[10], dstfile= ".../gdalMODIS_avhrr.tif", s_srs = crs(modis), t_srs =crs(avhrr) )
This returns a raster with essentially no spatial extent.
gdalMODISavhrr<-raster(".../gdalMODIS_avhrr.tif")
#class : RasterLayer
#dimensions : 357, 12850, 4587450 (nrow, ncol, ncell)
#resolution : 0.02801551, 0.02801573 (x, y)
#extent : -180, 179.9993, 79.99838, 90 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=longlat +ellps=clrk66 +no_defs
#values : -32768, 32767 (min, max)
Any ideas on why reprojecting this MODIS data is so difficult?
I have not tried this, but from looking at the package gdalUtils, the function gdalwarp() might do what you need?

Resources