I am trying to calculate the spatial correlation between two rasters. I have two large rasters with the same extent, resolution, etc
class : RasterLayer
dimensions : 45598, 53241, 2427683118 (nrow, ncol, ncell)
resolution : 30, 30 (x, y)
extent : 273366.8, 1870597, 367780.7, 1735721 (xmin, xmax, ymin, ymax)```
These layers have massive NAs cells
I tried to use terra::focalCor with the stack of those layers.
corr=focalCor(layerstack, w=9, cor)
But I have this issue
Error in v[[j - 1]] <- t(sapply(1:nrow(Y), function(i, ...) fun(X[i, ], :
more elements supplied than there are to replace
Any ideas or suggestions?
Cheers
It would have been easier to provide a specific answer with actual data provided to be able to reproduce your issue, but in this case it seems like you imported your gridded data using raster::raster() creating a RasterLayer object, but according to ?focalCor, x has clearly to be a SpatRaster with at least two layers.
So, try terra::rast(c("grid_1.tif", "grid_2.tif")) |> terra::focalCor(w = 9, cor) instead.
Edit:
Thanks for your reprex. I dared to reduce dimensions and modify the extent a little bit in order to reduce processing time:
library(terra)
r <- rast(ncols = 100, nrows = 100,
xmin = 0, xmax = 25, ymin = 0, ymax = 25,
crs = "epsg:4326")
r1 <- init(r, fun = runif)
r2 <- init(r, fun = runif)
r_stack <- c(r1, r2)
r_stack_cor_5 <- focalCor(r_stack, w = 5, cor)
r_stack_cor_5
#> class : SpatRaster
#> dimensions : 100, 100, 1 (nrow, ncol, nlyr)
#> resolution : 0.25, 0.25 (x, y)
#> extent : 0, 25, 0, 25 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326)
#> source : memory
#> name : lyr1
#> min value : -0.6476946
#> max value : 0.6948594
r_stack_cor_25 <- focalCor(r_stack, w = 25, cor)
r_stack_cor_25
#> class : SpatRaster
#> dimensions : 100, 100, 1 (nrow, ncol, nlyr)
#> resolution : 0.25, 0.25 (x, y)
#> extent : 0, 25, 0, 25 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326)
#> source : memory
#> name : lyr1
#> min value : -0.1020998
#> max value : 0.1045798
I used fun = cor instead of function(x, y) cor(x, y) but the result is the same according to all.equal(). However, your example seems to work - and I'm failing to recognize the issue at the moment.
Related
I have a raster raster_coarse at 0.25 resolution
class : SpatRaster
dimensions : 26, 31, 1 (nrow, ncol, nlyr)
resolution : 0.25, 0.25 (x, y)
extent : -87.75, -80, 24.5, 31 (xmin, xmax, ymin, ymax)
coord. ref. : lon/lat WGS 84 (EPSG:4326)
source : memory
name : layer1
min value : 0.08839285
max value : 0.11517857
I have another raster raster_fine at finer resolution
class : SpatRaster
dimensions : 2377, 2758, 1 (nrow, ncol, nlyr)
resolution : 0.002777778, 0.002777778 (x, y)
extent : -87.6361, -79.97499, 24.39723, 31.00001 (xmin, xmax, ymin, ymax)
coord. ref. : lon/lat WGS 84 (EPSG:4326)
source : memory
name : finer_res
min value : 0
max value : 1
Here's what I am trying to do. I want to populate every pixel in raster_fine with the values from raster_coarse. The way I approached this is:
# check the resolution difference
res_drop <- res(raster_coarse)[1]/res(raster_fine)[1] # 89.99999.
# disaggregate the raster_coarse
raster_coarse <- terra::disagg(raster_coarse, fact = res_drop)
# convert to points
raster_coarse_df <- as.data.frame(raster_coarse, xy = T, na.rm = TRUE)
raster_coarse_pts <- terra::vect(raster_coarse_df, geom = c("x", "y"), crs(raster_fine))
# create a blank canvas
canvas <- terra::rast(xmin = ext(raster_fine)[1],
xmax = ext(raster_fine)[2],
ymin = ext(raster_fine)[3],
ymax = ext(raster_fine)[4],
resolution = res(raster_fine),
crs = crs(raster_fine))
# populate the blank canvas with raster_coarse_pts
raster_coarse <- terra::rasterize(x = raster_coarse_pts , y = canvas, field = "layer1", fun = max, touches = T, background = NA)
raster_coarse
class : SpatRaster
dimensions : 2377, 2758, 1 (nrow, ncol, nlyr)
resolution : 0.002777778, 0.002777778 (x, y)
extent : -87.6361, -79.97499, 24.39723, 31.00001 (xmin, xmax, ymin, ymax)
coord. ref. : lon/lat WGS 84 (EPSG:4326)
source : memory
name : lyr.1
min value : 0.08839285
max value : 0.11517857
plot(raster_coarse)
I am not able to understand why the vertical white lines are coming from?
Example data
library(terra)
r <- rast(nrow=26, ncol=31, ext=c(-87.75, -80, 24.5, 31))
f <- geodata::gadm("USA", level=1, path=".")
f <- f[f$NAME_1 == "Florida", ]
values(r) <- runif(ncell(r))
r <- mask(r, f)
fine <- rast(nrow=2377, ncol=2758, ext=c(-87.6361, -79.97499, 24.39723, 31.00001))
The solution is to use resample
rr <- resample(r, x, "near")
Instead of the nearest neighbor you could use, for example, bilinear interpolation:
rr <- resample(r, x)
The white lines occur because with your method there will be cells that are not covered. To better understand that you could crop out a small area and plot the points on top of the raster.
I have a SpatRaster with 100 layers, each corresponding to a result of a bootstrapping analysis (all slightly different variations but pretty much the same). I want to reorder each pixel so that the first layer is the smallest of each respective pixel, second layer is second smallest, etc.
It can be converted to a data.frame with 102 columns (x, y, result1, ...), but I don't know how to order each row of a data.frame by size and exclude columns at the same time.
You can use sort to order the values of each grid cell. Illustrated below
Example data
library(terra)
r <- rast(ncol=5, nrow=5)
values(r) <- 30
x <- rast(c(a=r, b=r/10, c=r/100))
x
#class : SpatRaster
#dimensions : 5, 5, 3 (nrow, ncol, nlyr)
#resolution : 72, 36 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84
#sources : memory
# memory
# memory
#names : a, b, c
#min values : 30, 3, 0.3
#max values : 30, 3, 0.3
Solution
y <- sort(x)
y
#class : SpatRaster
#dimensions : 5, 5, 3 (nrow, ncol, nlyr)
#resolution : 72, 36 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84
#source : memory
#names : a, b, c
#min values : 0.3, 3, 30
#max values : 0.3, 3, 30
I found out the other day, more or less by chance, that it is possible to query layers from SpatRaster objects based on the time attribute in general (c.f. here), e.g based on years (r["2017"]) and yearmonths (r["2017-10"]).
Now I wanted to deep-dive a little bit into this because of the great flexibility you receive when working with spatio-temporal data. Unfortunately, I seem to be failing from the beginning and can't reproduce the behaviour from before because of the following error: [subset] no (valid) layer selected
library(terra)
#> terra 1.6.3
r <- rast(ncols = 10, nrows = 10, nlyr = 365)
time(r) <- seq(from = as.POSIXlt("2001-01-01", tz = "UTC"),
to = as.POSIXlt("2001-12-31", tz = "UTC"),
by = "day")
r
#> class : SpatRaster
#> dimensions : 10, 10, 365 (nrow, ncol, nlyr)
#> resolution : 36, 18 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> time : 2001-01-01 to 2001-12-31 UTC
time(r) |> class()
#> [1] "POSIXct" "POSIXt"
r["2001"]
#> Error: [subset] no (valid) layer selected
time(r) <- as.Date("2002-01-01") + 0:364
r
#> class : SpatRaster
#> dimensions : 10, 10, 365 (nrow, ncol, nlyr)
#> resolution : 36, 18 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> time (days) : 2002-01-01 to 2002-12-31
time(r) |> class()
#> [1] "Date"
r["2002"]
#> Error: [subset] no (valid) layer selected
I had a look at ?time as well as at the docs at rspatial.org but was not able to find relevant documentation related to indexing.
What am I missing here?
You are mixing up layer names (that may look like a time-stamp) with an actual time-stamp.
Here is how you can use time-stamps (using Date here, as it is a bit less verbose)
library(terra)
r <- rast(ncols = 10, nrows = 10, nlyr = 365)
time(r) <- as.Date("2001-01-01") + 0:364
head(names(r))
# "lyr.1" "lyr.2" "lyr.3" "lyr.4" "lyr.5" "lyr.6"
You can subset layers by name like this
r[["lyr.1"]]
Note the double brackets for sub-setting layers, although you can use single brackets when using a name as opposed to a numerical index.
To subset by time, you can do
r[[time(r) == as.Date("2001-06-01")]]
#class : SpatRaster
#dimensions : 10, 10, 1 (nrow, ncol, nlyr)
#resolution : 36, 18 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84
#time (days) : 2001-06-01
With dates, you do not even need to use "as.Date"
r[[time(r) >= "2001-12-01"]]
#class : SpatRaster
#dimensions : 10, 10, 31 (nrow, ncol, nlyr)
#resolution : 36, 18 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84
#time (days) : 2001-12-01 to 2001-12-31
I have a SpatRaster object in R called IDW3, estimated using IDW interpolation method. I have nlyr = 240, containing 12 months x 20 years. I need to calculate the long-term mean monthly rainfall from the layers, so that I get nlyr = 12 at the end, in which each layer represents one calendar month (Jan - Dec).
I have tried using the code below, following this thread calculating long term daily means from a RASTER in R, but I want to verify the code I used.
Any thoughts and comments please?
idw3
#> class : SpatRaster
#> dimensions : 723, 449, 240 (nrow, ncol, nlyr)
#> resolution : 100, 100 (x, y)
#> extent : 624698.7, 669598.7, 640507.8, 712807.8 (xmin, xmax, ymin, ymax)
#> coord. ref. :
#> sources : May 1998_masked_idw3.asc
#> May 1999_masked_idw3.asc
#> May 2000_masked_idw3.asc
#> ... and 237 more source(s)
#> names : Jan 1998, Jan 1999, Jan 2000, Jan 2001, #> Jan 2002, Jan 2003, ...
#> min values : ? , ? , ? , ? , ? , ? , ...
#> max values : ? , ? , ? , ? , ? , ? , ...
## CALCULATE THE LONGTERM MONTHLY MEANS
# get the months substring
month.ltm <- substr(my, 1,3)
# calculate the ltm using tapp funtion in terra
idw3.ltm <- tapp(idw3, month.ltm, mean)
names(idw3.ltm)
#> [1] "May" "Apr" "Aug" "Jan" "Sep" "Jul" "Jun" "Feb" "Dec"
#> [10] "Nov" "Oct" "Mar"
You can use tapp for that. Here are some example data
library(terra)
r <- rast(ncols=10, nrows=10, nlyr=24)
values(r) <- 1:size(r)
If the data are ordered by year, and then by month, you can now do
x <- tapp(r, 1:12, mean)
In other cases you may have to create another index to match the layers that need to be combined. If your data has a time-stamp, there are some shortcuts. In this case you can use index="months"
time(r) <- as.Date("2000-01-15") + 0:23 * 30.5
y <- tapp(r, "months", mean)
It's been some time since you posted this, but I'd try to solve this for the next one asking. Since your example does not seem to be fully reproducible, let my use my own data for this purpose.
I used GPCC v2022 data - the last two decades of monthly data at 0.5° resolution to be precise - from German Weather Service.
library(terra)
#> terra 1.5.21
# define filenames
files <- c("full_data_monthly_v2022_2001_2010_05.nc",
"full_data_monthly_v2022_2011_2020_05.nc")
# create SpatRaster object
nc_data <- rast(files)
# get variable names
varnames(nc_data)
#> [1] "precip" "numgauge"
#> [3] "infilled_numgauges" "interpolation_error"
#> [5] "interpolation_error_infilled" "diff_new_old_method"
#> [7] "precip" "numgauge"
#> [9] "infilled_numgauges" "interpolation_error"
#> [11] "interpolation_error_infilled" "diff_new_old_method"
# subset dataset to precipitation only
nc_precip <- nc_data["precip"]
# sneak peek
nc_precip
#> class : SpatRaster
#> dimensions : 360, 720, 240 (nrow, ncol, nlyr)
#> resolution : 0.5, 0.5 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> sources : full_data_monthly_v2022_2001_2010_05.nc:precip (120 layers)
#> full_data_monthly_v2022_2011_2020_05.nc:precip (120 layers)
#> varnames : precip (gpcc full data monthly product version 2022, precipitation per grid)
#> precip (gpcc full data monthly product version 2022, precipitation per grid)
#> names : precip_1, precip_2, precip_3, precip_4, precip_5, precip_6, ...
#> unit : mm/month, mm/month, mm/month, mm/month, mm/month, mm/month, ...
#> time : 2001-01-01 to 2020-12-01
As you can see, this dataset is quite similar to yours in terms of information at least (except for crs, extent and resolution). A stack of SpatRaster objects with nlyr = 240 containing monthly precipitation data. What differs most notably is the time attribute ranging from 2001-01-01 to 2020-12-01.
However, basically I approached your issue constructing an appropriate time-based index vector as input to tapp using fun = mean:
# get timestamps from your SpatRaster object
tst <- terra::time(nc_precip)
# calculate monthly means
lta <- tapp(nc_precip, index = 1:12, fun = mean)
# tidy your names a little bit
names(lta) <- format(tst, "%B") |> unique()
# inspect result
lta
#> class : SpatRaster
#> dimensions : 360, 720, 12 (nrow, ncol, nlyr)
#> resolution : 0.5, 0.5 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> sources : memory
#> memory
#> memory
#> ... and 9 more source(s)
#> names : January, February, March, April, May, June, ...
#> min values : 0, 0, 0, 0, 0, 0, ...
#> max values : 979.1880, 852.0020, 720.6245, 739.8225, 884.2455, 1590.6805, ...
The result seems plausible from my point of view, but since this is the first time I used tapp, I want to make sure the function behaves as expected by re-calculating manually:
# init an empty list for temporary storage purposes
lta <- list()
# loop monthly and calculate the long-term mean
for (i in 1:12) {
idx <- seq(from = i, by = 12, length.out = n_years)
lta[[i]] <- nc_precip[[idx]] |> terra::mean()
}
# create a SpatRast object with nlyr = 12
lta <- terra::rast(lta)
lta
#> class : SpatRaster
#> dimensions : 360, 720, 12 (nrow, ncol, nlyr)
#> resolution : 0.5, 0.5 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> sources : memory
#> memory
#> memory
#> ... and 9 more source(s)
#> names : mean, mean, mean, mean, mean, mean, ...
#> min values : 0, 0, 0, 0, 0, 0, ...
#> max values : 979.1880, 852.0020, 720.6245, 739.8225, 884.2455, 1590.6805, ...
Same results, phew.
Edit:
After some weird behaviour yesterday which cannot be reproduced today I can confirm that using index = months.abb gives you the same results as using index = "months" (as suggested by Robert below in the comments):
tapp(nc_precip, index = month.abb, fun = mean)
#> class : SpatRaster
#> dimensions : 360, 720, 12 (nrow, ncol, nlyr)
#> resolution : 0.5, 0.5 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> source : memory
#> names : Jan, Feb, Mar, Apr, May, Jun, ...
#> min values : 0, 0, 0, 0, 0, 0, ...
#> max values : 979.1880, 852.0020, 720.6245, 739.8225, 884.2455, 1590.6805, ...
tapp(nc_precip, index = "months", fun = mean)
#> class : SpatRaster
#> dimensions : 360, 720, 12 (nrow, ncol, nlyr)
#> resolution : 0.5, 0.5 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> source : memory
#> names : X1, X2, X3, X4, X5, X6, ...
#> min values : 0, 0, 0, 0, 0, 0, ...
#> max values : 979.1880, 852.0020, 720.6245, 739.8225, 884.2455, 1590.6805, ...
I have a categorical raster layer that I need to one-hot encode, as I'm using neural networks to run a species distribution model (for a class) and it only works on continuous predictors. I've already run the model with the one-hot encoded data itself, but in order to predict onto a map, the raster itself needs to be one-Hot encoded in the same way.
In the RStoolbox package, oneHotEncode() should do what I need it to do, but I can't get it to work
nn.raster<-oneHotEncode(newraster$NLCD_2016_Land_Cover_oreg, classes=values, background = 0, foreground = 1, na.rm = FALSE)
Error message:
Error in .calcTest(x[1:5], fun, na.rm, forcefun, forceapply) :
cannot use this function. Perhaps add '...' or 'na.rm' to the function arguments?
Has anybody used this function and can help me troubleshoot? I think the problem is coming from the class's argument. My categories are numerical (from the national land cover raster), which is why they show up as "values" in the raster info. Do I need to do something to reclassify them? I think I'm naming them wrong but I'm not sure how.
After giving a quick look at RStoolbox::oneHotEncode, I am under the impression that it does what raster::layerize also does.
library(raster)
r <- raster(nrow=20, ncol=20)
values(r) <- c(rep(NA, 50), rep(1:5, 70))
r
#class : RasterLayer
#dimensions : 20, 20, 400 (nrow, ncol, ncell)
#resolution : 18, 9 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#crs : +proj=longlat +datum=WGS84 +no_defs
#source : memory
#names : layer
#values : 1, 5 (min, max)
b <- layerize(r)
b
#class : RasterBrick
#dimensions : 20, 20, 400, 5 (nrow, ncol, ncell, nlayers)
#resolution : 18, 9 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#crs : +proj=longlat +datum=WGS84 +no_defs
#source : memory
#names : X1, X2, X3, X4, X5
#min values : 0, 0, 0, 0, 0
#max values : 1, 1, 1, 1, 1
Which is equivalent to terra::separate
library(terra)
r <- rast(nrow=5, ncol=5)
values(r) <- rep(c(1:4, NA), each=5)
b <- separate(r)