Raster: Calculation on RasterStack only if not NA in other RasterLayer - r

Question

I have a RasterStack s1 consisting of 400 layers with data from an island. The extent of the raster is cropped to the extent of the island, but due to its irregular shape, only around 20% of the pixels are actually land area and have data values; the other 80% are water and NA.
I also have a land-water-mask lwm (RasterLayer), where land is coded as 1 and water as NA.
I would like to do different kinds of cell-based calculations on s1, but noticed that these take a long time to finish. To speed things up, the calculations should only be carried out for cells that are land area, whereas water areas should always be NA. In pseudo-code:
for each cell:
if cell is land
do calculation
if cell is water
return(NA)
An requirement is memory-safety.
Here is some sample data to illustrate the problem:
library(raster)
# generate data
lwm <- raster(nrow = 5, ncol = 5)
lwm[] <- c(rep(NA, 10), rep(1, 5), rep(NA, 10))
r1 <- raster(nrow = 5, ncol = 5)
r1[] <- runif(ncell(r1)) * 10
r2 <- raster(nrow = 5, ncol = 5)
r2[] <- runif(ncell(r2)) * 10
s1 <- stack(r1, r2)
s1 <- mask(s1, lwm)
# this works, but all NA-values on water are also unnecessarily evaluated
calc(s1, function(x) {sum(!is.na(x))})

Answer 1

After some playing around, I finally found a solution which works very well in my case, and which takes off quite a nice amount from the overall processing time:
library(raster)
# generate data
lwm <- raster(nrow = 50, ncol = 50)
lwm[] <- 1
# replace 80% with NA values
lwm[sample(1:ncell(lwm), round(0.8 * ncell(lwm)))] <- NA
r1 <- raster(lwm)
r1[] <- runif(ncell(r1))
r1_list <- replicate(400 , r1)
s1 <- stack(r1_list)
s1 <- mask(s1, lwm)
# this works, but all NA-values on water are also unnecessarily evaluated
system.time(r_sum1 <- calc(s1, function(x) {sum(x)}))
#user system elapsed
#0.14 0.00 0.14
## new approach:
# stack land-water-mask with RasterStack
s1_lwm <- stack(lwm, s1)
# function to check if first element of vector is NA; if yes, return NA; if no, do calculation
fun1 <- function(y) {
if (!is.na(y[1])) {
y = y[-1]
return(sum(y))
} else {
return(NA)
}
}
system.time(
r_sum2 <- calc(s1_lwm, fun = fun1)
)
# user system elapsed
# 0.4 0.0 0.4
# results are identical
identical(r_sum1[], r_sum2[])

Answer 2

That's a tricky one and unfortunetely I don't have a straight forward solution for you.
You can either do multiple crops of the island (i.e. 2-3) to minimise NA values and do the calculcations separately on each cropped raster and mosaic the results.
Or another option is to do a parallel calculation, which will speed up the process significantly:
#initialize cluster
#number of cores to use for clusterR function (max recommended: ncores - 1)
beginCluster(3)
#calculation
result <- clusterR(s1, calc, args=list(fun=function(x) {sum(!is.na(x))}))
#end cluster
endCluster()
Since you asked for a memory-safe solution, you should look how much RAM is being allocated when you only a singe core and then estimate how many cores you can run your calculation on, so you won't run out of memory.
Good luck! Hope it helps.