I have ~100 different 4-band satellite imagery .tif rasters that have floating-point 32 bit depth. I need to convert these to 16-bit unsigned in R while scaling pixel values (not just trashing high values), but I have no idea where to begin for even a single raster, let alone the whole batch. Any help would be much appreciated!
Edit with more info: I searched the raster package documentation for bit, pixel, and depth keywords without much luck. Judging by min/max pixel values information found under dataType(), I want to go from FLT4S (32-bit floating point) to INT2U (16 bit). I tried setting datatype with writeRaster() as shown in the example there, but the output was just a black and white image rather than normal satellite imagery.
image
class : RasterStack
dimensions : 4300, 8909, 38308700, 4 (nrow, ncol, ncell, nlayers)
resolution : 3, 3 (x, y)
extent : 691032, 717759, 57492, 70392 (xmin, xmax, ymin, ymax)
crs : +proj=utm +zone=17 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
names : image.1, image.2, image.3, image.4
min values : 7.244503e-02, 8.278998e-02, 2.286164e-05, 8.571137e-02
max values : 0.5347134, 0.3522218, 0.4896736, 0.7308348
dataType(image) #[1] "FLT4S" "FLT4S" "FLT4S" "FLT4S"
image2 = writeRaster(image, 'new.tif', datatype='INT2U', overwrite=TRUE, format="GTiff")
dataType(image2) #[1] "INT2U"
image2
class : RasterBrick
dimensions : 4300, 8909, 38308700, 4 (nrow, ncol, ncell, nlayers)
resolution : 3, 3 (x, y)
extent : 691032, 717759, 57492, 70392 (xmin, xmax, ymin, ymax)
crs : +proj=utm +zone=17 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
source : new.tif
names : new.1, new.2, new.3, new.4
min values : 0, 0, 0, 0
max values : 1, 0, 0, 1
In your example you have real values between 0 and 1. By writing to an integer type, these are all truncated to 0. If you want to write to INT2U (unsigned byte) you could first scale the values to be between 0 and 255.
Example data
library(raster)
b <- brick(system.file("external/rlogo.grd", package="raster"))
image <- clamp(b/255, 0, 0.6)
#class : RasterBrick
#dimensions : 77, 101, 7777, 3 (nrow, ncol, ncell, nlayers)
#resolution : 1, 1 (x, y)
#extent : 0, 101, 0, 77 (xmin, xmax, ymin, ymax)
#crs : +proj=merc +lon_0=0 +k=1 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs
#source : memory
#names : red, green, blue
#min values : 0, 0, 0
#max values : 0.6, 0.6, 0.6
What you do (truncation)
fname <- paste0(tempfile(), ".tif")
x <- writeRaster(image, fname, datatype='INT2U', overwrite=TRUE)
x
#min values : 0, 0, 0
#max values : 1, 1, 1
But note the difference when you use rounding
fname <- paste0(tempfile(), ".tif")
y <- round(image)
z <- writeRaster(y, fname, datatype='INT2U', overwrite=TRUE)
s <- stack(x[[1]], z[[1]])
plot(s)
Now with some scaling
maxv <- 65535
r <- round(image * maxv)
fname <- paste0(tempfile(), ".tif")
s <- writeRaster(r, fname, datatype='INT2U', overwrite=TRUE)
#s
#min values : 0, 0, 0
#max values : 39321, 39321, 39321
With your data you would get max values of
round(maxv * c(0.5347134, 0.3522218, 0.4896736, 0.7308348 ))
#[1] 35042 23083 32091 47895
You can also choose to set the max values of all layers to maxv, to keep more variation (but make the values no longer comparable with other data) --- more relevant if you were using a smaller range such as 0-255.
ss <- round(maxv * image / maxValue(image))
#names : red, green, blue
#min values : 0, 0, 0
#max values : 65535, 65535, 65535
the above works because the lowest values are zero; if you have negative values you would do
ss <- image - minValue(image)
ss <- round(maxv * ss / maxValue(ss))
In other cases you might want to use clamp. So you need to decide how to scale
What I showed is linear scaling. There are other ways. For example, there is the is also the scale method, to improve the statistical distribution of numbers. That may be relevant; but it depends on what your goals are.
Note. You do not say why you do this, and that is OK. But if it is to save hard disk space, you can use compression instead (see ?writeRaster)
Related
I have a raster of 30m cell size, with either value 0 or 1. I try to aggregate this into a 1000m cell size. I can then see the sum of '1's, in a 1000m cell size raster.
But the Aggregate function only lets me use a round number as a factor, so the closest I get is with factor 33x (to cell size 990m).
I would then need to resample 990m to 1000m but then obviously the values will shift and be incorrect.
Is there a way to use a decimal factor such as 33,3333, or better yet, use a raster with resolution 1000m as a template for the aggregation?
FOR ILLUSTRATION:
Here's an example of one aggregated cell sourced from 30x30m cells with 0/1 values, now in an aggregate raster at 990x990m cell size:
The two original cells are correctly aggregated into a 990x990m cell, and the sum of all values that were of interest (value '1') is 2. You can see these same two cells in the resampled cell on the right, in the top of the cell. But due to the resampling, the raster has shifted and includes more of the 30x30m cells. Yet, the cell value for the now resampled aggregate raster is still 2, while this is an incorrect value and should be 5.
You can use resample.
Example data:
library(terra)
r <- rast(crs="+proj=utm +zone=1", resolution=30, xmin=0, ymin=0, xmax=1020, ymax=1020)
values(r) <- 1:ncell(r)
I first aggregate to a value close to what you want (in this example, I go from 30 to 100 m spatial resolution), and then use resample
a <- aggregate(r, 3, mean)
rr <- rast(crs="+proj=utm +zone=1", resolution=100, xmin=0, ymin=0, xmax=1000, ymax=1000)
b <- resample(r, rr)
r
#class : SpatRaster
#dimensions : 34, 34, 1 (nrow, ncol, nlyr)
#resolution : 30, 30 (x, y)
#extent : 0, 1020, 0, 1020 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=utm +zone=1 +datum=WGS84 +units=m +no_defs
#source : memory
#name : lyr.1
#min value : 1
#max value : 1156
b
#class : SpatRaster
#dimensions : 10, 10, 1 (nrow, ncol, nlyr)
#resolution : 100, 100 (x, y)
#extent : 0, 1000, 0, 1000 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=utm +zone=1 +datum=WGS84 +units=m +no_defs
#source : memory
#name : lyr.1
#min value : 68.89777
#max value : 1103.335
You could also first disaggregate and then aggregate. With the above example you could do
d <- disaggregate(r, 3)
da <- aggregate(d, 10)
da
#class : SpatRaster
#dimensions : 11, 11, 1 (nrow, ncol, nlyr)
#resolution : 100, 100 (x, y)
#extent : 0, 1100, -80, 1020 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=utm +zone=1 +datum=WGS84 +units=m +no_defs
#source : memory
#name : lyr.1
#min value : 43
#max value : 1093
The solution was pretty simple, and I highly suspect the phrasing of my question was just not very clear, apologies for that.
When using the aggregate function (whether it is by mean, sum,...) you cannot resample afterwards and expect the agreggated values to still be correct. The values will be preserved, but because cells shift a little or a lot due to resampling, they will not be correct.
The trick as such, is incredibly simple: resample before you aggregate, not afterwards.
I have a categorical raster layer that I need to one-hot encode, as I'm using neural networks to run a species distribution model (for a class) and it only works on continuous predictors. I've already run the model with the one-hot encoded data itself, but in order to predict onto a map, the raster itself needs to be one-Hot encoded in the same way.
In the RStoolbox package, oneHotEncode() should do what I need it to do, but I can't get it to work
nn.raster<-oneHotEncode(newraster$NLCD_2016_Land_Cover_oreg, classes=values, background = 0, foreground = 1, na.rm = FALSE)
Error message:
Error in .calcTest(x[1:5], fun, na.rm, forcefun, forceapply) :
cannot use this function. Perhaps add '...' or 'na.rm' to the function arguments?
Has anybody used this function and can help me troubleshoot? I think the problem is coming from the class's argument. My categories are numerical (from the national land cover raster), which is why they show up as "values" in the raster info. Do I need to do something to reclassify them? I think I'm naming them wrong but I'm not sure how.
After giving a quick look at RStoolbox::oneHotEncode, I am under the impression that it does what raster::layerize also does.
library(raster)
r <- raster(nrow=20, ncol=20)
values(r) <- c(rep(NA, 50), rep(1:5, 70))
r
#class : RasterLayer
#dimensions : 20, 20, 400 (nrow, ncol, ncell)
#resolution : 18, 9 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#crs : +proj=longlat +datum=WGS84 +no_defs
#source : memory
#names : layer
#values : 1, 5 (min, max)
b <- layerize(r)
b
#class : RasterBrick
#dimensions : 20, 20, 400, 5 (nrow, ncol, ncell, nlayers)
#resolution : 18, 9 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#crs : +proj=longlat +datum=WGS84 +no_defs
#source : memory
#names : X1, X2, X3, X4, X5
#min values : 0, 0, 0, 0, 0
#max values : 1, 1, 1, 1, 1
Which is equivalent to terra::separate
library(terra)
r <- rast(nrow=5, ncol=5)
values(r) <- rep(c(1:4, NA), each=5)
b <- separate(r)
I am working with a marine species that has a shallow distribution, and I would like to delimit modelling of several ascii layers (e.g.SST, SSS) to a 50m depth threshold along the coastline using a bathymetric dataset (e.g. Bio-Oracle, MARSPEC or GEBCO). I am working in R. I do not want bathymetry to be part of the model though.
Does anyone know how to do this?
This is the bathymetric raster (with values down to 100m depth):
class : RasterLayer
dimensions : 600, 420, 252000 (nrow, ncol, ncell)
resolution : 0.08333333, 0.08333333 (x, y)
extent : -20, 15, 10, 60 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
data source : in memory
names : bathy
values : -100, -1 (min, max)
AND THIS IS RASTER STACK
class : RasterStack
dimensions : 600, 420, 252000, 4 (nrow, ncol, ncell, nlayers)
resolution : 0.08333333, 0.08333333 (x, y)
extent : -20, 15, 10, 60 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
names : SST.min, SST.max, SST.range, BO_dissox
min values : -0.120, 10.940, 0.000, 4.052
max values : 26.640, 30.320, 21.460, 8.058
Cheers,
Rita
library(raster)
library(marmap)
library(rgdal)
# your coordinate reference setup
this_crs <- crs("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0")
this_extent <- extent(-20, 15, 10, 60)
# a simple raster stack for example purpose
r <- raster(ncol=420, nrow=600, ext = this_extent, crs = this_crs)
values(r) <- (1:ncell(r))/100
r_SST <- sin(r)
r_SSS <- r + runif(ncell(r), 0, 100)^2
r_stack <- stack(r_SST, r_SSS)
# Get some sample bathymetric data to match your stack
# the marmaps package is handy but you could use your own
download_bathy <- getNOAA.bathy(-20, 15, 10, 60, res = 5, keep = TRUE)
bath_mask <- marmap::as.raster(download_bathy)
bath_mask <- projectRaster(from = bath_mask, to = r_stack)
# clip your raster stack to the desired bathymetric limits
bath_mask[bath_mask > 0 | bath_mask < -50] <- NA
clipped_stack <- mask(r_stack, bath_mask)
# check the result with a plot
plot(clipped_stack)
First use SDMPlay:::delim.area in which the first layer of the stack is used to delimit the 100m depth.
Then isolate the bathymetry layer with subset.
User raster:::mask to delimit all rasters to bathymetric area.
And finally remove bathymetry from stack, raster:::dropLayer.
Thanks to all and specially to Guillaumot Charlene!
Rita
I would like to find the pixels coordinates in a rasterStack that have the same pattern of a vector. Below is a simple example.
> s<-r<-raster(ncol=5,nrow=5)
> r[]<-round(runif(ncell(r)))
> s[]<-2
> rs<-stack(r,s)
> rs
class : RasterStack
dimensions : 5, 5, 25, 2 (nrow, ncol, ncell, nlayers)
resolution : 72, 36 (x, y)
extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
names : layer.1, layer.2
min values : 0, 2
max values : 1, 2
If I have a vector vet<-c(0,2), which the pixels coordinates that have value 0 in the first layer and value 2 in the second one?
Playing with which might work. I'm sticking with matrices; you can rasterize them later :-)
foo<-which(r==0,arr.ind=TRUE)
s<-matrix(ncol=5,nrow=5)
set.seed(10)
r[]<-round(runif(ncell(r)))
s[1:3,]<-2
foo<-which(r==0,arr.ind=TRUE)
bar<-which(s==2,arr.ind=TRUE)
Then look for identical rows in foo and bar, perhaps by concatenating rows, or calculating foo-bar and identifying zero rows.
EDIT: a better way, giving you the coordinates right away.
which(s==2 & r==0, arr.ind=TRUE)
I have been trying to create a new raster object that contains only a couple of values from an existing raster.
I am using the class raster found here: https://www.ga.gov.au/products/servlet/controller?event=FILE_SELECTION&catno=71071.
class : RasterLayer dimensions : 14902, 19161, 285537222 (nrow, ncol, ncell)
resolution : 0.002349, 0.002349 (x, y)
extent : 110, 155.0092, -45.0048, -9.999999 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0
values : G:\Spatial data\environmental_layers\Australian data\Land cover\Class\DLCDv1_Class.tif
min value : 1
max value : 34
I have tried:
pr <- rasterToPoints(r) # but the file is to big
and
s <- r[r>30 & r<33] # but the file is to big
and
rc <- reclass(r, c(-Inf,30,NA, 31,32, 1, 33,Inf,NA))
which produces a raster with properties:
class : RasterLayer
dimensions : 14902, 19161, 285537222 (nrow, ncol, ncell)
resolution : 0.002349, 0.002349 (x, y)
extent : 110, 155.0092, -45.0048, -9.999999 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0
values : C:\Users\Adam\AppData\Local\Temp\R_raster_tmp\raster_tmp_61931056968.grd
min value : 1
max value : 33
I thought this would produced a raster layer with values of NA and 1, but it has 33 values. I have been struggling to find a way to 'extract by attribute' using R on such a large file. Does anyone have suggestions of how I could do this?
reclassify() may work for you with a very large raster, but you need to specify the "is" "becomes" matrix correctly. Though I am not exactly sure from your question whether this is in fact your goal when you say "raster extract."
However, here is how to do the reclassification:
For example:
## Create sample raster with values from 0 to 9
r <- raster(nrow=100, ncol=100)
r[] <- trunc(runif(ncell(r))*10)
## Create reclassification table
## Set values 0 to 4 equal to 1
## Set values 5 to 9 equal to NA
isBecomes <- cbind(c(0, 1, 2, 3, 4, 5, 6, 7, 8, 9),
c(1, 1, 1, 1, 1, NA, NA, NA, NA, NA))
r2 <- reclassify(r, rcl=isBecomes)
I have not tested this in a raster too large to fit in memory, however I believe that reclassify() may be able to handle this.