Subscript out of bounds error - r

I have an issue when running this script in Rstudio where I get the error
Error in TMX03_site[, 24] : subscript out of bounds
I have 5 scripts in total like this two of which work fine and three come up with the same error.
I have looked through other post related to this issue and have tried the suggestions but have found no solution to my problem yet. I hope someone out there could trouble shoot this for me and find out where I am going wrong.
Many thanks
setwd("G:/All ArcGIS/tmax03/03")
m03_1 <- raster("1320_setnull")
m03_2 <- raster("2320_setnull")
m03_3 <- raster("3320_setnull")
m03_4 <- raster("4320_setnull")
m03_5 <- raster("5320_setnull")
m03_6 <- raster("6320_setnull")
m03_7 <- raster("7320_setnull")
m03_8 <- raster("8320_setnull")
m03_9 <- raster("9320_setnull")
m03_10 <- raster("10320_setnull")
m03_11 <- raster("11320_setnull")
m03_12 <- raster("12320_setnull")
m03_13 <- raster("1380_setnull")
m03_14 <- raster("2380_setnull")
m03_15 <- raster("3380_setnull")
m03_16 <- raster("4380_setnull")
m03_17 <- raster("5380_setnull")
m03_18 <- raster("6380_setnull")
m03_19 <- raster("7380_setnull")
m03_20 <- raster("8380_setnull")
m03_21 <- raster("9380_setnull")
m03_22 <- raster("10380_setnull")
m03_23 <- raster("11380_setnull")
m03_24 <- raster("12380_setnull")
m03.stacked<-stack(m03_1,m03_2,m03_3,m03_4,m03_5,m03_6,m03_7,m03_8,m03_10,
m03_11,m03_12,m03_13,m03_14,m03_15,m03_16,m03_17,m03_18,m03_19,m03_20,
m03_21,m03_22,m03_23,m03_24)
writeRaster(m03.stacked, "m03.stacked.nc",overwrite=TRUE)
write03 <- 'm03.stacked.nc' # link to nc
site_03 <- read.csv("C:/Users/Vaughn/Desktop/suteri/tmax/sites_03.csv")
for (x in 1:nrow(site_03)) {
longlat <- c(site_03[x,1],site_03[x,2])
}
#convert to a matrix and transpose by
site_03 <- t(matrix(longlat))
TMX03 <- brick(write03)
TMX03_site <- extract(TMX03, site_03)
TMX03_site1 <- TMX03_site[, 1]
TMX03_site2 <- TMX03_site[, 2]
TMX03_site3 <- TMX03_site[, 3]
TMX03_site4 <- TMX03_site[, 4]
TMX03_site5 <- TMX03_site[, 5]
TMX03_site6 <- TMX03_site[, 6]
TMX03_site7 <- TMX03_site[, 7]
TMX03_site8 <- TMX03_site[, 8]
TMX03_site9 <- TMX03_site[, 9]
TMX03_site10 <- TMX03_site[, 10]
TMX03_site11 <- TMX03_site[, 11]
TMX03_site12 <- TMX03_site[, 12]
TMX03_site13 <- TMX03_site[, 13]
TMX03_site14 <- TMX03_site[, 14]
TMX03_site15 <- TMX03_site[, 15]
TMX03_site16 <- TMX03_site[, 16]
TMX03_site17 <- TMX03_site[, 17]
TMX03_site18 <- TMX03_site[, 18]
TMX03_site19 <- TMX03_site[, 19]
TMX03_site20 <- TMX03_site[, 20]
TMX03_site21 <- TMX03_site[, 21]
TMX03_site22 <- TMX03_site[, 22]
TMX03_site23 <- TMX03_site[, 23]
TMX03_site24 <- TMX03_site[, 24]
> str(TMX03)
Formal class 'RasterBrick' [package "raster"] with 12 slots
..# file :Formal class '.RasterFile' [package "raster"] with 13 slots
.. .. ..# name : chr "G:\\All ArcGIS\\tmax03\\03\\m03.stacked.nc"
.. .. ..# datanotation: chr "FLT4S"
.. .. ..# byteorder : chr "little"
.. .. ..# nodatavalue : num -3.4e+38
.. .. ..# NAchanged : logi FALSE
.. .. ..# nbands : int 23
.. .. ..# bandorder : chr "BIL"
.. .. ..# offset : int 0
.. .. ..# toptobottom : logi FALSE
.. .. ..# blockrows : int 0
.. .. ..# blockcols : int 0
.. .. ..# driver : chr "netcdf"
.. .. ..# open : logi FALSE
..# data :Formal class '.MultipleRasterData' [package "raster"] with 14
.. .. ..# values : logi[0 , 0 ] slots
.. .. ..# offset : num 0
.. .. ..# gain : num 1
.. .. ..# inmemory : logi FALSE
.. .. ..# fromdisk : logi TRUE
.. .. ..# nlayers : int 23
.. .. ..# dropped : NULL
.. .. ..# isfactor : logi FALSE
.. .. ..# attributes: list()
.. .. ..# haveminmax: logi FALSE
.. .. ..# min : num [1:23] Inf Inf Inf Inf Inf ...
.. .. ..# max : num [1:23] -Inf -Inf -Inf -Inf -Inf ...
.. .. ..# unit : chr ""
.. .. ..# names : chr [1:23] "X1" "X2" "X3" "X4" ...
..# legend :Formal class '.RasterLegend' [package "raster"] with 5 slots
.. .. ..# type : chr(0)
.. .. ..# values : logi(0)
.. .. ..# color : logi(0)
.. .. ..# names : logi(0)
.. .. ..# colortable: logi(0)
..# title : chr "variable"
..# extent :Formal class 'Extent' [package "raster"] with 4 slots
.. .. ..# xmin: num 1903933
.. .. ..# xmax: num 3021933
.. .. ..# ymin: num 5305094
.. .. ..# ymax: num 6757094
..# rotated : logi FALSE
..# rotation:Formal class '.Rotation' [package "raster"] with 2 slots
.. .. ..# geotrans: num(0)
.. .. ..# transfun:function ()
..# ncols : int 1118
..# nrows : int 1452
..# crs :Formal class 'CRS' [package "sp"] with 1 slot
.. .. ..# projargs: chr NA
..# history : list()
..# z :List of 1
.. ..$ unknown: int [1:23(1d)] 1 2 3 4 5 6 7 8 9 10 ...
packages that are loaded are raster and sp

Related

Plotting RasterLayer objects using ggplot

I am incredibly sorry, but I feel like there aren´t good example datasets for this question, and thus I will not be able to prdocue a reproducible example. If this makes this question untenable then I will definitely delete.
I am trying to plot a RasterLayer object in ggplot2 using geom_raster(), but get an error. I have seen that it seems like you have to convert RasterLayer objects to dataframes, but doing a simple conversion doesn't seem to work on this particular RasterLayer.
For reference, this data plots easily using the plot() function, but I'd like to do it in ggplot if possible due to its versatility.
Here is the code that attempts to plot the data. Of note, the object is stored in a list, along with many other model objects that I would like to plot:
library(ggplot2)
ggplot() + geom_raster(data = mylist$model_object)
And the error:
`data` must be a <data.frame>, or an object coercible by `fortify()`, not an S4 object with class
<RasterLayer>.
Run `rlang::last_error()` to see where the error occurred.
Here is what happens when I run mylist$model_object :
class : RasterLayer
dimensions : 114, 180, 20520 (nrow, ncol, ncell)
resolution : 0.1666667, 0.1666667 (x, y)
extent : -116, -86, 14, 33 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : layer
values : 0, 0.5845846 (min, max)
And here is the structure of the object:
str(mylist$model_object)
Formal class 'RasterLayer' [package "raster"] with 12 slots
..# file :Formal class '.RasterFile' [package "raster"] with 13 slots
.. .. ..# name : chr ""
.. .. ..# datanotation: chr "FLT4S"
.. .. ..# byteorder : chr "little"
.. .. ..# nodatavalue : num -Inf
.. .. ..# NAchanged : logi FALSE
.. .. ..# nbands : int 1
.. .. ..# bandorder : chr "BIL"
.. .. ..# offset : int 0
.. .. ..# toptobottom : logi TRUE
.. .. ..# blockrows : int 0
.. .. ..# blockcols : int 0
.. .. ..# driver : chr ""
.. .. ..# open : logi FALSE
..# data :Formal class '.SingleLayerData' [package "raster"] with 13 slots
.. .. ..# values : num [1:20520] 0.002 0 0 0 0 ...
.. .. ..# offset : num 0
.. .. ..# gain : num 1
.. .. ..# inmemory : logi TRUE
.. .. ..# fromdisk : logi FALSE
.. .. ..# isfactor : logi FALSE
.. .. ..# attributes: list()
.. .. ..# haveminmax: logi TRUE
.. .. ..# min : num 0
.. .. ..# max : num 0.585
.. .. ..# band : int 1
.. .. ..# unit : chr ""
.. .. ..# names : chr "layer"
..# legend :Formal class '.RasterLegend' [package "raster"] with 5 slots
.. .. ..# type : chr(0)
.. .. ..# values : logi(0)
.. .. ..# color : logi(0)
.. .. ..# names : logi(0)
.. .. ..# colortable: logi(0)
..# title : chr(0)
..# extent :Formal class 'Extent' [package "raster"] with 4 slots
.. .. ..# xmin: num -116
.. .. ..# xmax: num -86
.. .. ..# ymin: num 14
.. .. ..# ymax: num 33
..# rotated : logi FALSE
..# rotation:Formal class '.Rotation' [package "raster"] with 2 slots
.. .. ..# geotrans: num(0)
.. .. ..# transfun:function ()
..# ncols : int 180
..# nrows : int 114
..# crs :Formal class 'CRS' [package "sp"] with 1 slot
.. .. ..# projargs: chr "+proj=longlat +datum=WGS84 +no_defs"
.. .. ..$ comment: chr "GEOGCRS[\"unknown\",\n DATUM[\"World Geodetic System 1984\",\n ELLIPSOID[\"WGS 84\",6378137,298.25722"| __truncated__
..# history : list()
..# z : list()
I totally understand if this is not enough to go off of. If anyone has a suggestion about easy to access raster data online that I can use as a proxy repoducible example, it would be much appreciated. Thank you!
We may use the as.data.frame method for raster
> grep("data.frame", methods(class = "RasterLayer"), value = TRUE)
[1] "as.data.frame,Raster-method" "extract,Raster,data.frame-method" "rasterize,data.frame,Raster-method" "subs,Raster,data.frame-method"
as.data.frame(mylist$model_object, xy = TRUE)
I would use "terra" and "tidyterra"
library(raster)
library(terra)
library(tidyterra)
library(ggplot2)
# RasterLayer
f <- system.file("external/test.grd", package="raster")
r <- raster(f)
# create SpatRaster (can also do `x <- rast(f)`
x <- rast(r)
ggplot() + geom_spatraster(data = x)

Convert from spatialpixeldataframe to estUD

In short: I have a spatialpixeldataframe of a utilization distribution of an animal across a grid (udspdf) which I need to convert to estUD-class.
Background to how I got there: After calculating the KDE of a single animal, I then need to convert the estUD-class object to a spatialpixeldataframe in order to remove non-habitat cells, and rescale the utilisation distribution of the habitat-cells to 1. I then need to convert the spatialpixeldataframe back to an estUD class file so that I can run kerneloverlaphr() on it.
I have the following code, which converts it to an estUDm-class. But I need it in estUD-class, as there is only one animal.
re <- lapply(1:ncol(udspdf), function(i) {
so <- new("estUD", udspdf[,i])
so#h <- list(h=0, meth="specified") # specify dummy h values: they are only required to recreate the estUDm
so#vol <- FALSE
return(so)
})
names(re) <- names(udspdf)
class(re) <- "estUDm"
image(re)
If I just change
class(re) <- "estUD"
This seems to work, but then I can see there is an issue because
image(re)
renders the following error: Error in is(x, "GridTopology") :
trying to get slot "grid" from an object (class "estUD") that is not an S4 object
I am sorry I do not know how to provide a reproducible example for such an example as the data is quite complex. I hope that a general code exists.
Any pointers appreciated!
> str(re)
List of 1
$ ud:Formal class 'estUD' [package "adehabitatHR"] with 9 slots
.. ..# h :List of 2
.. .. ..$ h : num 0
.. .. ..$ meth: chr "specified"
.. ..# vol : logi FALSE
.. ..# data :'data.frame': 4400000 obs. of 1 variable:
.. .. ..$ ud: num [1:4400000] 0 0 0 0 0 0 0 0 0 0 ...
.. ..# coords.nrs : num(0)
.. ..# grid :Formal class 'GridTopology' [package "sp"] with 3 slots
.. .. .. ..# cellcentre.offset: Named num [1:2] -70 -60
.. .. .. .. ..- attr(*, "names")= chr [1:2] "Var2" "Var1"
.. .. .. ..# cellsize : Named num [1:2] 0.01 0.01
.. .. .. .. ..- attr(*, "names")= chr [1:2] "Var2" "Var1"
.. .. .. ..# cells.dim : Named int [1:2] 2000 2200
.. .. .. .. ..- attr(*, "names")= chr [1:2] "Var2" "Var1"
.. ..# grid.index : int [1:4400000] 1 2 3 4 5 6 7 8 9 10 ...
.. ..# coords : num [1:4400000, 1:2] -70 -70 -70 -70 -70 ...
.. .. ..- attr(*, "dimnames")=List of 2
.. .. .. ..$ : NULL
.. .. .. ..$ : chr [1:2] "Var2" "Var1"
.. ..# bbox : num [1:2, 1:2] -70 -60 -50 -38
.. .. ..- attr(*, "dimnames")=List of 2
.. .. .. ..$ : chr [1:2] "Var2" "Var1"
.. .. .. ..$ : chr [1:2] "min" "max"
.. ..# proj4string:Formal class 'CRS' [package "sp"] with 1 slot
.. .. .. ..# projargs: chr "+proj=longlat +datum=WGS84 +no_defs"
- attr(*, "class")= chr "estUDm"
my code for kerneloverlaphr(), which works with the outputs stright from kernelUD() (estUD-class) is as follows:
NWI15b, BCI15b and BCI15i are my individual animals
library(adehabitatHR)
# convert list of KDEs (estUDs) to class estUDm
tot <- list(NWI15b=NWI15b, BCI15b=BCI15b, BCI15i=BCI15i)
class(tot) <- "estUDm"
#calculate overlap using kerneloverlaphr
kerneloverlaphr(tot, method = c("BA"), percent = 95, conditional = FALSE)
dput(head(as.data.frame.estUD(re), n = 10))
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'x' in selecting a method for function 'head': no method or default for coercing “estUDm” to “SpatialPixelsDataFrame”
> unique(re#data$ud)
Error in h(simpleError(msg, call)) :
error in evaluating the argument 'x' in selecting a method for function 'unique': trying to get slot "data" from an object (class "estUDm") that is not an S4 object
structure of object fed to kernelUD
str(track_sp)
Formal class 'SpatialPoints' [package "sp"] with 3 slots
..# coords : num [1:9790, 1:2] -59.2 -59.2 -59.2 -59.2 -59.2 ...
.. ..- attr(*, "dimnames")=List of 2
.. .. ..$ : chr [1:9790] "1" "2" "3" "4" ...
.. .. ..$ : chr [1:2] "x" "y"
..# bbox : num [1:2, 1:2] -65.8 -55.6 -56.1 -52.2
.. ..- attr(*, "dimnames")=List of 2
.. .. ..$ : chr [1:2] "x" "y"
.. .. ..$ : chr [1:2] "min" "max"
..# proj4string:Formal class 'CRS' [package "sp"] with 1 slot
.. .. ..# projargs: chr "+proj=longlat +datum=WGS84 +no_defs"
Where we have
re <- lapply(1:ncol(udspdf), function(i) {
so <- new("estUD", udspdf[,i])
so#h <- list(h=0, meth="specified") # specify dummy h values: they are only required to recreate the estUDm
so#vol <- FALSE
return(so)
})
> names(re) <- names(udspdf)
> class(re) <- "estUDm"
> image(re)
We then need to simply add this:
colonyA <- re[[1]]
#and then do the same after running the code for the next colony
colonyB<- re[[1]]

Significantly different speeds on raster (netCDF) calculations in R

I have some WRF output data that was subsetted and masked using pythons xarray module.
I'm now performing calculations on raster bricks using R's raster package and finding very different speeds for very similar files.
Knowns:
There are 3 netCDF files, all the exact same size - 9.47 GB, that contain 9 variables
They all have the exact same dimensions (nrow 327, ncol 348, nlayer 365)
All calculations are on individual files (layer calculations)
All calculations are on the same variable with the same values (except for the second which is masked)
system.time(sum(d97[[1:365]]))
user system elapsed
5.428 2.771 8.840
The second file is the exact same file but a masked portion, with all the masked values converted to NaN.
system.time(sum(masked_d97[[1:365]]))
user system elapsed
10.784 2.157 13.052
The last file is a slightly modified version (daily values rather than cummulative values) of the first file. It was modified using Xarray in Python.
system.time(sum(mod_d97[[1:365]]))
user system elapsed
22.015 1.773 24.474
What on earth is happening here? I'm happy to provide more details (code, ncdumps, etc) as requested.
EDIT: added str() of files
d97 <- brick(files[8], varname = "TMIN")
masked_97 <- brick(files[3], varname = "TMIN")
d03 <- brick(files[11], varname = "TMIN")
str(d97)
Formal class 'RasterBrick' [package "raster"] with 12 slots
..# file :Formal class '.RasterFile' [package "raster"] with 13 slots
.. .. ..# name : chr "/Users/charlesbecker/Desktop/Data/Project Data/Shiny/WY1997_yearly_stats.nc"
.. .. ..# datanotation: chr "FLT4S"
.. .. ..# byteorder : chr "little"
.. .. ..# nodatavalue : num NaN
.. .. ..# NAchanged : logi FALSE
.. .. ..# nbands : int 365
.. .. ..# bandorder : chr "BIL"
.. .. ..# offset : int 0
.. .. ..# toptobottom : logi TRUE
.. .. ..# blockrows : int 0
.. .. ..# blockcols : int 0
.. .. ..# driver : chr "netcdf"
.. .. ..# open : logi FALSE
..# data :Formal class '.MultipleRasterData' [package "raster"] with 14 slots
.. .. ..# values : logi[0 , 0 ]
.. .. ..# offset : num 0
.. .. ..# gain : num 1
.. .. ..# inmemory : logi FALSE
.. .. ..# fromdisk : logi TRUE
.. .. ..# nlayers : int 365
.. .. ..# dropped : NULL
.. .. ..# isfactor : logi FALSE
.. .. ..# attributes: list()
.. .. ..# haveminmax: logi FALSE
.. .. ..# min : num [1:365] Inf Inf Inf Inf Inf ...
.. .. ..# max : num [1:365] -Inf -Inf -Inf -Inf -Inf ...
.. .. ..# unit : chr "K"
.. .. ..# names : chr [1:365] "X1" "X2" "X3" "X4" ...
..# legend :Formal class '.RasterLegend' [package "raster"] with 5 slots
.. .. ..# type : chr(0)
.. .. ..# values : logi(0)
.. .. ..# color : logi(0)
.. .. ..# names : logi(0)
.. .. ..# colortable: logi(0)
..# title : chr "TMIN"
..# extent :Formal class 'Extent' [package "raster"] with 4 slots
.. .. ..# xmin: num 0.5
.. .. ..# xmax: num 348
.. .. ..# ymin: num 0.5
.. .. ..# ymax: num 328
..# rotated : logi FALSE
..# rotation:Formal class '.Rotation' [package "raster"] with 2 slots
.. .. ..# geotrans: num(0)
.. .. ..# transfun:function ()
..# ncols : int 348
..# nrows : int 327
..# crs :Formal class 'CRS' [package "sp"] with 1 slot
.. .. ..# projargs: chr NA
..# history : list()
..# z :List of 1
.. ..$ : int [1:365] 1 2 3 4 5 6 7 8 9 10 ...
str(masked_d97)
Formal class 'RasterBrick' [package "raster"] with 12 slots
..# file :Formal class '.RasterFile' [package "raster"] with 13 slots
.. .. ..# name : chr "/Users/charlesbecker/Desktop/Data/Project Data/Shiny/AVA_WY1997_yearly_stats.nc"
.. .. ..# datanotation: chr "FLT4S"
.. .. ..# byteorder : chr "little"
.. .. ..# nodatavalue : num NaN
.. .. ..# NAchanged : logi FALSE
.. .. ..# nbands : int 365
.. .. ..# bandorder : chr "BIL"
.. .. ..# offset : int 0
.. .. ..# toptobottom : logi TRUE
.. .. ..# blockrows : int 0
.. .. ..# blockcols : int 0
.. .. ..# driver : chr "netcdf"
.. .. ..# open : logi FALSE
..# data :Formal class '.MultipleRasterData' [package "raster"] with 14 slots
.. .. ..# values : logi[0 , 0 ]
.. .. ..# offset : num 0
.. .. ..# gain : num 1
.. .. ..# inmemory : logi FALSE
.. .. ..# fromdisk : logi TRUE
.. .. ..# nlayers : int 365
.. .. ..# dropped : NULL
.. .. ..# isfactor : logi FALSE
.. .. ..# attributes: list()
.. .. ..# haveminmax: logi FALSE
.. .. ..# min : num [1:365] Inf Inf Inf Inf Inf ...
.. .. ..# max : num [1:365] -Inf -Inf -Inf -Inf -Inf ...
.. .. ..# unit : chr ""
.. .. ..# names : chr [1:365] "X1" "X2" "X3" "X4" ...
..# legend :Formal class '.RasterLegend' [package "raster"] with 5 slots
.. .. ..# type : chr(0)
.. .. ..# values : logi(0)
.. .. ..# color : logi(0)
.. .. ..# names : logi(0)
.. .. ..# colortable: logi(0)
..# title : chr "TMIN"
..# extent :Formal class 'Extent' [package "raster"] with 4 slots
.. .. ..# xmin: num 0.5
.. .. ..# xmax: num 348
.. .. ..# ymin: num 0.5
.. .. ..# ymax: num 328
..# rotated : logi FALSE
..# rotation:Formal class '.Rotation' [package "raster"] with 2 slots
.. .. ..# geotrans: num(0)
.. .. ..# transfun:function ()
..# ncols : int 348
..# nrows : int 327
..# crs :Formal class 'CRS' [package "sp"] with 1 slot
.. .. ..# projargs: chr NA
..# history : list()
..# z :List of 1
.. ..$ : int [1:365] 1 2 3 4 5 6 7 8 9 10 ...
str(d03)
Formal class 'RasterBrick' [package "raster"] with 12 slots
..# file :Formal class '.RasterFile' [package "raster"] with 13 slots
.. .. ..# name : chr "/Users/charlesbecker/Desktop/Data/Project Data/Shiny/WY2003_yearly_stats.nc"
.. .. ..# datanotation: chr "FLT4S"
.. .. ..# byteorder : chr "little"
.. .. ..# nodatavalue : num NaN
.. .. ..# NAchanged : logi FALSE
.. .. ..# nbands : int 365
.. .. ..# bandorder : chr "BIL"
.. .. ..# offset : int 0
.. .. ..# toptobottom : logi TRUE
.. .. ..# blockrows : int 0
.. .. ..# blockcols : int 0
.. .. ..# driver : chr "netcdf"
.. .. ..# open : logi FALSE
..# data :Formal class '.MultipleRasterData' [package "raster"] with 14 slots
.. .. ..# values : logi[0 , 0 ]
.. .. ..# offset : num 0
.. .. ..# gain : num 1
.. .. ..# inmemory : logi FALSE
.. .. ..# fromdisk : logi TRUE
.. .. ..# nlayers : int 365
.. .. ..# dropped : NULL
.. .. ..# isfactor : logi FALSE
.. .. ..# attributes: list()
.. .. ..# haveminmax: logi FALSE
.. .. ..# min : num [1:365] Inf Inf Inf Inf Inf ...
.. .. ..# max : num [1:365] -Inf -Inf -Inf -Inf -Inf ...
.. .. ..# unit : chr "K"
.. .. ..# names : chr [1:365] "X1" "X2" "X3" "X4" ...
..# legend :Formal class '.RasterLegend' [package "raster"] with 5 slots
.. .. ..# type : chr(0)
.. .. ..# values : logi(0)
.. .. ..# color : logi(0)
.. .. ..# names : logi(0)
.. .. ..# colortable: logi(0)
..# title : chr "TMIN"
..# extent :Formal class 'Extent' [package "raster"] with 4 slots
.. .. ..# xmin: num 0.5
.. .. ..# xmax: num 348
.. .. ..# ymin: num 0.5
.. .. ..# ymax: num 328
..# rotated : logi FALSE
..# rotation:Formal class '.Rotation' [package "raster"] with 2 slots
.. .. ..# geotrans: num(0)
.. .. ..# transfun:function ()
..# ncols : int 348
..# nrows : int 327
..# crs :Formal class 'CRS' [package "sp"] with 1 slot
.. .. ..# projargs: chr NA
..# history : list()
..# z :List of 1
.. ..$ : int [1:365] 1 2 3 4 5 6 7 8 9 10 ...
system.time(sum(d97[[1:365]]))
user system elapsed
5.569 2.219 8.048
system.time(sum(masked_97[[1:365]]))
user system elapsed
11.887 2.342 14.569
system.time(sum(d03[[1:365]]))
user system elapsed
22.253 1.772 24.879
The most likely difference is that data in your new netCDF file is now compressed differently. Two forms of compression are common with netCDF files:
scale/offset encoding, e.g., to decode from int16 via a formula like scale_factor * values + add_offset.
zlib compression on individual chunks of the array (only supported with netCDF4 files).
If you don't slice or manipulate your variables, xarray will preserve compression setting via the encoding attribute, but this is generally dropped by xarray operations. See the xarray docs on reading/writing encoded data for more details.

how to differentiate the colours for background and NA values in gplot

I am generating a chlorophyll map for lake. I want to fill the lake with blue colour where there is a very low chlorophyll concentration and light blue for NA values. I am using a code as given below
gplot(Chlorophyll_map_5) + geom_tile(aes(fill=value)) + scale_fill_gradient(low = 'blue', high = 'red', na.value='blue',name="Chl-a (ug/l)",limits=c(0,1000)) + coord_equal()+theme_bw()
Which gives me a plot like this for na.value='blue':
na.value='blue'
When I use na.value='transparent' I got this image:
na.value='transparent'
If I change the colour of the na.value it also changes the background. Is there a way to fill the lake with colour without changing the background?
The output of my data:`The output of my data:
Formal class 'RasterLayer' [package "raster"] with 12 slots
..# file :Formal class '.RasterFile' [package "raster"] with 13 slots
.. .. ..# name : chr "/private/var/folders/68/hm_5ts9x7psb6j3wnb91_bfr0000gn/T/RtmpZ3BLZD/raster/r_tmp_2017-07-18_133827_28365_34843.grd"
.. .. ..# datanotation: chr "FLT8S"
.. .. ..# byteorder : Named chr "little"
.. .. .. ..- attr(*, "names")= chr "value"
.. .. ..# nodatavalue : num -1.7e+308
.. .. ..# NAchanged : logi FALSE
.. .. ..# nbands : int 1
.. .. ..# bandorder : Named chr "BIL"
.. .. .. ..- attr(*, "names")= chr "value"
.. .. ..# offset : int 0
.. .. ..# toptobottom : logi TRUE
.. .. ..# blockrows : int 0
.. .. ..# blockcols : int 0
.. .. ..# driver : chr "raster"
.. .. ..# open : logi FALSE
..# data :Formal class '.SingleLayerData' [package "raster"] with 13 slots
.. .. ..# values : logi(0)
.. .. ..# offset : num 0
.. .. ..# gain : num 1
.. .. ..# inmemory : logi FALSE
.. .. ..# fromdisk : logi TRUE
.. .. ..# isfactor : logi FALSE
.. .. ..# attributes: list()
.. .. ..# haveminmax: logi TRUE
.. .. ..# min : num 0.00335
.. .. ..# max : num 3870657
.. .. ..# band : int 1
.. .. ..# unit : chr ""
.. .. ..# names : chr "layer"
..# legend :Formal class '.RasterLegend' [package "raster"] with 5 slots
.. .. ..# type : chr(0)
.. .. ..# values : logi(0)
.. .. ..# color : logi(0)
.. .. ..# names : logi(0)
.. .. ..# colortable: logi(0)
..# title : chr(0)
..# extent :Formal class 'Extent' [package "raster"] with 4 slots
.. .. ..# xmin: num 35.8
.. .. ..# xmax: num 36.7
.. .. ..# ymin: num 2.4
.. .. ..# ymax: num 4.65
..# rotated : logi FALSE
..# rotation:Formal class '.Rotation' [package "raster"] with 2 slots
.. .. ..# geotrans: num(0)
.. .. ..# transfun:function ()
..# ncols : int 3240
..# nrows : int 8321
..# crs :Formal class 'CRS' [package "sp"] with 1 slot
.. .. ..# projargs: chr "+proj=longlat +ellps=WGS84 +no_defs"
..# history : list()
..# z : list()
x <- trim(Chlorophyll_map_5)
ggplot(Chlorophyll_map_5) +
geom_tile(aes(fill=value)) +
scale_fill_gradient(low = 'blue', high = 'red', na.value='blue',name="Chl-a (ug/l)",limits=c(0,1000)) +
coord_equal()+theme_bw()
Per documentation, the trim function "crops a RasterLayer by removing the outer rows and columns that
only contain NA values"

R error while using cbind

I trying to combine 2 vectors using cbind, both vectors are the same size, and I am having an error while i run the code, the vectors are quite big, length = 57605.
final=cbind (counts1,tx_by_gene)
> > Error: cannot allocate vector of size 225 Kb R(473,0xa0cb8540) malloc: *** mmap(size=233472) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug R(473,0xa0cb8540) malloc: *** mmap(size=233472) failed (error code=12)
> *** error: can't allocate region
> *** set a breakpoint in malloc_error_break to debug
Can anyone help me why am I having this error? or some other way of combining the 2 vectors?
thank you
> str(counts1) = int [1:57605] 0 0 0 0 0 0 0 0 0 0 ...
>str(tx_by_gene)
> Formal class 'GRangesList' [package "GenomicRanges"] with 5 slots ..# partitioning :Formal class 'PartitioningByEnd' [package
> "IRanges"] with 5 slots .. .. ..# end : int [1:57605] 3 5
> 12 17 27 36 42 46 58 60 ... .. .. ..# NAMES : chr [1:57605]
> "ENSG00000000003" "ENSG00000000005" "ENSG00000000419"
> "ENSG00000000457" ... .. .. ..# elementMetadata: NULL .. .. ..#
> elementType : chr "integer" .. .. ..# metadata : list()
> ..# unlistData :Formal class 'GRanges' [package "GenomicRanges"]
> with 7 slots .. .. ..# seqnames :Formal class 'Rle' [package
> "IRanges"] with 5 slots .. .. .. .. ..# values : Factor w/
> 93 levels "chr1","chr2",..: 8 20 1 6 1 8 6 3 7 13 ... .. .. .. ..
> ..# lengths : int [1:41694] 5 7 30 18 21 6 2 9 43 23 ... ..
> .. .. .. ..# elementMetadata: NULL .. .. .. .. ..# elementType :
> chr "ANY" .. .. .. .. ..# metadata : list() .. .. ..# ranges
> :Formal class 'IRanges' [package "IRanges"] with 6 slots .. .. .. ..
> ..# start : int [1:191891] 99883667 99887538 99888439
> 99839799 99848621 49551404 49551404 49551404 49551433 49551482 ...
> .. .. .. .. ..# width : int [1:191891] 8137 4149 6550 15084
> 3908 23684 23684 23689 10966 23577 ... .. .. .. .. ..# NAMES
> : NULL .. .. .. .. ..# elementMetadata: NULL .. .. .. .. ..#
> elementType : chr "integer" .. .. .. .. ..# metadata :
> list() .. .. ..# strand :Formal class 'Rle' [package
> "IRanges"] with 5 slots .. .. .. .. ..# values : Factor w/ 3
> levels "+","-","*": 2 1 2 1 2 1 2 1 2 1 ... .. .. .. .. ..# lengths
> : int [1:28670] 3 2 12 10 9 6 16 2 13 8 ... .. .. .. .. ..#
> elementMetadata: NULL .. .. .. .. ..# elementType : chr "ANY"
> .. .. .. .. ..# metadata : list() .. .. ..# seqlengths :
> Named int [1:93] 249250621 243199373 198022430 191154276 180915260
> 171115067 159138663 155270560 146364022 141213431 ... .. .. .. ..-
> attr(*, "names")= chr [1:93] "chr1" "chr2" "chr3" "chr4" ... .. ..
> ..# elementMetadata:Formal class 'DataFrame' [package "IRanges"] with
> 6 slots .. .. .. .. ..# rownames : NULL .. .. .. .. ..#
> nrows : int 191891 .. .. .. .. ..# elementMetadata: NULL
> .. .. .. .. ..# elementType : chr "ANY" .. .. .. .. ..# metadata
> : list() .. .. .. .. ..# listData :List of 2 .. .. .. .. ..
> ..$ tx_id : int [1:191891] 93738 93739 93740 93736 93737 175481
> 175482 175480 175483 175484 ... .. .. .. .. .. ..$ tx_name: chr
> [1:191891] "ENST00000373020" "ENST00000496771" "ENST00000494424"
> "ENST00000373031" ... .. .. ..# elementType : chr "ANY" .. ..
> ..# metadata : list() ..# elementMetadata:Formal class
> 'DataFrame' [package "IRanges"] with 6 slots .. .. ..# rownames
> : NULL .. .. ..# nrows : int 57605 .. .. ..#
> elementMetadata: NULL .. .. ..# elementType : chr "ANY" .. ..
> ..# metadata : list() .. .. ..# listData : list() ..#
> elementType : chr "GRanges" ..# metadata : list()
The object tx_by_gene isn't a vector. You can check using the is.vector function
is.vector(counts1)
is.vector(tx_by_gene)
Of course, there could be method defined so that the two objects can be combined
Those vectors should not be too big for R. You probably used up a lot of memory before the cbind() operation. Look at what objects you currently have with ls() and delete those you don't need any more with rm().

Resources