I have these 3 files here. Trying to stack them and extract a variable called "OzoneTropColumn". I am able to extract data for single file but unable to extract for multiple files.
library(raster)
library(ncdf4)
list_col1 <- list.files("E:/TES", pattern = "*.hdf", full.names = TRUE)
ncin1 <- raster::stack(list_col1, varname = "Data Fields/OzoneTropColumn", ncdf=TRUE)
Any help would be much appreciated.
Thank you
I don't know if this really works since I am no expert in raster or importing hdf files. But this code did load the three files in R for me but still gave some warnings:-
library(raster)
library(ncdf4)
list_col1 <- as.list(list.files("E:/TES", pattern = "*.hdf",
full.names = TRUE))
ncin1 <- raster::stack(list_col1,
varname = "Data Fields/OzoneTropColumn",
ncdf=TRUE)
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_1 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_0 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_1 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_0 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_1 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_0 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_1 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
[1] "vobjtovarid4: **** WARNING **** I was asked to get a varid for dimension named Data Fields/phony_dim_0 BUT this dimension HAS NO DIMVAR! Code will probably fail at this point"
But does shows me that it has read the 3 files
ncin1[[1]]
class : RasterLayer
dimensions : 83, 90, 7470 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : 0.5, 90.5, 0.5, 83.5 (xmin, xmax, ymin, ymax)
crs : NA
source : /home/shawn/Downloads/TES/TES-Aura_L3-O3-M2004m09_F01_12.hdf
names : Data.Fields.OzoneTropColumn.1
zvar : Data Fields/OzoneTropColumn
ncin1[[2]]
class : RasterLayer
dimensions : 83, 90, 7470 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : 0.5, 90.5, 0.5, 83.5 (xmin, xmax, ymin, ymax)
crs : NA
source : /home/shawn/Downloads/TES/TES-Aura_L3-O3-M2004m10_F01_12.hdf
names : Data.Fields.OzoneTropColumn.2
zvar : Data Fields/OzoneTropColumn
ncin1[[3]]
class : RasterLayer
dimensions : 83, 90, 7470 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : 0.5, 90.5, 0.5, 83.5 (xmin, xmax, ymin, ymax)
crs : NA
source : /home/shawn/Downloads/TES/TES-Aura_L3-O3-M2004m11_F01_12.hdf
names : Data.Fields.OzoneTropColumn.3
zvar : Data Fields/OzoneTropColumn
The script posted by Robert works well the only difference there was using as.list and without using as.list]1]1. I tried before using without as.list. The extracted data loses its georeference for reason unknown. So had to give extent and projection after raster stack. Thanks Robert.
Related
I have a single simple raster in EPSG:7532 that I am trying to project to EPSG:4326 but is failing
The source data is a Lidar point clould that I am able to process using the lidR package. The data source is in the link below
https://rockyweb.usgs.gov/vdelivery/Datasets/Staged/Elevation/LPC/Projects/WI_BrownRusk_2020_B20/WI_Brown_2_2020/LAZ/USGS_LPC_WI_BrownRusk_2020_B20_02531702.laz
l1 = readLAS("USGS_LPC_WI_BrownRusk_2020_B20_02531702.laz")
> l1
class : LAS (v1.4 format 6)
memory : 2.5 Gb
extent : 25349, 26849, 170258, 171758 (xmin, xmax, ymin, ymax)
coord. ref. : NAD83(2011) / WISCRS Brown (m) + NAVD88 height - Geoid18 (m)
area : 2.25 km²
points : 35.57 million points
density : 15.79 points/m²
density : 12.89 pulses/m²
convert to a spatRaster:
dsm <- rasterize_canopy(l1, res = 1.0, pitfree(c(0,2,5,10,15), c(0, 1.5)))
> dsm
class : SpatRaster
dimensions : 1500, 1501, 1 (nrow, ncol, nlyr)
resolution : 1, 1 (x, y)
extent : 25349, 26850, 170258, 171758 (xmin, xmax, ymin, ymax)
coord. ref. : NAD83(2011) / WISCRS Brown (m) (EPSG:7532)
source : memory
name : Z
min value : 185.836
max value : 333.709
The point of failure is the attempt to project to geographic format:
dsm_test <- terra::project(dsm, "EPSG:4326", method="bilinear")
> dsm_test <- terra::project(dsm, "EPSG:4326", method="bilinear")
Error: [project] cannot get output boundaries
In addition: Warning messages:
1: In x#ptr$warp(SpatRaster$new(), y, method, mask, FALSE, opt) :
GDAL Error 1: PROJ: vgridshift: could not find required grid(s).
2: In x#ptr$warp(SpatRaster$new(), y, method, mask, FALSE, opt) :
GDAL Error 1: PROJ: pipeline: Pipeline: Bad step definition: proj=vgridshift (File not found or invalid)
3: In x#ptr$warp(SpatRaster$new(), y, method, mask, FALSE, opt) :
GDAL Error 1: Too many points (961 out of 961) failed to transform, unable to compute output bounds.
A similar topic here, but seems different.
https://stackoverflow.com/questions/72404897/what-is-causing-this-raster-reprojection-error
This issue is not resulting from reprojection from EPSG:7532 to EPSG:4326 per se, but seems rather connected to the fact, that your SpatRaster object created via rasterize_canopy() comes with a vertical datum, apparently causing problems downstream:
VERTCRS["NAVD88 height - Geoid18 (m)",
VDATUM["North American Vertical Datum 1988"],
CS[vertical,1],
AXIS["up",up,
LENGTHUNIT["meter",1]],
GEOIDMODEL["GEOID18"],
ID["EPSG",5703]]]
The quick & dirty solution would be to simply override crs definition by EPSG:7532 and dropping references in Z dimension, although this does not feel 100 % right. On the other hand, I'm not sure how terra handles vertical crs information and if it is possible to keep this information at all.
library(lidR)
library(terra)
#> terra 1.6.49
l1 = readLAS("USGS_LPC_WI_BrownRusk_2020_B20_02531702.laz")
#> Warning: There are 53206 points flagged 'withheld'.
dsm <- rasterize_canopy(l1, res = 1.0, pitfree(c(0,2,5,10,15), c(0, 1.5)))
crs(dsm) <- "epsg:7532"
dsm_4326 <- project(dsm, "epsg:4326", method="bilinear")
dsm_4326
#> class : SpatRaster
#> dimensions : 1235, 1727, 1 (nrow, ncol, nlyr)
#> resolution : 1.093727e-05, 1.093727e-05 (x, y)
#> extent : -88.0786, -88.05972, 44.49092, 44.50443 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326)
#> source(s) : memory
#> name : Z
#> min value : 185.8360
#> max value : 295.1357
From my personal point of view, it would be better to have both crs (xy, z) listed in the SpatRaster summary with reproject() being able to address z-transformations separately (or have e.g. terra::transform()), e.g. when you wanted to just transform your heights but still keep EPSG:7532 as your xy reference system.
coord. ref. xy: NAD83(2011) / WISCRS Brown (m) (EPSG:7532)
coord. ref. z: NAVD88 height - Geoid18 (m) (EPSG:5703)
I would like to write a RasterStack object and preserve names and metadata of the individual layers. How to preserve names is explained here. Is there a way to preserve metadata of individual layers when writing a RasterStack object?
Here is replicable code:
# load library
library(raster)
# create example rasters
ras_1 <- raster(nrows=180, ncols=360, xmn=-180, xmx=180, ymn=-90, ymx=90, resolution=, vals=1)
ras_2 <- raster(nrows=180, ncols=360, xmn=-180, xmx=180, ymn=-90, ymx=90, resolution=, vals=2)
ras_3 <- raster(nrows=180, ncols=360, xmn=-180, xmx=180, ymn=-90, ymx=90, resolution=, vals=3)
# assign names
names(ras_1) <- "raster_A"
names(ras_2) <- "raster_B"
names(ras_3) <- "raster_C"
# assign metadata
metadata(ras_1) <- list("metadata_raster_A")
metadata(ras_2) <- list("metadata_raster_B")
metadata(ras_3) <- list("metadata_raster_C")
# check
ras_1
ras_2
ras_3
metadata(ras_1)
metadata(ras_2)
metadata(ras_3)
# create and check stack
raster_stack <- stack(ras_1,
ras_2,
ras_3)
raster_stack
raster_stack[[1]]
metadata(raster_stack[[1]])
# write raster stack to disk
setwd("~")
# load library
library(terra)
# create rast object
raster_stack_terr <- rast(raster_stack)
# write raster stack
terra::writeRaster(raster_stack_terr, "raster_stack_terr_test.tif")
# load and check raster stack
raster_stack_check <- stack("raster_stack_terr_test.tif")
raster_stack_check
raster_stack_check[[1]]
names(raster_stack_check[[1]])
metadata(raster_stack_check[[1]])
Use terra to preseve names according to the 3rd answer from here.
When opening the RasterStack from disk, the metadata is not preserved. See console output:
> metadata(raster_stack_check[[1]])
list()
How to preserve metadata of individual layers when writing and re-loading a RasterStack object? Thanks!
It does not seem like {terra} offers an equivalent to raster::metadata(). However, from my perspective, the use cases would be limited here, because you would only be able to store structured information in corresponding format-specific tags (at least, this is my understanding) when writing to disk.
TIFF files (c.f. here) seem to offer the following tags:
TIFFTAG_DOCUMENTNAME
TIFFTAG_IMAGEDESCRIPTION
TIFFTAG_SOFTWARE
TIFFTAG_DATETIME
TIFFTAG_ARTIST
TIFFTAG_HOSTCOMPUTER
TIFFTAG_COPYRIGHT
TIFFTAG_XRESOLUTION
TIFFTAG_YRESOLUTION
TIFFTAG_RESOLUTIONUNIT
ESRI-Grids, on the other hand, do not offer any possibilities to store metadata except for the known header and maybe the filename as far as I know.
If you only wanted to store certain metadata with your raster object, you might as well make use of attr(r, "meta") <- "foobar". However, I don't see how this (random) information can be stored in specific formats and restored afterwards.
You already noticed names() when using {terra}, but there is also time() to be mentioned. Maybe this already suits your needs, since you did not specify what exactly you intend to store.
# set up a raster stack with three layers
library(terra)
#> terra 1.6.17
# create raster
r <- rast(nrows = 10, ncols = 10)
values(r) <- rnorm(100)
# set metadata
names(r) <- "foo"
time(r) <- as.Date("2000-01-01")
attr(r, "meta") <- "bar"
# inspect
r
#> class : SpatRaster
#> dimensions : 10, 10, 1 (nrow, ncol, nlyr)
#> resolution : 36, 18 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84
#> source : memory
#> name : foo
#> min value : -2.503790
#> max value : 1.998731
#> time (days) : 2000-01-01
# write to disk
writeRaster(r, "sample.tif", overwrite = TRUE)
# read from disk
r2 <- rast("sample.tif")
r2
#> class : SpatRaster
#> dimensions : 10, 10, 1 (nrow, ncol, nlyr)
#> resolution : 36, 18 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : lon/lat WGS 84 (EPSG:4326)
#> source : sample.tif
#> name : foo
#> min value : -2.503790
#> max value : 1.998731
#> time (days) : 2000-01-01
# try to access attributes
attr(r2, "meta")
#> NULL
As expected, data stored as attribute has been lost whereas information provided via names() and time() was sustained.
I have 365 .nc files located in a folder containing daily soil moisture information. I want to extract data at five-coordinate locations for the whole year and write them into five separate csv files. My code is attached below. However, I am getting this error after the line:
s <- stack(ff)
>Error in if (is.na(get("has_proj_def.dat", envir = .RGDAL_CACHE))) { : argument is of length zero In addition: Warning message: In .varName(nc, varname, warn = warn) : varname used is: sm If that is not correct, you can set it to one of: sm, sm_noise, flag, sensor
No idea how to proceed further.
library(raster)
library(ncdf4)
ptf <- "D://SMOS_ECV_SM//SMOS_ECV_SM//ECV_SM_Data_1978_2010//1978"
ff <- list.files(path=ptf, pattern="[.]nc$", full.names=TRUE)
s <- stack(ff)
points <- rbind(c(0,1), c(100,120), c(80,5), c(85,4), c(82,4))
v <- extract(s, points)
for (i in 1:ncol(v)) {
write.csv(v[,i,drop=FALSE], paste0("file", i, ".csv"))
}
library(raster)
#Loading required package: sp
f <- list.files("try", full=T)
First try for a single file
r <- raster(f[1])
#Loading required namespace: ncdf4
#Warning message:
#In .varName(nc, varname, warn = warn) : varname used is: sm
#If that is not correct, you can set it to one of: sm, sm_noise, flag, sensor
To get rid of the warning:
r <- raster(f[1], varname="sm")
Now for all files
s <- stack(f, varname="sm")
s
#class : RasterStack
#dimensions : 720, 1440, 1036800, 2 (nrow, ncol, ncell, nlayers)
#resolution : 0.25, 0.25 (x, y)
#extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#crs : +proj=longlat +datum=WGS84 +no_defs
#names : Soil.Moisture.1, Soil.Moisture.2
Extract values
points <- rbind(c(-96.7, 47), c(34.55, 54.85))
v <- extract(s, points)
v
# Soil.Moisture.1 Soil.Moisture.2
#[1,] 0.3254 0.3018
#[2,] 0.3386 0.3386
I have stacked 28 layers to a brick in R
brik
class : RasterBrick
dimensions : 720, 1440, 1036800, 28 (nrow, ncol, ncell, nlayers)
resolution : 0.25, 0.25 (x, y)
extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
source : C:/Users/Ujjal Baruah/AppData/Local/Temp/Rtmp0GaiPO/raster/r_tmp_2020-01-03_030159_46788_10398.grd
names : Data.Fiel//tNO2Trop.1, Data.Fiel//tNO2Trop.2, Data.Fiel//tNO2Trop.3, Data.Fiel//tNO2Trop.4, Data.Fiel//tNO2Trop.5, Data.Fiel//tNO2Trop.6, Data.Fiel//tNO2Trop.7, Data.Fiel//tNO2Trop.8, Data.Fiel//tNO2Trop.9, Data.Fiel//NO2Trop.10, Data.Fiel//NO2Trop.11, Data.Fiel//NO2Trop.12, Data.Fiel//NO2Trop.13, Data.Fiel//NO2Trop.14, Data.Fiel//NO2Trop.15, ...
Now, i want to save this individual layers in Geotiff using
writeRaster(brik, file.path('/output/filepath/', names(brik)), bylayer=TRUE, format('GTiff'))
Unfortunately, i get just one file instead of multiple layers in geotiff.
Any solution would be appreciated.
Thanks
writeRaster seems to strip off the dot-number before creating a raster file. Hence it tries to write your layers all to Data.Fiel//tNO2Trop.tif.
> writeRaster(r, "./test.2", format="GTiff")
> dir(".")
[1] "test.tif"
(Note for some reason your code has format("GTiff") for format="GTiff". This works by the fluke that format is a function and returns the string "GTiff" and writeRaster is expecting the format string here)
I don't know why and I don't know if this is documented or a bug. You can work round by using dashes instead of dots:
> writeRaster(r, "./test-2", format="GTiff")
> dir(".")
[1] "test-2.tif" "test.tif"
and if dots are important to you then do a file.rename afterwards.
Edit: If you add the .tif to the file names then all is well:
> writeRaster(s, names(s), bylayer=TRUE, format="GTiff")
Error in .getGDALtransient(x, filename = filename, options = options, :
filename exists; use overwrite=TRUE
fails on the second layer because dot-number is stripped:
> dir()
[1] "layer.tif"
add .tif to the names:
> writeRaster(s, paste0(names(s),".tif"), bylayer=TRUE, format="GTiff")
shazam:
> dir()
[1] "layer.1.tif" "layer.2.tif" "layer.3.tif" "layer.tif"
Good day everyone..
I have 13 bioclimatic variables (in .tiff format) that I will used to perform sdm by using dismo package.
I followed the tutorial written by Robert J. Hijmans and Jane Elith.
However, when I tried to stack all of the variables, I got the following error
Error in .local(.Object, ...) :
Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer", :
Cannot create a RasterLayer object from this file.
All of my file's coordinate system, extent, and cell size have been adjusted so they are all the same..
when I tried to used the alternative brick function, I got the following error :
Error in .rasterObjectFromFile(x, objecttype = "RasterBrick", ...) :
Cannot create a RasterLayer object from this file.
In addition: There were 12 warnings (use warnings() to see them)
I used the warning() message but it was empty..
do any of you have any hints regarding what may be the cause of such errors?
i've tried to google it, but unfortunately, no answer can solve it.
Thank you in advance..
Here presented is the fraction of the transcript
#setting the workspace
setwd("D:/Riset/MaxentSelaginella/newpaperproject_part2/MakalahVI/Workspace_R")
#Loading Libraries
library("sp")
library("raster")
library("maptools")
library("rgdal")
library("dismo")
library("rJava")
#open the csv file
obs.data <- read.csv(file = "data3/Selaginella_plana.csv", sep = ",")
#open Environmental Data
files <- list.files(path = "data3/tif/", pattern = ".tif", full.names=TRUE)
#stacking all the files
predictors <- brick(files)
I guess you need to use stack instead of brick. As per brick help, in fact:
A RasterBrick is a multi-layer raster object. They are typically created from
a multi-layer (band) file; but they can also exist entirely in memory.
They are similar to a RasterStack (that can be created with stack), but processing
time should be shorter when using a RasterBrick. Yet they are less flexible as they can only point to a single file.
So, if we try to “stack” multiple files:
library(raster)
r <- raster(ncols = 100, nrows = 100, vals = 1:10000)
rfile1 <- tempfile(fileext = ".tif")
writeRaster(r, filename = rfile1)
rfile2 <- tempfile(fileext = ".tif")
writeRaster(r, filename = rfile2)
files_to_stack <- c(rfile1, rfile2)
This fails:
brick(files_to_stack)
#> Warning in if (x == "" | x == ".") {: the condition has length > 1 and only
#> the first element will be used
#> Warning in if (!start %in% c("htt", "ftp")) {: the condition has length > 1
#> and only the first element will be used
#> Warning in if (fileext %in% c(".GRD", ".GRI")) {: the condition has length
#> > 1 and only the first element will be used
#> Warning in if (!file.exists(x)) {: the condition has length > 1 and only
#> the first element will be used
.....
#> Error in .rasterObjectFromFile(x, objecttype = "RasterBrick", ...): Cannot create a RasterLayer object from this file.
While this works:
stack(files_to_stack)
#> class : RasterStack
#> dimensions : 100, 100, 10000, 2 (nrow, ncol, ncell, nlayers)
#> resolution : 3.6, 1.8 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0
#> names : file46e41bcd78e3, file46e43ea75bad
#> min values : 1, 1
#> max values : 10000, 10000
If you want to have a brick to get some gain in “efficiency” in further
processing, you can save the different "layers" as a multiband tiff, and then open using brick:
rfile_multi <- tempfile(fileext = ".tif")
writeRaster(stack(files_to_stack), filename = rfile_multi)
brick(rfile_multi)
#> class : RasterBrick
#> dimensions : 100, 100, 10000, 2 (nrow, ncol, ncell, nlayers)
#> resolution : 3.6, 1.8 (x, y)
#> extent : -180, 180, -90, 90 (xmin, xmax, ymin, ymax)
#> coord. ref. : +proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0
#> data source : D:\RTemp\RtmpacXztJ\file4808784f268c.tif
#> names : file4808784f268c.1, file4808784f268c.2
#> min values : 1, 1
#> max values : 10000, 10000
Created on 2018-11-10 by the reprex package (v0.2.1)