I am still working in the same project where I asked this question.
And I am getting by allright, but my processing times are too long, for each NetCDF file I am reading it, getting a stack form several of the time-slices, and then cropping each of those time slices within r, using a code like the one bellow:
library(raster)
library(ncdf4)
library(ncdf4.helpers)
library(rworldxtra)
data("countriesHigh")
The NETCdf files I have are for all of the world, but I am using only South America, for that I use countiresHigh from the rworldxtra package to subset it:
NONA <- countriesHigh[!is.na(countriesHigh#data$GEO3),]
## get shapefile of South America
SA <- NONA[NONA#data$GEO3 == "South America",]
Then using the following code I crop every layer that I need.
##Open conection to the layer
nc <- nc_open("C:/Users/mean_temperature-15000BP-10000BP.nc")
Now I start a loop
for(i in 1:10){
message(paste("reading layer", i))
# Read the stack for year i
r <- stack("C:/Users/mean_temperaturemean_temperature-15000BP-10000BP.nc", varname = age[i])
#Change the extent to the correct one
extent(r) <- c(-180,180,-90,90)
#Crop it to South America
r <- crop(r, SA)
gc()
}
Problem 1: Can I crop the netcdf once before reading stacks to make this process faster?
I have looked here, here and here, and found no answer.
Problem 2: If I can crop it, how do I define the extent given that the NETCDF file contains only positive latitude and positive longitude
as stated in this question, the defined extent of the map is not the most usual, but I am basing my cropping in the extent of a shapefile that has the more typical c(-180,180,-90,90) extent
Related
I'm working with the Biophysical settings raster data set from landfire.gov. When I initially read in the data set with terra::rast(), everything appears to be working. However, when I attempt to crop the data using a polygon, all values outside of the range 0-255 are replaced with NA. Values outside of this range are also dropped if I attempt to project this raster to a new coordinate reference system. Could anyone explain why this raster is being limited to values of the data type INT1U and how I might bypass this?
Below I provide a reproducible code example of how I've attempted to process this data set. This example is dependent on two public data sets:
Biophysical Settings from Landfire.gov
North Carolina state boundary from nconemap.gov (download this as a shapefile)
library(terra)
library(dplyr)
library(sf)
# Establish paths to required files.
# ** These will need to be replaced with your local paths
bpsDirPath <- "./dataRaw/envTerr/LF2020_BPS_220_CONUS/"
ncBoundaryPath <- "C:/Users/Eliot-KDV/Desktop/NCDOT_State_Boundary/NCDOT_State_Boundary.shp"
# Read in biophysicall setting raster data
bpsRaw <- terra::rast(paste0(bpsDirPath, "Tif/LC20_BPS_220.tif"))
# Read in codebook for bps categories
codeBook <- read.csv(paste0(bpsDirPath, "CSV_Data/LF20_BPS_220.csv"))
# Read in North Carolina state boundary
ncBoundary <- read_sf(ncBoundaryPath)
# Set levels of biophysical setting to category names provided in codebook instead
# of category codes. This step is unnecessary but makes plot more readable
levels(bpsRaw) <- dplyr::select(codeBook, VALUE, BPS_NAME)
# Take a look before any spatial operations, note that North Carolina countains
# numerous different levels
plot(bpsRaw)
# Transform ncBoundary to epsg:5070 so bps and ncBoundary share the same CRS
ncBoundary <- st_transform(ncBoundary, "epsg:5070")
# Crop bps to north carolina boundary
bpsNc <- terra::crop(bpsRaw, vect(ncBoundary), mask = TRUE)
# Look after cropping to NC boundary, now it only contains Open Water and
# Barren-Rock/Sand/Clay
plot(bpsNc)
After cropping the biophysical setting raster to the North Carolina boundary the warning "detected values outside of the limits of datatype INT1U" is displayed.
I've attempted using terraOptions() to set the default datatype to INT2S to no avail. If anyone could explain why this is happening and how I could correct it, that would be great!
Update:
Code for Existing Vegetation Height:
# This does not work as expected
library(terra)
evhRaw <- terra::rast("./dataRaw/envTerr/LF2022_EVH_220_CONUS/Tif/LC22_EVH_220.tif")
nc <- terra::vect("./dataTidy/cadastral/NCDOT_State_Boundary/NCDOT_State_Boundary.shp")
ncp <- project(nc, evhRaw)
evhNc <- terra::crop(evhRaw, ncp, mask = TRUE)
# This is where the issue occurs
evhNcPlane <- terra::project(evhNc, "epsg:2264")
Code for Biophysical Setting:
# This code works as expected
library(terra)
bpsRaw <- terra::rast("./dataRaw/envTerr/LF2020_BPS_220_CONUS/Tif/LC20_BPS_220.tif")
nc <- terra::vect("./dataTidy/cadastral/NCDOT_State_Boundary/NCDOT_State_Boundary.shp")
ncp <- project(nc, bpsRaw)
bpsNc <- terra::crop(bpsRaw, ncp, mask = TRUE)
bpsNcPlane <- terra::project(bpsNc, "epsg:2264")
Are you using the current version of "terra"? I am asking because this works well for me:
library(terra)
bpsRaw <- terra::rast("./LF2020_BPS_220_CONUS/Tif/LC20_BPS_220.tif"))
## this is how you change the category of interest
activeCat(bpsRaw) <- "BPS_NAME"
ncp <- project(nc, bpsRaw)
bpsNc <- terra::crop(bpsRaw, ncp, mask = TRUE)
I have a shapefile that I made in qgis of a national park with landcover type. I clipped a large shapefile of Thailand with a smaller shapefile of just the park. This file (DPKY.lc5) is now 10.5 Mb. When I run the code it takes forever and has not been successful. Why is that?
library(raster)
DPKY.lc5 <- shapefile("dpky.lc5.shp")
DPKY.lc5 <- spTransform(DPKY.lc5, CRS('+init=EPSG:4326'))
DPKY.lc5$VALUE<-as.numeric(DPKY.lc5$VALUE)
rr <- raster(DPKY.lc5, res=0.01)
rr1 <- rasterize(DPKY.lc5, rr, field="VALUE")
rr1
I would expect this to work as it is only 10.5Mb but it takes forever. I need this raster to work so I can use data points in another data frame to show the frequency of elephant habitat use in the park. I succeeded with other raster files to get elevation and slope aspect but this time it doesn't like it.
When I create a raster of the world land based on wrld_simpl (or any other environmental layer coming from worldclim) always appear to be some "impossible" NAs on land. Why would that happen? I need a perfect mask of the world land to excerpt records that did not fall in the ocean. However, there are many records on land and still are considered NA.
My script goes like this:
require(raster)
require(maptools)
data(wrld_simpl)
x=read.csv("https://www.dropbox.com/s/ncvu64r2fxgfd4e/NAlocations.csv?dl=0")
r=raster(ncols=360,nrows=(180))
extent(r)=extent(wrld_simpl)
r=rasterize(wrld_simpl,r,wrld_simpl$AREA)
plot(r)
x=x[-which(is.na(extract(r,x$lon,x$lat))),]# This should eliminate all locations on land.
points(x$lon,x$lat, col="red", cex=.3)
How is that possible? And would it be a way to create a clean raster for the world land?
The direct read.csv from dropbox does not work for me.
If I do
z <- extract(r, x)
# NOT z <- extract(r, x[,1], x[,2]) !!!
i <- which(is.na(z))
points(x[i,])
I see a bunch of points in the water of the coast of Mozambique.
I have a NetCDF file of global oceanographic (OmegaA) data at relatively coarse spatial resolution with 33 depth levels. I also have a global bathymetry raster at much finer resolution. My goal is to use get the seabed OmegaA data from the NetCDF file, using the bathymetry data to determine the desired depth. My code so far;
library(raster)
library(rgdal)
library(ncdf4)
# Aragonite data. Defaults to CRS WGS84
ncin <- nc_open("C:/..../GLODAPv2.2016b.OmegaA.nc")
ncin.depth <- ncvar_get(ncin, "Depth")# 33 depth levels
omegaA.brk <- brick("C:/.../GLODAPv2.2016b.OmegaA.nc")
omegaA.brk <-rotate(omegaA.bkr)# because netCDF is in Lon 0-360.
# depth raster. CRS WGS84
r<-raster("C:/....GEBCO.tif")
# resample the raster brick to the resolution that matches the bathymetry raster
omegaA.brk <-resample(omegaA.brk, r, method="bilinear")
# create blank final raster
omegaA.rast <- raster(ncol = r#ncols, nrow = r#nrows)
extent(omegaA.rast) <- extent(r)
omegaA.rast[] <- NA_real_
# create vector of indices of desired depth values
depth.values<-getValues(r)
depth.values.index<-which(!is.na(depth.values))
# loop to find appropriate raster brick layer, and extract the value at the desired index, and insert into blank raster
for (p in depth.values.index) {
dep.index <-which(abs(ncin.depth+depth.values[p]) == min(abs(ncin.depth+depth.values[p]))) ## this sometimes results in multiple levels being selected
brk.level <-omegaA.brk[[dep.index]] # can be more than on level if multiple layers selected above.
omegaA.rast[p] <-omegaA.brk[[1]][p] ## here I choose the first level if multiple levels have been selected above
print(paste(p, "of", length(depth.values.index))) # counter to look at progress.
}
The problem: The result is a raster with massive gaps (NAs) in it where there should be data. The gaps often take a distinctive shape - eg, follow a contour, or along a long straight line. I've pasted a cropped example.
enter image description here
I think this could be because either 1) for some reason the 'which' statement in the loop is not finding a match or 2) a misalignment of the projections is created which I've read can happen when using 'Rotate'.
I've tried to make sure all the extents, resolutions, number of cells, and CRS's are all the same, which they seem to be.
To speed up the process I've cropped the global brick and bathy raster to my area of interest, again checking that all the spatial resolutions, etc etc match - I've not included those steps here for simplicity.
At a loss. Any help welcome!
Without a reproducible example, this kind of problems is hard to solve. I can't tell where your problem is but I'll present to you the approach I would try. Maybe it's good, maybe it's bad, I don't know but it may inspire you to find a way to go around your problem.
To my understanding, you have a brick of OmegaA (33 layers/depth) and a bathymetry raster. You want to get the OmegaA value at the bottom of the sea. Here is how I would do:
Make OmegaA raster to the same resolution and extent to the bathymetry one
Transforme the bathymetry raster into a raster brick of 33 three layers of 0-1. e.g. If the sea bottom is at 200m for one particular pixel, than this pixel on all depth layer other than 200 is 0 and 1 for the 200. To program this, I would go the long way, something like
:
r_1 <- r
values(r_1) <- values(r)==10 # where 10 is the depth (it could be a range with < or >)
r_2 <- r
values(r_2) <- values(r)==20
...
r_33 <- r
values(r_33) <- values(r)==250
r_brick <- brick(r_1, r_2, ..., r_33)
then you multiple both your raster bricks. They have the same dimension, it should be easy. The output should be a raster brick of 33 layers with 0 everywhere where it isn't the bottom of the sea and the value of OmegaA anywhere else.
Combine all the layer of the brick obtained previously into a simple raster with a sum.
This should work. If you have problem with dealing with raster brick, you could make the data into base R arrays, it could be simpler.
Good luck.
I have a raster layer with climate data from the north of Mexico, part of Canada and the US mainland. I am trying to restrict the climate data to only the zone of the US mainland. To do this I thought it would be easy to import a US mainland map and overlay it with my raster data (of course this is turning out to be a lot more difficult than I thought). So far, using the rgdal library, I have managed to import a shapefile including the USA mainland map with states divisions. Now I want to convert this into a raster layer so that can finally overlay it with my initial climate raster layer. This is the code that I am using:
setwd ("C:/Climate_data/USA map")
ogrInfo(".", "USA_mainland_states")
usa_state = readOGR(dsn=".", layer="USA_mainland_states")
##Convert to Raster
r_usa_state <- raster()
extent(r_usa_state) <- extent(usa_state)
rasterize(usa_state,r_usa_state, fun='last')
overlay (r_usa_state, sms_av, fun='mask')
However, now I get the following error:
Error in .readCells(x, cells, 1) : no data on disk or in memory
sms_av is the climate rasterlayer (103936 elements, 823.3 kb).
Also, when I do:
hasValues(r_usa_state)
I get:
FALSE
What am I doing wrong?? Any advice would be GREATLY appreciated!!!!