I'm new to the package stars for R and am trying to do basic spatial operations with curvilinear data. I am using netCDF climate data. I am able to read the netcdf into r along with a shapefile I would like to use to specify the area in which I want to conduct analyses. I have tried to crop the file directly using st_crop() but receive the following error:
Warning message:
In st_crop.stars(test, wrst) : crop only crops regular grids
I then tried to warp the stars object using code like this:
warp <- test %>% st_set_crs(3338) %>% st_warp(st_as_stars(st_bbox(), dx = 2))
but I get this error:
Error in colrow_from_xy(pts, x, NA_outside = TRUE) :
colrow_from_xy not supported for curvilinear objects
Do I need to 'flatten' my curvilinear grid in order to conduct analyses in a given region? If so, how do I do that? Or, conversely, if I am able to conduct operations like st_crop() or the equivalent of raster operations calc() or stackApply() using a curvilinear grid, can someone point me in the right direction? Thanks so much.
Well I figured it out and it was quite simple. I was able to subset the stars object using the shapefile with this simple code: test[wrst]. No warping or resampling necessary.
Is there an easy way to convert a Spatial Lines into a Spatial Polygon object within R?
Reproducible Example
I have put together a reusable dataset here, which is downloaded from OpenStreetMaps through the overpass package. This extracts the locations of a few airports in South England:
devtools::install_github("hrbrmstr/overpass")
library(overpass)
library(raster)
library(sp)
# Write Query
query_airport <- '
(node["aeroway"="aerodrome"](50.8, -1.6,51.1, -1.1);
way["aeroway"="aerodrome"](50.8, -1.6,51.1, -1.1);
relation["aeroway"="aerodrome"](50.8, -1.6,51.1, -1.1);
);
out body;
>;
out skel qt;
'
# Run query
shp_airports <- overpass::overpass_query(query_airport, quiet = TRUE)
crs(shp_airports) <- CRS("+init=epsg:4326") # Add coordinates
shp_airports <- shp_airports[,1]
# Plot Results
plot(shp_airports, axes = T)
However, the data is of the class "SpatialLinesDataFrame". This really messes things up if you want to do any form of spatial joins or intersections, as it only acknowledges the edge of the region.
Potential Leads
I was exploring the use of SpatialLines2PolySet within the maptools package, but in my time exploring I produced nothing but error codes, so I didn't think there would be any worth including these within the question. There is some guidance about these functions here: https://rdrr.io/rforge/maptools/man/SpatialLines2PolySet.html
Notes
I have searched the web and SO to see find similar questions and struggled to find any questions directly referring to this. A lot seem to reference converting SpatialPoints -> SpatialLineDataFrames , but not SpatialLineDataFrames -> SpatialPolygonDataFrames. This question is similar but lacks any answers (or a reproducible dataset): Close a spatial line into a polygon using a shapefile
In addition, it seems strange that this would be difficult as it is something which can be done so easily in ArcGIS using the "Feature to Polygon" tool. This function requires no additional arguments specified and it works perfectly.
A way to solve the problem would be to use the library sf. After your query
library(sp)
library(raster)
library(sf)
sf_airports <- st_as_sf(shp_airports)
sf_airports_polygons <- st_polygonize(sf_airports)
shp_airports <- as(sf_airports_polygons, "Spatial") # If you want sp
class(shp_airports)
We have a dataset that contains latitude and longitude coordinates, as well as attribute information, each in its own separate column, stored as numeric. These coordinates have been geocoded based on the geographic coordinate system WGS 1984.
We know that we have significant spatial autocorrelation in our data, which we are hoping to visualize in a bubble plot using the “sp” package. We are modeling our example off of others online, such as here: https://beckmw.wordpress.com/2013/01/07/breaking-the-rules-with-spatial-correlation/ . However, when we try to use the coordinates command within "sp", we keep getting an error message:
Code example:
coords <- data.frame(lead$X, lead$Y)
coordinates(coords) <- c("lead6.X","lead6.Y")
Error in if (nchar(projargs) == 0) projargs <- as.character(NA) missing value where TRUE/FALSE needed
We can't load our direct code because it's sensitive and hosted on a virtual environment without access to the internet. Does anyone have ideas for why this might be happening? We've looked into the proj4 package but can't figure out how to specify a projection system (or is that even the error that we are getting?). If anyone knows of any other packages in R or ways to visualize spatial autocorrelation, those would be much appreciated too.
Your code is a bit "strange": seems you are trying to build a dataset containing only coordinates. AFAIU, you may need something in this line :
data <- data.frame(lead$X, lead$Y, lead$Z)
,with lead$Z corresponding to a generic "variable" you want to inspect, then
coordinates(data) <- c('X','Y')`
proj4string(data) <- "+init=epsg:4326"
, which should give you a proper "SpatialPointsDataframe" with lat-lon WGS84 geographic coordinates (the first line could be also dropped, and you'll keep all variables in the data of the spatialpointsdataframe).
HTH
summary
How to limit display of an R raster::plot to the bounds of a Raster* object? Why I ask:
I'm an R beginner who must
convert unprojected (lon-lat) global spatial data from ASCII to netCDF (solved)
"regrid" it: i.e., restricted the data to a region, project it, and downscale (decrease the size of the gridcells) it (mostly solved)
Unfortunately, in scientific work, one mostly QAs data transformations such as this by visual inspection. Step 1 above looks good. But though step 2 now looks much better, I really want to plot only the bounds (or extents) of the regridded RasterLayer. I have R code driven by bash scripts in a public git repository (here) that does the conversion step, and plots the converted output, apparently correctly. However my attempts to plot the output of the regridding with raster::plot are not quite right (see first 3 pages of output here).
How to fix?
details
background
I need to take some data (a global marine emissions inventory (EI)), combine it with other data (mostly other EIs), and input that to a model. The model wants to consume netCDF, and it wants that netCDF over a spatial domain slightly bigger than the contiguous US (CONUS). The borders of this image are the boundaries of the domain. (Note that part, but only part, of the domain is oceanic.) The model also wants that netCDF projected a certain way (LCC at 12-km resolution). I'm treating this as 2 independent problems:
convert the global EI from its native ASCII format to netCDF
"regrid" the netCDF from global/unprojected to the downscaled (finer-resolution), projected subdomain.
I'm attempting to solve these problems with code I have archived at this public git repository. If you clone that git repo, and then configure/run the bash driver GEIA_to_netCDF.sh as described for the "first example" in the README (and presuming your R is appropriately configured, notably with packages=raster, ncdf4) it will output netCDF and plot (using fields::image.plot) that output to a PDF, which should look like this: . The distribution of the output data appears correct, so problem 1 seems solved.
problem
Solving problem 2 (aka the "second example" in the README) seems to require R package=raster. My bash driver regrid_GEIA_netCDF.sh calls regrid.global.to.AQMEII.r (both in repository), which runs without error, and displays a PDF of its output. GEIA_N2O_oceanic_regrid.pdf comprises 4 pages, corresponding to 4 blocks of code at the end of regrid.global.to.AQMEII.r. Only the first 3 pages are relevant here (the 4th page tries to use fields::image.plot and has bigger problems).
Page=1/4 results from a simple raster::plot of the regridded output, plus a projected version of a CONUS map from wrld_simpl:
plot(out.raster)
plot(map.us.proj, add=TRUE)
Unfortunately the output appears global, or nearly so: But the desired domain, to which the output was regridded, is much smaller: . So I have 3 questions (1 main question, 2 followups):
Does the image displayed by raster::plot by default display only the extents of the RasterLayer given in its arguments?
If so, what is wrong with my regrid? (more below)
If not (i.e., if raster::plot by default displays more than the extents of its RasterLayer), how to make raster::plot display only the extents of that RasterLayer?
The code in regrid.global.to.AQMEII.r (here) that does the regridding seems to get the correct output:
out.crs <- '+proj=lcc +lat_1=33 +lat_2=45 +lat_0=40 +lon_0=-97 +x_0=-2556000 +y_0=-1728000'
Note that CRS appears to match the definition of the output domain given here. (Once at the linked page, scroll past the map.)
out.raster <-
projectRaster(
from=in.raster, to=template.raster, method='bilinear', crs=out.crs,
overwrite=TRUE, progress='window', format='CDF',
# args from writeRaster
NAflag=-999.0, # match emi_n2o:missing_value,_FillValue (TODO: copy)
varname=data.var.name,
varunit='ton N2O-N/yr',
longname='N2O emissions',
xname='COL',
yname='ROW',
filename=out.fp)
out.raster
# made with CRS
#> class : RasterLayer
#> dimensions : 299, 459, 137241 (nrow, ncol, ncell)
#> resolution : 53369.55, 56883.69 (x, y)
#> extent : -14802449, 9694173, -6258782, 10749443 (xmin, xmax, ymin, ymax)
# ??? why still proj=longlat ???
#> coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
#> data source : /home/rtd/code/R/GEIA_to_netCDF/GEIA_N2O_oceanic_regrid.nc
#> names : N2O.emissions
#> zvar : emi_n2o
But, as noted, the regrid output (out.raster) shows itself to be lon-lat in its CRS: I'm not sure why that is, or if it implies out.raster is global in extents.
I also tried to restrict the plot itself in two ways:
Firstly, I tried adding an extents to the plot, i.e.,
plot(out.raster, ext=template.raster)
plot(map.us.proj, add=TRUE)
which generates page=2/4 of the PDF. Unfortunately, that does not change the output at all: AFAICS, it is completely identical to page=1/4 (above).
Secondly, I tried using raster::crop to bound the regridded netCDF itself, before plotting that, using the same RasterLayer object (template.raster) that I used to bound the regrid:
template.raster.extent <- extent(template.raster)
out.raster.crop <-
# overwrite the uncropped file
crop(out.raster, template.raster.extent, filename=out.fp, overwrite=TRUE)
...
plot(out.raster.crop)
plot(map.us.proj, add=TRUE)
which generates page=3/4 of the PDF. Unfortunately, that it is also apparently completely identical to page=1/4 (above).
(In case you're wondering, page 4 of the PDF was generated using fields::image.plot, which has a different problem, described here, unless StackOverflow whacks that link.)
Your assistance to this R newbie is appreciated!
summary
My question
How to limit display of an R `raster::plot` to the bounds of a `Raster*` object?
was based on my incorrect diagnosis of the problem. The solution was to set the bounds (or extent) of the underlying Raster* data object (i.e., the source of the data for the plot) correctly, and particularly
to get the bounds of the template object (used to create the data object) by querying its underlying file (make no assumptions!)
to get the CRS of the template object by querying its underlying file (make no assumptions!)
to set the bounds and CRS for the template object directly
But
don't try to just set the resolution on the Extent; for me at least, that hangs
there is still one minor bounding problem with the raster::plot map; at least, relative to the map I'm using with fields::image.plot
details
The answer to this question is actually the answer to a different question: how to correctly set the extent of a Raster* object? because the raster::plot problem mostly went away once I properly set the extent of the object I wanted to plot. (With one minor exception--see below.) This may have been what Robert J. Hijmans, the main raster developer, was trying to convey in this post, but if so, I did not perceive it at the time.
I solved the problem after getting two suggestions which weren't themselves what I needed, but which set me on the proper path. In this case, that path led to the R package M3, which is useful for dealing with data input to and output by CMAQ and WRF. The suggestions were
Use project.lonlat.to.M3 for the regridding task.
The plot problem may be related to the assumption by the generating model (CMAQ) of a spherical projection.
The second suggestion seemed plausible, since I knew that I was getting and setting a CRS with +ellps=WGS84. (See profusely-commented R in this repository, specifically regrid.global.to.AQMEII.r.) So I resolved to look into that.
I knew the first suggestion would only work with difficulty (since it only projects points), but looked at the M3 doc just to be sure. The following functions in its ToC immediately caught my eye:
get.proj.info.M3, which not only returned a spherical PROJ.4 string from the template file, but one without the false eastings and northings which I believed I needed to supply:
# use package=M3 to get CRS from template file
out.crs <- get.proj.info.M3(template.in.fp)
cat(sprintf('out.crs=%s\n', out.crs)) # debugging
# out.crs=+proj=lcc +lat_1=33 +lat_2=45 +lat_0=40 +lon_0=-97 +a=6370000 +b=6370000
get.grid.info.M3, which can, with a bit of effort, return the bounds/extent of the template file:
extents.info <- get.grid.info.M3(template.in.fp)
extents.xmin <- extents.info$x.orig
extents.xmax <- max(
get.coord.for.dimension(
file=template.in.fp, dimension="col", position="upper", units="m")$coords)
extents.ymin <- extents.info$y.orig
extents.ymax <- max(
get.coord.for.dimension(
file=template.in.fp, dimension="row", position="upper", units="m")$coords)
template.extents <-
extent(extents.xmin, extents.xmax, extents.ymin, extents.ymax)
One can then set those bounds on the template Raster*
template.in.raster <- raster(template.in.fp, ...)
template.raster <- projectExtent(template.in.raster, crs=out.crs)
template.raster#extent <- template.extents
and use the template to regrid the input Raster*
out.raster <-
projectRaster(
# give a template with extents--fast, but gotta calculate extents
from=in.raster, to=template.raster, crs=out.crs,
# give a resolution instead of a template? no, that hangs
# from=in.raster, res=grid.res, crs=out.crs,
method='bilinear', overwrite=TRUE, format='CDF',
# args from writeRaster
NAflag=-999.0, # match emi_n2o:missing_value,_FillValue (TODO: copy)
varname=data.var.name,
varunit='ton N2O-N/yr',
longname='N2O emissions',
xname='COL',
yname='ROW',
filename=out.fp)
# above fails to set CRS, so
out.raster#crs <- CRS(out.crs)
(As suggested above, regridding by setting the template took only 7 sec, but regridding by setting a grid resolution failed to complete after 2 hr, when I killed the job.) After that, the raster::plot
map.us.unproj <- wrld_simpl[wrld_simpl$ISO3 %in% c('CAN', 'MEX', 'USA'),]
map.us.proj <-
spTransform(map.us.unproj, CRS(out.crs)) # projected
...
pdf(file=pdf.fp, width=5.5, height=4.25)
...
plot(out.raster, # remaining args from image.plot
main=title, sub=subtitle,
xlab='', ylab='', axes=F, col=colors(100),
axis.args=list(at=quantiles, labels=quantiles.formatted))
# add a projected CONUS map
plot(map.us.proj, add=TRUE)
was nearly as expected, with one exception: the map extends beyond the bounds of the data to the north and south (though not to the east and west), causing the image to insufficiently resemble published images of the domain, e.g. this. Interestingly, when I plot with fields::image.plot like
# see code in
# https://github.com/TomRoche/GEIA_to_netCDF/blob/master/plotLayersForTimestep.r
plot.raster(
raster=out.raster,
title=title,
subtitle=subtitle,
q.vec=probabilities.vec,
colors,
map.cmaq
)
I don't get that problem: . So I'll probably use fields::image.plot for display, at least for the moment.
I'm trying to use a kriging function to create vertical maps of chemical parameters in an ocean transect, and I'm having a hard time getting started.
My data look like this:
horiz=rep(1:5, 5)
depth=runif(25)
value = horiz+runif(25)/5
df <- data.frame(horiz, depth, value)
The autoKrige function in the automap package looks like it should do the job for me but it takes an object of class SpatialPointsDataFrame. As far as I can tell, the function spTransform in package rgdal creates SpatialPointsDataFrame objects, but there are two problems:
OSX binaries of this aren't available from CRAN, and my copy of RStudio running on OXS 10.7 doesn't seem to be able to install it, and
This function seems to work on lat/long data and correct distance values for the curvature of the Earth. Since I'm dealing with a vertical plane (and short distances, scale of hundreds of meters) I don't want to correct my distances.
There's an excellent discussion of kriging in R here, but due to the issues listed above I don't quite understand how to apply it to my specific problem.
I want a matrix or dataframe describing a grid of points with interpolated values for my chemical parameters, which I can then plot (ideally using ggplot2). I suspect that the solution to my problem is considerably easier than I'm making it out to be.
So there a a few question you want answered:
The spTransform function does not create SPDF's, but transforms between projections. To create a SPDF you can use a simple data.frame as a start. To transform df to a SPDF:
coordinates(df) = c("horiz", "depth")
OS X binaries of rgdal can be found at http://www.kyngchaos.com. But I doubt if you need rgdal.
spTransform can operate on latlong data, but also on projected data. But I do not think you need rgdal, or spTransform, see also point 1.
After you create the SPDF using point 1, you can use the info at the post you mentioned to go on.