I was just wondering if anyone has a conversion table for the resolution of rasters in R. I would like to know what numbers like these refer to in meters:
resolution : 0.08333333, 0.08333333
resolution : 0.009398496, 0.009398496
resolution : 0.002349, 0.002349 = 250m (I think)
I would really like to know what resolution to set a raster object to make cell sizes 1km^2. I am using rasters that span the country of Australia.
Thanks in advance everyone.
Cheers,
Adam
It all depends on the units of your raster, and that depends on the projection. Rasters might not even be square grids in metres - they might be square in degrees which aren't square in metres!
1 degree longitude at the equator is 1/360 of the earth's circumference. Near the north pole 1 degree is a much smaller distance, and at the pole its pretty much zero. Degrees of latitude however are constant.
You could take the corner points of your raster, convert them to lat-long coordinates if not already, and then work out the great-circle distance between them (there's an rdist function somewhere that does this I recall). However this won't work if your raster spans the whole globe, since then your NW corner and your NE corner are at the same point... Ummm. Anyway, the answer is... 42.
If you want to make 1km rasters of Australia then.... you need a coordinate system of Australia in kilometres. In the UK we have a system called the OSGB National Grid, which is close enough to a metric grid. Australia might be trickier because it is slightly bigger than the UK... So Australia seems to have a few grid systems. See here:
http://www.spatialreference.org/ref/?search=AGD84
So you might want to use the system that is in the middle of the country to avoid the worst distortions, then work out the bounds of Australia in lat-long, convert to epsg:20353 and create a raster based on that:
In lat-long I reckon Australia is roughly:
> xtll
[,1] [,2]
[1,] 112.5162 -43.906900
[2,] 155.8425 -7.585619
make this into a SpatialPoints object:
> xtll=SpatialPoints(xtll,CRS("+init=epsg:4326"))
convert to that AGD84 in the middle of the country:
> spTransform(xtll,CRS("+init=epsg:20353"))
SpatialPoints:
coords.x1 coords.x2
[1,] -1306200 4886041
[2,] 2849956 9103124
Make a raster extent object rounded to km:
> ext = extent(-1306000,2850000,4886000,9103000)
How many rows and columns do we need?
> length(-1306:2850)
[1] 4157
> length(4886:9103)
[1] 4218
Create a raster:
> r = raster(ext,ncol=4156,nrow=4217,crs="+init=epsg:20353")
> r
class : RasterLayer
dimensions : 4217, 4156, 17525852 (nrow, ncol, ncell)
resolution : 1000, 1000 (x, y)
extent : -1306000, 2850000, 4886000, 9103000 (xmin, xmax, ymin, ymax)
coord. ref. : +init=epsg:20353 +proj=utm +zone=53 +south +ellps=aust_SA +units=m +no_defs
values : none
Note that the ncol and nrow values are one less than the values from the bounds - this would be a fencepost error to put those values in.
See how my resolution is 1000? This is a 1km grid. The problem is that this is possibly going to be a bit distorted on the coasts. You could work out how distorted by converting to lat-long (epsg:4326), then to the proper AGD zone for points on the coast, and seeing how different they are. They might be very close, except for an offset.
Anyway, nuff said.
Finding the meta-data that gives meaning to your raster can be a bit of a challenge. I have spent lots of time hunting for this. If the raster was published by a government agency, then I would hope that this information is posted somewhere prominently.
The good news is that once you know the projection used on the various rasters, you can convert them to a common projection using projectRaster() in the raster package. You need to find the proj.4 string describing the original and the desired projections in each case. You can get this from: http://www.spatialreference.org.
When you know your projection, then the resolution information you seek will have meaning.
Related
After some extensive googling, I wasn't able to find my answer (first time I couldn't surmount the issue by looking at others questions/answers). I am new to asking questions, so forgive any missteps.
I am attempting to perform what ArcGIS or QGIS performs with the slope tool, just within R. To do so I have been importing a raster that I exported from ArcGIS in GRID format with the following characteristics:
class : RasterLayer
dimensions : 821, 581, 477001 (nrow, ncol, ncell)
resolution : 4.996121, 4.996121 (x, y)
extent : 2832147, 2835049, 14234048, 14238150 (xmin, xmax, ymin, ymax)
crs : +proj=tmerc +lat_0=34.75 +lon_0=-118.583333333333 +k=0.9999 +x_0=800000.000000001 +y_0=3999999.99999999 +datum=NAD83 +units=us-ft +no_defs
source : rr_2020_shell
names : rr_2020_shell
values : 5623.253, 6401.356 (min, max)
It is already projected in the correct coordinate system (EPSG: 3423) but when I go to find the slope using the following code:
RR_2020_Slope = terrain(RR_2020_St1_Raster,'slope', units = 'degrees', neighbors = 8, filename = 'RR_2020_Slope.grd', overwrite = T)
The result is a slope raster that ranges from 0 to 1.28°, which is very different from what I have calculated in ArcGIS using the slope tool. Using the same DEM raster in the same projection in ArcGIS I used the slope tool with an input of 'Degree' for the output measurement, 'Planar' for the method, and 1 for Z factor and my resulting slope raster ranges from 0.001 to 73.396°.
Overall I am wondering where my mistake in R originates from, is it an elevation resolution problem? Are there issues with my projection? Forgive me, I can't necessarily include the data as they are sensitive materials but perhaps there is a clear and obvious mistake in my approach or assumptions about the functions I have used?
The only red flag I see is that you say "it is already projected in the correct coordinate system". Projecting raster data degrades the quality. As cell values get smoothed, the slopes will get smaller. This may be particularly pronounced if the relief is at the scale of the cell size (e.g. sand dunes vs mountain chains). Have you compared with what you get with the original data?
Another source of error could be that the units of the values are different from the units of the coordinate reference system. But it would appear that in your case both are in feet.
Can you also try this with terra::terrain()?
I'm working in R - I have a directory of raster (.tif) tiles at 60cm resolution, downloaded from Google Earth Engine (NAIP 2018 NDVI). (I am running my analysis on pc rather than in Google Earth Engine due to human subjects requirements of my polygon data.) The 52 tiles are each 1.2-3.8GB in size. I also have 982 polygons, for which I'm trying to calculate the zonal means from these rasters. My code (below) uses the terra package, and instead of mosaic-ing the tiles into a very large singular raster, I've chosen to create a VRT (virtual raster) file.
I am running this code on a Xeon Gold 6134 # 3.2GHZ and have 128GB of ram. No matter what I set my terraOptions() to, R doesn't even come close to using a significant proportion of my processor or ram potential.
With this code, all 982 polygons will take 11.8 days to run. I would GREATLY appreciate if anyone could point me at specific tricks/tools that I may not have already tried (e.g., I've tried working with all the terraOptions, I've tried the raster package and the exact_extract package. The exact_extract() function won't work for me as I am using a SpatRaster/VRT and a sf polygon object as inputs - again to avoid mosaic-ing a very large singular raster.)
Thank you. (I apologize that I cannot share data, as its either too large or human subjects related...) Here is the un-looped code:
Edit: 52 tiles of 1.2-3.8GB EACH. My original quote of 150GB total directory size was incorrect as this was the compressed size in ArcGIS.
library(terra)
c <- "path/to directory of raster tiles"
v <- "path/new.vrt" # name of virtual raster
ras_lst <- list.files(c, full.names=T, pattern=".tif$")
terra::vrt(ras_lst, v, overwrite = T)
ras <- rast(v)
w <- vect("path to polygon shapefile")
w2 <- terra::project(w, terra::crs(ras)) # transform proj to same as raster tiles
e2 <- terra::extract(ras, w2, fun="mean")
e2 # zonal mean value for 1 polygon (of 982)
show(ras) produces:
class : SpatRaster
dimensions : 212556, 247793, 1 (nrow, ncol, nlyr)
resolution : 5.389892e-06, 5.389892e-06 (x, y)
extent : -118.964, -117.6284, 33.68984, 34.8355 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +no_defs
source : naip2018mos.vrt
name : naip2018mos
I have a large raster file (5GB) containing only 1's and NA's. I would like to convert this into a multipolygon of the areas with 1's, with adjacent cells dissolved into one polygon.
I have imported the file to R using
r = raster::raster(my_filename)
r
class : RasterLayer
dimensions : 17452, 45000, 785340000 (nrow, ncol, ncell)
resolution : 0.008, 0.008 (x, y)
extent : -180, 180, -55.9875, 83.6285 (xmin, xmax, ymin, ymax)
crs : NA
source : C://...binary_X01_januarysnow.asc
names : binary_X01_januarysnow
and
I have tried several methods to create the polygon:
rasterToPolygons from raster with dissolve==TRUE option (R crashes)
isoband from the isoband package (R crashes),
Both of the approaches have worked as expected when I've tried them on a subset of my raster covering appr. the area of Spain, so I assume the problem is only with the size of the data and not my code.
Then I have tried to read my raster with read_stars, and use stars::st_as_sf(st, as_points = FALSE, merge = TRUE, connect8 = TRUE). This returned an empty polygon, possibly because the file was read as a stars proxy object, but I'm not sure, I couldn't find any information about that online.
Then I have force-read the raster as stars and not as stars proxy by using read_stars(my_filename, proxy=FALSE) and have tried to use the st_as_sf command as above but got the message "Error: cannot allocate vector of size 2.9 Gb"
I know that in the worst case I can probably just decrease the raster resolution and therefore size and will be able to create the polygons I want (but with less precise resolution), but I was wondering if anyone has another suggestion I could try? Both the 1's and NA's are located in large continous areas, so it would be enough to have high resolution on the edges, if that helps.
PS This is my very first question on StackOverflow so I apologize if my problem is not clearly described. I don't know how to provide a reproducible example of a large dataset.
What you are looking for is as.polygons() from the terra package, the raster package's successor. terra handles large data sets better than raster does.
I've downloaded precipitation data from the TRMM (rainfall across the tropics) satellite as a netCDF file and have been trying to plot the data in R as a rasterstack. However, R insists on plotting the latitude and longitude axes incorrectly, such that longitude is plotted on the x-axis (as it should be) but uses the latitude coordinates, while latitude is on the y-axis, but uses the longitude coordinates. I've tried using both the plot() and levelplot() functions but neither seems to work. Can anyone help me correct this?
These are the characteristic of the stack:
class : RasterStack
dimensions : 1440, 186, 267840, 12 (nrow, ncol, ncell, nlayers)
resolution : 0.25, 0.25 (x, y)
extent : -23.25, 23.25, -180, 180 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
names : X2016.01.16, X2016.02.15, X2016.03.16, X2016.04.15, X2016.05.16, X2016.06.15, X2016.07.16, X2016.08.16, X2016.09.15, X2016.10.16, X2016.11.15, X2016.12.16
Date : 2016-01-16, 2016-02-15, 2016-03-16, 2016-04-15, 2016-05-16, 2016-06-15, 2016-07-16, 2016-08-16, 2016-09-15, 2016-10-16, 2016-11-15, 2016-12-16
In the following image you can see the current output. It should show rainfall over the tropics from -23 to 23 degrees latitude, and -180 to 180 degrees longitude.
It's odd if the coordinates are switched prior to any processing. Maybe you want to asses the source you downloaded the data from and if there's maybe a better one.
Anyways, (in the meantime) the raster package can be of help for you .. specifically the transpose t() function. Here's an example:
# data before transpose
x <- getData('worldclim',var='tmean',res=10)
plot(x)
# data after transpose
y <- t(x)
plot(y)
There are also a couple of other functions in raster which could be of interest for you: flip and rotate
HTH
thanks for your response. It does seem strange that the coordinates are messed up right out of the box, and I did try downloading a fresh set of data and the same problem occurred. However, thanks to your input, I was able to rectify the problem through using the transpose() and flip() functions. I had to transpose the data, then flip it along both the x and y dimensions as the image was 'mirrored'. Here's the code I used in case anyone else encounters this problem with the TRMM data sets:
a.t = t(test.rasterstack)
a.flipy = flip(a.t, direction = 2)
a.t.flipxy = flip(a.t.flipy, direction = 1)
levelplot(a.t.flipxy)
I have downloaded a text file of data from the following link: http://radon.unibas.ch/files/us_rn_50km.zip
After unzipping I use the following lines of code to plot up the data:
# load libraries
library(fields)
# function to rotate a matrix (and transpose)
rotate <- function(x) t(apply(x, 2, rev))
# read data
data <- as.matrix(read.table("~/Downloads/us_rn_50km.txt", skip=6))
data[data<=0] <- NA
# rotate data
data <- rotate(data)
# plot data
mean.rn <- mean(data, na.rm=T)
image.plot(data, main=paste("Mean Rn emissions =", sprintf("%.3f", mean.rn)) )
This all looks OK, but I want to be able to plot the data on a lat-long grid. I think I need to convert this array into an sp class object but I don't know how. I know the following (from the web site): "The projection used to project the latitude and longitude coordinates is that used for the Decade of North American Geology (DNAG) maps. The projection type is Spherical Transverse Mercator with a base latitude of zero degrees and a reference longitude of 100 degrees W. The scale factor used is 0.926 with no false easting or northing. The longitude-latitude datum is NAD27 and the units of the xy-coordinates are in meters. The ellipsoid used is Clarke 1866. The resolution of the map is 50x50km". But don't know what to do with this data. I tried:
proj4string(data)=CRS("+init=epsg:4267")
data.sp <- SpatialPoints(data, CRS("+proj=longlat+ellps=clrk66+datum=NAD27") )
But had various problems (with NA's) and fundamentally I think that the data isn't in the right format.. I think that the SpatialPoints function wants a data on location (in 2-D) and a third array of values associated with those locations (x,y,z data - I guess my problem is working out the x and the y's from my data!)
Any help greatly appreciated!
Thanks,
Alex
The file in question is an ASCII raster grid. Coordinates are implicit in this format; a header describes the position of the (usually) lower left corner, as well as the grid dimensions and resolution. After this header section, values separated by white space describe how the variable varies across the grid, with values given in row-major order. Open it in a text editor if you're interested.
You can import such files to R with the fantastic raster package, as follows:
download.file('http://radon.unibas.ch/files/us_rn_50km.zip',
destfile={f <- tempfile()})
unzip(f, exdir=tempdir())
r <- raster(file.path(tempdir(), 'us_rn_50km.txt'))
You can plot it immediately, without assigning the projection:
If you didn't want to transform it to another CRS, you wouldn't necessarily need to assign the current coordinate system. But since you do want to transform it to WGS84 (geographic), you need to first assign the CRS:
proj4string(r) <- '+proj=tmerc +lon_0=-100 +lat_0=0 +k_0=0.926 +datum=NAD27 +ellps=clrk66 +a=6378137 +b=6378137 +units=m +no_defs'
Unfortunately I'm not entirely sure whether this proj4string correctly reflects the info given at the website that provided the data (it would be great if they actually provided the definition in a standard format).
After assigning the CRS, you can project the raster with projectRaster:
r.wgs84 <- projectRaster(r, crs='+init=epsg:4326')
And if you want, write it out to a raster format of your choice, e.g.:
writeRaster(r.wgs84, filename='whatever.tif')