I have a big list of raster stacks and I want to reproject and then clip them in R. I have done the same procedure in ArcGIS using batch processing which was significantly faster.
It is not going well in R. Any suggestion to improve the process?
my rstack.lst consist of 19 raster stacks like this:
class : RasterStack
dimensions : 4800, 7200, 34560000, 46 (nrow, ncol, ncell, nlayers)
resolution : 463.3127, 463.3127 (x, y)
extent : 11119505, 14455357, -5559753, -3335852 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs
names : ...
I am using this code:
for (i in 1:length(rstack.lst)){
rstack.lst[[i]] <- projectRaster(rstack.lst[[i]], crs = m.crs) # reproject
rstack.lst[[i]] <- crop(rstack.lst[[i]], brdshp) # crop
rstack.lst[[i]] <- mask(rstack.lst[[i]], brdshp) # mask
print (i)
}
not even one I is printed after two hours!
Related
I'm merging two MODIS DSR tiles using a R script that I developed, these are the products:
https://drive.google.com/drive/folders/1RG3JkXlbaotBax-h5lEMT7lEn-ObwWsD?usp=sharing
So, I open both products (tile h15v05 and tile h16v05) from same date (2019180), then I open each SDS and merge them together (00h from h15v05 with 00h from h16v05 and so on...)
Visualisation on Panoply (using the merge option) of the two products:
Purple square is the location of the division line that separates the two tiles.
With my code I obtain a plot with pixels with different resolution (and different min/max values) and I don't understand why:
I suspect that the results obtained are due to:
1- Changing from Sinusoidal CRS to longlat WGS84 CRS;
2- Using resample (method ngb) to work with mosaic.
My code is extensive, but here are some parts of it:
# Open scientific dataset as raster
SDSs <- sds(HDFfile)
SDS <- SDSs[SDSnumber]
crs(SDS) <- crs("+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs")
SDSreprojected <- project(SDS, DesiredCRS)
SDSasRaster <- as(SDSreprojected, "Raster")
# Resample SDS based on a reference SDS (SDS GMT_1200_DSR of a first product), I need to do this to be able to use mosaic
SDSresampled <- resample(SDSasRaster,ResampleReference_Raster,method='ngb')
# Create mosaic of same SDS, but first convert stack to list to use mosaic
ListWith_SameSDS_OfGroupFiles <- as.list(StackWith_SameSDS_OfGroupFiles)
ListWith_SameSDS_OfGroupFiles.mosaicargs <- ListWith_SameSDS_OfGroupFiles
ListWith_SameSDS_OfGroupFiles.mosaicargs$fun <- mean
SDSmosaic <- do.call(mosaic, ListWith_SameSDS_OfGroupFiles.mosaicargs)
# Save SDSs mosaic stack to netCDF
writeRaster(StackWith_AllMosaicSDSs_OfGroupFiles, NetCDFpath, overwrite=TRUE, format="CDF", varname= "DSR", varunit="w/m2", longname="Downward Shortwave Radiation", xname="Longitude", yname="Latitude", zname="TimeGMT", zunit="GMT")
Does anyone have an idea of what could be the cause of this mismatch between results?
print(ResampleReference_Raster)
class : RasterLayer
dimensions : 1441, 897, 1292577 (nrow, ncol, ncell)
resolution : 0.01791556, 0.006942043 (x, y)
extent : -39.16222, -23.09196, 29.99652, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : MCD18A1.A2019180.h15v05.061.2020343034815
values : 227.5543, 970.2346 (min, max)
print(SDSasRaster)
class : RasterLayer
dimensions : 1399, 961, 1344439 (nrow, ncol, ncell)
resolution : 0.01515284, 0.007149989 (x, y)
extent : -26.10815, -11.54627, 29.99717, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : MCD18A1.A2019180.h16v05.061.2020343040755
values : 0, 0 (min, max)
print(SDSmosaic)
class : RasterLayer
dimensions : 1441, 897, 1292577 (nrow, ncol, ncell)
resolution : 0.01791556, 0.006942043 (x, y)
extent : -39.16222, -23.09196, 29.99652, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : layer
values : 0, 62.7663 (min, max)
Also, some of the islands were ignored by the script (bottom right)...
sorry I didn't reply earlier. So I think you're right that this issue is extent to which you are resampling. I think you might be able to get around this by creating a dummy raster that has the extent of the raster you want to resample, but has the resolution of the raster you want to mosaic to.Try:
dummy<-raster(ext = SDSasRaster#extent, resolution=ResampledReference_Raster#res, crs=SDSasRaster#crs)
SDS2<-resample(SDSasRaster, dummy, method="ngb")
Final<-moasic(SDS2, ResampledReference_Raster, fun=mean)
I have a DTM and DSM with different resolutions.
Here are the summaries of each Raster layer.
> raster_dsm
class : RasterLayer
dimensions : 2001, 2501, 5004501 (nrow, ncol, ncell)
resolution : 0.5, 0.5 (x, y)
extent : -112500.2, -111249.8, 388999.8, 390000.2 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=tmerc +lat_0=0 +lon_0=16.33333333333333 +k=1 +x_0=0 +y_0=-5000000 +ellps=bessel +units=m +no_defs
data source : D:/Test_Raster/DSM/dsm.asc
names : dsm
>raster_dtm
class : RasterLayer
dimensions : 1001, 1251, 1252251 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : -112500.5, -111249.5, 388999.5, 390000.5 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=tmerc +lat_0=0 +lon_0=16.33333333333333 +k=1 +x_0=0 +y_0=-5000000 +ellps=bessel +units=m +no_defs
data source : D:/Test_Raster/DTM/dtm.asc
names : dtm
As you can see, the resolution of the dtm is 1 m and the resolution of the dsm is 0.5m.
I want to calculate a Crown Heigth Model (CHM).
The easiest way is to
CHM = dsm - dtm
But when I try in R the following error code appear:
Error in compareRaster(e1, e2, extent = FALSE, rowcol = FALSE, crs = TRUE, :
different resolution
Is there a simple way to ignore the resolution? Or must I do a resampling of the data, before further calculation?
In ArcGis you can do this kind of raster calculation easily, because you don't have to resample the data first.
Any suggestions will be appreciated!
Yes, Arc*** will do this for you, but what does it actually do? I think it is better to avoid that kind of ambiguity. In this case you cannot use dis/aggregate because the extents are different. So you need to use resample
I have the following raster:
class : RasterLayer
dimensions : 1446, 1243, 1797378 (nrow, ncol, ncell)
resolution : 1000, 1000 (x, y)
extent : 4210000, 5453000, 3372000, 4818000 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=laea +lat_0=52 +lon_0=10 +x_0=4321000 +y_0=3210000 +ellps=GRS80 +units=m +no_defs
data source : -
names : ALL_2016_year_Raster
values : 0, 36293 (min, max)
It seems like it is a multiband raster but I would like to export it as a single band raster. when I import it, i used raster <- raster(file, band=1) but it does not change anything...
Is there a way to make it single band?
library(raster)
library(ncdf)
library(rgdal)
I am facing some issues converting UTM to latlon in a rasterBrick object. My code is below:
Example file can be found here:
https://www.dropbox.com/s/yv4opmrx1v4whpt/2013_000_CaPA_continental_v2.3-analyse_final_10km_6h_APCP.nc.7z?dl=0
rasnc<- brick('file.nc', varname = "APCP_surface")
rasnc
#class : RasterBrick
#dimensions : 824, 935, 770440, 1460 (nrow, ncol, ncell, nlayers)
#resolution : 10000, 10000 (x, y)
#extent : -5000, 9345000, -5000, 8235000 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
wgs84.p4s <- "+proj=longlat +datum=NAD83 +ellps=GRS80 +no_defs"
# reprojecting
rx <- projectRaster(from=rasnc, crs=wgs84.p4s,method="ngb")
The above does not convert from UTM to latlon. I even tried to write the file to GeoTIFF and then re-project but it still did not work.
writeRaster(rasnc, file="myfil.tif", format="GTiff", overwrite=TRUE)
Basically I am trying to:
read a netCDF file.nc using brick
convert the UTM coordinates to latlon
crop the RasterBrick object to ext <- extent(-141.01807,-52.61941,41.68144.0,83.13550)
display any of the layers using levelplot function.
The time series is 6-hourly from Jan-01-2013 to dec-01-2013
It is true my use of the older raster package version was the source of the problem as suggested by #RobertH. rasnc has no coord.ref. From the webpage (http://weather.gc.ca/grib/grib2_RDPA_ps10km_e.html) of the data modellers, it is stated a polar-stereographic (PS) grid covering North America and adjacent waters with a 10 km resolution at 60 degrees north is used. How can I get the right PS for rasnc after which I will apply projectRaster to rasnc to get latlon coordinates
The problem is that rasnc reports that it has a lon/lat coordinate reference system (#coord. ref. : +proj=longlat +datum=WGS84); which looks incorrect. If you know what it should be, (you say UTM, but what is the zone?), assign it:
crs(rasnc) <- '+proj=utm +zone=??? +datum=WGS84'
and then things should work.
The question is, how did this go wrong?
I get:
library(raster)
r <- brick("2013_000_CaPA_continental_v2.3-analyse_final_10km_6h_APCP.nc")
#Loading required namespace: ncdf4
#r
#class : RasterBrick
#dimensions : 824, 935, 770440, 1460 (nrow, ncol, ncell, nlayers)
#resolution : 10000, 10000 (x, y)
#extent : -5000, 9345000, -5000, 8235000 (xmin, xmax, ymin, ymax)
#coord. ref. : NA
which is correct as this file does not provide coordinate reference system information.
First update your version of raster (you are using ncdf instead of ncdf4, so we can see it is old); and do not leave out code (if you set the coord. ref. system yourself)
I have a large raster file (245295396) cells and stacks of rasters having 4 layers each which lie in the extent of this large raster. To start with, I am trying to get value from one stack (3 channels) and for the same zone from the large raster. Every things works fine, just the extraction from large raster takes 5 mins. So, if I repeat this process for 4000 more times it will take 13 days.
cld<- raster("cdl_30m_r_il_2014_albers.tif") #this is the large raster
r<- stack(paste(path,"/data_robin/", fl,sep="")) #1 stack,I have 4000 similar
mat<-as.data.frame(getValues(r)) # getting values from the stack
xy<-xyFromCell(r,c(1:ncell(r)),spatial = TRUE)
clip1 <- crop(cld, extent(r)) # Tried to crop it to a smaller size
cells<-cellFromXY(clip1,xy)
mat$landuse<- NA
# mat$landuse<-cld[cells]
mat$landuse<- extract(clip1,cells) #this line takes 5 mins based on profiling
> cld
class : RasterLayer
dimensions : 20862, 11758, 245295396 (nrow, ncol, ncell)
resolution : 30, 30 (x, y)
extent : 378585, 731325, 1569045, 2194905 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=aea +lat_1=29.5 +lat_2=45.5 +lat_0=23 +lon_0=-96 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs +ellps=GRS80 +towgs84=0,0,0
data source : /Users/kaswani/R/Image/cdl_30m_r_il_2014_albers.tif
names : cdl_30m_r_il_2014_albers
values : 0, 255 (min, max)
> r
class : RasterStack
dimensions : 9230, 7502, 69243460, 4 (nrow, ncol, ncell, nlayers)
resolution : 0.7995722, 0.7995722 (x, y)
extent : 589084.4, 595082.8, 1564504, 1571884 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=aea +lat_1=29.5 +lat_2=45.5 +lat_0=23 +lon_0=-96 +x_0=0 +y_0=0 +ellps=GRS80 +towgs84=0,0,0,0,0,0,0 +units=m +no_defs
names : m_3608906_ne_16.1, m_3608906_ne_16.2, m_3608906_ne_16.3, m_3608906_ne_16.4
min values : 0, 0, 0, 0
max values : 255, 255, 255, 255
My data is in .tiff format and I am new to geospatial coding. Will really appreciate any suggestions to improve the speed. I have also tried this approach but during the masking part it gives the error:
Error in compareRaster(x, mask) : different extent.