I have a list containing 375 raster tiles that I would like to mosaic into one raster:
filelist_lc <- list.files("Northern_Land_Cover_2000/")
lc_2000_tiles <- lapply(filelist_lc, rast)
> lc_2000_tiles[[1]]
class : SpatRaster
dimensions : 4962, 5049, 1 (nrow, ncol, nlyr)
resolution : 30, 30 (x, y)
extent : 1614258, 1765728, 8094905, 8243765 (xmin, xmax, ymin, ymax)
coord. ref. : LCC E008
source : 014M.tif
name : 014M
lc_2000_tiles[[2]]
class : SpatRaster
dimensions : 4791, 4747, 1 (nrow, ncol, nlyr)
resolution : 30, 30 (x, y)
extent : 1462288, 1604698, 8381835, 8525565 (xmin, xmax, ymin, ymax)
coord. ref. : LCC E008
source : 015L.tif
name : 015L
....
I was trying to figure out a way to use mosaic them. This was the solution I came up with. However, after 7 tiles are combined, the computer runs out of disk space.
Error: [merge] internal error: insufficient disk space (perhaps from temporary files?)
Is there a more efficient way to do this?
You do not show what you do with lc_2000_tiles. Do you provide a filename argument? If not, the output goes to the temp folder, and perhaps this is on disk with not much space (and it is inefficient). You can set the temp folder with terraOptions(tempdir = "....")
Also, do you need mosaic or can you use merge (equivalent if you have non-overlapping areas)? If so, you can also use vrt:
v <- vrt(filelist_lc)
A "VRT" is a virtual raster. It can treat different files (typically adjacent non-overlapping tiles, but that is not required) as a single data source (file). With that, you could then do
x <- writeRaster(v, "combined.tif")
Related
I am trying to create a automating process in R to analyse a large number of tiff files from Sentinel-1. I read the raster in R using
r <-raster("s1a-iw-grd-vh-20230208t052407-20230208t052432-047140-05a7dd-002.tiff")
The file is read but R shows that it has no projection. Note that R raster thinks that the extent of the raster is the dimensions (rows,columns) and assigns values 1:nrow and 1.ncol as coordinates and also gives a wrong resolution.
class : RasterLayer
dimensions : 16668, 26588, 443168784 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : 0, 26588, 0, 16668 (xmin, xmax, ymin, ymax)
crs : NA
source : s1a-iw-grd-vh-20230208t052407-20230208t052432-047140-05a7dd-002.tiff
names : s1a.iw.grd.vh.20230208t052407.20230208t052432.047140.05a7dd.002
values : 0, 25771 (min, max)
Plotting the raster using mapview(r) plots it south of Ghana and Togo in Africa yet it is supposed to be in the south of Sweden.
Here is the mapview image.
Using ArcGIS placed the image in the correct geogrtaphical region which implies that R raster package is not able to read ceratin attributes of the "tiff" file.
Here is the ArcGIS plot.
ArcGIS shows that the source file has spatial attributes as shown here
How can I get R to read and map this "tiff" file properly.
I would love solutions only in R. I have also tried the terra and stars packages without much luck
here is a link to the tiff file TIFF_FILE
terra v 1.7.10 can now handle files that are georeferenced with Ground Control Points (GCPs). This is currently the development version that you can install with install.packages('terra', repos='https://rspatial.r-universe.dev')
With your file, I see
library(terra)
#terra 1.7.10
x <- rast("s1a-iw-grd-vh-20230208t052407-20230208t052432-047140-05a7dd-002.tiff"
x <- rast(f)
#Warning message:
#[rast] the data in this file are rotated. Use 'rectify' to fix that
r <- rectify(x)
r
#class : SpatRaster
#dimensions : 21480, 29751, 1 (nrow, ncol, nlyr)
#resolution : 0.0001656102, 8.934863e-05 (x, y)
#extent : 11.47043, 16.3975, 56.93446, 58.85367 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84
#source : spat_kxvbxy0YqI5GrWE_4932.tif
#name : spat_IbFSRebuQNX93z9_4932_rect
#min value : 0.00
#max value : 21275.04
See
plet(r)
This is still somewhat experimental. And feedback appreciated.
When I directly read into R the jp2 band files I get unusual high values compared to when I read the files in SNAP (version 9). To read the bands into R I use terra package (you can also use raster package) and the values range from 0 to 18000 more or less. I was wondering if SNAP is doing some conversion that I am not aware of to show values that range from 0 to 0.15 more or less.
> r_10
class : SpatRaster
dimensions : 10980, 10980, 4 (nrow, ncol, nlyr)
resolution : 10, 10 (x, y)
extent : 499980, 609780, 6690240, 6800040 (xmin, xmax, ymin, ymax)
coord. ref. : WGS 84 / UTM zone 32N (EPSG:32632)
sources : T32VNN_20181018T105031_B02_10m.jp2
T32VNN_20181018T105031_B03_10m.jp2
T32VNN_20181018T105031_B04_10m.jp2
... and 1 more source(s)
names : B02_10m_m10_2018, B03_10m_m10_2018, B04_10m_m10_2018, B08_10m_m10_2018
min values : 0, 0, 0, 0
max values : 18815, 17880, 17023, 15608
>
I have tried to export the bands from SNAP into TIF to see if it is a problem of format but it takes forever. I was hoping that there is a convesion factor to show the actual values that I need for my analysis.
SNAP applies a scaling factor of 1.0E4 so that values can be stored more efficiently as integers. You will need to either divide the values by this scaling factor in R, or else use the scaled units to benefit from more efficient integer arithmetic. See https://step.esa.int/docs/tutorials/Exporting%20data%20from%20SNAP.pdf for more details
Legacy formats ([Geo]TIFF, JPEG) have incomplete metadata support. You may lose missing value codes, offset and scale factors, processing history, etc. NetCDF4-CF has good metadata support for applications that actually use what is available. The R terra package can import NetCDF4-CF format, but is selective about what metadata are imported. For the files I have tested, missing data, scale, and offset values are used by the rast() function, but other metadata are lost.
With terra you can also set a scale/offset if you want:
library(terra)
#terra 1.6.21
f <- system.file("ex/elev.tif", package="terra")
r <- rast(f)
r
#class : SpatRaster
#dimensions : 90, 95, 1 (nrow, ncol, nlyr)
#resolution : 0.008333333, 0.008333333 (x, y)
#extent : 5.741667, 6.533333, 49.44167, 50.19167 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84 (EPSG:4326)
#source : elev.tif
#name : elevation
#min value : 141
#max value : 547
scoff(r) <- cbind(10, 0)
r
#class : SpatRaster
#dimensions : 90, 95, 1 (nrow, ncol, nlyr)
#resolution : 0.008333333, 0.008333333 (x, y)
#extent : 5.741667, 6.533333, 49.44167, 50.19167 (xmin, xmax, ymin, ymax)
#coord. ref. : lon/lat WGS 84 (EPSG:4326)
#source : elev.tif
#name : elevation
#min value : 1410
#max value : 5470
Essentially this is way to delay the evaluation of value * scale + offset, but now it may need to be done multiple times (each time that r is used), so it is not something I would generally recommend doing.
I'm merging two MODIS DSR tiles using a R script that I developed, these are the products:
https://drive.google.com/drive/folders/1RG3JkXlbaotBax-h5lEMT7lEn-ObwWsD?usp=sharing
So, I open both products (tile h15v05 and tile h16v05) from same date (2019180), then I open each SDS and merge them together (00h from h15v05 with 00h from h16v05 and so on...)
Visualisation on Panoply (using the merge option) of the two products:
Purple square is the location of the division line that separates the two tiles.
With my code I obtain a plot with pixels with different resolution (and different min/max values) and I don't understand why:
I suspect that the results obtained are due to:
1- Changing from Sinusoidal CRS to longlat WGS84 CRS;
2- Using resample (method ngb) to work with mosaic.
My code is extensive, but here are some parts of it:
# Open scientific dataset as raster
SDSs <- sds(HDFfile)
SDS <- SDSs[SDSnumber]
crs(SDS) <- crs("+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs")
SDSreprojected <- project(SDS, DesiredCRS)
SDSasRaster <- as(SDSreprojected, "Raster")
# Resample SDS based on a reference SDS (SDS GMT_1200_DSR of a first product), I need to do this to be able to use mosaic
SDSresampled <- resample(SDSasRaster,ResampleReference_Raster,method='ngb')
# Create mosaic of same SDS, but first convert stack to list to use mosaic
ListWith_SameSDS_OfGroupFiles <- as.list(StackWith_SameSDS_OfGroupFiles)
ListWith_SameSDS_OfGroupFiles.mosaicargs <- ListWith_SameSDS_OfGroupFiles
ListWith_SameSDS_OfGroupFiles.mosaicargs$fun <- mean
SDSmosaic <- do.call(mosaic, ListWith_SameSDS_OfGroupFiles.mosaicargs)
# Save SDSs mosaic stack to netCDF
writeRaster(StackWith_AllMosaicSDSs_OfGroupFiles, NetCDFpath, overwrite=TRUE, format="CDF", varname= "DSR", varunit="w/m2", longname="Downward Shortwave Radiation", xname="Longitude", yname="Latitude", zname="TimeGMT", zunit="GMT")
Does anyone have an idea of what could be the cause of this mismatch between results?
print(ResampleReference_Raster)
class : RasterLayer
dimensions : 1441, 897, 1292577 (nrow, ncol, ncell)
resolution : 0.01791556, 0.006942043 (x, y)
extent : -39.16222, -23.09196, 29.99652, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : MCD18A1.A2019180.h15v05.061.2020343034815
values : 227.5543, 970.2346 (min, max)
print(SDSasRaster)
class : RasterLayer
dimensions : 1399, 961, 1344439 (nrow, ncol, ncell)
resolution : 0.01515284, 0.007149989 (x, y)
extent : -26.10815, -11.54627, 29.99717, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : MCD18A1.A2019180.h16v05.061.2020343040755
values : 0, 0 (min, max)
print(SDSmosaic)
class : RasterLayer
dimensions : 1441, 897, 1292577 (nrow, ncol, ncell)
resolution : 0.01791556, 0.006942043 (x, y)
extent : -39.16222, -23.09196, 29.99652, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : layer
values : 0, 62.7663 (min, max)
Also, some of the islands were ignored by the script (bottom right)...
sorry I didn't reply earlier. So I think you're right that this issue is extent to which you are resampling. I think you might be able to get around this by creating a dummy raster that has the extent of the raster you want to resample, but has the resolution of the raster you want to mosaic to.Try:
dummy<-raster(ext = SDSasRaster#extent, resolution=ResampledReference_Raster#res, crs=SDSasRaster#crs)
SDS2<-resample(SDSasRaster, dummy, method="ngb")
Final<-moasic(SDS2, ResampledReference_Raster, fun=mean)
I am finding the average production days per year for maple syrup. My maple distribution data is in an ascii file. I have a raster (created from NetCDF files) called brick.Tmax. I want to match the specs of brick.Tmax to my maple distribution data.
## These are the specs I want to use for my maple distribution
brick.Tmax
class : RasterBrick
dimensions : 222, 462, 102564, 366 (nrow, ncol, ncell, nlayers)
resolution : 0.125, 0.125 (x, y)
extent : -124.75, -67, 25.125, 52.875 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
data source : E:\all_files\gridded_obs.daily.Tmax.1980.nc
names : X1980.01.01, X1980.01.02, X1980.01.03, X1980.01.04, X1980.01.05, X1980.01.06, X1980.01.07, X1980.01.08, X1980.01.09, X1980.01.10, X1980.01.11, X1980.01.12, X1980.01.13, X1980.01.14, X1980.01.15, ...
Date : 1980-01-01, 1980-12-31 (min, max)
varname : Tmax
## reading in red maple data from ascii file into rasterLayer
red_raster <- raster("E:/all_files/Maple_Data/redmaple.asc")
red_raster
class : RasterLayer
dimensions : 140, 150, 21000 (nrow, ncol, ncell)
resolution : 20000, 20000 (x, y)
extent : -1793092, 1206908, -1650894, 1149106 (xmin, xmax, ymin, ymax)
coord. ref. : NA
data source : E:\all_files\Maple_Data\redmaple.asc
names : redmaple
values : -2147483648, 2147483647 (min, max)
How can I project all specs (dimension, crs, resoluion, and extent) from brick.Tmax onto red_raster, while still preserving the values of red_raster? It seems like the two are mutually exclusive.
NOTE: In an attempt to streamline my question, I edited my question quite a bit from my original post, so apologies if the comments below are confusing in the current context. (I removed the raster prodavg_rastwhich was acting like a middleman).
The two rasters clearly do not have the same extent. In fact are in different universes (coordinate reference systems). brick.Tmax has angular (longitude/latitude) coordinates: +proj=longlat +datum=WGS84
but red_raster clearly does not given extent : -1793092, 1206908, -1650894, 1149106. So to use these data together, one of the two needs to be transformed (projected into the the coordinate reference system of the other). The problem is that we do not know what the the crs of red_raster is (esri ascii files do not store that information!). So you need to find out what it is from your data source, or by guessing giving the area covered and conventions. After you find out, you could do something like:
library(raster)
tmax <- raster(nrow=222, ncol=462, xmn=-124.75, xmx=-67, ymn=25.125, ymx=52.875, crs="+proj=longlat +datum=WGS84")
red <- raster(nrow=140, ncol=150, xmn=-1793092, xmx=1206908, ymn=-1650894, ymx=1149106, crs=NA)
crs(red) <- " ?????? "
redLL <- projectRaster(red, tmax)
Projectiong rasters takes time. A good way to test whether you figured out the crs would be to transform some polygons that can show whether things align.
library(rgdal)
states <- shapefile('states.shp')
sr <- spTransform(states, crs(red)
plot(red)
plot(sr, add=TRUE)
I have a raster called 'xx' and is the output from using the 'Distance' function within R Raster. As part of running the distance function, the ocean/sea surrounding my island has been set to a value of 0. In order to change this value from 0 back to NA, I have used the function 'Mask' like so:
asdf<-mask(xx,jamaica.raster, filename="distancesurface.asc", prj=TRUE, keepres=TRUE, overwrite=TRUE)
In the above example, jamaica.raster is being used to replace the 0 values with NA. I run the command and plot the result. As expected, it has left my distance surface in place and set the ocean to NA, exactly as I wanted.
After, when I attempt to writeRaster the object 'asdf' I get the error:
Error in .local(.Object, ...) :
Couldn't find data values in ASCII Grid file.
If I get a summary of the objects 'xx' and 'jamaica.raster' before applying 'mask' function, both have a 'values' section like so:
class : RasterLayer
dimensions : 135, 306, 41310 (nrow, ncol, ncell)
resolution : 0.008333333, 0.008333333 (x, y)
extent : -78.49359, -75.94359, 17.59729, 18.72229 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=WGS84 +no_defs
data source : D:\Dropbox\\...
names : jamaicapc1
values : -3.419589, 14.17305 (min, max)
If I do the same for my new object 'asdf' there are no values- They seem to have gone missing since using the Mask function. For example:
class : RasterLayer
dimensions : 135, 306, 41310 (nrow, ncol, ncell)
resolution : 0.008333333, 0.008333333 (x, y)
extent : -78.49359, -75.94359, 17.59729, 18.72229 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=longlat +ellps=WGS84 +no_defs
data source : C:\Users\Simon\Desktop\distancesurface.asc
names : distancesurface
I assume this is expected behaviour so my question is, how can I write my new raster (asdf) to disk? Is there a step I need to carry out before I can use the writeRaster function?
Thanks in advance for any help you may be able to provide.