I am processing hundreds of 4-band images in R and need help on what is probably a very simple task. As part of the processing, I need to export a single band RGB composite that maintains the spatial information of the original GeoTiff. In other software I've exported a .jgw file but I need to be able to do this in R. These images will be used as basemaps and fed into another mapping interface. I have searched and searched and can only find how to plotRGB() and how to writeRaster(). PlotRGB loses the spatial information and writeRaster() produces a multi-band image.
Any ideas? There is a built in raster in R that can be used.
library(raster)
library(rgdal)
r <- raster(system.file("external/test.grd", package="raster"))
x <- RGB(r)
plotRGB(x) #Is there a way to output this where it will maintain spatial information?
writeRaster(x, filename="file.tif") #This produces a 3-band tiff, not a composite
The writeRaster function can take an options argument to pass options to the underlying GDAL library (e.g., GeoTIFF options are documented here). The option TFW=YES writes out a .tfw world file which appears to be the same thing as a .jgw file.
Now, "composite RGB" isn't standard terminology in the TIFF world; it seems to be specific to "ArcMap" and friends, so it's hard to tell what's really meant by this, but you can generate what one would normally think of as a "standard" RGB TIFF format by specifying that the datatype for the color components be 1-byte unsigned integers (datatype="INT1U"), so the following may do what you want:
writeRaster(RGB(r), filename="file2.tif", datatype="INT1U",
options="TFW=YES", format="GTiff")
As far as I can tell, unrecognized or misspelled options values don't generate any error messages, so you need to be careful they're all spelled correctly.
Just noting an update to the process utilizing the terra package. Process is very similar but there are some different parameters.
r <- rast(system.file("ex/logo.tif", package="terra"))
# a little forced as RGB is already assigned in this image...
RGB(r) <- 1:3
# export as geotiff -- again force due to input file example...
writeRaster(x = r, filename = "rgb2.tif",datatype="INT1U",filetype = "GTiff")
I've been using with NAIP imagery successfully.
Related
For a question that is specific to my particular dataset, how can I make a reproducible example of that dataset if I have it stored as a SpatRaster in R?
The data structure is complex enough that I don't know how to freehand invent a simpler version and read it as a SpatRast (i.e. x <- rast(????????)
I also haven't been able to figure out how I could use a package or command to extract enough information to provide what is functionally a reproducible example either
See my previous question for an example: How can I add a class name to numeric raster values in a terra SpatRaster?
You can create objects from scratch like this:
library(terra)
r <- rast()
s <- rast(ncols=22, nrows=25, nlyrs=5, xmin=0)
See ?terra::rast for additional arguments you can use and for alternative approaches.
You can also use a file that ships with R. For example:
f <- system.file("ex/elev.tif", package="terra")
r <- rast(f)
You can also create from scratch a new SpatRaster with (mostly) the same properties with what is returned by
as.character(r)
And then recreate it with something like
r <- rast(ncols=95, nrows=90, nlyrs=1, xmin=5.74166666666667, xmax=6.53333333333333, ymin=49.4416666666667, ymax=50.1916666666667, names=c('elevation'), crs='GEOGCRS[\"WGS 84\",DATUM[\"World Geodetic System 1984\",ELLIPSOID[\"WGS 84\",6378137,298.257223563,LENGTHUNIT[\"metre\",1]]],PRIMEM[\"Greenwich\",0,ANGLEUNIT[\"degree\",0.0174532925199433]],CS[ellipsoidal,2],AXIS[\"geodetic latitude (Lat)\",north,ORDER[1],ANGLEUNIT[\"degree\",0.0174532925199433]],AXIS[\"geodetic longitude (Lon)\",east,ORDER[2],ANGLEUNIT[\"degree\",0.0174532925199433]],ID[\"EPSG\",4326]]')
r <- init(r, "cell")
If you cannot replicate your error with example data, this may give you a hint about the problem. Does it have to do with NAs? The file being on disk? The file format? One tricky situation is where there is a difference if the real file is much larger. You can simulate a large file by setting terraOptions(todisk=TRUE) and using a steps argument in a function, e.g.
b <- clamp(x, steps=5)
If none of that allows you to replicate an error, your last resort is to provide a link to the file so that others can download it. If you cannot do that, then at least show the content of the SpatRaster x with show(x) and provide the code to create a similar object with as.character(x)
I am desesperate, because my problem seems very simple, but I cannot find out how to manage it.
Aim:
I would like to read 1 to 4 raster files from a folder. The names of the one that I need are listed in a list as character.
After having opened the files, I would like to create a new raster corresponding to the mean of the files.
I can manage it on QGIS, but I need to automatize hte process, as I have a lot of individuals!
1) It should work with list.files(pattern = ) but as the names are in a list, I do not know how to do.
Ex: for the first individual, I have to read 2 files named 2018-12-27_sic.tif and 2018-12-27_sic_con.tif
I tried to read with readGDAL , open.GDAL it didn't work
thanks a lot for your valuable help
I would use the stack and calc functions from the raster package. The function stack creates a stack of rasters, all with the same resolution and extent, and makes it easy to do operations like take the mean of every cell. So:
library(raster)
fs <- list.files(pattern='tif$')
rasterstack <- stack(fs)
rastermean <- calc(rasterstack, fun=mean)
Note, if your rasters are not the same resolution, you will have to use the resample function, and if they are not the same extent, you will have to use crop. Typing in ?resample and ?crop in RStudio will show you instructions for using those functions.
I successfully wrote a for loop in R. That is okay and I am very happy that it works. But I also want to understand what I've done exactly because I will have to work with loops later on in my analysis as well.
I work with Raster Data (DEMs). I load them into the environment as rasters and then I use the getValues function in the loop as I want to do some calculations. Looks as follows:
list <- dir(pattern=".tif", full.names=T)
tif.files <- list()
tif.files.values <- tif.files
for (i in 1: length(list)){
tif.files[[i]] <- raster (list[[i]])
tif.files.values[[i]] <- getValues(tif.files[[i]])
}
Okay, so far so good. I don't get why I have to specify tif.files and tif.files.values before I use them in the loop and I don't know why to specify them exactly how I did that. For the first part, the raster operation, I had a pattern. Maybe someone can explain the context. I really want to understand R.
When you do:
tif.files[[i]] <- raster (list[[i]])
then tif.files[[i]] is the result of running raster(list[[i]]), so that is storing the raster object. This object contains the metadata (extent, number of rows, cols etc) and the data, although if the tiff is huge it doesn't actually read it in at the time.
tif.files.values[[i]] <- getValues(tif.files[[i]])
that line calls getValues on the raster object, which reads the values from the raster and returns a vector. The values of the grid cells are now in tif.files.values[[i]].
Experiment by printing tif.files[[1]] and tif.files.values[[1]] at the R prompt.
Note
This is R, not RStudio, which is the interface you are using that has all the buttons and menus. The R language exists quite happily without it, and your question is just a language question. I've edited and tagged it now for you.
I'm having trouble extracting point data from a large shape file (916.2 Mb, 4618197 elements - from here: https://earthdata.nasa.gov/data/near-real-time-data/firms/active-fire-data) in R. I'm using readShapeSpatial in maptools to read in the shape file which takes a while but eventually works:
worldmap <- readShapeSpatial("shp_file_name")
I then have a data.frame of coordinates that I want extract data for. However R is really struggling with this and either loses connection or freezes, even with just one set of coordinates!
pt <-data.frame(lat=-64,long=-13.5)
pt<-SpatialPoints(pt)
e<-over(pt,worldmap)
Could anyone advise me on a more efficient way of doing this?
Or is it the case that I need to run this script on something more powerful (currently using a mac mini with 2.3 GHz processor)?
Many thanks!
By 'point data' do you mean the longitude and latitude coordinates? If that's the case, you can obtain the data underlying the shapefile with:
worldmap#data
You can view this in the same way you would any other data frame, for example:
View(worldmap#data)
You can also access columns in this data frame in the same way you normally would, except you don't need the #data, e.g.:
worldmap$LATITUDE
Finally, it is recommended to use readOGR from the rgdal package rather than maptools::readShapeSpatial as the former reads in the CRS/projection information.
I am currently doing a large amount of data analysis in Fortran. I have been using R to plot most of my results, as Fortran is ill-suited for visualization. Up until now, the data sets have been two-dimensional and rather small, so I've gotten away with routines that write the data-to-be-plotted and various plot parameters to a .CSV file, and using a system call to run an R script that reads the file and generates the required plot.
However, I find myself now dealing with somewhat larger 3D data sets, and I do not know if I can feasibly continue in this manner (notably, sending and properly reading in a 3D array via .CSV is rather more difficult, and takes up a lot of excess memory which is a problem given the size of the data sets).
Does anyone know any efficient way of sending data from Fortran to R? The only utility I found for this (RFortran) is windows-only, and my work computer is a mac. I know that R possesses a rudimentary Fortran interface, but I am calling R from Fortran, not vice-versa, and moreover given the number of plot parameters I am sending (axis lables, plot titles, axis units and limits, etc., many of which are optional and have default values in the current routines I'm using) I am not sure that it has the features I require.
I would go for writing NetCDF files from Fortran. These files can contain large amounts of multi-dimensional data. There are also good bindings for creating NetCDF files form within Fortran (it is used a lot in climate models). In addition, R has excellent support for working with NetCDF files in the form of the ncdf package. It is for example very easy to only read a small portion of the data cube into memory (only some timesteps, or some geographic region). Finally, NetCDF works across all platforms.
In terms of workflow, I would let the fortran program generate NetCDF files plus some graphics parameters in a separate file (data.nc and data.plt for example), and then as a post-processing step call R. In this way you do not need to directly interface R and Fortran. Managing the entire workflow could be done by a separate script (e.g. Python), which calls the Fortran model, makes a list of the NetCDF/.plt files and creates the plots.
So, it turns out that sending arrays via. unformatted files between Fortran and R is trivially easy. Both are column-major, so one needs to do no more than pass an unformatted file containing the array and another containing array shape and size information, and then read the data directly into an array of proper size and shape in R.
Sample code for an n-dimensional array of integers, a, with dimension i having size s(i).
Fortran-side (access must be set to "stream," else you will have extra bytes inserted after every write):
open(unit = 1, file="testheader.dat", form="unformatted", access="stream", status="unknown")
open(unit = 2, file="testdata.dat", form="unformatted", access="stream", status="unknown")
write(1) n
do i=1,n
write(1) s(i)
enddo
write(2) a
R-side (be sure that you have endianness correct, or this will fail miserably):
testheader = file("testheader.dat", "rb")
testdata = file("testdata.dat", "rb")
dims <- readBin(testheader, integer(), endian="big")
sizes <- readBin(testheader, integer(), n=dims, endian="big")
dim(sizes) <- c(dims)
a <- readBin(testdata, integer(), n=prod(sizes), endian="big")
dim(a) <- sizes
You can put the header and data in the same file if you want.