I'm getting an error when merging raster tiles inside a custom function. The following code works every time:
files = list.files("data_folder/")
tiles = list()
for(f in files) tiles = c(tiles, list(raster(readGDAL(paste0(dir,f)), layer=1, values=T)))
ras = do.call(merge, tiles)
But when I try to run this inside a function:
read_and_merge_rasters = function(dir){
files = list.files(dir)
tiles = list()
for(f in files) tiles = c(tiles, list(raster(readGDAL(paste0(dir,f)), layer=1, values=T)))
return(do.call(merge, tiles))
}
.. it reads the files in ok (confirmed by print report) but then fails with the error: Error in as.data.frame.default(x) :
cannot coerce class "structure("RasterLayer", package = "raster")" to a data.frame
I can't think where/why as.data.frame is being called. Any idea why this is happening? Thanks in advance.
You can try this approach (not tested, since you didn't not provide a reproductible example).
read_and_merge_rasters <- function(dir) {
library(raster)
files <- list.files(dir, full.names = TRUE)
tiles <- lapply(files, raster)
do.call(merge, tiles)
}
You also have check that your differents raster in your directory are comparable (extent, origin and resolution).
There are multiple merge functions. do.call will tend to send it to merge for data.frames unless you set the names of the first two arguments:
tiles <- lapply(files, raster)
names(tiles)[1:2] <- c('x', 'y')
x <- do.call(merge, tiles)
Related
I'm working with 1.460 HDF files (4 years daily data). I'm interested in obtaining MEAN AOD from all the files. With the following code I only obtain information from the last file, not the combination of all the files I'm working with and I'm not sure why this is not working. (I'm not interested in getting TIF files in the process, but I'm not sure if this is necessary to get what I want)
read_hdf = function(file, n) {
sds = get_subdatasets(file)
f = tempfile(fileext = ".tif")
gdal_translate(sds[1], f)
brick(f)
}
files <- dir(pattern = ".hdf")
for(f in files) {
sds = get_subdatasets(f)
Optical_Depth_047 = read_hdf(f, grep("grid1km:Optical_Depth_047", sds))
names(Optical_Depth_047) = paste0("Optical_Depth_047.", letters[1:nlayers(Optical_Depth_047)])
r = stack(Optical_Depth_047)
}
meanAOD <- stackApply(r, indices = rep(1,nlayers(r)), fun = "mean", na.rm = T)
Welcome to Stack Overflow M. Fernanda Valle Seijo! For future reference, please try to post a reproducible example. You can see guidelines for them here. These help people answering your questions better diagnose the issue.
I think your code is on the right track, but it looks like the reason you are getting just the last item is that you have included the stack() function in you loop. This will overwrite r in each iteration of the loop. Also, you may need to setup Optical_Depth_047 as a list first and save each iteration of the loop as an element of that list. Try running your code like this:
read_hdf = function(file, n) {
sds = get_subdatasets(file)
f = tempfile(fileext = ".tif")
gdal_translate(sds[1], f)
brick(f)
}
files <- dir(pattern = ".hdf")
Optical_Depth_047<-list()
for(f in files) {
sds = get_subdatasets(f)
Optical_Depth_047[[f]] = read_hdf(f, grep("grid1km:Optical_Depth_047", sds))
names(Optical_Depth_047)[f] = paste0("Optical_Depth_047.", letters[f])
}
r = stack(Optical_Depth_047)
meanAOD <- stackApply(r, indices = rep(1,nlayers(r)), fun = "mean", na.rm = T)
with terra you can take some shortcuts, as you can read the HDF files directly. You can do
library(terra)
r <- rast(f, "grid1km:Optical_Depth_047")
Perhaps what you are after can be as simple as this:
x <- rast(files, "grid1km:Optical_Depth_047")
m <- mean(x)
I'm trying to upload multiple images to do some machine learning in R. I can upload a single image just fine, but when I try to upload multiple images using either lapply or a for loop, I get the following error: "Error in wrap.url(file, load.image.internal) : File not found". I did a check to make sure the files do exist, my WD is set correctly and R recognizes that the files and directory do exist. No matter what I change, the error is always the same. It doesn't change the outcome if I list the path from the drive it originates in or from the WD onward. I've asked many people for help with no success. I've posted my code using lapply and a for loop below. I'm still relatively new to R so if there is something I'm missing I'd greatly appreciate knowing. Also, I'm using imager here to load the files.
eggs2015 <- list()
file_list <- list.files(path="~/Grad School/Thesis Work/Machine Learning R/a2015_experimental_clustering_R/*.jpg", pattern="*.jpg", full.names = TRUE)
for (i in 1:length(file_list)){
Path <- paste0("a2015_experimental_clustering_R",file_list[i])
eggs2015 <- c(eggs2015, list(load.image(Path)))
}
names(eggs2015) <- file_list
eggs2015 <- list.files(path = "~/Grad School/Thesis Work/Machine Learning R/2015_experimental_clustering_R", pattern = ".jpg", all.files = TRUE, full.names = TRUE)
eggs2015 <- lapply(list, FUN = load.image("~/Grad School/Thesis Work/Machine Learning R/a2015_experimental_clustering_R/*.jpg"))
eggs2015 <- as.data.frame(eggs2015)
Personally for this kind of operation I prefer to use sapply so I can identify images with the original file names later on (if needed):
FilesToRead <- list.files(path = "~/Grad School/Thesis Work/Machine Learning R/2015_experimental_clustering_R", pattern = ".jpg", all.files = TRUE, full.names = TRUE)
ListOfImages <- sapply(FilesToRead, FUN = load.image, simplify = FALSE, USE.NAMES = TRUE)
should work and give you a list of elements with your images using the file paths as names
Or using lapply (sapply is just a wrapper for lapply)
ListOfImages <- lapply(FilesToRead, FUN = load.image)
As you can see, your code just needed a little tweaking.
Hope it helps
I am so close to getting my code to work, but cannot seem to figure out how to get a dynamic file name. Here is what Ivve got:
require(ncdf)
require(raster)
require(rgdal)
## For multiple files, use a for loop
## Input directory
dir.nc <- 'inputdirectoy'
files.nc <- list.files(dir.nc, full.names = T, recursive = T)
## Output directory
dir.output <- 'outputdirectory'
## For simplicity, I use "i" as the file name, but would like to have a dynamic one
for (i in 1:length(files.nc)) {
r.nc <- raster(files.nc[i], varname = "precipitation")
writeRaster(r.nc, paste(dir.output, i, '.tiff', sep = ''), format = 'GTiff', prj = T, overwrite = T)
}
## END
I appreciate any help. So close!!
You can do this in different ways, but I think it is generally easiest to first create all the output filenames (and check if they are correct) and then use these in the loop.
So something like this:
library(raster)
infiles <- list.files('inputpath', full.names=TRUE)
ff <- extension(basename(infiles), '.tif')
outpath <- 'outputpath'
outfiles <- file.path(outpath, ff)
To assure that you are writing to an existing folder, you can create it first.
dir.create(outpath, showWarnings=FALSE, recursive=TRUE)
And then loop over the files
for (i in 1:length(infiles)) {
r <- raster(infiles[i])
writeRaster(r, paste(outfiles[i], overwrite = TRUE)
}
You might also use something along these lines
outfiles <- gsub('in', 'out', infiles)
Here is the code that finally worked:
# Imports
library(raster)
#Set source file
infiles <- list.files('infilepath', full.names=TRUE)
#create dynamic file names and choose outfiles to view list
ff <- extension(basename(infiles), '.tif')
outpath <- 'outfilepath'
outfiles <- file.path(outpath, ff)
#run da loop
for (i in 1:length(infiles)) {
r <- raster(infiles[i])
writeRaster(r, paste(outfiles[i]), format ='GTiff', overwrite = T)
}
## END
I have 14 raster files in Tiff format and I want to read values of a series of pixels (same file location). However, when I ran the R code, the expected results did not show up. Could you tell me why?
#set working directory#
path <- 'E:/TSL_VCF/Tiffs'
setwd(path)
#list tiff files in the working directory#
list.files(path, pattern = 'tif')
#count the number of tiff files#
mylist <- list.files(path, pattern = 'tif')
mylength <- length(mylist)
#get values for certain "location"#
for (i in 1:mylength){
myraster <- raster(mylist[i])
mymatrix <- as.matrix(myraster)
mymatrix[1,771]
}
results are not printed because the instruction
mymatrix[1,771]
is inside the "for" loop. This:
#set working directory#
path <- 'E:/TSL_VCF/Tiffs'
setwd(path)
#list tiff files in the working directory#
list.files(path, pattern = 'tif')
#count the number of tiff files#
mylist <- list.files(path, pattern = 'tif')
mylength <- length(mylist)
#get values for certain "location"#
for (i in 1:mylength){
myraster <- raster(mylist[i])
mymatrix <- as.matrix(myraster)
print(mymatrix[1,771])
}
should work.
However, it won't store your resulting array anywhere but on the screen.
I'd suggest you to have a look at the extract function of the raster package for a better solution. If you build a rasterstack in advance using something like:
mystack <- stack(mylist)
you can also avoid looping over the files and just do something like:
result <- extract(mystack, as.matrix(c(1,771), nrow = 1))
, and you should get the results in the "result" variable
HTH,
Lorenzo
HTH,
Lorenzo
You need to explicitly call print if you want R to print stuff in a loop. For example:
m = rnorm(10)
for (i in 1:10) m[i] # doesn't print
for (i in 1:10) print(m[i]) # print
I started recently to work with R so this question has probably a simple solution.
I have some .tif satellite images from different scenes. I can create a test raster brick with it but the process needs to be automatised because of the huge amount of files. Therefore I have been trying to create a function to read the list of .tif files and to output a list of rasters.
You can find here below the code I have been using:
# Description: Prepare a raster brick with ordered acquisitions
# from all the scenes of the study area
library(raster)
library(rgdal)
library(sp)
library(rtiff)
rm(list = ls())
setwd=getwd()
# If you want to download the .tif files of the 2 scenes from dropbox:
dl_from_dropbox <- function(x, key) {
require(RCurl)
bin <- getBinaryURL(paste0("https://dl.dropboxusercontent.com/s/", key, "/", x),
ssl.verifypeer = FALSE)
con <- file(x, open = "wb")
writeBin(bin, con)
close(con)
message(noquote(paste(x, "read into", getwd())))
}
dl_from_dropbox("lndsr.LT52210611985245CUB00-vi.NDVI.tif", "qb1bap9rghwivwy")
dl_from_dropbox("lndsr.LT52210611985309CUB00-vi.NDVI.tif", "sbhcffotirwnnc6")
dl_from_dropbox("lndsr.LT52210611987283CUB00-vi.NDVI.tif", "2zrkoo00ngigfzm")
dl_from_dropbox("lndsr.LT42240631992198XXX02-vi.NDVI.tif", "gx0ctxn2mca3u5v")
dl_from_dropbox("lndsr.LT42240631992214XXX02-vi.NDVI.tif", "pqnjw2dpz9beeo5")
dl_from_dropbox("lndsr.LT52240631986157CUB02-vi.NDVI.tif", "rrka10yaktv8la8")
# 1- Create a list of .tif files with names ordered chronologically (for time series analysis later on)
pathdir= # change
# List all the images from any scene in that folder and
# make a dataframe with a column for the date
a <- list.files(path=pathdir,pattern="lndsr.LT", all.files=FALSE,full.names=FALSE)
a1 <- as.data.frame(a, row.names=NULL, optional=FALSE, stringsAsFactors=FALSE) # class(a1$a) # character
# Create date column with julean date and order it in ascending order
a1$date <- substr(a1$a, 16, 22) # class(a1$date) = character
a1 <- a1[order(a1$date),]
# Keep only the column with the name of the scene
a1 <- subset(a1, select=1) # class(a1$a): character
# retrieve an ordered list from the dataframe
ord_dates <- as.list(as.data.frame(t(a1$a))) # length(ord_dates): 4 (correct)
# class(odd_dates) # list
# 2- Create rasters from elements of a list
for (i in 1:(length(ord_dates))){
# Point to each individual .tif file
tif_file <- ord_dates[i] # Problem: accesses only the first item of ord_dates
# Make a raster out of it
r <- raster(tif_file) # we cant use here a list as an input. Gives error:
# Error in .local(x, ...) : list has no "x"
# Give it a standardised name (r1,r2,r3, etc)
name <- paste("r", 1:length(ord_dates),sep = "")
# Write the raster to file
writeRaster (r , filename = name,format = "GTiff", overwrite =T )
}
I have also tried to use lapply() without much success.
r = lapply(ord_dates, raster)
Can you give me an advice on what concept to follow? I am guessing I should be using matrices but I don't really understand which are their advantages here or in what step they are required.
Any help is really appreciated!
Thanks in advance
Assuming ord_dates is a list of file names (that have full path or are in your getwd()), you can apply a (any) function to this list using lapply. I haven't tested this, unfortunately.
convertAllToRaster <- function(tif_file) {
r <- raster(tif_file)
# Give it a standardised name (r1,r2,r3, etc)
name <- paste("r", 1:length(ord_dates),sep = "")
# Write the raster to file
writeRaster (r , filename = name,format = "GTiff", overwrite =T )
message("Eeee, maybe it was written successfully.")
}
lapply(ord_dates, FUN = convertAllToRaster)
After solving the issues with factors and with the name, this is the code that worked for me. I added a for loop also inside the function you proposed, Roman. Thankyou very much for your kind help!!
convertAllToRaster <- function(ord_dates) {
for (i in 1:(length(ord_dates))){
tif_file <- ord_dates[i]
r <- raster(tif_file)
# Keep the original name
name <- paste(tif_file, ".grd", sep ="")
# Write the raster to file
writeRaster (r , filename = name,format = "raster", overwrite =T ) # in .grd format
}
}
lapply(ord_dates, FUN = convertAllToRaster)