i want to do some tests with my rasterdata in R. I need numeric values. But R shows me only integers. How can i change this? Any idea? Thanks in advance :D
#libraries
library(raster)
library(rgdal)
setwd("C:/Users/cathe/Documents/Cropped_raster")
## polygon with crop-extend ##
shp <- readOGR("C:/Users/cathe/OneDrive/Documents/ArcGIS/Projects/CLC2000_shapefiles/CLC2000_Nichtdurchgängig städtische Prägung.shp")
## load tif files ##
infiles = list.files(path=getwd(),
pattern="*.tif$|*.TIF$")
## Filenames with desired suffix and output place ##
outfiles = file.path("C:/Users/cathe/Documents/Cropped_raster_nicht_durchgängig",
paste0(basename(tools::file_path_sans_ext(infiles)),
".tif"))
outfiles[outfiles == -9999] <- NA #alle -9999 auf NA setzen, wenn nötig
## crop and output settings (compression and datatype)
for (i in seq_along(infiles)) {
r = crop(stack(infiles[i]), shp)
writeRaster(r, filename=outfiles[i],
bylayer=FALSE,
format="GTiff",
datatype="numeric",
options="COMPRESS=ZIP",
x, NAflag=-9999,
overwrite=TRUE)
}
Your issue is in the datatype argument of writeRaster().
From the documentation:
datatype: Character. Output data type (e.g. 'INT2S' or 'FLT4S'). See
dataType. If no datatype is specified, 'FLT4S' is used, unless this
default value was changed with rasterOptions
Numeric only exists in the R world. Outside you have integers and floats, what you need depends on your data.
Related
I am trying to export a raster file (.tif) as an ascii in R for subsequent analysis. The goal is to replicate this methodology (area-to-point-regression kriging), but the produced file is full of NA's. My original raster has "normal" values.
Here is a link to my ascii and tif files.
The code I have used so far:
library(raster)
#import raster
ntl = raster("mydir/ntl.tif")
#export as ascii format
writeRaster(ntl, "mydir/ntl.asc", format="ascii", overwrite = T)
I am using Windows 10, RStudio Version 1.4.1717
Your raster mostly consists of NAs, as can be illustrated like this (I prefer to use terra the replacement of raster):
library(terra)
x <- rast("ntl.tif")
plot(x, colNA="light blue")
Most of the image is light blue, that is, covered with cells that are NA
You can remove most NAs with trim
y <- trim(x)
plot(y, colNA="light blue")
You are not saying why you are creating an ascii file. I assume that you want to read the values with some other tool that does not know about spatial data file formats. In that case you might consider as.data.frame with na.rm=TRUE instead.
d <- as.data.frame(x, na.rm=TRUE, cells=TRUE)
head(d)
# cell ntl
#44592 44592 3.615484
#44593 44593 6.819953
#45010 45010 2.256919
#45011 45011 3.350195
#45012 45012 9.617457
#45013 45013 8.812189
And then save it to file, for example with
write.csv(d, "test.csv", row.names=FALSE)
When doing raster math, for example raster1-raster2, the datatype of the output raster is 'FLT4S', even if the datatype ot both raster1 and raster 2 is 'INT2S'. How can I force the output to be 'INT2S', without writing to disk? Is there a global way of doing it saying that all raster processing shall result in INT2S data?
The reason for wanting 'INT2S' instead of 'FLT4S' is to save memory space and speed up processing when using for loops on larger raster datasets.
In rasterOptions() one can specify dataType, but as far as I understand that only applies when writing to disk, right?
#load package raster
require (raster)
#create sample rasters
r1<-raster::raster(ext=extent(c(0,10,0,10)), res=1, vals=1:100)
r2<-raster::raster(ext=extent(c(0,10,0,10)), res=1, vals=100:1)
#set dataType of sample rasters to 'INT2S'
dataType(r1)<-'INT2S'
dataType(r2)<-'INT2S'
#check dataType of sample rasters
dataType(r1)
dataType(r2)
#do some simple arithmetics
r3<-r2-r1
#check the dataType of the output raster
dataType(r3)
I would like dataType(r3) to be 'INT2S' as well
In your example, it actually does work. Notice the L after the numbers if the vals argument. That is equivalent to as.integer
library(raster)
r1 <-raster::raster(ext=extent(c(0,10,0,10)), res=1, vals=1:100L)
r2 <-raster::raster(ext=extent(c(0,10,0,10)), res=1, vals=100:1L)
r3 <- r2 - r1
class(values(r3))
#[1] "integer"
But in other cases it does not work
class(values(r3 - 2L))
[1] "numeric"
And you cannot control that behavior.
Note that dataType provides information about the file that the RasterLayer refers to. If there is no file, like in the above example, the value is meaningless. You should also not set it, except for debugging when dealing with an existing file.
So the best you can do is set the datatype when writing to disk as in
writeRaster(r3, filename="test.tif", datatype="INT2S")
I have the same problem with 'INT2U' data and I don't believe it's possible. AFAIK, R supports 'numeric'. Floating point can be coerced to 'integer' with 'as.integer()' but I think it's just truncated.
I've had this same question too. Hopefully someone will chime in if I'm wrong, but I believe you really only have one option and that is to convert the output when you write to disk:
writeRaster(r3,filename="r3.tif", format="GTiff", datatype='INT2S')
then load back into R.
You can convert the output after the calculation, but it does not change the size of the object:
dataType(r3)<-'INT2S'
You can check the object size using:
object.size(r3)
If you write the data to disk and reload the object size will be smaller.
Another option is to use the calc or overlay functions in raster to do some math, save to disk and return the resulting raster in one go. I wrote the example rasters to disk to start since Robert said you should not assign it and in real cases you will probably be reading from disk.
# load package raster
library(raster)
# create sample rasters on disk
r1 <- raster::raster(ext = extent(c(0, 10, 0, 10)), res = 1, vals = 1:100)
r2 <- raster::raster(ext = extent(c(0, 10, 0, 10)), res = 1, vals = 100:1)
r1_fl <- tempfile(fileext = ".tif")
r2_fl <- tempfile(fileext = ".tif")
r3_fl <- tempfile(fileext = ".tif")
writeRaster(r1, r1_fl, datatype = "INT2S")
writeRaster(r2, r2_fl, datatype = "INT2S")
# read rasters from disk
r1 <- raster(r1_fl)
r2 <- raster(r2_fl)
# check dataType of sample rasters
dataType(r1)
dataType(r2)
# do some simple (or complex) arithmetic
r3 <- overlay(r1, r2, fun = function(x, y){x - y}, filename = r3_fl,
datatype = "INT2S")
# check the dataType of the output raster
dataType(r3)
I have more than 50 raster files (ASCII format) that I need to crop. I already exported the mask from ArcMap in ASCII format as well and loaded it into R. How can I make it work for all rasters in a row and export them with the same name as before (of course in a different folder to not overwrite)?
I know there is a crop function in the raster package, but I never used it so far. I only stacked them for further habitat analysis.
My Code so far:
#### Use only part of area
files2 <- list.files(path="D:/",full.names=TRUE, pattern = "\\.asc$")
files2
# Create a RasterLayer from first file
mask_raster <- raster(files2[1])
# Crop. But how??
crop(x = , y=mask_raster)
writeRaster(...)`
I did not find an easy solution to crop multiple rasters by a raster, but by a shape file. So I just went back to ArcMap and converted the raster into a shapefile. Then in R, crop and mask were the crucial steps. See code below (modified from Mauricio Zambrano-Bigiarini's code). Hope this helps.
# Reading the shapefile (mask to crop later)
Maskshp <- readOGR("mask.shp")
# Getting the spatial extent of the shapefile
e <- extent(Maskshp)
# for loop with (in this case) 60 runs
for (i in 1:60) {
# Reading the raster to crop
files <- list.files(path="...your path",full.names=TRUE, pattern = "\\.asc$")
Env_raster <- raster(files[i])
# Save the filename
filename <- (paste(basename(files[i]), sep=""))
# Crop the raster
Env_raster.crop <- crop(Env_raster, e, snap="out")
# Dummy raster with a spatial extension equal to the cropped raster,
# but full of NA values
crop <- setValues(Env_raster.crop, NA)
# Rasterize the catchment boundaries, with NA outside the catchment boundaries
Maskshp.r <- rasterize(Maskshp, crop)
# Putting NA values in all the raster cells outside the shapefile boundaries
Maskshp.masked <- mask(x=Env_raster.crop, mask=Maskshp.r)
plot(Maskshp.masked)
#Export file to working directory with original name as new name
writeRaster(Maskshp.masked, filename)
}
I'm running into an unusual issue with the cover function in R. We are trying to fill cloudy pixels with values from another layer. I can make it work just fine with stacks as follows -
library(raster)
r1 <- raster(ncols=36, nrows=18)
r1[] <- 1:ncell(r1)
r1b <- r1a <- r1
r1_stack <- stack(r1, r1a, r1b)
r2 <- setValues(r1, runif(ncell(r1)))
r2b <- r2a <- r2
r_stack <- stack(r2, r2a, r2b)
r_stack[r_stack < 0.5] <- NA
r3 <- cover(r_stack, r1_stack)
But then i try to do the same thing with a raster stack and i get the error:
Error in as.character(x) :
cannot coerce type 'closure' to vector of type 'character'
The code:
# get all tifs
LS5_032_032_2008_09_21 <- list.files("LT050340302008090301T1-SC20170526100900/",
pattern = glob2rx("*band*.tif$"), full.names = T)
# stack bands
cloudy_scene <- stack(LS5_032_032_2008_09_21)
# import cloud mask
cloud_mask <- raster('LT050340302008090301T1-SC20170526100900/LT05_L1TP_034030_20080903_20160905_01_T1_sr_cloud_qa.tif')
# mask data
masked_data <- mask(cloudy_scene, mask = cloud_mask, maskvalue=0, inverse=TRUE)
####### get cloud free data
# get files
LS5_2008_09_19 <- list.files("LT050340302008091901T1-SC20170526101124/",
pattern = glob2rx("*band*.tif$"), full.names = T)
# subset and stack cloud free bands
cloud_free_data <- stack(LS5_2008_09_19)
# use cover function to assign NA pixels to corresponding pixels in other scene
cover <- cover(masked_data, cloud_free_data)
TRACEBACK() output:
9: toupper(format)
8: .defaultExtension(format)
7: .getExtension(filename, filetype)
6: .local(x, filename, ...)
5: writeStart(outRaster, filename = filename, format = format, datatype = datatype,
overwrite = overwrite)
4: writeStart(outRaster, filename = filename, format = format, datatype = datatype,
overwrite = overwrite)
3: .local(x, y, ...)
2: cover(masked_data, cloud_free_data)
1: cover(masked_data, cloud_free_data)
UPDATE: I tried to resample the data - still doesn't work
cloud_free_resam <- resample(cloud_free_data, masked_data)
cover <- cover(masked_data, cloud_free_resam)
ERROR:
Error in as.character(x) :
cannot coerce type 'closure' to vector of type 'character'
I also tried to crop both layers - same error
# find intersection boundary
crop_extent <- intersect(extent(cloud_free_data), extent(masked_data))
cloud_free_data <- crop(cloud_free_data, crop_extent)
masked_data <- crop(masked_data, crop_extent)
# use cover function to assign NA pixels to corresponding pixels in other scene
cover <- cover(masked_data, cloud_free_data)
GET THE DATA: (WARNING: 317mb download - unpacks to ~1gb)
https://ndownloader.figshare.com/files/8561230
Any ideas what might be causing this error with this particular dataset?
I'm sure we're missing something quite basic but...what?
Thank you in advance.
Leah
This is a bug. Cover with two multi-layered Raster* objects cannot write to disk. This can be seen in the simple example by setting
rasterOptions(todisk=TRUE)
I have fixed this in version 2-6.1 (forthcoming)
I think it is to do with when the object is converted to a rasterBrick and the raster written to a temporary file. i.e., masked_data <- mask(cloudy_scene, mask = cloud_mask)
Using Spacedman's crop creates a 'in memory' rasterBrick objects that do not rely on accessing the files; in that case the example works with no error.
But using the full extent (or cropping at end of process), raster writes and accesses (temporary) files and the error occurs.
Maybe as a temporary fix is to explicitly split up the images into memory sized chunks, mask/cover and then restitch with mosaic.
The extents of the two objects are also slightly different, so that should be fixed. Also generally a good idea to avoid issues by explicitly setting band names, min-max values and values < 0 to NA.
Trying to convert NOAA Snow data (NetCDF) into Raster format in R. This data has been pre-processed by me in CDO (interpolated from weekly-daily).
library(raster)
library(ncdf4)
nc<-nc_open('NOAA_Snow_JanJune2016.nc')
# extract variable name, size and dimension
v <- nc$var[[1]]
size <- v$varsize
dims <- v$ndims
nt <- size[dims] # length of time dimension
lat <- nc$dim$latitude$vals # latitude position
lon <- nc$dim$longitude$vals # longitude position
# read sea ice variable
r<-list()
for (i in 1:nt) {
start <- rep(1,dims) # begin with start=(1,1,...,1)
start[dims] <- i # change to start=(1,1,...,i) to read timestep i
count <- size # begin with count=(nx,ny,...,nt), reads entire var
count[dims] <- 1 # change to count=(nx,ny,...,1) to read 1 tstep
dt<-ncvar_get(nc, varid = 'snow_cover_extent', start = start, count = count)
# convert to raster
r[i]<-raster(dt)
}
Returns the following error:
Error in ncvar_get_inner(ncid2use, varid2use, nc$var[[li]]$missval, addOffset, :
Error: variable has 3 dims, but start has 2 entries. They must match!
Has anyone else had and solved this problem? I wonder if prepping the file in CDO is causing the problem. The (.nc) data can be accessed here:
https://drive.google.com/file/d/0Bz0W7Ut_SNfjeE9ObXpySzJ5UWs/view?usp=sharing
Many thanks!
Use ncdf4 to explore your ncdf attributes
library(ncdf4)
nc<-nc_open('D:/NOAA_Snow_JanJune2016.nc')
print(nc)
varname<-names(nc$var)
Use raster to convert your ncdf to raster
library (raster)
r<-brick('D:/NOAA_Snow_JanJune2016.nc',varname='snow_cover_extent')
Here the plot
spplot(r[1])