Given a raster timeseries (satellite images within the same spatial extent each taken on a different date), how can I add the date as name for each layer/band in a (Geo)Tiff? Is it probably underspecified?
Using R as an example:
library(rgdal)
library(raster)
## example data (taken from https://rspatial.org/raster/rs/1-introduction.html#data)
dir.create('/tmp/data', showWarnings = FALSE)
if (!file.exists('/tmp/data/rs/samples.rds')) {
download.file('https://biogeo.ucdavis.edu/data/rspatial/rsdata.zip', dest = '/tmp/data/rsdata.zip')
unzip('/tmp/data/rsdata.zip', exdir='/tmp/data')
}
## just two raster layers
layers <- c("/tmp/data/rs/LC08_044034_20170614_B1.tif", "/tmp/data/rs/LC08_044034_20170614_B2.tif")
lc_stack <- raster::stack(layers)
## set arbitrary timestamps
lc_stack_z <- setZ(lc_stack,c("2019-09-20", "2019-09-21"))
timeseries_rs <- writeRaster(lc_stack_z, "/tmp/data/rs/timeseries-raster.tiff", "GTiff")
The gdalinfo output is as follows (no band name information):
[...]
Band 1 Block=1497x1 Type=Float32, ColorInterp=Gray
Min=0.096 Max=0.735
Minimum=0.096, Maximum=0.735, Mean=nan, StdDev=nan
NoData Value=-3.39999999999999996e+38
Metadata:
STATISTICS_MAXIMUM=0.73462820053101
STATISTICS_MEAN=nan
STATISTICS_MINIMUM=0.096417911350727
STATISTICS_STDDEV=nan
Band 2 Block=1497x1 Type=Float32, ColorInterp=Undefined
Min=0.075 Max=0.718
Minimum=0.075, Maximum=0.718, Mean=nan, StdDev=nan
NoData Value=-3.39999999999999996e+38
Metadata:
STATISTICS_MAXIMUM=0.71775615215302
STATISTICS_MEAN=nan
STATISTICS_MINIMUM=0.074839904904366
STATISTICS_STDDEV=nan
There is a Tiff Tag Library but there I can't find any (semantically appropriate) tag which would serve my need.
Is there a good practice how to add layer/band names to a (Geo)Tiff -- optimally which can be interpreted/read by other Software like QGis.
Related
I have a shapefile of population estimates of different administrative levels on Nigeria and I want to create a cartogram out of it.
I used the cartogram package and tried the following
library(cartogram)
admin_lvl2_cartogram <- cartogram(admin_level2_shape, "mean", itermax=5)
However this gives me an error stating "Error: Using an unprojected map. This function does not give correct centroids and distances for longitude/latitude data:
Use "st_transform()" to transform coordinates to another projection." I'm not sure how to resolve this
To recreate the initial data
Download the data using the wopr package
library(wopr)
catalogue <- getCatalogue()
# Select files from the catalogue by subsetting the data frame
selection <- subset(catalogue,
country == 'NGA' &
category == 'Population' &
version == 'v1.2')
# Download selected files
downloadData(selection)
Manually unzip the downloaded zip file (NGA_population_v1_2_admin.zip) and read in the data
library(rgdal)
library(here)
admin_level2_shape <- readOGR(here::here("wopr/NGA/population/v1.2/NGA_population_v1_2_admin/NGA_population_v1_2_admin_level2_boundaries.shp"))
The function spTransform in the sp package is probably easiest because the readOGR call returns a spatial polygon defined in that package.
Here's a full example that transforms to a suitable projection for Nigeria, "+init=epsg:26331". You'll probably have to Google to find the exact one for your needs.
#devtools::install_github('wpgp/wopr')
library(wopr)
library(cartogram)
library(rgdal)
library(sp)
library(here)
catalogue <- getCatalogue()
# Select files from the catalogue by subsetting the data frame
selection <- subset(catalogue, country == 'NGA' & category == 'Population' & version == 'v1.2')
# Download selected files
downloadData(selection)
unzip(here::here("wopr/NGA/population/v1.2/NGA_population_v1_2_admin.zip"),
overwrite = T,
exdir = here::here("wopr/NGA/population/v1.2"))
admin_level2_shape <- readOGR(here::here("wopr/NGA/population/v1.2/NGA_population_v1_2_admin/NGA_population_v1_2_admin_level2_boundaries.shp"))
transformed <- spTransform(admin_level2_shape, CRS("+init=epsg:26331"))
admin_lvl2_cartogram <- cartogram(transformed, "mean", itermax=5)
I confess I don't know anything about the specific packages so I don't know if what is produced is correct, but at least it transforms.
I have more than 50 raster files (ASCII format) that I need to crop. I already exported the mask from ArcMap in ASCII format as well and loaded it into R. How can I make it work for all rasters in a row and export them with the same name as before (of course in a different folder to not overwrite)?
I know there is a crop function in the raster package, but I never used it so far. I only stacked them for further habitat analysis.
My Code so far:
#### Use only part of area
files2 <- list.files(path="D:/",full.names=TRUE, pattern = "\\.asc$")
files2
# Create a RasterLayer from first file
mask_raster <- raster(files2[1])
# Crop. But how??
crop(x = , y=mask_raster)
writeRaster(...)`
I did not find an easy solution to crop multiple rasters by a raster, but by a shape file. So I just went back to ArcMap and converted the raster into a shapefile. Then in R, crop and mask were the crucial steps. See code below (modified from Mauricio Zambrano-Bigiarini's code). Hope this helps.
# Reading the shapefile (mask to crop later)
Maskshp <- readOGR("mask.shp")
# Getting the spatial extent of the shapefile
e <- extent(Maskshp)
# for loop with (in this case) 60 runs
for (i in 1:60) {
# Reading the raster to crop
files <- list.files(path="...your path",full.names=TRUE, pattern = "\\.asc$")
Env_raster <- raster(files[i])
# Save the filename
filename <- (paste(basename(files[i]), sep=""))
# Crop the raster
Env_raster.crop <- crop(Env_raster, e, snap="out")
# Dummy raster with a spatial extension equal to the cropped raster,
# but full of NA values
crop <- setValues(Env_raster.crop, NA)
# Rasterize the catchment boundaries, with NA outside the catchment boundaries
Maskshp.r <- rasterize(Maskshp, crop)
# Putting NA values in all the raster cells outside the shapefile boundaries
Maskshp.masked <- mask(x=Env_raster.crop, mask=Maskshp.r)
plot(Maskshp.masked)
#Export file to working directory with original name as new name
writeRaster(Maskshp.masked, filename)
}
I'd like to crop GeoTiff Raster Files using the two mentioned packages, "rgdal" and "raster". Everything works fine, except that the quality of the resulting output tif is very poor and in greyscale rather than colour. The original data are high quality raster maps from the swiss federal office of Topography, example files can be downloaded here.
This is my code:
## install.packages("rgdal")
## install.packages("raster")
library("rgdal")
library("raster")
tobecroped <- raster("C:/files/krel_1129_2012_254dpi_LZW.tif")
ex <- raster(xmn=648000, xmx=649000, ymn=224000, ymx=225000)
projection(ex) <- proj4string(tobecroped)
output <- "c:/files/output.tif"
crop(x = tobecroped, y = ex, filename = output)
In order to reproduce this example, download the sample data and extract it to the folder "c:/files/". Oddly enough, using the sample data the quality of the croped image is alright, but still greyscale.
I played around using the options "datatype", "format", but didnt get anywhere with that. Can anybody point out a solution? Should I supply more information the the input data?
EDIT:
Josh's example works superb with the sample data 2. Unfortunately, the data I have seems to be older and somewhat different. Can you tell me what option I choose if you read the following GDALinfo:
# packages same as above
OldInFile = "C:/files/krel1111.tif"
dataType(raster(OldInFile)
[1] "INT1U"
GDALinfo(OldInFile)
rows 4800
columns 7000
bands 1
lower left origin.x 672500
lower left origin.y 230000
res.x 2.5
res.y 2.5
ysign -1
oblique.x 0
oblique.y 0
driver GTiff
projection +proj=somerc +lat_0=46.95240555555556 +lon_0=7.439583333333333+k_0=1 +x_0=600000+y_0=200000 +ellps=bessel +units=m+no_defs
file C:/files/krel1111.tif
apparent band summary:
GDType hasNoDataValue NoDataValue blockSize1 blockSize2
1 Byte FALSE 0 1 7000
apparent band statistics:
Bmin Bmax Bmean Bsd
1 0 255 NA NA
Metadata:
AREA_OR_POINT=Area
TIFFTAG_RESOLUTIONUNIT=2 (pixels/inch)
TIFFTAG_XRESOLUTION=254
TIFFTAG_YRESOLUTION=254
Warning message:
statistics not supported by this driver
Edit (2015-03-10):
If one simply wants to crop out a subset of an existing GeoTIFF and save the cropped part to a new *.tif file, using gdalUtils::gdal_translate() may be the most straightforward solution:
library(raster) # For extent(), xmin(), ymax(), et al.
library(gdalUtils) # For gdal_translate()
inFile <- "C:/files/krel_1129_2012_254dpi_LZW.tif"
outFile <- "subset.tif"
ex <- extent(c(686040.1, 689715.9, 238156.3, 241774.2))
gdal_translate(inFile, outFile,
projwin=c(xmin(ex), ymax(ex), xmax(ex), ymin(ex)))
Looks like you need to change two details.
First, the *.tif file you're reading in has three bands, so should be read in using stack(). (Using raster() on it will only read in a single band (the first one, by default) producing a monochromatic or 'greyscale' output).
Second (for reasons mentioned here) writeRaster() will by default write out the values as real numbers (Float64 on my machine). To explicitly tell it you instead want to use bytes, give it the argument datatype='INT1U'.
library("rgdal")
library("raster")
inFile <- "C:/files/krel_1129_2012_254dpi_LZW.tif"
outFile <- "out.tif"
## Have a look at the format of your input file to:
## (1) Learn that it contains three bands (so should be read in as a RasterStack)
## (2) Contains values written as Bytes (so you should write output with datatype='INT1U')
GDALinfo(inFile)
## Read in as three separate layers (red, green, blue)
s <- stack(inFile)
## Crop the RasterStack to the desired extent
ex <- raster(xmn=648000, xmx=649000, ymn=224000, ymx=225000)
projection(ex) <- proj4string(s)
s2 <- crop(s, ex)
## Write it out as a GTiff, using Bytes
writeRaster(s2, outFile, format="GTiff", datatype='INT1U', overwrite=TRUE)
All of which outputs the following tiff file:
Have a Question on Mapping with R, specifically around the choropleth maps in R.
I have a dataset of ZIP codes assigned to an are and some associated data (dataset is here).
My final data format is: Area ID, ZIP, Probability Value, Customer Count, Area Probability and Area Customer Total. I am attempting to present this data by plotting area probability and Area Customer Total on a Map. I have tried to do this by using the census TIGER Shapefiles but I guess R cannot handle the complete country.
I am comfortable with the Statistical capabilities and now I am moving all my Mapping from third party GIS focused applications to doing all my Mapping in R. Does anyone have any pointers to how to achieve this from within R?
To be a little more detailed, here's the point where R stops working -
shapes <- readShapeSpatial("tl_2013_us_zcta510.shp")
(where the shp file is the census/TIGER) shape file.
Edit - Providing further details. I am trying to first read the TIGER shapefiles, hoping to combine this spatial dataset with my data and eventually plot. I am having an issue at the very beginning when attempting to read the shape file. Below is the code with the output
require(maptools)
shapes<-readShapeSpatial("tl_2013_us_zcta510.shp")
Error: cannot allocate vector of size 317 Kb
There are several examples and tutorials on making maps using R, but most are very general and, unfortunately, most map projects have nuances that create inscrutable problems. Yours is a case in point.
The biggest issue I came across was that the US Census Bureau zip code tabulation area shapefile for the whole US is huge: ~800MB. When loaded using readOGR(...) the R SpatialPolygonDataFrame object is about 913MB. Trying to process a file this size, (e.g., converting to a data frame using fortify(...)), at least on my system, resulted in errors like the one you identified above. So the solution is to subset the file based in the zip codes that are actually in your data.
This map:
was made from your data using the following code.
library(rgdal)
library(ggplot2)
library(stringr)
library(RColorBrewer)
setwd("<directory containing shapfiles and sample data>")
data <- read.csv("Sample.csv",header=T) # your sample data, downloaded as csv
data$ZIP <- str_pad(data$ZIP,5,"left","0") # convert ZIP to char(5) w/leading zeros
zips <- readOGR(dsn=".","tl_2013_us_zcta510") # import zip code polygon shapefile
map <- zips[zips$ZCTA5CE10 %in% data$ZIP,] # extract only zips in your Sample.csv
map.df <- fortify(map) # convert to data frame suitable for plotting
# merge data from Samples.csv into map data frame
map.data <- data.frame(id=rownames(map#data),ZIP=map#data$ZCTA5CE10)
map.data <- merge(map.data,data,by="ZIP")
map.df <- merge(map.df,map.data,by="id")
# load state boundaries
states <- readOGR(dsn=".","gz_2010_us_040_00_5m")
states <- states[states$NAME %in% c("New York","New Jersey"),] # extract NY and NJ
states.df <- fortify(states) # convert to data frame suitable for plotting
ggMap <- ggplot(data = map.df, aes(long, lat, group = group))
ggMap <- ggMap + geom_polygon(aes(fill = Probability_1))
ggMap <- ggMap + geom_path(data=states.df, aes(x=long,y=lat,group=group))
ggMap <- ggMap + scale_fill_gradientn(name="Probability",colours=brewer.pal(9,"Reds"))
ggMap <- ggMap + coord_equal()
ggMap
Explanation:
The rgdal package facilitates the creation of R Spatial objects from ESRI shapefiles. In your case we are importing a polygon shapefile into a SpatialPolygonDataFrame object in R. The latter has two main parts: a polygon section, which contains the latitude and longitude points that will be joined to create the polygons on the map, and a data section which contains information about the polygons (so, one row for each polygon). If, e.g., we call the Spatial object map, then the two sections can be referenced as map#polygons and map#data. The basic challenge in making choropleth maps is to associate data from your Sample.csv file, with the relevant polygons (zip codes).
So the basic workflow is as follows:
1. Load polygon shapefiles into Spatial object ( => zips)
2. Subset if appropriate ( => map).
3. Convert to data frame suitable for plotting ( => map.df).
4. Merge data from Sample.csv into map.df.
5. Draw the map.
Step 4 is the one that causes all the problems. First we have to associate zip codes with each polygon. Then we have to associate Probability_1 with each zip code. This is a three step process.
Each polygon in the Spatial data file has a unique ID, but these ID's are not the zip codes. The polygon ID's are stored as row names in map#data. The zip codes are stored in map#data, in column ZCTA5CE10. So first we must create a data frame that associates the map#data row names (id) with map#data$ZCTA5CE10 (ZIP). Then we merge your Sample.csv with the result using the ZIP field in both data frames. Then we merge the result of that into map.df. This can be done in 3 lines of code.
Drawing the map involves telling ggplot what dataset to use (map.df), which columns to use for x and y (long and lat) and how to group the data by polygon (group=group). The columns long, lat, and group in map.df are all created by the call to fortify(...). The call to geom_polygon(...) tells ggplot to draw polygons and fill using the information in map.df$Probability_1. The call to geom_path(...) tells ggplot to create a layer with state boundaries. The call to scale_fill_gradientn(...) tells ggplot to use a color scheme based on the color brewer "Reds" palette. Finally, the call to coord_equal(...) tells ggplot to use the same scale for x and y so the map is not distorted.
NB: The state boundary layer, uses the US States TIGER file.
I would advise the following.
Use readOGR from the rgdal package rather than readShapeSpatial.
Consider using ggplot2 for good-looking maps - many of the examples use this.
Refer to one of the existing examples of creating a choropleth such as this one to get an overview.
Start with a simple choropleth and gradually add your own data; don't try and get it all right at once.
If you need more help, create a reproducible example with a SMALL fake dataset and with links to the shapefiles in question. The idea is that you make it easy to help us help you rather than discourage us by not supplying code and data in your question.
I want to to convert two .shp files into one database that would allow me to draw the maps together.
Also, is there a way to convert .shp files into .csv files? I want to be able to personalize and add some data which is easier for me under a .csv format. What I have in mind if to add overlay yield data and precipitation data on the maps.
Here are the shapefiles for Morocco, and Western Sahara.
Code to plot the two files:
# This is code for mapping of CGE_Morocco results
# Loading administrative coordinates for Morocco maps
library(sp)
library(maptools)
library(mapdata)
# Loading shape files
Mor <- readShapeSpatial("F:/Purdue University/RA_Position/PhD_ResearchandDissert/PhD_Draft/Country-CGE/MAR_adm1.shp")
Sah <- readShapeSpatial("F:/Purdue University/RA_Position/PhD_ResearchandDissert/PhD_Draft/Country-CGE/ESH_adm1.shp")
# Ploting the maps (raw)
png("Morocco.png")
Morocco <- readShapePoly("F:/Purdue University/RA_Position/PhD_ResearchandDissert/PhD_Draft/Country-CGE/MAR_adm1.shp")
plot(Morocco)
dev.off()
png("WesternSahara.png")
WesternSahara <- readShapePoly("F:/Purdue University/RA_Position/PhD_ResearchandDissert/PhD_Draft/Country-CGE/ESH_adm1.shp")
plot(WesternSahara)
dev.off()
After looking into suggestions from #AriBFriedman and #PaulHiemstra and subsequently figuring out how to merge .shp files, I have managed to produce the following map using the following code and data (For .shp data, cf. links above)
code:
# Merging Mor and Sah .shp files into one .shp file
MoroccoData <- rbind(Mor#data,Sah#data) # First, 'stack' the attribute list rows using rbind()
MoroccoPolys <- c(Mor#polygons,Sah#polygons) # Next, combine the two polygon lists into a single list using c()
summary(MoroccoData)
summary(MoroccoPolys)
offset <- length(MoroccoPolys) # Next, generate a new polygon ID for the new SpatialPolygonDataFrame object
browser()
for (i in 1: offset)
{
sNew = as.character(i)
MoroccoPolys[[i]]#ID = sNew
}
ID <- c(as.character(1:length(MoroccoPolys))) # Create an identical ID field and append it to the merged Data component
MoroccoDataWithID <- cbind(ID,MoroccoData)
MoroccoPolysSP <- SpatialPolygons(MoroccoPolys,proj4string=CRS(proj4string(Sah))) # Promote the merged list to a SpatialPolygons data object
Morocco <- SpatialPolygonsDataFrame(MoroccoPolysSP,data = MoroccoDataWithID,match.ID = FALSE) # Combine the merged Data and Polygon components into a new SpatialPolygonsDataFrame.
Morocco#data$id <- rownames(Morocco#data)
Morocco.fort <- fortify(Morocco, region='id')
Morocco.fort <- Morocco.fort[order(Morocco.fort$order), ]
MoroccoMap <- ggplot(data=Morocco.fort, aes(long, lat, group=group)) +
geom_polygon(colour='black',fill='white') +
theme_bw()
Results:
New Question:
1- How to eliminate the boundaries data that cuts though the map in half?
2- How to combine different regions within a .shp file?
Thanks you all.
P.S: the community in stackoverflow.com is wonderful and very helpful, and especially toward beginners like :) Just thought of emphasizing it.
Once you have loaded your shapefiles into Spatial{Lines/Polygons}DataFrames (classes from the sp-package), you can use the fortify generic function to transform them to flat data.frame format. The specific functions for the fortify generic are included in the ggplot2 package, so you'll need to load that first. A code example:
library(ggplot2)
polygon_dataframe = fortify(polygon_spdf)
where polygon_spdf is a SpatialPolygonsDataFrame. A similar approach works for SpatialLinesDataFrame's.
The difference between my solution and that of #AriBFriedman is that mine includes the x and y coordinates of the polygons/lines, in addition to the data associated to those polgons/lines. I really like visualising my spatial data with the ggplot2 package.
Once you have your data in a normal data.frame you can simply use write.csv to generate a csv file on disk.
I think you mean you want the associated data.frame from each?
If so, it can be accessed with the # slot access function. The slot is called data:
write.csv( WesternSahara#data, file="/home/wherever/myWesternSahara.csv")
Then when you read it back in with read.csv, you can try assigning:
myEdits <- read.csv("/home/wherever/myWesternSahara_modified.csv")
WesternSahara#data <- myEdits
You may need to do some massaging of row names and so forth to get it to accept the new data.frame as valid. I'd probably try to merge the existing data.frame with a csv you read in in R, rather than making edits destructively....