I am dealing with 28 HDF4 files on ocean primary productivity (annual .tar files can be found here: http://orca.science.oregonstate.edu/1080.by.2160.monthly.hdf.cbpm2.v.php)
My goal is to do some calculations (I need to calculate concentrations per area and obtain the mean over several years, i.e. combine all files spatially) and then convert them to a georeferenced file I can work with in ArcGIS (preferably shapefile, or geotiff).
I have tried several ways to convert to ASCII or raster files and then add a projection using gdalUtils tools such as gdal_translate and get_subdatasets. However, as the HDF4 files are not named after the standards (unlike the MODIS files), the latter doesn't work and I cannot access the subsets.
Here's the code I used to convert to raster:
library(raster)
library(gdalUtils)
setwd("...path_to_files...")
gdalinfo("cbpm.2015060.hdf")
hdf_file <- "cbpm.2015060.hdf"
outfile="testout"
gdal_translate(hdf_file,outfile,sds=TRUE,verbose=TRUE)
file.rename(outfile,paste("CBPM_test",".tif",sep=""))
rast <- raster("CBPM_test.tif")
wgs1984 <- CRS("+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0")
projection(rast) <- wgs1984
#crs(rast) <- "+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0"
plot(rast)
writeRaster(rast, file="CBPM_geo.tif", format='GTiff', overwrite=TRUE)
The resulting projection is completely off. I'd appreciate help how to do this (converting through any format that works), preferably as batch process.
You've not set the extent of your raster, so its assumed to be 1:ncols, 1:nrows, and that's not right for a lat-long data set...
The gdalinfo implies its meant to be a full sphere, so if I do:
extent(rast)=c(xmn=-180, xmx=180, ymn=-90, ymx=90)
plot(rast)
writeRaster(rast, "output.tif")
I see a raster with full global lat-long extent and when I load the raster into QGIS it overlays nicely with an OpenStreetMap.
There doesn't seem to be enough metadata in the file to do a precise projection (what's the Earth radius and eccentricity?) so don't try and do anything small-scale with this data...
Here's how it looks:
Also you've jumped through a few unnecessary hoops to read this. You can read the HDF directly and set its extent and projection:
> r = raster("./cbpm.2017001.hdf")
What do we get:
> r
class : RasterLayer
dimensions : 1080, 2160, 2332800 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : 0, 2160, 0, 1080 (xmin, xmax, ymin, ymax)
coord. ref. : NA
data source : /home/rowlings/Downloads/HDF/cbpm.2017001.hdf
names : cbpm.2017001
Set extent:
> extent(r)=c(xmn=-180, xmx=180, ymn=-90, ymx=90)
And projection:
> projection(r)="+init=epsg:4326"
And land values to NA:
> r[r==-9999]=NA
Write it, plot it:
> writeRaster(r,"r.tif")
> plot(r)
Related
I have a shapefile of 10 CA counties projected in NAD83(Hard)/CA Albers. I have a raster (a netCDF file of temperature) for the entire US projected in WGS84/WGS84. I want to use the shapefile to clip the raster. I know that I need to get them on the same datum/projection first. But I've tried re-projecting the raster using raster::projectRaster(). That failed (as in the data disappeared). So then I tried re-projecting the shapefile instead using sp::spTransform(). This also failed (as in the data don't overlap). I've searched through stackoverflow but didn't see anything that seemed to help. I'm not getting an error, but projectRaster is not working and re-projecting the shapefile using spTransform doesn't produce the desired outcome. I feel like there is something specific going on here, like the transformation from WGS84 to NAD83 or loading the raster in using raster() is the problem ... but then again, it could easily be something stupid that I'm missing! =)
my shapefile and raster are here: https://www.dropbox.com/sh/l3b2syzcioeqmyy/AAA5CstBZty4ofOcVFkAumNYa?dl=0
here is my code:
library(raster) #for creating rasters from .bil files
library(rgdal) #for reading .bil files and .gdb files
library(ncdf4) #for working with ncdf files
library(sp) #for working with spatial data files
load(my_counties.RData)
myraster <- raster(myraster.nc)
my.crs <- CRS("+init=EPSG:3311") #NAD83(HARN) / California Albers (HARN is high resolution)
newraster <- projectRaster(myraster, res = 6000, crs = my.crs) #raster resolution is 1/16th of a degree
#There is data in the raster.
plot(myraster)
#but none in newraster
plot(newraster)
#Now try re-projecting the shapefile
my.crs2 <- crs(myraster)
newshapefile <- spTransform(my_counties, my.crs2)
#but the data don't overlap
plot(newshapefile); plot(myraster, add = T)
You can do
library(raster)
library(rgdal)
load("my_counties.RData")
b <- brick("myraster.nc")
Now look at b
b
#class : RasterBrick
#dimensions : 490, 960, 470400, 365 (nrow, ncol, ncell, nlayers)
#resolution : 0.0625, 0.0625 (x, y)
#extent : 234, 294, 23.375, 54 (xmin, xmax, ymin, ymax)
#crs : +proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0
#source : myraster.nc
#names : X2005.01.01, X2005.01.02, X2005.01.03, X2005.01.04, X2005.01.05, X2005.01.06, X2005.01.07, X2005.01.08, X2005.01.09, X2005.01.10, X2005.01.11, X2005.01.12, X2005.01.13, X2005.01.14, X2005.01.15, ...
#Date : 2005-01-01, 2005-12-31 (min, max)
#varname : tasmax
The horizontal extent is between 234 and 294 degrees. That points to a system with longitudes that start in Greenwich at 0 and continues to 360 (again in Greenwich). Climatologist do that. To go to the more conventional -180 to 180 degrees system:
r <- shift(b, -360)
(if your data had a global extent, you would use raster::rotate instead)
Now, transform the counties to lonlat and show that they overlap.
counties <- spTransform(my_counties, crs(r))
plot(r, 1)
lines(counties)
It is generally best to transform vector data, not raster data if you can avoid it.
I have been attempting to export a geographic raster layer which is projected in terms of the Lambert Conformal Conic projection (with a resolution of 1000 m by 1000 m) as an image file (e.g. as an *.eps) after having plotted it with some overlaid points (which is straightforward enough), but upon opening the image after export, it is clear that it has a resolution of a lesser quality than the aforementioned.
I've been using the 24 Bioclimate layers downloadable from: https://sites.ualberta.ca/~ahamann/data/climatewna.html. Here is an example of one of those layers (which was saved as a *.tif), and its resolution:
Below is the section of the script I have been running which is of relevance:
> projection <- raster("prediction.grd")
> projection
class : RasterLayer
dimensions : 3132, 2359, 7388388 (nrow, ncol, ncell)
resolution : 1000, 1000 (x, y)
extent : -3594000, -1235000, 4703000, 7835000 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=lcc +lat_1=49 +lat_2=77 +lat_0=0 +lon_0=-95 +x_0=0 +y_0=0 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
data source : C:\Users\User\Documents\Project\prediction.grd
names : layer
values : 7.006775e-12, 0.001495079 (min, max)
> plot(projection, col = colorPalette(14, "jc"), colNA = 'black', axes = FALSE);
> points(train_locality_points, col='white', pch=15, cex=0.2)
Following the running of this code, I've gone about exporting the plot generated via the 'export' dropdown menu of R Studio to save it as an image with an extension that can be viewed outside of R instead of as a file with its original extension or something similar. I can open the image which is saved, but it evidently is not of the same quality as what the resolution of the layer is stated as above.
The image below is one I've saved in the way just described. To illustrate the trouble I've been having, Vancouver Island ( i.e. the large one with many white dots clustered at its southern end) is about 500 kilometres in length, which should correspond to about 500 pixels in the image if the resolution were 1 km by 1 km as the raster layer is. In this image the length of the island is represented by about a fifth of that number (...yes, I counted).
Any help as to how this issue could be remedied I would greatly appreciate. Thank-you for taking the time to read through and consider this question.
To get an image with same resolution as RasterLayer r, you can do
png('file.png', height=nrow(r), width=ncol(r))
plot(r, maxpixels=ncell(r))
dev.off()
While I am not new to R, I am new to visualizing spatial data in it rather than ArcGIS.
I am attempting to import a GeoTIFF image from a public-access GIS website that is already projected in Mercator (image can be found here). Using the raster package in R, I can successfully import the GeoTIFF into R and plot it (see below).
> library(raster)
> mb <- raster("/Maps/mb_srelief10m/mb_srelief10m.tif")
> mb
class : RasterLayer
dimensions : 8060, 7220, 58193200 (nrow, ncol, ncell)
resolution : 10, 10 (x, y)
extent : -48598.54, 23601.46, 3850014, 3930614 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=merc +lon_0=-70.31666666666668 +lat_ts=41.65 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs +ellps=GRS80 +towgs84=0,0,0
data source : C:\Users\Connor\Documents\ArcGIS\Maps\mb_srelief10m\mb_srelief10m.tif
names : mb_srelief10m
values : 0, 255 (min, max)
attributes :
ID COUNT BinValues Value
from: 0 1460797 0 0
to : 51 14795988 255 255
As you can see, the x- and y-axis labels represent the NAD83 values from the raster file (red boxes in image). Yet, when I attempt to use the crop function to use only a portion of this raster file, I cannot select the appropriate area because my user-defined extent object is in latitude and longitude coordinates. I understand I can use the drawExtent function to manually select the area but I want to automate this process.
My question is:
How do I transform the raster file so that my axes are in latitude and longitude and I can crop the file using latitude and longitude coordinates?
If this post would be more appropriate elsewhere, please let me know and I will remove the file. Thank you!
I've recently started using R for spatial data. I'd be really grateful if you could could help me with this. Thanks!
I've extracted data from a multidimensional netCDF file. This file had longitude, latitude and temperature data (for 12 months of a specific year).
From this netCDF I've got a data frame for January with these variables: longitude, latitude, temperature.
With this data frame I've created a raster.
# Packages
library("sp")
library("raster")
library("rgdal")
library("ncdf")
library("maptools")
library("rgeos")
library("sm")
library("chron")
# Dataframe to raster
# Create spatial points data frame
coordinates(tmp.df01) <- ~ lon + lat
# Coerce to SpatialPixelsDataFrame
gridded(tmp.df01) <- T
# Coerce to raster
rasterDF1 <- raster(tmp.df01)
> print(tmp.df01)
class : SpatialPixelsDataFrame
dimensions : 103, 241, 24823, 1 (nrow, ncol, npixels, nlayers)
resolution : 0.02083333, 0.02083333 (x, y)
extent : 5.739583, 10.76042, 45.73958, 47.88542 (xmin, xmax, ymin, ymax)
coord. ref. : NA
names : TabsM_1
min values : -18.1389980316162
max values : 2.26920962333679
There is no value for 'coord. ref.'
The projection of the original netCDF was WGS84. So I gave this projection to the raster.
proj4string(rasterDF1) <- "+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0"
Then, I wanted to reproject my raster to another projection:
# Reprojecting into CH1903_LV03
# First, change the coordinate reference system (crs)
proj4string(rasterDF1) <- "+init=epsg:21781"
# Second, reproject the raster
rasterDF1.CH <- spTransform(rasterDF1, crs("+init=epsg:21781"))
At this point I get the following error:
Error in spTransform(rasterDF1, crs("+init=epsg:21781")) :
load package rgdal for spTransform methods
But the package rgdal is already uploaded! It must be something wrong in the code!
Here the code to solve the problem described.
Solution provided by Frede Aakmann Tøgersen.
tmp.df01 # tmp.df01 is a data.frame
coordinates(tmp.df01) <- ~ lon + lat # tmp.df01 is now a SpatialPointsDataFrame
# Assign orignial data projection
proj4string(tmp.df01) <- CRS("+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0")
gridded(tmp.df01) <- T # tmp.df01 is now a SpatialPixelFrame
# Coerce to raster
rasterDF1 <- raster(tmp.df01) # rasterDF1 is a RasterLayer
# To reproject the raster layer
rasterDF1.proj <- projectRaster(rasterDF1, crs=CRS("+init=epsg:21781"))
Here is another option GDAL. Using gdal_translate you can convert netcdf file with bands to geotiffs.
gdal_translate -a_srs EPSG:4326 NETCDF:File_Name.nc:Band_Name -of ‘Gtiff’ Output_FileName.geotiff
To explore more options in gdal_translate you can visit this link
I have a nc file (which I can read it as a raster in R and ArcGIS) and a shapefile (polygon) for Canada.
When I open both files in ArcMap, it re-projects them on-fly and the polygon would be located on the right location:
Then, I use R to open and plot them using the code:
## set the working directory:
setwd("C:/Users/Desktop/Data")
## required libraries
library(raster)
library(rgdal)
library(ncdf4)
## reading the polygon:
My_study_area <- readOGR(dsn = ".", layer = "Canada_AB")
## create a connection to the downloaded NC file:
fn <- "tasmin_day_CanESM5_ssp126_r1i1p1f1_gn_21010101-23001231_cliped_AB.nc"
ncin <- nc_open(fn)
ncin
## create a raster from the nc file:
fn_brick <- brick(fn)
fn_brick
extent(fn_brick)
Here is the output:
class : Extent
xmin : 229.2188
xmax : 271.4062
ymin : 39.06842
ymax : 64.18258
## since it has 73,000 layers, let us only work on one layer: fn_brick[[1]]
My Goal:
## I want to shift this raster in the x direction for -177.1876 units:
## why -177.1876 units ?
## because I know if I shift it for -177.1876 units, the raster and the polygon will be in the right place
fn_brick_shifted <- shift(fn_brick[[1]], dx= -177.1876, dy= 0, filename="new_nc_file", format="GTiff", datatype='FLT4S', overwrite=T)
extent(fn_brick_shifted)
Despite shifting the raster, its extent is still remained unchanged:
class : Extent
xmin : 229.2188
xmax : 271.4062
ymin : 39.06842
ymax : 64.18258
However, let us plot them:
plot(fn_brick_shifted)
lines(My_study_area)
The result in the R is like this:
As can be seen, the raster is NOT shifted, therefore the polygon is not plotted on the right place.
However, we can see the coordinate info for the files:
For the raster:
crs(fn_brick_shifted)
CRS arguments:
+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0
For the polygon:
crs(My_study_area)
CRS arguments:
+proj=longlat +datum=WGS84 +no_defs +ellps=WGS84 +towgs84=0,0,0
Did I make any mistake in using the shift function for the raster?
Do I need to employ another function for obtaining my goal?
Any comment or help would be highly appreciated.
If you think having the raster and polygon files may help, you could download them here in a zip file:
nc and polygon.zip