I am trying to convent HDF4 files (representing daily Sea Ice Concentration) to raster object in R. However, the HDF files themselves do not contain the longitude/latitude grids or projection information, and such information should be extracted from another hdf file.
The website on data format says:
Data Format
Sea ice concentration maps with two different color scales are available as PNG image. The NIC color scale uses the same colors as the National Ice Center, the "visual" color scale uses white and shades of grey.
There is one file per day per region per color scale.
Sea ice concentration data are available as HDF4 files: There is one file per day per region. Each file contains one two-dimensional array of the sea ice concentration in a polar stereographic grid.
The longitude and latitude coordinates of each pixel in a the HDF4 file are saved in extra files, one file per region for each available resolution.
They are found here: https://seaice.uni-bremen.de/data/grid_coordinates/, sorted by hemisphere and grid resolution (see also the README file https://seaice.uni-bremen.de/data/grid_coordinates/README).
GEOTIFF files use the NIC color scale and were tested to work with QGIS. Ice concentrations are scaled between 0 and 100, land and missing values are set to 120 (older files: SIC: 0-200, land/NaN: 255).
I tried to use R to load this map using this code:
> require(raster)
> CurrTemp <- tempfile()
> download.file(url = "https://seaice.uni-bremen.de/data/amsre/asi_daygrid_swath/s6250/2003/feb/Antarctic/asi-s6250-20030214-v5.hdf", destfile = CurrTemp, mode = "wb", quiet = T)
> Map1 <- readAll(raster(CurrTemp))
> plot(Map1)
> Map1
class : RasterLayer
dimensions : 1328, 1264, 1678592 (nrow, ncol, ncell)
resolution : 1, 1 (x, y)
extent : 0, 1264, 0, 1328 (xmin, xmax, ymin, ymax)
coord. ref. : NA
data source : in memory
names : file43fc5b4e68de
values : 0, 100 (min, max)
The map is loaded into R as raster object but with wrong coordinates and with no projection. According to this page, coordinates should be extracted from another hdf file.
Could you please let me know how to convert these hdf files into raster obejcts with correct coordinates and projection.
Thanks.
I used one of the geotiff files that they also make available to find the extent and crs.
library(raster)
raster('asi-AMSR2-s6250-20180922-v5.tif')
#class : RasterLayer
#dimensions : 1328, 1264, 1678592 (nrow, ncol, ncell)
#resolution : 6250, 6250 (x, y)
#extent : -3950000, 3950000, -3950000, 4350000 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=stere +lat_0=-90 +lat_ts=-70 +lon_0=0 +k=1 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs
#data source : asi-AMSR2-s6250-20180922-v5.tif
#names : asi.AMSR2.s6250.20180922.v5
#values : 0, 255 (min, max)
Now I know I can do
library(raster)
CurrTemp <- tempfile()
download.file(url = "https://seaice.uni-bremen.de/data/amsre/asi_daygrid_swath/s6250/2003/feb/Antarctic/asi-s6250-20030214-v5.hdf", destfile = CurrTemp, mode = "wb", quiet = T)
r <- raster(CurrTemp)
extent(r) <- c(-3950000, 3950000, -3950000, 4350000)
crs(r) <- "+proj=stere +lat_0=-90 +lat_ts=-70 +lon_0=0 +k=1 +x_0=0 +y_0=0 +a=6378273 +b=6356889.449 +units=m +no_defs "
# writeRaster(r, 'my_asi-s6250-20030214-v5.tif')
The "other hdf" file has longitude / latitude values for the cells, but that is not what you are after as the data do not have a lon/lat coordinate reference system.
Related
I'm merging two MODIS DSR tiles using a R script that I developed, these are the products:
https://drive.google.com/drive/folders/1RG3JkXlbaotBax-h5lEMT7lEn-ObwWsD?usp=sharing
So, I open both products (tile h15v05 and tile h16v05) from same date (2019180), then I open each SDS and merge them together (00h from h15v05 with 00h from h16v05 and so on...)
Visualisation on Panoply (using the merge option) of the two products:
Purple square is the location of the division line that separates the two tiles.
With my code I obtain a plot with pixels with different resolution (and different min/max values) and I don't understand why:
I suspect that the results obtained are due to:
1- Changing from Sinusoidal CRS to longlat WGS84 CRS;
2- Using resample (method ngb) to work with mosaic.
My code is extensive, but here are some parts of it:
# Open scientific dataset as raster
SDSs <- sds(HDFfile)
SDS <- SDSs[SDSnumber]
crs(SDS) <- crs("+proj=sinu +lon_0=0 +x_0=0 +y_0=0 +a=6371007.181 +b=6371007.181 +units=m +no_defs")
SDSreprojected <- project(SDS, DesiredCRS)
SDSasRaster <- as(SDSreprojected, "Raster")
# Resample SDS based on a reference SDS (SDS GMT_1200_DSR of a first product), I need to do this to be able to use mosaic
SDSresampled <- resample(SDSasRaster,ResampleReference_Raster,method='ngb')
# Create mosaic of same SDS, but first convert stack to list to use mosaic
ListWith_SameSDS_OfGroupFiles <- as.list(StackWith_SameSDS_OfGroupFiles)
ListWith_SameSDS_OfGroupFiles.mosaicargs <- ListWith_SameSDS_OfGroupFiles
ListWith_SameSDS_OfGroupFiles.mosaicargs$fun <- mean
SDSmosaic <- do.call(mosaic, ListWith_SameSDS_OfGroupFiles.mosaicargs)
# Save SDSs mosaic stack to netCDF
writeRaster(StackWith_AllMosaicSDSs_OfGroupFiles, NetCDFpath, overwrite=TRUE, format="CDF", varname= "DSR", varunit="w/m2", longname="Downward Shortwave Radiation", xname="Longitude", yname="Latitude", zname="TimeGMT", zunit="GMT")
Does anyone have an idea of what could be the cause of this mismatch between results?
print(ResampleReference_Raster)
class : RasterLayer
dimensions : 1441, 897, 1292577 (nrow, ncol, ncell)
resolution : 0.01791556, 0.006942043 (x, y)
extent : -39.16222, -23.09196, 29.99652, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : MCD18A1.A2019180.h15v05.061.2020343034815
values : 227.5543, 970.2346 (min, max)
print(SDSasRaster)
class : RasterLayer
dimensions : 1399, 961, 1344439 (nrow, ncol, ncell)
resolution : 0.01515284, 0.007149989 (x, y)
extent : -26.10815, -11.54627, 29.99717, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : MCD18A1.A2019180.h16v05.061.2020343040755
values : 0, 0 (min, max)
print(SDSmosaic)
class : RasterLayer
dimensions : 1441, 897, 1292577 (nrow, ncol, ncell)
resolution : 0.01791556, 0.006942043 (x, y)
extent : -39.16222, -23.09196, 29.99652, 40 (xmin, xmax, ymin, ymax)
crs : +proj=longlat +datum=WGS84 +no_defs
source : memory
names : layer
values : 0, 62.7663 (min, max)
Also, some of the islands were ignored by the script (bottom right)...
sorry I didn't reply earlier. So I think you're right that this issue is extent to which you are resampling. I think you might be able to get around this by creating a dummy raster that has the extent of the raster you want to resample, but has the resolution of the raster you want to mosaic to.Try:
dummy<-raster(ext = SDSasRaster#extent, resolution=ResampledReference_Raster#res, crs=SDSasRaster#crs)
SDS2<-resample(SDSasRaster, dummy, method="ngb")
Final<-moasic(SDS2, ResampledReference_Raster, fun=mean)
I would like to make a piece of code I have more efficient.
Right now I download Sentinel-2 data in jp2 format via the open acceshub. The jp2 files that I download have, for some reason, a wrong extent. Right now I correct this in the following way (in which file is the filename of the jp2):
r = raster(file)
extent(r) = new_extent
writeRaster(r, file)
This method, however, writes the entire raster (which takes ages) whereas I only changed a minor detail.
Is there a neat way using gdal or the raster package to do this more efficiently?
If I print the raster I see:
class : RasterLayer
dimensions : 1830, 1830, 3348900 (nrow, ncol, ncell)
resolution : 60, 60 (x, y)
extent : 499980, 609780, 6690240, 6800040 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=utm +zone=55 +datum=WGS84 +units=m +no_defs +ellps=WGS84 +towgs84=0,0,0
data source : /home/daniel/R/farmhack/x.jp2
names : x
values : 0, 65535 (min, max)
I do not know what this extent means.
I have a raster that I have reprojected from Cylindrical Equal-Area (Lambert) Central Meridian: -160. Datum: WGS 1984 (+proj=cea +lon_0=Central Meridian +lat_ts=Standard Parallel +x_0=False Easting +y_0=False Northing +ellps=WGS84) to latitude and longitude.
The original raster looks like this:
library(rasterVis)
levelplot(r)
Original raster without missing points
r
#class : RasterLayer
#dimensions : 64, 200, 12800 (nrow, ncol, ncell)
#resolution : 2e+05, 2e+05 (x, y)
#extent : -20037507, 19962493, -6363885, 6436115 (xmin, xmax, ymin, ymax)
#coord. ref. : +proj=cea +lon_0=-160 +lat_ts=Standard Parallel +x_0=False Easting +y_0=False Northing +ellps=WGS84
#data source : N:\My documents\Data\Exposure\Reefs at risk\Global_Threats\Acidification\arag_380\w001001.adf
#names : w001001
#values : 1.025163, 4.11939 (min, max)
I have been able to use the projectRaster function from the Raster package by cutting the extent of the y-axis of the raster by 0.95 to the North and South. I was having a few problems reprojecting the raster without cutting the extent (see here: https://gis.stackexchange.com/questions/220589/error-using-projectraster-in-r-error-in-if-maxy-miny-missing-value-whe/220741#220741).
#Cut y-axis values because projectRaster failed using full extent
extent(r) <- c(xmin= -20037507, xmax= 19962493, ymin= 0.95*(-6363885), ymax= 0.95*(6436115))
# Define the new Proj.4 spatial reference
sr <- "+proj=longlat +datum=WGS84 +ellps=WGS84 +towgs84=0,0,0"
# Project Raster
projected_r <- projectRaster(r, crs = sr, method = 'bilinear')
The problem I am having is that there are missing data in the new, reprojected raster.
levelplot(projected_r)
reprojected raster with missing points
The missing coordinates are not located in the 5% North and Souththat I cut from the extent so I am not sure why these data are missing? Any help would be appreciated!
library(raster)
library(ncdf4)
library(rgdal)
I am trying to crop a raster layer based on polygon: sample data (~45MB) can be found here sample data for illustration:
dat<- raster('data.nc')# make a subset
dat
class : RasterLayer
dimensions : 824, 935, 770440 (nrow, ncol, ncell)
resolution : 10000, 10000 (x, y)
extent : -5000, 9345000, -5000, 8235000 (xmin, xmax, ymin, ymax)
coord. ref. : NA
data source : C:\Users\data.nc
names : Total.Precipitation
zvar : Total.Precipitation
As coord. ref is = NA, I assign a polar stereographic projection based on information from the website of the data provider.
projj=CRS("+proj=stere +lat_0=90 +lat_ts=90 +lon_0=-100 +k=0.994 +x_0=2000000 +y_0=2000000 +datum=WGS84 +units=m +no_defs")
proj4string(dat) <- projj
#rx <- projectRaster(from=dat, crs=wgs84.p4s)
Then transform the bbox shapefile to projj
shgrid <- spTransform(Prairie.Boundaries, projj)# transform to dat spat reference
plot(dat)
plot(shgrid,add=T)# this gives
CROPPING
cr <- crop(dat, extent(shgrid))
plot(cr)
fr <- rasterize(shgrid, cr)
dat1<- mask(x=cr, mask=fr)
levelplot(dat1)
OUTPUT:
The cropped image shows only part of canada but I want all of Canada.
What did I miss within my code? Could it be the projection?
Besides I would prefer the axes labels in latlon coordinates.
Any suggestions?
While I am not new to R, I am new to visualizing spatial data in it rather than ArcGIS.
I am attempting to import a GeoTIFF image from a public-access GIS website that is already projected in Mercator (image can be found here). Using the raster package in R, I can successfully import the GeoTIFF into R and plot it (see below).
> library(raster)
> mb <- raster("/Maps/mb_srelief10m/mb_srelief10m.tif")
> mb
class : RasterLayer
dimensions : 8060, 7220, 58193200 (nrow, ncol, ncell)
resolution : 10, 10 (x, y)
extent : -48598.54, 23601.46, 3850014, 3930614 (xmin, xmax, ymin, ymax)
coord. ref. : +proj=merc +lon_0=-70.31666666666668 +lat_ts=41.65 +x_0=0 +y_0=0 +datum=NAD83 +units=m +no_defs +ellps=GRS80 +towgs84=0,0,0
data source : C:\Users\Connor\Documents\ArcGIS\Maps\mb_srelief10m\mb_srelief10m.tif
names : mb_srelief10m
values : 0, 255 (min, max)
attributes :
ID COUNT BinValues Value
from: 0 1460797 0 0
to : 51 14795988 255 255
As you can see, the x- and y-axis labels represent the NAD83 values from the raster file (red boxes in image). Yet, when I attempt to use the crop function to use only a portion of this raster file, I cannot select the appropriate area because my user-defined extent object is in latitude and longitude coordinates. I understand I can use the drawExtent function to manually select the area but I want to automate this process.
My question is:
How do I transform the raster file so that my axes are in latitude and longitude and I can crop the file using latitude and longitude coordinates?
If this post would be more appropriate elsewhere, please let me know and I will remove the file. Thank you!