How can one make a Rasterbrick in R from several hdf5 files? Often, data are provided in hdf5 format and one has to convert it to a more friendly format for easy handling.
At the moment I know of the rhdf5 package but how to get a RasterBrick is that which I am unsure about.
source("http://bioconductor.org/biocLite.R")
biocLite("rhdf5")
library("rhdf5")
library("raster")
You can access several hdf5 files on this link http://mirador.gsfc.nasa.gov/cgi-bin/mirador/cart.pl?C1=GPM_3IMERGHH&CGISESSID=fb3b45e091f081aba8823f3e3f85a7d9&LBT_THRESHOLD=4000000.
You can use two files for illustration.
Thanks!
AT.
One option is using gdalUtils to convert the hdf5 files into GTiff. Once you did that you can read them in a stack. Here is an example code:
# list all the `hdf5` files
files <- list.files(path=".", pattern=paste(".*.h5",sep=""), all.files=FALSE, full.names=TRUE)
#choose the band that you want using the sds[] option and write GTiff files.
for (i in (files)) {
sds <- get_subdatasets(i)
r2 <- gdal_translate(sds[1], dst_dataset =paste(i,".tif",sep=""))}
Related
I am using one of the files in here: http://orca.science.oregonstate.edu/1080.by.2160.monthly.hdf.vgpm.m.chl.m.sst.php:
untar(tarfile = "http://orca.science.oregonstate.edu/data/1x2/monthly/vgpm.r2018.m.chl.m.sst/hdf/vgpm.m.2010.tar", exdir = "./foo")
I get error: ar.exe: Error opening archive: Failed to open 'http://orca.science.oregonstate.edu/data/1x2/monthly/vgpm.r2018.m.chl.m.sst/hdf/vgpm.m.2010.tar'
so I manually had to download the file and untar it ( that is why cant provide a reproducible example here). Inside there are files of .hdf format:
I also was not able to read them:
library(ncdf4)
ncin <- nc_open(".\\vgpm.m.2010\\vgpm.2010001.hdf")
ncin
lon <- ncvar_get(ncin,"fakeDim0")
head(lon)
lat <- ncvar_get(ncin,"fakeDim1")
head(lat)
fillvalue <- ncatt_get(ncin,"npp","_FillValue")
Can you please help to explain why i cant utar the file and why .hdf files have no fill value?
You should be able to untar the file once you have downloaded it. Download the file first to your working directory, then untar from your working directory: untar("vgpm.m.2002.tar", exdir = "mydir"). Your issue is likely with the connection. There can be many reasons for that which are specific to your computer's settings. You'll need to troubleshoot that separately.
Once you untar the directory, the contents inside are not .hdf files. They are compressed .hdf files (thus why their file names end in .gz). You'll need to decompress:
library(R.utils)
gunzip("mydir/vgpm.2002335.hdf.gz", remove = FALSE)
Once you actually have the .hdf file, you need to open it and then read it. You are correct to use ncdf4 because it accommodates multiple .hdf file formats. Some of the older formats would need different packages or software.
To open and read it, you need two different functions, nc_open() and ncvar_get():
library(ncdf4)
dat <- nc_open("mydir/vgpm.2002335.hdf", write = TRUE)
ncvar_get(dat)
Note that these functions will NOT work if you have not completed the pre-requisite set-up explained in detail in the documentation. For example:
Both the netCDF library and the HDF5 library must already be installed on your machine for this R interface to the library to work.
I also tried to rasterize it: it also works great:
library(raster)
x <- raster("~.\\vgpm.2010001.hdf")
extent(x) <- extent(-180, 180, -90, 90)
crs(x) <- "+proj=longlat +datum=WGS84"
NAvalue(x) <- -9999
#plot(x)
f1<- as.data.frame(x, xy=TRUE)
I have downloaded the latest R package and am using RStudio and am trying to convert a pgm image into a csv file and am using a readImage function.
Although any time I do
img <- readImage(file)
where file is the filepath
I get
Error in readImage(file) : could not find function "readImage"
Is there some other pack I need to download or am I using it wrong?
You can use the magick package to read pgm files.
First, you need to do:
install.packages("magick")
Now you call
library(magick)
In my case, I have a pgm file in my R home directory, so I make the file path with:
file <- path.expand("~/cat.pgm")
Now I can read the image and convert it into a matrix of RGB strings by doing:
img <- image_read(file)
ras <- as.raster(img)
mat <- as.matrix(ras)
To write this to csv format, I can do:
write.csv(mat, "cat.csv", row.names = FALSE)
So now I have the image saved as a csv file. To read this back in, and prove it works, I can do:
cat_csv <- read.csv("cat.csv")
cat_ras <- as.raster(as.matrix(cat_csv))
plot(cat_ras)
Note though that the csv file is very large - 9MB, which is one of the reasons why it is rarely a good idea to store an image as csv.
Created on 2022-02-05 by the reprex package (v2.0.1)
I have a bam file does anyone know how to convert a bam file to a csv file? I am trying to use R-software to open the bam file but I am not sure how to get the variables from the bam files so far I have used the below mentioned coding:
rm(list=ls())
#install bam packages
source("http://bioconductor.org/biocLite.R")
biocLite("Rsamtools",suppressUpdates=TRUE)
biocLite("RNAseqData.HNRNPC.bam.chr14",suppressUpdates=TRUE)
biocLite("GenomicAlignments",suppressUpdates=TRUE)
#load library
library(Rsamtools)
library(RNAseqData.HNRNPC.bam.chr14)
library(GenomicAlignments)
bamfile <- file.path("C:","Users","azzop","Desktop","I16-1144-01-esd_m1_CGCTCATT-AGGCGAAG_tophat2","accepted_hits.bam")
gal<-readGAlignments(bamfile)
gal
length(gal)
names(gal)
When I inserted names(gal) it gave me NULL not sure it is the correct.
I would like to convert the bam to csv and it would be easier to read the data
I would suggest converting BAM to BED and then reading BED file into R.
You can convert BAM to BED using bedtools.
This abstract code should work:
bamfile <- "C:/Users/azzop/Desktop/I16-1144-01-esd_m1_CGCTCATT-AGGCGAAG_tophat2/accepted_hits.bam"
# This code line sends command to convert BAM to BED (might take some time)
system(paste("bedtools bamtobed -i", bamfile, "> myBed.bed"))
library(data.table)
myData <- fread("myBed.bed")
Here I'm using function fread from a data.table package for a fast data read.
I want to stack and write some Landsat bands/tiff files to BIP interleave in ENVI format. However, the results always come as BSQ, even though I change the bandorder to BIP.
Below is my code:
library(raster)
library(rgdal)
library(gdalUtils)
inbands <-list.files(pattern= "*.tif")
stk<-stack(inbands[2], inbands[3], inbands[4])
writeRaster(stk, "BIP_test", format="ENVI", bandorder='BIP')
This also did not work
writeRaster(stk, "BIP_test", format="ENVI", options="INTERLEAVE=PIXEL", overwrite=TRUE)
Any assistance is appreciated.
I think this should work:
writeRaster(stk, "BIP_test", format="ENVI", options="INTERLEAVE=BIP", overwrite=TRUE)
per the GDAL formats page on the ENVI format, "BIP" (not "PIXEL") is the argument to "INTERLEAVE". Based on my reading of the WriteRaster helpfile, bandorder='BIP' only works for the raster package's native file format.
I am a relatively new R user. I am having problems extracting the same file from several HDF5 files. This is what I did using the rhdf5 package:
files <- list.files(pattern = ".he5",full.names=TRUE)
The file I can't extract from all the HDF5 Files has the extension
"/HDFEOS/SWATHS/ColumnAmountNO2/Data Fields/ColumnAmountNO2Trop"
When I run the command below for individual files, it works:
NO2he5<-h5read("MA.he5","/HDFEOS/SWATHS/ColumnAmountNO2/Data Fields/ColumnAmountNO2Trop")
How do I translate this to a function that can extract the layer simultaneously for multiple hdf5 files, please?
See lapply:
library(rhdf5)
files <- list.files(pattern = ".he5", full.names = TRUE)
attribute <- "/HDFEOS/SWATHS/ColumnAmountNO2/Data Fields/ColumnAmountNO2Trop"
out.list <- lapply(files, h5read, attribute)
This will store all the read objects into a single object: a list. It is the preferred approach versus creating as many objects as you have files.