I was trying to convert a dataset from netcdf to csv format using R.
although I have installed 'raster and 'netcdf4' in R.
but still it doesn't find nc.brick.
rm(list=ls())
library(raster)
library(ncdf4)
nc.brick <- brick(file.choose)
dim(nc.brick)
nc.df <- as.data.frame(nc.brick[[1]],xy=T)
head(nc.df)
write.csv(nc.df,file.choose())
test <- read.csv(file.choose())
Can you see the difference between your usage of file.choose in these two lines:
nc.brick <- brick(file.choose)
write.csv(nc.df,file.choose())
there's your problem.
Related
I wrote a function to extract values from ncdf files, as shown below:
precresults <- function(x){
library(magrittr)
library(ncdf4)
library(raster)
library(ncdf.tools)
##library(ncf)
re1 <- brick(nl1a[x])
re <- extract(re1,zuobiao)
####zuobiao stands for the coordinate of the targeted sites
###extract(zuobiao)
return(re)
}
###precresults(20)
precresults11 <- lapply(1:420, precresults)
"lapply" function is used to extract values for multiple site.
However, an error occurs:
Error in UseMethod("extract_") : no applicable method for
'extract_' applied to an object of class "c('RasterBrick', 'Raster',
'RasterStackBrick', 'BasicRaster')"
How can we solve such error?
I think Roman is correct, magrittr::extract is masking raster::extract. In the example provided you do not use magrittr, but you may use it elsewhere, so use raster::extract rather then extract. Your function could be rewritten as:
library(raster)
precresults <- function(x){
re1 <- brick(nl1a[x])
raster::extract(re1, zuobiao)
}
precresults11 <- lapply(1:420, precresults)
Or like this:
library(raster)
x <- matrix(nrow=length(zuobiao), ncol=420)
for (i in 1:420) {
re1 <- brick(nl1a[i])
x[,i] <- raster::extract(re1, zuobiao)
}
In R, how can I export a khrud object from function kernelUD in package adehabitat to a raster file (geoTiff)?
I tried following this thread (R: how to create raster layer from an estUDm object) using the code here:
writeRaster(raster(as(udbis1,"SpatialPixelsDataFrame")), "udbis1.tif")
where udbis1 is a khrud object, but I get "Error in as(udbis1, "SpatialPixelsDataFrame") : no method or default for coercing “khrud” to “SpatialPixelsDataFrame."
I think the issue may be that the old thread was before an update to the adehabitat package changed the data format from estUD to khrud. Maybe?
You do not provide a reproducible example. The following works for me:
library(adehabitatHR)
library(raster)
data(puechabonsp)
loc <- puechabonsp$relocs
ud <- kernelUD(loc[, 1])
r <- raster(as(ud[[1]], "SpatialPixelsDataFrame"))
writeRaster(r, filename = file.path(tempdir(), "ud1.tif"))
AdehabitatHR solutions work well for data that are in the required format or when using multiple animals. Though when wanting to create KDE with data organized differently or for only one source, it can be frustrating. For some reason, #johaness' answer doesn't work for my case so here is an alternative solution that avoids the headaches of going into adehabitatHR's innards.
library(adehabitatHR)
library(raster)
# Recreating an example for only one animal
# with a basic xy dataset like one would get from tracking
loc<-puechabonsp$relocs
loc<-as.data.frame(loc)
loc<-loc[loc$Name=="Brock",]
coordinates(loc)<-~X+Y
ud<-kernelUD(loc)
# Extract the UD values and coordinates into a data frame
udval<-data.frame("value" = ud$ud, "lon" = ud#coords[,1], "lat" = ud#coords[,2])
coordinates(udval)<-~lon+lat
# coerce to SpatialPixelsDataFrame
gridded(udval) <- TRUE
# coerce to raster
udr <- raster(udval)
plot(udr)
I am new to R and NetCDF files. I am trying to open data on surface sea temperatures that are in a .nc file from here. My code is as follows:
rm(list=ls())
#install.packages(c("netcdf", "chron", "RColorBrewer", "lattice"))
library(chron)
library(RColorBrewer)
library(lattice)
library(ncdf4)
# set path and filename
ncpath <- "C:/Users/Mihir Sharma/Dropbox/data/"
ncname <- "20140101023458-NCEI-L3C_GHRSST-SSTskin-AVHRR_Pathfinder-PFV5.3_NOAA19_G_2014001_night-v02.0-fv01.0"
ncfname <- paste(ncpath, ncname, ".nc", sep="")
dname <- "tmp" # note: tmp means temperature (not temporary)
# open a NetCDF file
ncin <- nc_open(ncfname)
But I am getting the following error:
Error in nc_open(ncfname) :
Error in nc_open trying to open file C:/Users/Mihir Sharma/Dropbox/1 EPIC/MPA/data/20140101023458-NCEI-L3C_GHRSST-SSTskin-AVHRR_Pathfinder-PFV5.3_NOAA19_G_2014001_night-v02.0-fv01.0.nc
I have followed the code from here and here. what am I doing wrong?
Many thanks,
Mihir
Path problems when using someone else's code:
# set path and filename
ncpath <- "C:/Users/Mihir Sharma/Dropbox/data/"
When you're following code on a blog or tutorial, if it's written well, they'll use a platform independent way to describe, but often they won't.
The platform independent way to write a path is:
file.path("Users", "Mihir Sharma", "Dropbox", "data"). This will always use the correct file separator for your Platform, a value stored in .Platform$file.sep
I'm trying to read MODIS 17 data files into R, manipulate them (cropping etc.) and then save them as geoTIFF's. The data files come in .hdf format and there doesn't seem to be an easy way to read them into R.
Compared to other topics there isn't a lot of advice out there and most of it is several years old. Some of it also advises using additional programmes but I want to stick with just using R.
What package/s do people use for dealing with .hdf files in R?
Ok, so my MODIS hdf files were hdf4 rather than hdf5 format. It was surprisingly difficult to discover this, MODIS don't mention it on their website but there are a few hints in various blogs and stack exchange posts. In the end I had to download HDFView to find out for sure.
R doesn't do hdf4 files and pretty much all the packages (like rgdal) only support hdf5 files. There are a few posts about downloading drivers and compiling rgdal from source but it all seemed rather complicated and the posts were for MAC or Unix and I'm using Windows.
Basically gdal_translate from the gdalUtils package is the saving grace for anyone who wants to use hdf4 files in R. It converts hdf4 files into geoTIFFs without reading them into R. This means that you can't manipulate them at all e.g. by cropping them, so its worth getting the smallest tiles you can (for MODIS data through something like Reverb) to minimise computing time.
Here's and example of the code:
library(gdalUtils)
# Provides detailed data on hdf4 files but takes ages
gdalinfo("MOD17A3H.A2000001.h21v09.006.2015141183401.hdf")
# Tells me what subdatasets are within my hdf4 MODIS files and makes them into a list
sds <- get_subdatasets("MOD17A3H.A2000001.h21v09.006.2015141183401.hdf")
sds
[1] "HDF4_EOS:EOS_GRID:MOD17A3H.A2000001.h21v09.006.2015141183401.hdf:MOD_Grid_MOD17A3H:Npp_500m"
[2] "HDF4_EOS:EOS_GRID:MOD17A3H.A2000001.h21v09.006.2015141183401.hdf:MOD_Grid_MOD17A3H:Npp_QC_500m"
# I'm only interested in the first subdataset and I can use gdal_translate to convert it to a .tif
gdal_translate(sds[1], dst_dataset = "NPP2000.tif")
# Load and plot the new .tif
rast <- raster("NPP2000.tif")
plot(rast)
# If you have lots of files then you can make a loop to do all this for you
files <- dir(pattern = ".hdf")
files
[1] "MOD17A3H.A2000001.h21v09.006.2015141183401.hdf" "MOD17A3H.A2001001.h21v09.006.2015148124025.hdf"
[3] "MOD17A3H.A2002001.h21v09.006.2015153182349.hdf" "MOD17A3H.A2003001.h21v09.006.2015166203852.hdf"
[5] "MOD17A3H.A2004001.h21v09.006.2015099031743.hdf" "MOD17A3H.A2005001.h21v09.006.2015113012334.hdf"
[7] "MOD17A3H.A2006001.h21v09.006.2015125163852.hdf" "MOD17A3H.A2007001.h21v09.006.2015169164508.hdf"
[9] "MOD17A3H.A2008001.h21v09.006.2015186104744.hdf" "MOD17A3H.A2009001.h21v09.006.2015198113503.hdf"
[11] "MOD17A3H.A2010001.h21v09.006.2015216071137.hdf" "MOD17A3H.A2011001.h21v09.006.2015230092603.hdf"
[13] "MOD17A3H.A2012001.h21v09.006.2015254070417.hdf" "MOD17A3H.A2013001.h21v09.006.2015272075433.hdf"
[15] "MOD17A3H.A2014001.h21v09.006.2015295062210.hdf"
filename <- substr(files,11,14)
filename <- paste0("NPP", filename, ".tif")
filename
[1] "NPP2000.tif" "NPP2001.tif" "NPP2002.tif" "NPP2003.tif" "NPP2004.tif" "NPP2005.tif" "NPP2006.tif" "NPP2007.tif" "NPP2008.tif"
[10] "NPP2009.tif" "NPP2010.tif" "NPP2011.tif" "NPP2012.tif" "NPP2013.tif" "NPP2014.tif"
i <- 1
for (i in 1:15){
sds <- get_subdatasets(files[i])
gdal_translate(sds[1], dst_dataset = filename[i])
}
Now you can read your .tif files into R using, for example, raster from the raster package and work as normal. I've checked the resulting files against a few I converted manually using QGIS and they match so I'm confident the code is doing what I think it is. Thanks to Loïc Dutrieux and this for the help!
These days you can use the terra package with HDF files
Either get sub-datasets
library(terra)
s <- sds("file.hdf")
s
That can be extracted as SpatRasters like this
s[1]
Or create a SpatRaster of all subdatasets like this
r <- rast("file.hdf")
The following worked for me. It's a short program and just takes in the input folder name. Make sure you know which sub data you want. I was interested in sub data 1.
library(raster)
library(gdalUtils)
inpath <- "E:/aster200102/ast_200102"
setwd(inpath)
filenames <- list.files(,pattern=".hdf$",full.names = FALSE)
for (filename in filenames)
{
sds <- get_subdatasets(filename)
gdal_translate(sds[1], dst_dataset=paste0(substr(filename, 1, nchar(filename)-4) ,".tif"))
}
Use the HEG toolkit provided by NASA to convert your hdf file to geotiff and then use any package ("raster" for example) to read the file. I do the same for both old and new hdf files.
Heres the link: https://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGHome.html
Take a look at the NASA products supported here: https://newsroom.gsfc.nasa.gov/sdptoolkit/HEG/HEGProductList.html
Hope this helps.
This script has been very useful and I managed to convert a batch of 36 files using it. However, my problem is that the conversion does not seem correct. When I do it using ArcGIS 'Make NetCDF Raster Layer tool', I get different results + I am able to convert the numbers to C from Kelvin using simple formula: RasterValue * 0.02 - 273.15. With the results from R conversion I don't get the right results after conversion which leads me to believe ArcGIS conversion is good, and R conversion returns an error.
library(gdalUtils)
library(raster)
setwd("D:/Data/Climate/MODIS")
# Get a list of sds names
sds <- get_subdatasets('MOD11C3.A2009001.006.2016006051904.hdf')
# Isolate the name of the first sds
name <- sds[1]
filename <- 'Rasterinr.tif'
gdal_translate(sds[1], dst_dataset = filename)
# Load the Geotiff created into R
r <- raster(filename)
# Identify files to read:
rlist=list.files(getwd(), pattern="hdf$", full.names=FALSE)
# Substract last 5 digits from MODIS filename for use in a new .img filename
substrRight <- function(x, n){
substr(x, nchar(x)-n+1, nchar(x))
}
filenames0 <- substrRight(rlist,9)
# Suffixes for MODIS files for identyfication:
filenamessuffix <- substr(filenames0,1,5)
listofnewnames <- c("2009.01.MODIS_","2009.02.MODIS_","2009.03.MODIS_","2009.04.MODIS_","2009.05.MODIS_",
"2009.06.MODIS_","2009.07.MODIS_","2009.08.MODIS_","2009.09.MODIS_","2009.10.MODIS_",
"2009.11.MODIS_","2009.12.MODIS_",
"2010.01.MODIS_","2010.02.MODIS_","2010.03.MODIS_","2010.04.MODIS_","2010.05.MODIS_",
"2010.06.MODIS_","2010.07.MODIS_","2010.08.MODIS_","2010.09.MODIS_","2010.10.MODIS_",
"2010.11.MODIS_","2010.12.MODIS_",
"2011.01.MODIS_","2011.02.MODIS_","2011.03.MODIS_","2011.04.MODIS_","2011.05.MODIS_",
"2011.06.MODIS_","2011.07.MODIS_","2011.08.MODIS_","2011.09.MODIS_","2011.10.MODIS_",
"2011.11.MODIS_","2011.12.MODIS_")
# Final new names for converted files:
newnames <- vector()
for (i in 1:length(listofnewnames)) {
newnames[i] <- paste0(listofnewnames[i],filenamessuffix[i],".img")
}
# Loop converting files to raster from NetCDF
for (i in 1:length(rlist)) {
sds <- get_subdatasets(rlist[i])
gdal_translate(sds[1], dst_dataset = newnames[i])
}
I am trying to perform a raster stack using three spatial raster that are in .grd format, in R using the dismo package.The three grid files are in the path "D:~" . I am using this code,
files= list.files(path = "D:~",pattern = 'grd',full.names = TRUE)
the error output am getting is
Error in x[[1]] : subscript out of bounds
Can anyone help? thanks in advance
If the files are in sub-directories then you can use recursive=TRUE:
files <- list.files(path="D:", pattern="grd", all.files=FALSE, full.names=TRUE,recursive=TRUE)
s <- stack(files)
Simply use the stack function from the raster package:
files=c("layer1.grd", "layer2.grd", "layer3.grd")
library(raster)
raster_stack=stack(files)