I'm trying to visualize monthly averages of RegCM output I joined using CDO but I'm not being able to do it.
In order to do that I was trying to find a way to plot de monthly averages of my variable "pr" as you could do using GrADS.
I found that a way to do this was using the brick function and the raster library. So I was trying to use the code suggested in another question to convert my netcdf file into a raster brick:
NetCDF to Raster Brick "Unable to find inherited method for function 'brick' for 'ncdf4'"
# load package
library(sp)
library(raster)
library(ncdf4)
# read ncdf file
nc<-nc_open('dat.nc')
# extract variable name, size and dimension
v <- nc$var[[1]]
size <- v$varsize
dims <- v$ndims
nt <- size[dims] # length of time dimension
lat <- nc$dim$xlat$vals # latitude position
lon <- nc$dim$xlong$vals # longitude position
# read pr variable
r<-list()
for (i in 1:nt) {
start <- rep(1,dims) # begin with start=(1,1,...,1)
start[dims] <- i # change to start=(1,1,...,i) to read timestep i
count <- size # begin with count=(nx,ny,...,nt), reads entire var
count[dims] <- 1 # change to count=(nx,ny,...,1) to read 1 tstep
dt<-ncvar_get(nc, varid = 'pr', start = start, count = count)
# convert to raster
r[i]<-raster(dt)
}
# create layer stack with time dimension
r<-stack(r)
# transpose the raster to have correct orientation
rt<-t(r)
extent(rt)<-extent(c(range(lon), range(lat)))
# plot the result
spplot(rt)
But once I tried to run the for loop in the code, I get the following error:
Error in ncvar_get_inner(ncid2use, varid2use, nc$var[[li]]$missval, addOffset, :
Error: variable has 3 dims, but start has 2 entries. They must match!
The file I'm trying to visualize can be found in the following link:
https://drive.google.com/file/d/13KsOpnt-Wk2v93WwGcOU6AHw8KGOFlai/view?usp=sharing
I would really appreciate any insights with this problem!
Related
I am trying to extract summed raster cell values from a single big file for various SpatialPolygonsDataFrames (SPDF) objects in R stored in a list, then add the extracted values to the SPDF objects attribute tables. I would like to iterate this process, and have no idea how to do so. I have found an efficient solution for multiple polygons stored in a single SPDF object (see: https://gis.stackexchange.com/questions/130522/increasing-speed-of-crop-mask-extract-raster-by-many-polygons-in-r), but do not know how to apply the crop>mask>extract procedure to a LIST of SPDF objects, each containing multiple polygons. Here is a reproducible example:
library(maptools) ## For wrld_simpl
library(raster)
## Example SpatialPolygonsDataFrame
data(wrld_simpl) #polygon of world countries
bound <- wrld_simpl[1:25,] #country subset 1
bound2 <- wrld_simpl[26:36,] #subset 2
## Example RasterLayer
c <- raster(nrow=2e3, ncol=2e3, crs=proj4string(wrld_simpl), xmn=-180,
xmx=180, ymn=-90, ymx=90)
c[] <- 1:length(c)
#plot, so you can see it
plot(c)
plot(bound, add=TRUE)
plot(bound2, add=TRUE, col=3)
#make list of two SPDF objects
boundl<-list()
boundl[[1]]<-bound1
boundl[[2]]<-bound2
#confirm creation of SPDF list
boundl
The following is what I would like to run for the entire list, in a forloop format. For a single SPDF from the list, the following series of functions seem to work:
clip1 <- crop(c, extent(boundl[[1]])) #crops the raster to the extent of the polygon, I do this first because it speeds the mask up
clip2 <- mask(clip1, boundl[[1]]) #crops the raster to the polygon boundary
extract_clip <- extract(clip2, boundl[[1]], fun=sum)
#add column + extracted raster values to polygon dataframe
boundl[[1]]#data["newcolumn"] = extract_clip
But when I try to isolate the first function for the SPDF list (raster::crop), it does not return a raster object:
crop1 <- crop(c, extent(boundl[[1]])) #correctly returns object class 'RasterLayer'
cropl <- lapply(boundl, crop, c, extent(boundl)) #incorrectly returns objects of class 'SpatialPolygonsDataFrame'
When I try to isolate the mask function for the SPDF list (raster::mask), it returns an error:
maskl <- lapply(boundl, mask, c)
#Error in (function (classes, fdef, mtable) : unable to find an inherited method for function ‘mask’ for signature ‘"SpatialPolygonsDataFrame", "RasterLayer"’
I would like to correct these errors, and efficiently iterate the entire procedure within a single loop (i.e., crop>mask>extract>add extracted values to SPDF attribute tables. I am really new to R and don't know where to go from here. Please help!
One approach is to take what is working and simply put the desired "crop -> mask -> extract -> add" into a for loop:
for(i in seq_along(boundl)) {
clip1 <- crop(c, extent(boundl[[i]]))
clip2 <- mask(clip1, boundl[[i]])
extract_clip <- extract(clip2, boundl[[i]], fun=sum)
boundl[[i]]#data["newcolumn"] <- extract_clip
}
One can speed-up the loop with parallel execution, e.g., with the R package foreach. Conversely, the speed gain of using lapply() instead of the for loop will be small.
Why the error occurs:
cropl <- lapply(boundl, crop, c, extent(boundl))
applies the function crop() to each element of the list boundl. The performed operation is
tmp <- crop(boundl[[1]], c)
## test if equal to first element
all.equal(cropl[[1]], tmp)
[1] TRUE
To get the desired result use
cropl <- lapply(boundl, function(x, c) crop(c, extent(x)), c=c)
## test if the first element is as expected
all.equal(cropl[[1]], crop(c, extent(boundl[[1]])))
[1] TRUE
Note:
Using c to denote an R object is a bade choice, because it can be easily confused with c().
Trying to convert NOAA Snow data (NetCDF) into Raster format in R. This data has been pre-processed by me in CDO (interpolated from weekly-daily).
library(raster)
library(ncdf4)
nc<-nc_open('NOAA_Snow_JanJune2016.nc')
# extract variable name, size and dimension
v <- nc$var[[1]]
size <- v$varsize
dims <- v$ndims
nt <- size[dims] # length of time dimension
lat <- nc$dim$latitude$vals # latitude position
lon <- nc$dim$longitude$vals # longitude position
# read sea ice variable
r<-list()
for (i in 1:nt) {
start <- rep(1,dims) # begin with start=(1,1,...,1)
start[dims] <- i # change to start=(1,1,...,i) to read timestep i
count <- size # begin with count=(nx,ny,...,nt), reads entire var
count[dims] <- 1 # change to count=(nx,ny,...,1) to read 1 tstep
dt<-ncvar_get(nc, varid = 'snow_cover_extent', start = start, count = count)
# convert to raster
r[i]<-raster(dt)
}
Returns the following error:
Error in ncvar_get_inner(ncid2use, varid2use, nc$var[[li]]$missval, addOffset, :
Error: variable has 3 dims, but start has 2 entries. They must match!
Has anyone else had and solved this problem? I wonder if prepping the file in CDO is causing the problem. The (.nc) data can be accessed here:
https://drive.google.com/file/d/0Bz0W7Ut_SNfjeE9ObXpySzJ5UWs/view?usp=sharing
Many thanks!
Use ncdf4 to explore your ncdf attributes
library(ncdf4)
nc<-nc_open('D:/NOAA_Snow_JanJune2016.nc')
print(nc)
varname<-names(nc$var)
Use raster to convert your ncdf to raster
library (raster)
r<-brick('D:/NOAA_Snow_JanJune2016.nc',varname='snow_cover_extent')
Here the plot
spplot(r[1])
I am new to R programming. I was trying to visualize some dataset. I was using Googlevis in R and was unable to visualize it.
The error I got was:
Error: Length of logical index vector must be 1 or 8, got: 14835
Can someone help?
Dataset is here:
https://www.kaggle.com/c/predict-west-nile-virus/data
Code is below
# Read competition data files:
library(readr)
data_dir <- "C:/Users/Wesley/Desktop/input"
train <- read_csv(file.path(data_dir, "train.csv"))
spray <- read_csv(file.path(data_dir, "spray.csv"))
# Generate output files with write_csv(), plot() or ggplot()
# Any files you write to the current directory get shown as outputs
# Install and read packages
library(lubridate)
library(googleVis)
# Create useful date columns
spray$Date <- as.Date(as.character(spray$Date),format="%Y-%m-%d")
spray$Week <- isoweek(spray$Date)
spray$Year <- year(spray$Date)
# Create a total count of measurements
spray$Total <- 1
for(i in 1:nrow(spray)) {
spray$Total[i] = i
}
# Aggregate data by Year, Week, Trap and order by old-new
spray_agg <- aggregate(cbind(Total)~Year+Week+Latitude+Longitude,data=spray,sum)
spray_agg <- spray[order(spray$Year,spray$Week),]
# Create a misc format for Week for Google Vis Motion Chart
spray_agg$Week_Format <- paste(spray_agg$Year,"W",spray_agg$Week,sep="")
# Function to create a motion chart together with a overview table
# It takes the aggregated data as input as well as a year of choice (2007,2009,2011,2013)
# It filters out "no presence" weeks since they distort the graphical view
# Next to that it creates an overview table of that year
# With gvisMerge you can merge the 3 html outputs into 1
create_motion <- function(data=spray_agg,year=2011){
data_motion <- data[data$Year==year]
motion <- gvisMotionChart(data=data_motion,idvar="Total",timevar="Week_Format",xvar="Longitude",yvar="Latitude"
,sizevar=0.1,colorvar="Blue",options=list(width="600"))
return(motion)
}
# Get the per year motion charts
#motion1 <- create_motion(spray_agg,2007)
#motion2 <- create_motion(spray_agg,2009)
motion3 <- create_motion(spray_agg,2011) : (Error: Length of logical index vector must be 1 or 8, got: 14835)
motion4 <- create_motion(spray_agg,2013) :(Error: Length of logical index vector must be 1 or 8, got: 14835)
# Merge them together into 1 dashboard
output <- gvisMerge(gvisMerge(motion1,motion2,horizontal=TRUE),gvisMerge(motion3,motion4,horizontal=TRUE),horizontal=FALSE)
plot(output)
# Plot the output in your browser
I also posted this question on stack gis 1. From the netcdf4 data that have sub categories, I want to be able to read "Retrieval/fs" variable. I also want to read them and convert to raster girds, but it seems that raster doesn't support netcdf4. I appreciate any suggestions.
library(ncdf4)
library(raster)
file <- "http://140906_B7101Ar_150909171225s.nc4"
names(file$var)
"latitude" ... "longitude"... "Retrieval/fs"
lat <- raster(file, varname="latitude")
lon <- raster(file, varname="longitude")
Error in (function (classes, fdef, mtable) :
unable to find an inherited method for function ‘raster’ for signature ‘"ncdf4"’
raster does work with ncdf4 You are now showing actual code. file is a character vector. You cannot do names(file$var) with that (at least you won't get "latitude" ... "longitude"... "Retrieval/fs". So file is probably ncdf4 object (see the error message), while the raster function expects a filename (but not a url).
If you download the file and then do
library(raster)
x <- brick(filename, var="Retrieval/fs")
Things should work if the ncdf file had regular raster data.
However, it does not so you cannot directly import this as a raster. Instead you can get the lat and lon and values from the files, treat these as points and then rasterize (interpolate) these to get a regular raster.
Here is the answer to the question I asked. Since the data is not gridded, I retrieve the lon and lat information along with the variables to create a dataframe.
fs <- ncvar_get(ncfile, "Retrieval/fs")
xlon <- ncvar_get(ncfile, "longitude")
xlat <- ncvar_get(ncfile, "latitude")
d <- data.frame( as.vector(xlon),as.vector(xlat), as.vector(fs))# create a dataframe
coordinates(d) <- c("xlon","xlat")
proj4string(d) <- CRS("+proj=longlat")
spoint <- SpatialPoints(coords = d) #create a spatial point object
I'm having real difficulty with exporting data from GrADS to a .csv file although it should be really easy. The file in question is from the APHRODITE project relating to rainfall over Asia. Basically I can read this file into GrADS using:
open d:/aphro/aphro.ctl
and it tells me that:
Data file d:/aphro/APHRO_MA_025deg_V1101R2.%y4 is open as file 1
Lon set to 60.125 149.875
Lat set to -14.875 54.875
Lev set to 1 1
Time values set: 1961:1:1:0 1961:1:1:0
E set to 1 1
If I execute:
q ctlinfo
it also tells me that I have three variables:
precip 1 0 daily precipitation analysis
rstn 1 0 ratio of 0.05 degree grids with station
flag 1 0 ratio of 0.05 degree grids with snow
Okay, now all I want to do is produce a list in a .csv file (or .txt) file with the following information:
Precipitation Lon Lat Time(date)
It sounds really easy but I just can't do it. One method is to use:
fprintf precip d:/output.csv %g 1
This gives me an .csv file with the entire data for that day in one long column (which is what I want). I can also do the same for lon and lat in different files and combine them. The problem is that this takes for ages for the output file - it is much faster if you don't mind lots of columns but this becomes a pain to manage. Basically, this method is too slow.
Another method is to export the data as a NetCDF file by:
Set sdfwrite -4d d:/output.nc
define var = precip
sdfwrite precip
This then very quickly writes a file called output.nc which contains all the data I need. Using R I can then read all the variables individually e.g.
f <- open.ncdf("D:/aphro/test.nc")
A <- get.var.ncdf(nc=f,varid="time")
B <- get.var.ncdf(nc=f,varid="rain")
D <- get.var.ncdf(nc=f,varid="lon")
E <- get.var.ncdf(nc=f,varid="lat")
But what I want is to make an output file where each row tells me the time, rain amount, lon and lat. I tried rbind but it doesn't associate the correct time(date) with the right rain amount, and similarly messes up the lon and lat as there are hundreds of thousand of rain data but only a few dates and only 360 lon points and 280 lat points (i.e. the rain data is a grid of data for each day over several days). I'm sure this should be easy but how to do it?
Please help
Tony
Up to my knowledge, you can change the GrAD file to NetCDF file by using climate data operator and R together. Details can be found here. Further a NetCDF file can be converted in to a .csv file. For this I am providing a dummy code.
library(ncdf)
nc <- open.ncdf("foo.nc") #open ncdf file and read variables
lon <- get.var.ncdf(nc, "lon") # Lon lat and Time
lat <- get.var.ncdf(nc, "lat")
time <- get.var.ncdf(nc, "time")
dname <- "t" # name of variable which can be found by using print(nc)
nlon <- dim(lon)
nlat<- dim(lat)
nt<- dim(time)
lonlat <- expand.grid(lon, lat) # make grid of given longitude and latitude
mintemp.array <- get.var.ncdf(nc, dname)
dlname <- att.get.ncdf(nc, dname, "long_name")
dunits <- att.get.ncdf(nc, dname, "units")
fillvalue <- att.get.ncdf(nc, dname, "_FillValue")
mintemp.vec.long <- as.vector(mintemp.array)
mintemp.mat <- matrix(mintemp.vec.long, nrow = nlon * nlat, ncol = nt)
mintemp.df <- data.frame(cbind(lonlat, mintemp.mat))
options(width = 110)
write.csv(mintemp.df, "mintemp_my.csv")
I hope, it explains your question.