tl;dr
First of all, all necessary code to reproduce the problem is available below. I'm experiencing some challenges with the grib part, but also with the more basic things with download.file()
Problem
There's a nice article on using GFS weather data in R here. It would be a great starting point for further analysis, but I'm having several problems. The article contains all R code required to reproduce it, but here are the first few lines which also is the most important part to get things going:
#STEP 1 (doesn't work either way because of bad link)
loc=file.path("ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2009121700/gfs.t00z.sfluxgrbf03.grib2")
My first problem was that the link didn't work, but here's the url to a file of the same format from the same source, so this should work:
#PART 1 with working link (at least in google chrome):
loc=file.path("ftp://ftp.ncep.noaa.gov/pub/data/nccf/com/gfs/prod/gfs.2015081112/gfs.t12z.sfluxgrbf00.grib2")
download.file(loc,"temp.grb",mode="wb")
#PART 2
shell("wgrib2 -s temp03.grb | grep :LAND: | wgrib2 -i temp00.grb -netcdf LAND.nc",intern=T)
#PART3
library(ncdf)
landFrac <-open.ncdf("LAND.nc")
land <- get.var.ncdf(landFrac,"LAND_surface")
x <- get.var.ncdf(landFrac,"longitude")
y <- get.var.ncdf(landFrac,"latitude")
#PART4
rgb.palette <- colorRampPalette(c("snow1","snow2","snow3","seagreen","orange","firebrick"), space = "rgb")#colors
image.plot(x,y,t2m.mean,col=rgb.palette(200),axes=F,main=as.expression(paste("GFS 24hr Average 2M Temperature",day,"00 UTC",sep="")),axes=F,legend.lab="o C")
contour(x,y,land,add=TRUE,lwd=1,levels=0.99,drawlabels=FALSE,col="grey30")
My system
Windows 7, 64 bit
Rstudio 0.99.467
Question 1 - download.file()
I'm having problems with download.file(), even with a link (or ftp path) that works fine in Chrome. Here's a part of the error message:
In download.file(url = loc, destfile = "temp.grb", mode = "wb") :
InternetOpenUrl failed:
Anyone know what causes this?
Question: 2 - wgrib2
I've downloaded a sample file manually to get things going, but get stuck again in Part 2. How would the following r command look like directly in a command window?
shell("wgrib2 -s temp03.grb | grep :LAND: | wgrib2 -i temp00.grb -netcdf LAND.nc",intern=T)
I've installed the necessary dlls available under the links below, and I've also got wgrib 2 up and running (apparently). Information on wgrib2 is provided in the article, and available here. Source for Windows 7.
I hope some of you find this interesting!
Related
I have long-read RNA sequencing data from PacBio IsoSeq and trying to use IsoPops software (https://kellycochran.github.io/IsoPops/site_files/walkthrough.html) to generate graphs to interpret the data. I am new to R-studio so have limited proficiency. There is a walkthrough for the program which seems quite intuitive. I have successsfully installed the package on R Studio, and have opened the library and assigned variables to the files I have (which include a fasta file, gff file and abundance files). Once I've successfully assigned variables to the files, I'm supposed to generate a rawdb using the function compile_raw_db. However, when I do this function I get the error:
Error in system(command = paste("head -n1 ", filename, step = ""), intern = T) :
'head' not found
I've used the head -n1 function in linux command line, and can see that there definitely is a header and contents in the file. The error occurs for any file that I give to R studio, so my reasoning is that the programme is not recognizing the files.
Thought the files might have been empty, opened on Ubuntu and head -n2 function worked fine for each file, so I don't think it's an issue with the files.
My data was originally stored on an online shared network drive, moved files to local computer to eliminate any possibility for path errors. Still have the error message.
Have tried the same function with a number of different files, and I always have the same error message. So I think the issue is with how R studio is reading the files, and not the files themselves.
Would really appreciate some support with this, at the end of my PhD trying to analyse the last piece of my data and really struggling with this. Thanks in advance for any feedback!
> library(IsoPops)
Welcome to IsoPops version 0.3.1.
> transcript_AD3 <- "C:/R_Package_for_IsoSeq/IsoPops/IsoPops_Data/AD3-hq_transcripts.fasta"
> abundance_AD3 <- "C:/R_Package_for_IsoSeq/IsoPops/IsoPops_Data/AD3_Collapsed_Filtered_Isoform_Counts.abundance.txt"
> GFF_AD3 <- "C:/R_Package_for_IsoSeq/IsoPops/IsoPops_Data/AD3-collapse_isoforms.gff"
> rawDB <- compile_raw_db(transcript_AD3, abundance_AD3, GFF_AD3)
[1] "Loading sequences..."
Error in system(command = paste("head -n1 ", filename, step = ""), intern = T) :
'head' not found
I am getting an error when I am trying to run the code below in R-Studio 3.3.2 on a Mac (OS Sierra)
devtools::install_github('ramnathv/rCharts')
install.packages("knitr")
require(rCharts)
require(knitr)
haireye <- as.data.frame(HairEyeColor)
n1 <- nPlot(Freq ~ Hair, group = 'Eye', type = 'multiBarChart',
data = subset(haireye, Sex == 'Male')
)
n1$save('fig/n1.html', cdn = TRUE)
cat('<iframe src="fig/n1.html" width= 100%, height=600</iframe>')
Pls see output below:
Error in file(con, "w") : cannot open the connection
In addition: Warning message: In file(con, "w") : cannot open file 'fig/n1.html': No such file or directory
But I am able to generate the reqd bar graph in the viewer when I use:
n1$show(cdn = TRUE)
in lieu of n1$save('fig/n1.html', cdn = TRUE)
To take care of write permission issues (if any), I also tried including the line below, altering the WD path wherever necessary.
knitr::knit2html('Users/documents/n1.html')
But it did not help. I see the n1.html file created but it only opens an empty browser.
Any help to resolve this is appreciated.
Best,
S
A lot of times we face this error due to caching in RStudio and in that case, actual code errors don't show up. Restart RStudio and this error will be gone and actual code errors would show.
You have two separate problems.
The connection error appears because the fig/ folder does not exist. Create the folder and the save command will work. R has functions to check the existance of directories and create new ones if you would like to do it in your code.
The second problem comes from the way you save, you should use n1$save('fig/n1.html', standalone = TRUE). Here you have a similar situation.
As a side-note, I would say rCharts is not currently developed or mantained at all, so I would recommend you to use another library for your charts. In my opinion Plotly is quite nice. rCharts brought the NVD3 project to R and the chart style is in my opinion really nice. However, as far as I know both projects are stopped so I would look for a library that is still alive.
I have fixed this problem with good old rm(list=ls()) . I know I have
fallen into sequences where the error stops execution of my script. I fix the error, and then it won't run. This is likely due to lazy evaluation but it is a near impossible problem to diagnose, so the solution at the top works almost all the time.
The ZevRoss blog is as follow:
http://zevross.com/blog/2014/04/11/using-r-to-quickly-create-an-interactive-online-map-using-the-leafletr-package/
The code with error is:
# ----- Write data to GeoJSON
leafdat<-paste(downloaddir, "/", filename, ".geojson", sep="")
writeOGR(subdat, leafdat, layer="", driver="GeoJSON")
And the error is:
Error in writeOGR(subdat, leafdat, layer = "", driver = "GeoJSON") :
GDAL Error 3: Cannot open file 'd:/Leaflet/County_2010Census_DP1.geojson'
Because I am a freshman in R, I searched for this problem a lot and didn't get any good answer.
I am using Rstudio R version 3.1.1(2014-07-10) on windows 7 32bit.
My rgdal version is 0.9-1.
The other code in the blog runs successfully, this sentence seems to be the only difficult point.
You could create GeoJSON using leafletR package:
library('leafletR')
Your_GeoJSON <- toGeoJSON(data=YourData, dest=getwd())
I've tried to find a solution for this mysterious error for some time.
Eventually I found this post on the Gdal package errors' tickets site that clarified the problem and gave a solution.
Basically the problem is in the interface between rgdal and Gdal (Gdal changed their way to work and the latest version of rgdal hasn't watched up yet):
writeOGR() calls ogrCheckExists("foo.geojson") to check first if the file exists before creating a new dataset.
In the 1.11 version the OGR GeoJSON driver will emit an error message that this file doesn't exists, whereas previous version didn't emit an error message.
rgdal catches this error as a fatal one and doesn't go to the writing step. This should be fixed in rgdal.
Meanwhile you have an easy workaround : add check_exists = FALSE as a parameter to writeOGR()
Therefore the following code will work:
writeOGR(spDf,'foo.geojson','spDf', driver='GeoJSON',check_exists = FALSE)
Of course if there is already a geojson file with the chosen name at the location writeOGR still fails.
Even though you already have a 'd:' drive on your computer and you have permission to write to that drive, try the following:
--------------------------------
leafdat<-paste(downloaddir, "/", ".geojson", sep="")
> leafdat
> "d:/Leaflet/.geojson"
writeOGR(subdat, leafdat, layer="", driver="GeoJSON")
--------------------------------
Then you may get ".geojson" file on "d:/Leaflet". Change the file name ".geojson" to "County_2010Census_DP1.geojson".
I am trying to run an R script from the Windows command prompt (the reason is that later on I would like to run the script by using VBA).
After having set up the R environment variable (see the end of the post), the following lines of code saved in R_code.R work perfectly:
library('xlsx')
x <- cbind(rnorm(10),rnorm(10))
write.xlsx(x, 'C:/Temp/output.xlsx')
(in order to run the script and get the resulting xlsx output, I simply type the following command in the Windows command prompt: Rscript C:\Temp\R_code.R).
Now, given that this "toy example" is working as expected, I tried to move to my main goal, which is indeed very similar (to run a simple R script from the command line), but for some reason I cannot succeed.
Again I have to use a specific R package (-copula-, used to sample some correlated random variables) and export the R output into a csv file.
The following script (R_code2.R) works perfectly in R:
library('copula')
par_1 <- list(mean=0, sd=1)
par_2 <- list(mean=0, sd=1)
myCop.norm <- ellipCopula(family='normal', dim=2, dispstr='un', param=c(0.2))
myMvd <- mvdc(myCop.norm,margins=c('norm','norm'),paramMargins=list(par_1,par_2))
x <- rMvdc(10, myMvd)
write.table(x, 'C:/Temp/R_output.csv', row.names=FALSE, col.names=FALSE, sep=',')
Unfortunately, when I try to run the same script from the command prompt as before (Rscript C:\Temp\R_code2.R) I get the following error:
Error in FUN(c("norm", "norm"))[[1L]], ...) :
cannot find the function "existsFunction"
Calls: mvdc -> mvdcCheckM -> mvd.has.marF -> vapply -> FUN
Do you have any idea idea on how to proceed to fix the problem?
Any help is highly appreciated, S.
Setting up the R environment variable (Windows)
For those of you that want to replicate the code, in order to set up the environment variable you have to:
Right click on Computer -> Properties -> Advanced System Settings -> Environment variables
Double click on 'PATH' and add ; followed by the path to your Rscript.exe folder. In my case it is ;C:\Program Files\R\R-3.1.1\bin\x64.
This is a tricky one that has bitten me before. According to the documentation (?Rscript),
Rscript omits the methods package as it takes about 60% of the startup time.
So your better solution IMHO is to add library(methods) near the top of your script.
For those interested, I solved the problem by simply typing the following in the command prompt:
R CMD BATCH C:\Temp\R_code2.R
It is still not clear to me why the previous command does not work. Anyway, once again searching into the R documentation (see here) proves to be an excellent choice!
When I want to create the map using the gvisGeoMap() from googleVis, I get error:
## Using the google visualization API with R
library(googleVis)
input<- read.csv("data.csv")
select<- input[which(input$Subgroup=="Total 5-14"),]
select<- input[which(input$Subgroup=="Total 5-14 yr"),]
Map<- data.frame(select$Country.or.Area, select$Value)
names(Map)<- c("Country", "Percentage")
Geo=gvisGeoMap(Map, locationvar="Country", numvar="Percentage",
options=list(height=350, dataMode='regions'))
plot(Geo)
#starting httpd help server ... done
#Error in ifelse(interactive(), getOption("browser"), "false") :
#replacement has length zero
The above is the error in the "RGui". The error message in "RStudio" is differet:
#object of type 'closure' is not subsettable
and the browser does not fire at all. The HTTP server works fine since I can simply call help pages.( for example ?googleVis will fire up the browser and give the help page). The "Geo" object in the code above is fine and contains the html code only that the plot() does not do what it is supposed to do (I can manually run the html file in the temp folder and see the results). The example above is available here.
I would appreciate your clues.
Thank you
This is the result of correspondance with the the Authors of the packages. It seems that there was a bug that prevented the plot to work properly. The released a new version. You can find the link below.
TQ
.....
Yesterday evening I realised that with version 0.3.0 of googleVis I unfortunately introduced a bug in RStudio and R on Windows.
The bug has been fixed already and a new version (0.3.1) is available from our project site (here is the link), but not on CRAN yet.
I have put a note on my blog (here is the link) to inform others as well.
I hope this helps.
Best regards
Markus