I am working with a couple of R packages for genetic pathway enrichment analyses and the two packages that I am using are now throwing errors when trying to connect to each package's respective server for downloading the reference data for the analysis.
In the first package gage, I am getting the following error when attempting to download:
library(gage)
> kg.ko = kegg.gsets("ko") # ("ko" is KEGG ortholog pathway)
Error in curl::curl_fetch_memory(url, handle = handle) :
Failure when receiving data from the peer
In the second package clusterProfiler, I am getting the following error:
library(clusterProfiler)
# the data
dput(head(de_kegg_chr))
c("K14847", "K19009", "K00078", "K21407", "K23285", "K06972")
# KEGG enrichment (which will pull relevant reference data during this step)
# over-representation analysis (fisher's)
> enrich <- enrichKEGG(gene = de_kegg_chr,
+ organism = "ko",
+ keyType='kegg',
+ pvalueCutoff = 0.01)
Reading KEGG annotation online:
fail to download KEGG data...
Error in download.KEGG.Path(species) :
'species' should be one of organisms listed in 'http://www.genome.jp/kegg/catalog/org_list.html'...
In addition: Warning message:
In utils::download.file(url, quiet = TRUE, method = method, ...) :
URL 'https://rest.kegg.jp/link/ko/pathway': status was 'Failure when receiving data from the peer'
After the first error, I thought it was something specific to the gage package and found a simple work-around because these data are downloaded from the server prior to the analysis function.
This is more of a problem with the second package because the reference data are downloaded within the function that conducts the analysis.
Now that this is happening with more than one package (both of these scripts were working perfectly before yesterday), I'm thinking it is something systematic within R or R studio.
Related
I am unable to download a large SSURGO dataset using the FedData package in R. I get the same error using the example code from the package:
vepPolygon <- polygon_from_extent(raster::extent(672800,740000,4102000,4170000),
proj4string='+proj=utm +datum=NAD83 +zone=12')
# Get the NRCS SSURGO data (USA ONLY)
SSURGO.VEPIIN <- get_ssurgo(template=vepPolygon, label='VEPIIN')
produces the error:
Error in { :
task 1 failed - "Download of https://sdmdataaccess.nrcs.usda.gov/Spatial/SDMNAD83Geographic.wfs?Service=WFS&Version=1.0.0&Request=GetFeature&Typename=SurveyAreaPoly&BBOX=-109.056777043215,37.033556543269,-108.279725827617,37.6609211495406 failed!"
It does work if I use extremely small templates, like 20'x20', but my understaning is that this package is supposed to enable larger downloads. Has something changed with the web soil survey since this package was created?
I'm trying to get TRMM data from NASA OPenDAP server using the raster package in R. Initially I had some difficulty regarding authentication, but that issue was resolved.
NASA OPenDAP server publishes TRMM 3B42_daily data as individual files, one for each day and an aggregated annual data (using ncml). So, my problem now is that, using R raster package and the authentication files .dodsrc and .netrc I can download individual NetCDF files but I can't download the aggregated data.
So, this works:
library(raster)
single_date_opendap <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/TRMM_L3/TRMM_3B42_Daily.7/2002/04/3B42_Daily.20020405.7.nc4'
test <- stack(single_date_opendap, varname = 'precipitation')
This doesn't:
library(raster)
url_opendap_no_brkt <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml'
test <- stack(url_opendap_no_brkt, varname = 'precipitation')
And gives me the error message:
Error in .local(.Object, ...) :
An error occurred while creating a virtual connection to the DAP server:
Error while reading the URL: https://disc2.gesdisc.eosdis.nasa.gov:443/openda
p/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml.
ver.
The OPeNDAP server returned the following message:
Unauthorized: Contact the server administrator.
Error in .rasterObjectFromFile(x, band = band, objecttype = "RasterLayer",
Cannot create a RasterLayer object from this file. (file does not exist)
Is it possible to get data from a OPenDAP server that publishes aggregated data?
After some exchange with NASA support and with Antonio's tip, found out that R raster package will not work with the aggregated datasets. But ncdf4::nc_open is able to handle it. Strange because, from what I understand, raster package calls nc_open in the background.
Anyway, this works:
library(ncdf4)
url_opendap <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml'
trmm <- nc_open(url_opendap)
and this doesn't
library(raster)
url_opendap <- 'https://disc2.gesdisc.eosdis.nasa.gov:443/opendap/ncml/aggregation/TRMM_3B42_Daily.7/TRMM_3B42_daily.7_Aggregation_2001.ncml'
trmm <- stack(url_opendap, varname = "precipitation")
I'm trying to download financial statements in R using a package at:
Financial statements in R
I'm trying to modify the example in their read me for other companies. I have tried to download the last two Tesla Q's.
The code I modified so far is:
xbrl_url2017Q3 <- "https://www.sec.gov/Archives/edgar/data/1318605/000156459018026353/tsla-20180930.xml"
xbrl_url2017Q2 <- "https://www.sec.gov/Archives/edgar/data/1318605/000156459018019254/tsla-20180630.xml"
old_o <- options(stringsAsFactors = FALSE)
xbrl_data_tsla2017Q3 <- xbrlDoAll(xbrl_url2017Q3)
Error from the line above is:
Error in fileFromCache(file) :
Error in download.file(file, cached.file, quiet = !verbose) :
cannot open URL 'https://www.sec.gov/Archives/edgar/data/1318605/000156459018026353/https://xbrl.sec.gov/dei/2018/dei-2018-01-31.xsd'
In addition: Warning message:
In download.file(file, cached.file, quiet = !verbose) :
cannot open URL 'https://www.sec.gov/Archives/edgar/data/1318605/000156459018026353/https://xbrl.sec.gov/dei/2018/dei-2018-01-31.xsd': HTTP status was '403 Forbidden'
xbrl_data_tsla2017Q2 <- xbrlDoAll(xbrl_url2017Q2)
options(old_o)
tsla2017Q3 <- xbrl_get_statements(xbrl_data_tsla2017Q3)
tsla2017Q2 <- xbrl_get_statements(xbrl_data_tsla2017Q2 )
tsla2017Q2
balance_sheet2017Q2 <- tsla2017Q2$StatementOfFinancialPositionClassified
balance_sheet2017Q3<- tsla2017Q3$StatementOfFinancialPositionClassified
income2017Q2 <- tsla2017Q2$StatementOfIncome
income2017Q3 <- tsla2017Q3$StatementOfIncome
balance_sheet2017Q3
Returns "NULL"
See the 10-Q at tesla's SEC fillings.
The last 10-Q.
Any recommendations on how I can go about this?
I'm looking to download the financial data to play around it with and would like it in tidy formate.
This is a common problem with the XBRL package where not all XML schemas are downloaded in the cache for some SEC filings. Download the missing schema in your cache folder and retry the xbrlDoAll call - it should work this time.
I'm using the Rglimclim package on RGui. When I run the command:
sim.summary <-
summary(sim,season.defs=seasons,thresholds=0,which.regions=0)
I received the following warning:
Error in file(FILENAME, "a") : cannot open the connection
In addition: Warning message:
In file(FILENAME, "a") :
cannot open file 'C:\Users\ÎÄÐù\AppData\Local\Temp\RtmpKyOJZM\Rfwf.2a2c50a61c18': No such file or directory
I haven't been using R for quite a long period of time, and this really bugs me.
Rglimclim is a weather generator based on generalised linear model. I used this package to simulate 100 daily rainfall sequence that is stored in 100 files under ./SimFiles.The command I used to generate the sequence is:
sim <- GLCsim(list(Occurrence=Model7.fitted,Intensity=Intensity.fitted),
nsims=100,start=198001,end=198912,impute.until=197912,
simdir="./SimFiles",file.prefix="SimDemo")
I want to plot these data using the command summary()and the problem occurs.
I refered to the link given below for doing sentiment analysis
http://heuristically.wordpress.com/2011/04/08/text-data-mining-twitter-r/
And when I ran the code that is given below :`for (page in c(1:15)){
# search parameter
twitter_q <- URLencode('#prolife OR #prochoice')
twitter_url =
# fetch remote URL and parse
mydata.xml <- xmlParseDoc(twitter_url, asText=F)
# extract the titles
mydata.vector <- xpathSApply(mydata.xml, '//s:entry/s:title', xmlValue, namespaces =c('s'='http://www.w3.org/2005/Atom'))
# aggregate new tweets with previous tweets
mydata.vectors <- c(mydata.vector, mydata.vectors)
}
After running the code it is prompting me for an error
Error:Error in UseMethod("xpathApply") :
no applicable method for 'xpathApply' applied to an object of class "NULL"
I/O warning : failed to load HTTP resource
I installed the packages Roath,stringr,XML,plyr which was required.And I am using R Ver 3.0.3
Kindly help me out pleas how to go about it . I am struggling for this . It would be a great help if anyone guides me properly in right direction.