Suddenly, over the weekend, my code is no longer working.
when I run it, I receive the following message:
Error in parse_con(txt, bigint_as_char) :
Failure when receiving data from the peer
the code is the following:
raiz <- "https://olinda.bcb.gov.br/olinda/servico/Expectativas/versao/v1/odata/"
tipo <- "ExpectativaMercadoMensais?%24format=json&%24select="
indicador <- "Indicador,Data,DataReferencia,Mediana,numeroRespondentes"
restricao <- "&%24orderby=Data%20desc&%24filter=Indicador%20eq%20'IPCA'&%24top=10"
library("jsonlite")
jsonlite::fromJSON(paste0(raiz,tipo,indicador,restricao), simplifyVector = FALSE)
There is a problem with the GET function that jsonlite uses to read the website. Use readLines instead.
raiz <- "https://olinda.bcb.gov.br/olinda/servico/Expectativas/versao/v1/odata/"
tipo <- "ExpectativaMercadoMensais?%24format=json&%24select="
indicador <- "Indicador,Data,DataReferencia,Mediana,numeroRespondentes"
restricao <- "&%24orderby=Data%20desc&%24filter=Indicador%20eq%20'IPCA'&%24top=10"
library("jsonlite")
web <- readLines(paste0(raiz,tipo,indicador,restricao), warn = FALSE)
df <- jsonlite::fromJSON(web, simplifyVector = FALSE)
I didn't understand your query, but here we have one that works:
web <- readLines("https://olinda.bcb.gov.br/olinda/servico/Expectativas/versao/v1/odata/ExpectativasMercadoInflacao12Meses?$format=json", warn = FALSE)
df <- fromJSON(web)
df$value
Related
:)
I m struggling with the package RD com client. I got an error:
(please check attached code)
library(RDCOMClient)
t = Sys.Date()-1
outlook_app <- COMCreate("Outlook.Application")
fx_date = paste0(substr(as.Date(t),9,10),"/",substr(as.Date(t),6,7),"/",substr(as.Date(t),1,4))
search <- outlook_app$AdvancedSearch("Inbox", paste0("urn:schemas:httpmail:subject = ","'string",
paste0(fx_date, "'")))
results <- search$Results()
Sys.sleep(15)
email <- results$Item(1)
attachment_file <- tempfile()
email$Attachments(1)$SaveAsFile(attachment_file)
fx_rates <- read.csv(attachment_file)
Error: Exception occurred. 80020009 No support for
InterfaceSupportsErrorInfo checkErrorInfo -2147352567
Please can you try run attached code and let me know if you obtain the same output error?
Thanks a lot
I want to run an R code on Google Colaboratory to generate hsa.txt file.
First cell:
%load_ext rpy2.ipython
Second cell:
%%R
install.packages("BiocManager")
BiocManager::install("KEGGREST")
BiocManager::install("EnrichmentBrowser")
library(KEGGREST)
library(EnrichmentBrowser)
library(org.Hs.eg.db)
MRSA252 <- keggList("hsa")
sarpathway <- downloadPathways("hsa")
hsa <- getGenesets(org="hsa", db="kegg", cache=TRUE, return.type="list")
for (i in 1:length(hsa)) {
t <- unlist(mget(hsa[[i]], envir=org.Hs.egSYMBOL, ifnotfound=NA),use.names=FALSE)
hsa[[i]] <- t[!is.na(t)]
# Remove disease-specific pathways
hsa[[i]] <- hsa[!grep("disease$", hsa)]
}
writeGMT(hsa, gmt.file="hsa.txt")
Traceback:
RInterpreterError: Failed to parse and evaluate line '\ninstall.packages("BiocManager")\nBiocManager::install("KEGGREST")\nBiocManager::install("EnrichmentBrowser")\n\nlibrary(KEGGREST)\nlibrary(EnrichmentBrowser)\nlibrary(org.Hs.eg.db)\n\nMRSA252 <- keggList("hsa")\nsarpathway <- downloadPathways("hsa")\n\n\n\nhsa <- getGenesets(org="hsa", db="kegg", cache=TRUE, return.type="list")\nfor (i in 1:length(hsa)) {\n t <- unlist(mget(hsa[[i]], envir=org.Hs.egSYMBOL, ifnotfound=NA),use.names=FALSE)\n hsa[[i]] <- t[!is.na(t)]\n # Remove disease-specific pathways\n hsa[[i]] <- hsa[!grep("disease$", hsa)]\n}\nwriteGMT(hsa, gmt.file="hsa.txt")'.
R error message: 'Error in .getUrl(url, .textParser) : Forbidden (HTTP 403).'
I am using the following code. I create a list of first names and then generate links to an API for each name and then try to capture the data from each link.
mydata$NameGenderURL2 <- paste ("https://gender-api.com/get?name=",mydata$firstname, "&key=suZrzhrNJRvrkWFXAG", sep="")
mynamegenderfunction <- function(x){
GET(url= mydata$NameGenderURL2[x])
this.raw.content <- genderdata$content
this.raw.content <- rawToChar(genderdata$content)
this.content <- fromJSON(this.raw.content)
name1[x] <- this.content$name
gender1[x] <- this.content$gender}
namelist <- mydata$firstname[1:100]
genderdata <- lapply(namelist, mynamegenderfunction)
Oddly enough I receive the following message:
>Error in curl::curl_fetch_memory(url, handle = handle) :
>Could not resolve host: NA`
I tried another API and got the same issue. Any suggestions?
Here is a data sample:
namesurl
https://api.genderize.io/?name=kaan
https://api.genderize.io/?name=Joan
https://api.genderize.io/?name=homeblitz
https://api.genderize.io/?name=Flatmax
https://api.genderize.io/?name=BRYAN
https://api.genderize.io/?name=James
https://api.genderize.io/?name=Dion
https://api.genderize.io/?name=Flintu
https://api.genderize.io/?name=Adriana
The output that I need is the gender for each link, which would be :Male/Female, Null
I am trying to use the below code to make API calls in a parallel process to speed up the API calls. (I know this isn't the best way to speed up API calls but it works)
It only fails when I try to use parallel, otherwise it works. In the ldply function I am getting the below error:
Error in do.ply(i) :
task 1 failed - "object of type 'closure' is not subsettable"
In addition:
Warning messages:
1: : ... may be used in an incorrect context: ‘.fun(piece, ...)’
2: : ... may be used in an incorrect context: ‘.fun(piece, ...)’
any help would be appreciated!
One <- 26
cl<-makeCluster(4)
registerDoSNOW(cl)
func.time <- Sys.time()
## API CALL ONE FOR "kline"
url <- "https://api.binance.com"
path <- paste("/api/v1/klines?symbol=",pairs[1],"&interval=1m&limit=1", sep = "")
raw.results <- GET(url = url, path = path)
text_content <- content(raw.results, as = "text", encoding = "UTF-8")
kline <- data.frame(text_content %>% fromJSON())
kline$symbol <- pairs[1]
## API FUNCTION TO BE APPLIED FOR REST
loopfunction <- function(i){
url <- "https://api.binance.com"
path <- paste("/api/v1/klines?symbol=",pairs[i],"&interval=1m&limit=1", sep = "")
raw.results <- GET(url = url, path = path)
text_content <- content(raw.results, as = "text", encoding = "UTF-8")
kline_temp <- data.frame(text_content %>% fromJSON())
kline_temp$symbol <- pairs[i]
kline <- rbind(kline,kline_temp)
return(kline)
}
## DPLY PARALLEL FUNCTION
kline2 <- data.frame(ldply(2:(One - 1), .fun = loopfunction, .parallel = T, .paropts = c("httr", "jsonlite", "dplyr"))) ##"ONE" is a list varriable created earlier
stopCluster(cl)
func.end.time <- Sys.time()
func.tot.time <- func.end.time - func.time
Your question isn't fully reproducible, so the following is an educated guess.
Your loopfunction() references an object called pairs. It seems from your script that a variable called pairs is defined somewhere in your local environment. However, when loopfunction() is passed to ldply(), it no longer has access to that variable (ordinarily, it would, but parallelization requires fresh R environments to be created). Having failed to find an object called pairs in the environment, R continues searching, and finds a match in stats::pairs(). This is a plotting function, not a subsettable object like a vector or data frame. Hence the error message, "object of type 'closure' is not subsettable".
I'm not especially familiar with how ldply implements parallel processing, but you could probably modify your function definition like this:
loopfunction <- function(i, pairs) {
...[body of function]...
}
And pass pairs as an extra parameter in your ldply call:
kline2 <- data.frame(ldply(2:(One - 1), .fun = loopfunction, pairs = pairs, .parallel = T, .paropts = list(.packages = c("httr", "jsonlite", "dplyr"))))
I am trying to obtain the number of results obtained from specific google searches.
For example for stackoverflow there are "About 28,200,000 results (0.12 seconds)".
Normally I would use the xpathSApply function from the XML R package but I am having errors and am not sure how to solve them or know if there is an alternative approach
library(XML)
googleURL <- "https://www.google.ca/search?q=stackoverflow"
googleInfo <- htmlParse(googleURL, isURL = TRUE)
Error: failed to load external entity "https://www.google.ca/search?q=stackoverflow"
#use of RCurl which I am not that familiar with
library(RCurl)
getURL(googleURL)
#Error in function (type, msg, asError = TRUE) :
#SSL certificate problem, verify that the CA cert is OK. Details:
#error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed
# final effort
library(httr)
x <- GET(googleURL)
# no error but am not sure how to proceed
# the relevant HTML code to parse is
# <div id=resultStats>About 28,200,000 results<nobr> (0.12 seconds) </nobr></div>
Ay help in solving errors or parsing the httr object would be much appreciated
You are asking for a secure http connection
https://www.google.ca/search?q=stackoverflow
XML is complaining about this as is RCurl. httr will download the page.
XML ask for an unsecured connection
library(XML)
googleURL <- "http://www.google.ca/search?q=stackoverflow"
googleInfo <- htmlParse(googleURL, isURL = TRUE)
xpathSApply(googleInfo,'//*/div[#id="resultStats"]')
#[[1]]
#<div id="resultStats">About 28,200,000 results</div>
RCurl use ssl.verifypeer = FALSE thou it worked without for me
library(RCurl)
googleURL <- "https://www.google.ca/search?q=stackoverflow"
googleInfo <- getURL(googleURL,ssl.verifypeer = FALSE)
googleInfo <- htmlParse(googleInfo)
# or if you want to use a cert
# system.file("CurlSSL/cacert.pem", package = "RCurl")
# googleInfo <- getURL(googleURL, cainfo = cert)
# googleInfo <- htmlParse(googleInfo)
xpathSApply(googleInfo,'//*/div[#id="resultStats"]')
#[[1]]
#<div id="resultStats">About 28,200,000 results</div>
httr use content
library(httr)
x <- GET(googleURL)
googleInfo <- htmlParse(content(x, as = 'text'))
xpathSApply(googleInfo,'//*/div[#id="resultStats"]')
#[[1]]
#<div id="resultStats">About 28,200,000 results</div>