I am getting some data from a server in json or xml format , and the server is sending status code in header like 500,200,404 etc. So how can I get the error status code or the error and handle it in R. If I can get a simple sample code or any reference, it would work.
Or if there is some other way around that will also work.
If you are just looking to collect the status responses, you just need to inspect/parse what you get back.
library(httr)
GET("www.google.com")$status
# [1] 200
As a starting point for error handling... if you just want console warnings then you can use the function of the same name.
info_get <- GET("www.google.com")
if (info_get$status == 200) {
warning(paste0("Response ", info_get$status, " received from target."))
}
# Warning message:
# Response 200 received from target.
Related
I have just received an Academic Twitter Developer privileges and am attempting to scrape some tweets. I updated RStudio, regenerated a new bearer token once I got updated Academic Twitter access, and get_bearer() returns my new bearer token. However, I continue to get the following error:
Error in make_query(url = endpoint_url, params = params, bearer_token = bearer_token, :
something went wrong. Status code: 403
In addition: Warning messages:
1: Recommended to specify a data path in order to mitigate data loss when ingesting large amounts of data.
2: Tweets will not be stored as JSONs or as a .rds file and will only be available in local memory if assigned to an object.
Additionally, I have tried specifying a data path, but I think I am confused as to what these means? I think that's where my issue lies, but does the data path way mean like a specific file pathway on my computer?
Below is the code I was attempting to use.This code worked previously with my professor's bearer token that they used to just show the output:
`tweets <-
get_all_tweets(
query = "#BlackLivesMatter",
start_tweets = "2020-01-01T00:00:00Z",
end_tweets = "2020-01-05T00:00:00Z",
n = 100)`
Thanks in advance!
Status code 403 means forbidden. You may want to check the error codes reference page of Twitter API here.
Perhaps your bearer token is misspelled?
I have a code where I make requests for an API using the jsonlite package.
My request is:
aux <- fromJSON (www ... js)
The problem is that there is a time limit on requests and sometimes the error is returned:
*Error in open.connection (con, "rb"): HTTP error 429.*
I need that, when there is an error the code wait X seconds and make a new request and this is repeated until I get the requested data.
I found the try and tryCatch functions and the retry package. But I couldn't make it work as I need it.
Try this approach :
aux <- tryCatch(fromJSON (www ... js), error = function(e) {return(NA)})
while(all(is.na(aux))) {
Sys.sleep(30) #Change as per requirement.
aux <- tryCatch(fromJSON(www ... js), error = function(e) {return(NA)})
}
Whereas options(warn=2) will prompt an error and hence enable debugging, I'm struggling with doing the same for messages.
For example, somewhere in my codebase, an unknown function seems to use jsonlite-package, which triggers the following message.
So my question is: Is there a convenient way to trace back the origins of messages?
Note: Using browser() doesn't seem to help, since messages are not shown in browser-mode.
You can use wrap your code in a call to withCallingHandlers to turn messages into errors:
withCallingHandlers(
message("example message"),
message = function(m) stop(m)
)
#Error in message("example message") : example message
I try to send a request to API, use RCurl library.
My code:
start = "2018-07-30"
end = "2018-08-15"
api_request <- paste("https://api-metrika.yandex.ru/stat/v1/data.csv?id=34904255&date1=",
start,
"&date2=",
end,
"&dimensions=ym:s:searchEngine&metrics=ym:s:visits&dimensions=ym:s:<attribution>SearchPhrase&filters=ym:s:<attribution>SearchPhrase!~'some|phrases|here'&limit=100000&oauth_token=OAuth_token_here", sep="")
s <- getURL(api_request)
And every time I try to do it I have the response "Error 400" or "Bad Request" if I use getUrlContent instead. When I just open this url in my browser - it works correctly.
I still couldn't find any solution for this problem, so if somebody knows something about it - please help me, kind man =)
There are several approaches you can use in case the URL is right. First you can add the following parameter to the getURL function. Setting the parameter followlocation equal TRUE allows to follow any "Location:" header that the server returns as part of the HTTP headers. Processing is recursive, PHP will follow any "Location:" header.
> s <- getURL(url1, .opts=curlOptions(followlocation = TRUE))
If this is not working an alternative way is to use the XML package by calling the htmlParse method instead of getURL
> library(XML)
> s <- htmlParse(api_request)
Another approach would be to use the httr package and call the GET function:
> library(httr)
> s <- GET(api_request)
I am trying and failing to use RCurl to automate the process of fetching a spreadsheet from a web site, China Labour Bulletin's Strike Map.
Here is the URL for the spreadsheet with the options set as I'd like them:
http://strikemap.clb.org.hk/strikes/api.v4/export?FromYear=2011&FromMonth=1&ToYear=2015&ToMonth=6&_lang=en
Here is the code I'm using:
library(RCurl)
temp <- tempfile()
temp <- getForm("http://strikemap.clb.org.hk/strikes/api.v4/export",
FromYear="2011", FromMonth="1",
ToYear="2015", ToMonth="6",
_lang="en")
And here is the error message I get in response:
Error: unexpected input in:
" ToYear=2015, ToMonth=6,
_"
Any suggestions on how to get this to work?
Try enclosing _lang with a backtick.
temp <- getForm("http://strikemap.clb.org.hk/strikes/api.v4/export",
FromYear="2011",
FromMonth="1",
ToYear="2015",
ToMonth="6",
`_lang`="en")
I think R has trouble on the argument starting with an underscore. This seems to have worked for me.