Using a browser, I can navigate to https://myaccount.draftkings.com/login and login using a username and password. Then if I navigate to https://api.draftkings.com/scores/v1/leaderboards/9000?format=json&embed=leaderboard in another tab, I can see the API response.
I want to do this programmatically.
There is a python package that I previously used successfully for this purpose. But for some reason, I can't get that package to work now.
I believe the python package hits an API endpoint with a post request to authenticate and saves the cookies to a file that are later used to make requests.
I tried sending a post request to the login endpoint the python package uses with httr2:
req <- httr2::request("https://api.draftkings.com/users/v3/providers/draftkings/logins?format=json") %>%
httr2::req_method("POST") %>%
httr2::req_headers(
"Contest-Type" = "application/json",
"Accept" = "*/*",
"Accept-Encoding" = "gzip, deflate, br"
) %>%
httr2::req_body_json(
list(
"login" = "email",
"password" = "password",
"host" = "api.draftkings.com",
"challengeResponse" = list("solution" = "", "type" = "Recaptcha")
)
)
req %>%
httr2::req_perform()
but I get a 403 error.
I also tried logging in using rvest:
library(rvest)
url <- "myaccount.draftkings.com/login"
session <- session(url)
form <- session %>%
html_form() %>%
magrittr::extract2(1)
form$action <- url
filled_form <- form %>%
html_form_set(!!! list(EmailOrUsername = "user",
Password = 'password'))
html_form_submit(filled_form, submit = 3)
session_jump_to(session, "https://www.draftkings.com/lobby")
but that didn't seem to do what I want either. Note that the form object returned an empty action element. I'm not sure what to replace the action element with, but if I leave it null I get an error.
I also tried saving the cookies from a browser session after logging into draftkings.com and passing all the cookies to a GET request with httr:
cook <- jsonlite::read_json("cookies.json")
clean_cook <- unlist(lapply(cook, function(x) {stats::setNames(x$value, x$name)}))
resp <- httr::GET(
"https://api.draftkings.com/scores/v1/leaderboards/9000?format=json&embed=leaderboard",
httr::set_cookies(.cookies = clean_cook)
)
httr::content(resp)
but this returns a 400 error code and message "Invalid userKey.". This is the same error message I get if I clear my cache in the browser and then visit https://api.draftkings.com/scores/v1/leaderboards/9000?format=json&embed=leaderboard. I don't think the issue is related to URL encoding of the cache values. I tried restarting my RStudio session.
Update
I figured out how to successfully perform the GET request using the cookies saved from my browser and httr2:
cook <- jsonlite::read_json("cookies.json")
clean_cook <- paste0(unlist(lapply(cook, function(x) {paste0(x$name, "=", x$value)})), collapse = ";")
req <- httr2::request(
"https://api.draftkings.com/scores/v1/leaderboards/9000?format=json&embed=leaderboard"
)
req <- httr2::req_headers(req, "cookie" = clean_cook)
resp <- httr2::req_perform(req)
str(httr2::resp_body_json(resp))
For some reason, I was unable to get httr::set_cookies() to work. Passing cookies in the header (using httr::add_headers) was also not reliably successful using httr.
Related
I'm trying to create authenticate into the Letterboxd API using R and the httr package. The Letterboxd docs give instructions, but I am not sure how to put everything together into a URL.
I know the url is:
url <- "https://api.letterboxd.com/api/v0/auth/token"
And then they want my username and password, presumably as JSON, but what I'll write as a named list since I'm doing this in R:
login_info <- list(
grant_type = "password",
username = "myemail#gmail.com",
password = "extremelysecurepassword"
)
I've tried various calls, using GET(), oauth2.0_token(), oauth_endpoint() functions from the httr package.
I feel like I have all the necessary information and am circling around a solution, but I can't quite nail it.
The docs contain this information:
When generating or refreshing an access token, make a form request to the /auth/token endpoint with Content-Type: application/x-www-form-urlencoded and Accept: application/json headers
(Full text is linked to above)
And I'm not sure how to add that information; in working with APIs through R, I'm used to just sending URLs with UTM parameters, but the inputs they want don't work here using ? and &.
I'm aware of this related post, but it looks like it relies on having a secret token already. And I don't seem to be able to generate a secret token inside of the GUI of Letterboxd.com, which is again what I'm used to doing with authentication. I think I need to feed it those sources of information above in login_info to the url, but I don't quite know how to connect the dots.
How do I authenticate to the Letterboxd API using R?
This runs for me but I get a 401 Unauthorized since you correctly did not supply valid credentials. It looks like there is a python library for this API https://github.com/swizzlevixen/letterboxd if you need hints how to make subsequent requests.
sign_request() is mimicking python library's api.py#L295-L304
sign_request <- function(apisecret, url, method, body = "") {
signing_bytes <- as.raw(c(charToRaw(method), 0, charToRaw(url), 0, charToRaw(body)))
# https://stackoverflow.com/a/31209556/8996878
# https://stackoverflow.com/q/54606193/8996878
digest::hmac(key = apisecret, object = signing_bytes, algo = "sha256", serialize = FALSE)
}
url <- "https://api.letterboxd.com/api/v0/auth/token"
login_info <- list(
grant_type = "password",
username = "myemail#gmail.com",
password = "extremelysecurepassword"
)
apikey <- "mytopsecretapikey"
apisecret <- "YOUR_API_SECRET"
method <- "POST"
params <- list(
apikey = apikey,
nonce = uuid::UUIDgenerate(),
timestamp = round(as.numeric(Sys.time()))
)
# now we need to sign the request
body <- paste(names(login_info), login_info, sep = "=", collapse = "&")
body <- URLencode(body)
body <- gsub("#","%40", body) # something URLencode doesn't do but post does
destination <- httr::parse_url(url)
destination$query <- params
post_url_with_params <- httr::build_url(destination)
signature <- sign_request(apikey, post_url_with_params, method, body)
token_request <- httr::POST(url, httr::add_headers(
"Accept" = "application/json",
"Authorization" = paste0("Signature ", signature)
),
query = params,
body = login_info, encode = "form", httr::verbose()
)
token_body <- httr::content(token_request, type = "application/json")
# look for the value of"access_token"
Let me start by saying that I understand how to do a POST request using "httr" and "crul" packages. I am working on developing an asynchronous method to sending multiple POST request with unique JSON body requests using the basic "curl" package. I have legitimate reasons for trying this with this package, but more importantly I'm just determined to get it to work. This may not be possible, or I may even be trying to wrong functions in "curl"...but wanted to see if anyone had any ideas.
I am trying to send a post request using curl_fetch_multi() as a POST request with a JSON in the body like this...
{
"configuration": {
"Id": 4507
},
"age": 0,
"zip": 32411,
"Date": "2020-12-23"
}
I have succeeded in at least getting getting error messages back form the API indicating an invalid body input using something along the lines of starting with an object containing each body i need to submit
library(curl)
library(jsonlite)
library(magrittr)
pool <- new_pool()
# results only available through call back function
cb <- function(req){cat("done:", req$url, ": HTTP:", req$status, "\n", "content:", rawToChar(req$content), "\n")}
# Create request for each body
for(i in 1:nrow(df)){
curl_fetch_multi(
"http://api.com/values?api_key=1234",
done = cb,
pool = pool,
handle = new_handle() %>%
handle_setopt(post = TRUE) %>%
handle_setheaders("Content-Type"="application/vnd.v1+json") %>%
handle_setform(body = df$body[[i]]) ###df$body[[i]] is a JSON string
)
}
# This actually performs requests
out <- multi_run(pool = pool)
done: http://api.com/values?api_key=1234 : HTTP: 400
content: {"errors":[{"code":"Service.input.invalid","message":"Invalid input"}]}
done: http://api.com/values?api_key=1234 : HTTP: 400
content: {"errors":[{"code":"Service.input.invalid","message":"Invalid input"}]}
....
I'm 90% positive it has to do with how it's attempting to call the JSON in handle_setform() setting of the handle. This is about where I am over my head and documentation is scarce.
Also, I am pretty sure the JSON is structured properly, as I can use them in other packages with no problem.
Any assistance would be greatly appreciated.
Found the solution!!
Needed to use following settings with handle_setopts()
for(i in 1:nrow(df)){
curl_fetch_multi(
"http://api.com/values?api_key=1234",
done = cb,
pool = pool,
handle = new_handle() %>%
handle_setheaders("Content-Type"="application/v1+json") %>%
handle_setopt(customrequest = "POST") %>%
handle_setopt(postfields = df$body[[i]]) #df$body is list of JSON
)
}
out <- multi_run(pool = pool)
I am trying to access a RapidAPI with the httr package in R as follows:
library(httr)
url <- "https://extract-news.p.rapidapi.com/v0/article"
queryString <- list(url = "https://www.theverge.com/2020/4/17/21224728/bill-gates-coronavirus-lies-5g-covid-19")
response <- VERB("GET", url, addheaders(x_rapidapi_key ="my-api-key", x_rapidapi_host = "extract-news.p.rapidapi.com"), query = queryString, contenttype("application/octet-stream"))
content(response, "text")
I tried accessing other APIs too. However, I keep getting this error message:
Status: 401
Could you please help me solve this issue?
Thanks a lot in advance!
Try using content_type() and add_headers instead of contenttype and addheaders to tell the server what sort of data you are sending. Read more about the VERB method here.
library(httr)
url <- "https://extract-news.p.rapidapi.com/v0/article"
queryString <- list(url = "https://www.theverge.com/2020/4/17/21224728/bill-gates-coronavirus-lies-5g-covid-19")
response <- VERB("GET", url, add_headers(x_rapidapi-host = 'extract-news.p.rapidapi.com', x_rapidapi-key = '*************************', '), query = queryString, content_type("application/octet-stream"))
content(response, "text")
I am having trouble with DELETE requests in R. I have been successful in making GET and POST requests using the below code. Any help / pointers will be appreciated.
It will require an api.key, secret & passphrase from GDAX to work.
Here is my function:
library(RCurl)
library(jsonlite)
library(httr)
library(digest)
cancel_order <- function(api.key,
secret,
passphrase) {
api.url <- "https://api.gdax.com"
#get url extension----
req.url <- "/orders/"
#define method----
method = "DELETE"
url <- paste0(api.url, req.url)
timestamp <-
format(as.numeric(Sys.time()), digits = 13) # create nonce
key <- base64Decode(secret, mode = "raw") # encode api secret
#create final end point----
what <- paste0(timestamp, method, req.url)
#create encoded signature----
sign <-
base64Encode(hmac(key, what, algo = "sha256", raw = TRUE)) # hash
#define headers----
httpheader <- list(
'CB-ACCESS-KEY' = api.key,
'CB-ACCESS-SIGN' = sign,
'CB-ACCESS-TIMESTAMP' = timestamp,
'CB-ACCESS-PASSPHRASE' = passphrase,
'Content-Type' = 'application/json'
)
##------------------------------------------------
response <- getURL(
url = url,
curl = getCurlHandle(useragent = "R"),
httpheader = httpheader
)
print(rawToChar(response)) #rawToChar only on macOS and not on Win
}
The error I get is "{\"message\":\"invalid signature\"}" even though the same command will code and signature will work with GET & POST.
Ref: GDAX API DOCs
just a guess as I am not familiar with the API, but perhaps you are missing the 'order-id' ...
look at: https://docs.gdax.com/?javascript#cancel-an-order
Ok. I took #mrflick's advise and pointed my connection to requestbin based on his feedback on a different but related question.
After careful inspection, I realized that the my request for some reason was treated as a POST request and not a DELETE request. So I decided to replace the getURL function with another higher level function from RCurl for it to work.
response <- httpDELETE(
url = url,
curl = getCurlHandle(useragent = "R"),
httpheader = httpheader
)
Everything else remains the same. Apparently there never was an issue with the signature.
I have added this function now to my unofficial wrapper rgdax
EDIT::
The unofficial wrapper is now official and on CRAN.
I am trying to post an update to Twitter using the api. Following the steps at https://github.com/hadley/httr/blob/master/demo/oauth1-twitter.r, I can get the GET call to work, but I haven't managed to get the POST to work. Here is the relevant code:
library(httr)
library(httpuv)
oauth_endpoints("twitter")
myapp <- oauth_app("twitter",
key = "key"
secret = "secret"
)
twitter_token <- oauth1.0_token(oauth_endpoints("twitter"), myapp)
#this works fine
req <- GET("https://api.twitter.com/1.1/statuses/home_timeline.json",
config(token = twitter_token))
#this doesn't work. Returns error 403: status is forbidden
POST(url = "https://api.twitter.com/1.1/statuses/update.json",
body = "test",
config(token = twitter_token)
)
Any solution that allows a status update (Tweet) from R is acceptable.
The docs say you need to pass a parameter status as a query parameter, not in the body, which makes sense b/c it can only be 140 characters long anyway.
POST("https://api.twitter.com/1.1/statuses/update.json",
query = list(status = "testing from R"),
config(token = twitter_token)
)