I have some url, some raw data that looks like json and auth creds.
Everything together works great via postman, but now i need to automate it in R.
For example:
url1 = 'https://site.io/v2/passes/?values=true'
bod = paste0('{"values": [{"value": "0","label": "label1"},{"value": "5","label": "label2"}] }')
I've already tried this one
r <- POST(url1, body = bod, encode = "raw", httr::add_headers(.headers=headers)
, httr::authenticate("user", "password", type="digest"))
But it seems incorrect (and also getting 400 error)
I assume your bod is a text string. When you use encode="raw", you have to also specify the Content-encoding header if the host is expecting one (and in this case, I assume it's expecting "application/json":
headers <- c(headers, `Content-encoding`="application/json")
POST(url1, body=bod, encode="raw", add_headers(.headers=headers), ...)
Related
I'm trying to create authenticate into the Letterboxd API using R and the httr package. The Letterboxd docs give instructions, but I am not sure how to put everything together into a URL.
I know the url is:
url <- "https://api.letterboxd.com/api/v0/auth/token"
And then they want my username and password, presumably as JSON, but what I'll write as a named list since I'm doing this in R:
login_info <- list(
grant_type = "password",
username = "myemail#gmail.com",
password = "extremelysecurepassword"
)
I've tried various calls, using GET(), oauth2.0_token(), oauth_endpoint() functions from the httr package.
I feel like I have all the necessary information and am circling around a solution, but I can't quite nail it.
The docs contain this information:
When generating or refreshing an access token, make a form request to the /auth/token endpoint with Content-Type: application/x-www-form-urlencoded and Accept: application/json headers
(Full text is linked to above)
And I'm not sure how to add that information; in working with APIs through R, I'm used to just sending URLs with UTM parameters, but the inputs they want don't work here using ? and &.
I'm aware of this related post, but it looks like it relies on having a secret token already. And I don't seem to be able to generate a secret token inside of the GUI of Letterboxd.com, which is again what I'm used to doing with authentication. I think I need to feed it those sources of information above in login_info to the url, but I don't quite know how to connect the dots.
How do I authenticate to the Letterboxd API using R?
This runs for me but I get a 401 Unauthorized since you correctly did not supply valid credentials. It looks like there is a python library for this API https://github.com/swizzlevixen/letterboxd if you need hints how to make subsequent requests.
sign_request() is mimicking python library's api.py#L295-L304
sign_request <- function(apisecret, url, method, body = "") {
signing_bytes <- as.raw(c(charToRaw(method), 0, charToRaw(url), 0, charToRaw(body)))
# https://stackoverflow.com/a/31209556/8996878
# https://stackoverflow.com/q/54606193/8996878
digest::hmac(key = apisecret, object = signing_bytes, algo = "sha256", serialize = FALSE)
}
url <- "https://api.letterboxd.com/api/v0/auth/token"
login_info <- list(
grant_type = "password",
username = "myemail#gmail.com",
password = "extremelysecurepassword"
)
apikey <- "mytopsecretapikey"
apisecret <- "YOUR_API_SECRET"
method <- "POST"
params <- list(
apikey = apikey,
nonce = uuid::UUIDgenerate(),
timestamp = round(as.numeric(Sys.time()))
)
# now we need to sign the request
body <- paste(names(login_info), login_info, sep = "=", collapse = "&")
body <- URLencode(body)
body <- gsub("#","%40", body) # something URLencode doesn't do but post does
destination <- httr::parse_url(url)
destination$query <- params
post_url_with_params <- httr::build_url(destination)
signature <- sign_request(apikey, post_url_with_params, method, body)
token_request <- httr::POST(url, httr::add_headers(
"Accept" = "application/json",
"Authorization" = paste0("Signature ", signature)
),
query = params,
body = login_info, encode = "form", httr::verbose()
)
token_body <- httr::content(token_request, type = "application/json")
# look for the value of"access_token"
Trying to send POST request in R but didn't work. I tried:
require(httr)
token_endpoint = '...'
client_id = '...'
client_secret = 'pgGj9n4Sdl8cfM4cKNnjYcLVGSIyQxhm3ydCX3IRbdc='
scope = 'icdapi_access'
grant_type = 'client_credentials'
r <- POST(token_endpoint, add_headers('client_id'= client_id,
'client_secret'= client_secret,
'scope'= scope,
'grant_type'= grant_type))
with package httr following a Python version of this https://github.com/ICD-API/Python-samples/blob/master/sample.py
but I don't know where to find access_token in r
Also I don't know where should I
Essentially, you need to retrieve the content object in your response data and have it parsed. Consider sending JSON content type as a config parameter (i.e., unnamed parameter). Since it is json, convert the content with rawToChar and pass it into jsonlite::fromJSON(). You may need to dig inside api_json to retrieve needed data.
library(httr)
library(jsonlite)
...
response <- httr::POST(
token_endpoint,
add_headers(
'client_id'= client_id,
'client_secret'= client_secret,
'scope'= scope,
'grant_type'= grant_type)
),
httr::content_type_json()
)
api_json <- jsonlite::fromJSON(rawToChar(response$content))
I have a question about to add body to GET request.
In Postman, I easily add body raw json to tab Body
In R, I use httr::GETand I can not find any option to add them. I try to use option query, but it return error 500 "Missing parameter username"
This is my code (sorry about fake data):
library(httr)
api <- ".........................../users/authorize"
query <- list("username": "xxxxx", "password": "xxxxxxxxxxxx")
resp <- GET(api, query = query)
stop_for_status(resp)
# Error: Internal Server Error (HTTP 500).
content(resp, type = "text", encoding = "utf-8")
# [1] "{\"status\":\"0\",\"error_code\":\"S002\",\"errors\":\"Missing param [username].\"}"
Can anybody help me to this case? Thank you so much.
Try this:
resp <- httr::POST(body = query, encode = "json")
I'm using R to interact with the Gemini exchange API (https://docs.gemini.com/rest-api/) for private endpoints. I've been able to reduce my problem to endpoints which require more than 2 parameters in the payload. In particular I'm attempting to query the /v1/mytrades endpoint (https://docs.gemini.com/rest-api/#get-past-trades) which I believe requires the 'request', 'nonce' and 'symbol' parameters at a minimum. The error code I receive is HTTP 400 which Gemini describes as:
Auction not open or paused, ineligible timing, market not open, or the request was malformed; in the case of a private API request, missing or malformed Gemini private API authentication headers
I have no issues with other endpoints which require only the 'request' and 'nonce' parameters, so I'm struggling to understand which step is a problem since those queries require similar steps to create a base64 encoding of the payload, a signature of that encoding using the API secret and headers with that data plus the API key.
Below is my example code where MY_API_SECRET and MY_API_KEY are placeholders for the actual secret and key strings
# Set variable for the gemini api URL
geminiHost <- "https://api.gemini.com"
# Set variable for the gemini endpoint
geminiEndpoint <- "/v1/mytrades"
# Create the symbol parameter
symbol <- 'btcusd'
# Create nonce parameter
currentTimeNonce <- round(as.numeric(Sys.time()) * 1000, 0)
# Create JSON payload
payload <-
toJSON(data.frame(
request = geminiEndpoint,
nonce = currentTimeNonce,
symbol = symbol
)) %>% gsub("\\[|\\]", "", .)
# Convert payload to base64
payloadBase64Enc <- base64_enc(payload)
# Create signature
signatureString <- sha384(payloadBase64Enc, key = 'MY_API_SECRET')
# Construct the complete URL
completeURL <- paste0(geminiHost, geminiEndpoint)
# Create header
hdr = c(
"Content-Type" = "text/plain",
"Content-Length" = "0",
"Cache-Control" = "no-cache",
"X-GEMINI-APIKEY" = "MY_API_KEY",
"X-GEMINI-PAYLOAD" = payloadBase64Enc,
"X-GEMINI-SIGNATURE" = signatureString
)
# Request API using the complete URL and the required headers
mytradesAPIResult <- fromJSON(httpPOST(completeURL,
httpheader = hdr,
verbose = TRUE))
For comparison, the following code which requests the /v1/orders endpoint (https://docs.gemini.com/rest-api/#get-active-orders) does indeed come back with a response:
# Set variable for the gemini api URL
geminiHost <- "https://api.gemini.com"
# Set variable for the gemini endpoint
geminiEndpoint <- "/v1/orders"
# Create nonce parameter
currentTimeNonce <- round(as.numeric(Sys.time()) * 1000, 0)
# Create JSON payload
payload <-
toJSON(data.frame(request = geminiEndpoint, nonce = currentTimeNonce)) %>%
gsub("\\[|\\]", "", .)
# Convert payload to base64
payloadBase64Enc <- base64_enc(payload)
# Create signature
signatureString <- sha384(payloadBase64Enc, key = 'MY_API_SECRET')
# Construct the complete URL
completeURL <- paste0(geminiHost, geminiEndpoint)
# Create header
hdr = c(
"Content-Type" = "text/plain",
"Content-Length" = "0",
"Cache-Control" = "no-cache",
"X-GEMINI-APIKEY" = "MY_API_KEY",
"X-GEMINI-PAYLOAD" = payloadBase64Enc,
"X-GEMINI-SIGNATURE" = signatureString
)
# Request API using the complete URL and the required headers
mytradesAPIResult <- fromJSON(httpPOST(completeURL,
httpheader = hdr,
verbose = TRUE))
So in the latter code the only difference is the geminiEndpoint and payload construction which as mentioned above only has 2 required parameters (request and nonce). To expand further, I'm successfully hitting other endpoints such as /v1/tradevolume, /v1/heartbeat, and /v1/balances that also require those 2 parameters only while /v1/order/status is another example of an endpoint requiring at least 3 parameters that doesn't work for me.
I appreciate any help to understand where I'm going wrong with this. Thank you in advance!
As mentioned in the comments, I started to work on an equivalent bash script based on r2evans' reply when I found my issue. It was the base64 encoding step in R that resulted in some unusual output. As seen in my original scripts, I was using the "base64_enc()" function which is part of the jsonlite package. As a simple check I was trying to confirm that the encoding from R was equal to an equivalent encoding using base64 in shell so I started by trying to decode the R result.
In R, the encoding of the payload for the 3 parameter example was coming out with a backslash character '\' which is not valid Base64 and points to my misunderstanding of what the base64_enc function is doing. I replaced this function with base64_encode from the openssl package and now my 3 parameter queries are coming back with results.
I'm attempting to request data from a google spreadsheet using the googleAuthR. I need to use this library instead of Jenny Bryan's googlesheets because the request is part of a shiny app with multiple user authentication. When the request range does not contain spaces (e.g. "Sheet1!A:B" the request succeeds. However, when the tab name contains spaces (e.g. "'Sheet 1'!A:B" or "\'Sheet 1\'!A:B", the request fails and throws this error:
Request Status Code: 400
Error : lexical error: invalid char in json text.
<!DOCTYPE html> <html lang=en>
(right here) ------^
Mark Edmondson's googleAuthR uses jsonlite for parsing JSON. I assume this error is coming from jsonlite, but I'm at a loss for how to fix it. Here is a minimal example to recreate the issue:
library(googleAuthR)
# scopes
options("googleAuthR.scopes.selected" = "https://www.googleapis.com/auth/spreadsheets.readonly")
# client id and secret
options("googleAuthR.client_id" = "XXXX")
options("googleAuthR.client_secret" = "XXXX")
# request
get_data <- function(spreadsheetId, range) {
l <- googleAuthR::gar_api_generator(
baseURI = "https://sheets.googleapis.com/v4/",
http_header = 'GET',
path_args = list(spreadsheets = spreadsheetId,
values = range),
pars_args = list(majorDimension = 'ROWS',
valueRenderOption = 'UNFORMATTED_VALUE'),
data_parse_function = function(x) x)
req <- l()
req
}
# authenticate
gar_auth(new_user = TRUE)
# input
spreadsheet_id <- "XXXX"
range <- "'Sheet 1'!A:B"
# get data
df <- get_data(spreadsheet_id, range)
How should I format range variable for the request to work? Thanks in advance for the help.
Use URLencode() to percent-encode spaces.
Details:
Using options(googleAuthR.verbose = 1) shows that the GET request was of the form:
GET /v4/spreadsheets/.../values/'Sheet 1'!A:B?majorDimension=ROWS&valueRenderOption=UNFORMATTED_VALUE HTTP/1.1
I had assumed the space would be encoded, but I guess not. In this github issue from August 2016, Mark states URLencode() was going to be the default for later versions of googleAuthR. Not sure if that will still happen, but it's an easy fix in the meantime.