With a paginated oAuth2.0 API, how do I use the response headers to view the next page?
So far I have successfully created an endpoint, app, token, and response:
# here's the key and username
API_KEY = 'my key'
API_USER = 'my username'
# set up the endpoint
api_endpoint= httr::oauth_endpoint(
authorize = "https://www.hydrovu.com/public-api/oauth/authorize",
access = "https://www.hydrovu.com/public-api/oauth/token"
)
# heres the app we will use to download data to
App = httr::oauth_app("xxx", key = API_USER, secret = API_KEY)
# heres the token we will use as a password
API.token = httr::oauth2.0_token(api_endpoint,
App,
scope = "https://www.hydrovu.com/public-api/oauth/token",
client_credentials=TRUE,
cache = FALSE,
config_init = user_agent("Testing Oauth with httr"))
# this is the response or the data
res <- httr::GET(url = "https://www.hydrovu.com/public-api/v1/locations/{id}/data?startTime=0",
user_agent("Testing Oauth with httr"),
config(token = API.token))
The response headers from res include next-page, which has a long string as the value. E.g. 6haGnbeLO09jlwe6poy7erT43qp
How to I use this next-page information to actually get the next page?
I am using the HydroVu API: https://www.hydrovu.com/public-api/docs/index.html
Let me know if you need more information!
After speaking to the excellent Hadley Wickham from httr maintenance, I have found the solution.
I was originally trying to pass x-isi-next-page to x-isi-next-page through add_headers() like so:
# original GET
res <- httr::GET(url,
user_agent("Testing Oauth with httr"),
config(token = API.token))
# write x-isi-next-page to object
nextpage = res$headers$`x-isi-next-page`
# pass to add_headers()
res <- httr::GET(url,
user_agent("Testing Oauth with httr"),
config(token = API.token),
add_headers(`x-isi-next-page` = nextpage))
When in fact I should have been passing it to x-isi-start-page
resp <- httr::GET(url,
user_agent("Testing Oauth with httr"),
config(token = API.token),
add_headers(`x-isi-start-page` = nextpage))
Thus, giving me the next page of results.
Now, to decode the json and create a compiled csv...
Related
Hi and thanks for reading me. I'm trying to connect to a database in firestore (firebase) following the steps of a tutorial I found on the internet, but I got a problem right now, and that is when trying to write the database path for the function , then it returns me an empty JSON. It does not give me any error message, so I assume that the connection to the firestore works, the problem arises when writing the path to the database.
The functions to connect to firestore are the following:
library(jsonlite)
library(httr)
sign.up <- function(email, password, api_key) {
#https://firebase.google.com/docs/reference/rest/auth/#section-create-email-password
r <- POST(paste0("https://identitytoolkit.googleapis.com/v1/accounts:signUp?key=", api_key),
add_headers("Content-Type" = "application/json"),
body = toJSON(list(email = email, password = password, returnSecureToken = TRUE),auto_unbox=TRUE))
return(content(r))
}
sign.in <- function(email, password, api_key) {
#https://firebase.google.com/docs/reference/rest/auth/#section-sign-in-email-password
r <- POST(paste0("https://identitytoolkit.googleapis.com/v1/accounts:signInWithPassword?key=", api_key),
add_headers("Content-Type" = "application/json"),
body = toJSON(list(email = email, password = password, returnSecureToken = TRUE),auto_unbox=TRUE))
return(content(r))
}
my_auth_info<-sign.in(
Sys.getenv("mail"),
Sys.getenv("password"),
Sys.getenv("token")
)
And the function to get the firestore data are the following:
read.db <- function(db_endpoint, auth_token) {
r <- GET(sprintf("https://firestore.googleapis.com/v1beta1/%s", db_endpoint),
add_headers("Content-Type" = "application/json",
"Authorization" = paste("Bearer", auth_token)))
return(r)
}
When i run the function with the following information I got an empty JSON:
read.db(
"projects/projectname/databases/(default)/documents/databasename",
my_auth_info$idToken
)
For the project name I use the following:
And for database name im using the following:
I need to use another information for the database path?
Thanks for the help
I am trying to use rtweet's search_fullarchive using my premium token registered at Twitter Developer. However, I got an error message of:
Warning: list(message = "Forbidden: Authentication succeeded but account is not authorized to access this resource.", sent = "2019-07-14T14:30:11+00:00", transactionId = "xxxxxxxxxx")
How do I get past this problem? Is there an error in the way i code?
I ensured my token is working well by testing it using httr's POST method and it worked completely fine.
I also tested my token using normal search_tweets, and also worked fine.
cons_key = "xxx"
cons_sec = "xxx"
acc_tok = "xxx"
acc_sec = "xxx"
app = "abc"
token = rtweet::create_token(app,cons_key,cons_sec,acc_tok,acc_sec)
manutd = search_fullarchive("manchester united",n=500,
fromDate = "201812010000",toDate = "201902010000",
env_name = app,token = token)
I expected a tibble dataframe as usually Rtweet's returns. However this is what i received:
"Warning: list(message = "Forbidden: Authentication succeeded but account is not authorized to access this resource.", sent = "2019-07-14T14:30:11+00:00", transactionId = "xxxxxxxxxx")"
data frame with 0 columns and 0 rows
To help anyone who might have trouble with the same thing, i'll write down a bit of explanation. Firstly, if you have registered a premium account, go to
To create the token, you need the App Name.
token = rtweet::create_token(app_name,cons_key,cons_sec,acc_tok,acc_sec)
To do query, you need the environment name.
search_fullarchive(q, n = 100, fromDate = NULL, toDate = NULL, env_name = "dev_name", safedir = NULL, parse = TRUE, token = token)
Hope it helps beginners or anyone who might have the same troubles.
I'm struggling to import the orders for a Shopify development store using the httr package in R. Here's what I've tried.
I've created a development store and made some fake orders.
Within my development store, I added a private app and generated my API key and Password
Following this article, I tried implementing the following request
Code
apikey <- "foo"
pass <- "bar"
shop <- GET(
url = "my-test-store.myshopify.com/orders.json",
authenticate(user = apikey, password = pass)
)
But this gives a 401 status code. However, this works but returns xml instead of json
shop <- GET(
url = "my-test-store.myshopify.com/orders",
authenticate(user = apikey, password = pass)
)
How can I retrieve the results as JSON instead of XML?
Note that I can also fetch the orders using the R package shopifyr but would rather not use that package as it is no longer maintained.
You're close. Try this:
library(httr)
apikey <- "foo"
pass <- "bar"
orders <- GET(
url = "https://yourshop.myshopify.com/admin/orders.json",
authenticate(user = apikey, password = pass)
)
content(orders)
Update 2019-05-13
I created an R package called shopr for querying data via the Shopify API. Fetching orders looks like this
library(shopr)
shopr_get_orders(
shopURL = "https://my-test-store.myshopify.com",
APIKey = "abc123",
APIPassword = "def456"
)
Old Answer
Figured it out.
orders <- GET(
url = "https://my-test-store.myshopify.com/admin/orders",
add_headers(Accept = "application/json"),
authenticate(user = apikey, password = pass)
)
orders
The trick was to explicitly put "https://..." in the url otherwise httr was prepending "http://" to the url, causing my 401 problem.
I'm trying to make my first POST request to an API. For some reason, I always get status 403 in return. I suspect it's the signature that is being incorrectly generated. The api-key and client id is for sure correct.
My code
nonce <-as.integer(Sys.time())
post_message <- paste0(nonce, data_client.id, data_key) # data_client.id = client id # data_key = key
sha.message <- toupper(digest::hmac(data_secret, object = post_message, algo = 'sha256', serialize = TRUE))
url <- 'https://www.bitstamp.net/api/v2/balance/'
body = list('API-KEY' = data_key, 'nonce' = nonce, 'signature' = sha.message)
httr::POST(url, body = body, verbose())
Output
<- HTTP/1.1 403 Authentication Failed
I'm trying to access the Bitstamp API: https://www.bitstamp.net/api/?package=Rbitcoin&version=0.9.2
All private API calls require authentication. For a successful
authentication you need to provide your API key, a signature and a
nonce parameter.
API KEY
To get an API key, go to "Account", "Security" and then "API Access".
Set permissions and click "Generate key".
NONCEN
once is a regular integer number. It must be increased with every
request you make. Read more about it here. Example: if you set nonce
to 1 in your first request, you must set it to at least 2 in your
second request. You are not required to start with 1. A common
practice is to use unix time for that parameter.
SIGNATURE
Signature is a HMAC-SHA256 encoded message containing nonce, customer
ID (can be found here) and API key. The HMAC-SHA256 code must be
generated using a secret key that was generated with your API key.
This code must be converted to it's hexadecimal representation (64
uppercase characters).
I'm not sure if your question is still standing, but based on your code, I managed to get it working. In fact, the main problem is in the body, the API documentation shows it expects 'key' instead of 'API-KEY'.
Also, serialize should be FALSE instead of TRUE.
At the moment this works (but the API may change):
nonce <-as.integer(Sys.time())
post_message <- paste0(nonce, data_client.id, data_key) # data_client.id = client id # data_key = key
sha.message <- toupper(digest::hmac(data_secret, object = post_message, algo = 'sha256', serialize = FALSE))
url <- 'https://www.bitstamp.net/api/v2/balance/'
body = list('key' = data_key, 'nonce' = nonce, 'signature' = sha.message)
httr::POST(url, body = body, verbose())
A refresh token is not available when I follow Hadley's R google Oauth2.0 demo to access Fusion tables.
Demo: https://github.com/hadley/httr/blob/master/demo/oauth2-google.r
Example of modified "offline" attempt:
google_token <- oauth2.0_token(oauth_endpoints("google"), myapp,
scope = "https://www.googleapis.com/auth/fusiontables",
type= "offline",
use_oob = FALSE,
cache = TRUE)
Any direction on how to retrieve a refresh token is much appreciated.
UPDATE:
Using the follow code a character string is returned with google_token$credentials. Is this the authorization code referenced here:https://developers.google.com/accounts/docs/OAuth2WebServer#offline
google_token <- oauth2.0_token(oauth_endpoints("google"), myapp,
scope = "https://www.googleapis.com/auth/fusiontables",
type= "access_type='offline'",
use_oob = FALSE,
cache = TRUE)
Thank you.
I'm a bit late to the party here but hopefully this helps someone. I found this question last week because I was struggling with the same issue. Like you, I read the API documentation and tried the "offline" in the "type" field of the "oauth2.0_token()" function but it messed up the response. I downloaded the httr package source files from the github repository and had a look around. After some digging I found a way around it.
if you modify the "authorize-url" variable in oauth-init from this:
authorize_url <- modify_url(endpoint$authorize, query = compact(list(
client_id = app$key,
scope = scope_arg,
redirect_uri = redirect_uri,
response_type = "code",
state = state)))
to this:
authorize_url <- modify_url(endpoint$authorize, query = compact(list(
client_id = app$key,
scope = scope_arg,
redirect_uri = redirect_uri,
response_type = "code",
state = state,
access_type="offline")))
and then source the oauth-token and all the dependent functions (including oauth-init) you'll get a refresh token. For some reason, when oauth_token calls init_oauth2.0 it doesn't pass the "type" argument.
It's a nasty workaround and I've probably committed several sins but it does work. and you do get a refresh token.