Where to include OAuth when creating an API function in R - r

I'm trying to create a function that I can apply to a data frame that will query an API for each row in my data frame.
After hours of trying I have got a response back from the API after having struggled with the whole authentication piece.
The steps needed to get a response back from the API include:
Send JSON payload with client id & secret to OAuth2.0 temp token generation endpoint
Parse access token from the response
Include access token in header of request to data endpoint which requires three parameters (ICN,SFFX & ENG values) to get a valid response
I'm now trying to build a function that incorporates all of these steps above and that I can apply/map_df to a data frame which contains the three input parameters (ICN,SFFX & ENG values) needed for the API.
What I wish to know is whether I should include steps 1 & 2 above in my function or just step 3 alone. I'm assuming if I'm iterating over rows in a data frame that I do not need to apply for temp tokens each time I send an API request for data??
In my code below I have defined the function after the OAuth phase (after first POST request).
What is the best practice for creating functions with OAuth2.0, should auth be done outside of a function?
My code looks as follows:
# INSTALL PACMAN IF NOT ALREADY INSTALLED
if (!require(pacman)) install.packages("pacman")
# LOAD LIBARIES
pacman::p_load(tidyverse, httr)
# ENABLE TOKEN CACHE TO COMPLY WITH API GUIDELINES
options(httr_oauth_cache = TRUE)
# DEFINE A USER AGENT
ua <- user_agent("https://github.mycompany.com/me")
# OAUTH2.0 TEMP TOKEN GENERATION ENDPOINT
url <- "https://gateway-stage-dmz.mycompany.com/auth/oauth2/cached/token"
# PAYLOAD TO SEND WITH THE AUTH REQUEST
body <- list(
client_id = "foo",
client_secret = "bar",
grant_type = "client_credentials"
)
# ACCESS TOKEN REQUEST
token <- POST(url,
body = jsonlite::toJSON(body,
pretty = TRUE,
auto_unbox = TRUE
),
httr::add_headers(`accept` = "application/json"),
httr::content_type("application/json")
)
# CREATE A FUNCTION TO QUERY THE API
api_fun <- function(ICN, SFFX, ENG) {
# HEADERS TO BE INCLUDED IN API CALL
headers <- c("Authorization" = sprintf("Bearer %s", content(token)$access_token))
# QUERY PARAMETERS FOR API CALL
params <- list(
icn = ICN,
suffix = SFFX,
EngineCd = ENG
)
# API RESPONSE
response <- httr::POST(
url = "https://gateway-stage-dmz.mycompany.com/api/endpoint_name_here",
httr::add_headers(.headers = headers),
query = params
)
df <- content(response,
as = "text",
encoding = "UTF-8"
) %>%
jsonlite::fromJSON(.,
flatten = TRUE
) %>%
data.frame()
}

Tokens typically last for a set amount of time, the most common default being 3600 s. Either the response holding the access token or the API's docs should tell you exactly how long the access token is fresh.
Here's an example from Google's APIs for Mobile and Desktop apps:
{
"access_token": "1/fFAGRNJru1FTz70BzhT3Zg",
"expires_in": 3920,
"token_type": "Bearer",
"scope": "https://www.googleapis.com/auth/drive.metadata.readonly",
"refresh_token": "1//xEoDL4iW3cxlI7yDbSRFYNG01kVKM2C-259HOF2aQbI"
}
The expires_in field represents the number of seconds until the access token expires. At that point, you can use the refresh token to get a new access token.
If you are confident that all of your calls should be completed by then, getting the access token once and using it for all the rows should work fine (i.e., you should not need to refresh the access token for each request).
If there is a chance that the process will take longer than the lifetime of the token, you will may need to refresh the token at some point. The best way to do that depends on lots of factors, so you would need to think about the best way to implement it on a row-by-row basis, which defeats a lot of the niceness of a dataframe.

Related

Adding arbitrary http headers with defaults in Swagger for R Plumber

I have added Bearer token authorization to my Swagger (created with Plumber), using the solution here.
Now I would like to add arbitrary headers that are a part of the request. My plumber deployment is used by a React dashboard, where parsing req$HTTP_TENANT gives me the desired value. And it is this I would like to recreate in Swagger.
Consider this example:
library(plumber)
r <- plumber::pr("plumber.R") %>%
plumber::pr_set_docs("swagger") %>%
plumber::pr_set_api_spec(function(spec) {
spec$components$securitySchemes$bearerAuth$type <- "http"
spec$components$securitySchemes$bearerAuth$scheme <- "bearer"
spec$components$securitySchemes$bearerAuth$bearerFormat <- "JWT"
spec$security[[1]]$bearerAuth <- ""
spec$components$parameters$HTTP_TENANT <- "HTTP_TENANT"
spec$parameters[[1]]$HTTP_TENANT <- "Customer name"
spec
})
r %>% plumber::pr_run(port = 8000, host = '0.0.0.0')
This gives the following result:
How can arbitrary headers be requested, for example HTTP_TENANT, maybe typed below the Bearer token input? It could also be somewhere else at the top of Swagger.
How can default values be provided for the headers, e.g. Customer name, but also the Bearer token (i.e. it could be programmatically entered from R)?
In Swagger, you can add a parameter to the API endpoint and change the "in" property to "header" to request any header, such as HTTP_TENANT. For instance:
spec$paths$"/endpoint"$get$parameters[[1]]$name <- "HTTP_TENANT"
spec$paths$"/endpoint"$get$parameters[[1]]$in <- "header"
spec$paths$"/endpoint"$get$parameters[[1]]$description <- "Customer name"
Under the "parameters" section of the Swagger UI, an input field for the "HTTP_TENANT" header will be added as a result of this.
A "default" property on the parameter object can be added to headers to provide default values. For instance:
spec$paths$"/endpoint"$get$parameters[[1]]$default <- "customer1"
The default values could also be set programmatically by reading them from an external source (like a configuration file) or by using an R package that handles authentication (like httr) to handle the bearer token automatically.

api authorization header r httr

I am trying to access the Open Apparel Registry api using httr.
NB: It's free to sign up (need to login + get authentication code on profile page).
But you can see the swagger api docs here: https://openapparel.org/api/docs/#!/facilities/facilities_list
Here is how you authorize on web version:
oar_root_api <- "https://openapparel.org/api/facilities/"
oar_token <- XXX
oar_api_facilities_GET <- httr::GET(url = oar_root_api,
add_headers(
`Authorization` = oar_token),
verbose()
)
The code I receive back is 401 so something is wrong with my authorization, but I've tried so many ways. I can't figure out how to specify this correctly.
Sorry to hear you've been having difficulties. This additional documentation may help: https://docs.google.com/document/d/1ZKCN84Eu9WDAXUokojOw7Dcg5TAJw0vKnVk7RPrTPZ0/edit?usp=sharing
We tend to find that users need to add the "Token" prefix (see page 3), which I appreciate isn't standard practice - this is something we intend to change!
Let us know how you get on.
OAR
The Open Apparel Registry (OAR) uses Django REST Framework to provide API enpoints. The TokenAuthentication class requires that the Authorization header value have a "Token " prefix. From the documentation
For clients to authenticate, the token key should be included in the Authorization HTTP header. The key should be prefixed by the string literal "Token", with whitespace separating the two strings. For example:
Authorization: Token 9944b09.....
I am not familiar with R, but I searched for string concatenation and it looks like the paste function will build the header value that you need.
oar_root_api <- "https://openapparel.org/api/facilities/"
oar_token <- XXX
oar_api_facilities_GET <- httr::GET(url = oar_root_api,
add_headers(
`Authorization` = paste("Token ", oar_token)),
verbose()
)

R: How to use Bing free tier web search using R

Assuming user provided card and phone and has valid Azure account. Created a free tier service. (has key and endpoint, something like xyz.cognitiveservices.azure.com/bing/v7.0
Using free tier (3 searchers per second and max per month or so) (see here https://azure.microsoft.com/en-us/pricing/details/cognitive-services/ )
Is it a GET or POST call and what are the right header parameters?
They only have Python example that is not working.
https://learn.microsoft.com/en-us/azure/cognitive-services/bing-web-search/quickstarts/python
https://github.com/Azure-Samples/cognitive-services-REST-api-samples/blob/master/python/Search/BingWebSearchv7.py
The questions is how to do it in R.
This code does not work
library(httr)
token='xxxxx'
server='https://xxxxx.cognitiveservices.azure.com/bing/v7.0/'
url=paste0(server,'search')
response = GET(url = url,
authenticate('',token, type = 'basic'))
response
res = content(response, encoding = 'json')
For /search endpoint, a GET request with a non-empty search parameter (q) is required.
Basic Authentication is not supported at all. Instead, as shown in the Python example, an HTTP header Ocp-Apim-Subscription-Key containing your subscription key is required.
So, I succeeded with the following code. It should work for you too.
library(httr)
server = "https://xxxxx.cognitiveservices.azure.com/bing/v7.0/"
token = "subscription key for Bing Search APIs v7"
search_term = "search term"
url = paste0(server, "search")
response = GET(url = url,
query = list(q = search_term),
add_headers(`Ocp-Apim-Subscription-Key` = token)
)
res = content(response, encoding = "json")
res
See Web Search API v7 reference for more information about headers and query parameters.

What is the Correct Endpoint for the Form Recognizer API?

Docs about using the Azure's Form Recognizer seem unclear. What is the correct ENDPOINT to send my request?
I'm using python and have followed the docs to use the prebuilt receipt model of Form Recognizer but don't get the expected output. I'm unsure if I'm using the correct endpoint. I tried two things:
Reading this tutorial, it is stated that I need to look up the <ENDPOINT> in my resource's overview page. In my case it is: formextractor.cognitiveservices.azure.com. So I tried this:
import http.client, urllib.request, urllib.parse, urllib.error, base64
params = urllib.parse.urlencode({
})
# key = '...'
headers = {
# Request headers
'Content-Type': 'application/json',
'Ocp-Apim-Subscription-Key': key,
}
source = r"https://www.w3.org/WAI/WCAG20/Techniques/working-examples/PDF20/table.pdf"
body = {"url":source}
body = json.dumps(body)
try:
conn = http.client.HTTPSConnection('formextractor.cognitiveservices.azure.com')
conn.request("POST", "/formrecognizer/v1.0-preview/prebuilt/receipt/asyncBatchAnalyze?s" % params, f"{body}", headers)
response = conn.getresponse()
data = response.read()
operationURL = "" + response.getheader("Operation-Location")
print ("Operation-Location header: " + operationURL)
conn.close()
except Exception as e:
print(e)
This returns:
[Errno 8] nodename nor servname provided, or not known
But in the API Docs the ENDPOINT is already fixed to westeurope.api.cognitive.microsoft.com, which is where my resource is located. So I tried this:
# ... same headers, body and params as before
try:
conn = http.client.HTTPSConnection('westeurope.api.cognitive.microsoft.com')
conn.request("POST", "/formrecognizer/v1.0-preview/prebuilt/receipt/asyncBatchAnalyze?%s" % params, f"{body}", headers)
response = conn.getresponse()
data = response.read()
operationURL = "" + response.getheader("Operation-Location")
print ("Operation-Location header: " + operationURL)
conn.close()
except Exception as e:
print("[Errno {0}] {1}".format(e.errno, e.strerror))
Outputs an URL where I see:
{"error":{"code":"401","message": "Access denied due to invalid subscription key or wrong API endpoint. Make sure to provide a valid key for an active subscription and use a correct regional API endpoint for your resource."}}
I am certain I'm using the correct key. But in any case neither of the two seems to work. Could you help me? Thank you.
It looks like the initial documentation you are mentioning is misleading. You can find the endpoint of your resource in Azure portal, in the resource overview. Sample for mine:
Form Recognizer API is (at the time of writing this answer) hosted in the following Azure regions:
West US 2 - westus2.api.cognitive.microsoft.com
West Europe - westeurope.api.cognitive.microsoft.com
So in my case it's WestEurope, and as you mentioned it is the same on your resource. Once you got it, you then got a 401 Unauthorized
Several possibilities:
You made an error in the way you are passing the header (wrong key name, wrong key value), but it looks okay based on your code above (but I'm not used to Python)
Your resource is not from the same region as the endpoint you are querying (please double check)
You are using the right root, but something is wrong in how you are calling it
Once you have checked your region / key values, can you remove your ?%s"%params from your query? The Analyze Receipt method don't have params in query string (given documentation)

Consume a REST API with R using a Token

I want to consume a REST API and have to perform the following steps for it.
I have to get a Token using my UserName and Password
(What I have accomplished successfully and the token is stored in a variable)
I have to use this Token to get data from the API and here I stuck.
I have tried
req_token <- THE TOKEN I HAVE RECIEVED ALREADY
url <- 'https://myService.com/web/api/datasources/{identifier}/data'
mydata <- GET(url, config = add_headers(paste0("Basic ", req_token)))
The identifier is there to specify a datasource within many, so in my case I had to replace it with EdQVFcgRGF0 (sort like). So I end up with the url
https://myService.com/web/api/datasources/{EdQVFcgRGF0}/data
All specification I got from the provider was
/datasources/{identifier]/data (GET)
● get data for one datasource (full data)
I tried consume the api with vb.net first and sending the token in the header works
request.AddHeader("Authorization", "Basic " + _token)
Now I get an 401 Unauthorized using R and do not know, what is wrong, anyone who can help me out?
Depending on the API configuration, I think you'll add it in where there's the curly brackets for {identifier} in the URL.
req_token <- THE TOKEN I HAVE RECIEVED ALREADY
url <- paste('https://myService.com/web/api/datasources/', req_token, '/data', sep='')
That's how some API's do it. Which means your headers might not look like this any more.
mydata <- GET(url, config = add_headers(paste0("Basic ", req_token)))
They probably just won't be there any more. So like :
mydata <- GET(url)
If the token is required in the headers it might look more like this:
mydata <- GET(url, config = add_headers("Basic " = req_token))
But I doubt the token will be both in the URL and in the header. You'll have to find out which is required from the docs.
Edit
I believe your headers should look like this:
mydata <- GET(url, config = add_headers("Authorization " = paste( "Basic", req_token, sep = ' ' ))

Resources