Posting a file to an outside API with a JSON web token - r

I am trying to upload a database file to an outside organization's API using R. I have a username and password, as well as an separate address to get the token from, and then to upload the file.
usr<-"username"
pw<-"passwood"
url <- "https:/routurl/api/"
Token='Token'
UploadFile='UploadFile'
#Get Token
r <- httr::POST(url = paste0(url,Token),
body = list(
UserName = usr,
Password = pw,
grant_type = "password"
), verbose())
tkn=jsonlite::prettify(httr::content(r, "text"))
This seems to work, as I can extract a token from the content.
> tkn
{
"result": {
"token": "eyJhbGciOiJIUzFAKEIsInR5cCI6IkpCJ9.eyJodHRwOi8vc2NoZW1hcy54bWxzb2FwLm9yZy93cy8yMDA1LzA1L2lkZW50aXR5L2NsYWltcy9uYW1lIjoiZ3JphzZSIsImp0aSI6IjUwNmIwN2MyLTTHISISFAKEIwMDUvMDUvaWRlbnRpdHkvY2xhaW1zL2VIVECHANGEDTHINGScyI6ImVtaWx5dGdyaWZmaXRoc0BiaW9zLmF1LmRrIiwiZXhwIjoxNTk4NzEwMTU3LCJpc3MiOiJ2bXNhcHAiLCJhdWQiOiJ2bXN1c2VycyJ9.z8sr-HT21u1bN7qCEXAMPLEONLY-TKAluO3k",
"expiration": "29 August 2020 16:09:17"
},
"id": 2,
"exception": null,
"status": 5,
"isCanceled": false,
"isCompleted": true,
"isCompletedSuccessfully": true,
"creationOptions": 0,
"asyncState": null,
"isFaulted": false
}
#re-formatting
tkn=jsonlite::fromJSON(content(r, "text"), simplifyVector = FALSE)
So, this all seems ok, however, if I try to double check this on the JSON DeCoder, my correct web information comes up in the payload, but at the bottom it claims it is an invalid signature.
Also, the auth_token variable is NULL in the request, and that doesn't seem right.
> r$request$auth_token
NULL
However, I can't test this because I cannot, for the life of me, figure out how to use this JWT to POST a file to the rooturl/UploadFile. Every document I look at that goes over how to POST to an API does not include how to include your JWT in the POST, or at least it isn't very clear. Is it in the header? Is it like this?
r2=POST(url=paste0(url,UploadFile), body = list(y = upload_file('O:/Igoturfilerighthere.h5')),
add_headers('Authorization' = paste("Bearer", tkn$result$token, sep = " ")), encode = "json", verbose())
Am I setting the headers incorrectly?
r3=POST(url=paste0(url,UploadFile), body = list(y = upload_file('O:/Igoturfilerighthere.h5')),
httr::add_headers("x-auth-token"=tkn$result$token), verbose())
For the r3 request I get a 401 error, which makes me think that I am on the correct path and that I am entering my token information incorrectly. If anyone could help guide me on the next step, I'd appreciate it. I just don't know where else to place that information.
Cheers,
etg
UPDATE:
If, in the initial request, I add 'encode = "json"', it throws a 400 Bad Request Error. This is how the website I am trying to upload to writes its own code. I've double checked my username and password, and they are correct.
r <- httr::POST(url = paste0(url,Token),
body = list(
UserName = usr,
Password = pw,
grant_type = "password"
),encode = "json", verbose())
HTTP/1.1 400 Bad Request
Transfer-Encoding: chunked
Content-Type: application/problem+json; charset=utf-8
Server: Microsoft-IIS/10.0
Strict-Transport-Security: max-age=2592000
X-Powered-By: ASP.NET

So, I reached out to the org behind the API I was trying to access, and there few a few problems with my JWT request. This is the correct code:
r <- httr::POST(paste0(url,Token),
body = list(UserName = usr, password = pw),
encode = "form", verbose())
The big difference is 'grant_type' is removed and 'encode="form"', as I was trying to log in via a form on their site. With that difference, I was able to upload a file using the following:
r2=POST(url=paste0(url,UploadFile), body = list(fileToUpload = httr::upload_file('O:/IGotUrFileHere.h5')),
httr::add_headers('Authorization' = paste("Bearer", tkn$result$token, sep = " ")), verbose())
Again, the verbose() function isn't necessary. It just helps you troubleshoot. Good luck!

Related

Getting 401 error from Post request in R with Bearer token

Im trying to update a body on a postman POST which ask for Bearer Token (that I have) but still getting 401 error..
I first request the Token
login <- list(
"corporateEmail" = " xxx#xxx.com",
"password" = "xxx"
)
test <- httr::POST("https://xxxx", body = login, encode = "json")
and succesfully the the access token in the response.
then trying to do the post to change the body
test2 <- httr::POST(url = urls,
add_headers(Authorization = token),
body = bodys,
encode = "json",
verbose())
and i get 401...
the body is correct and I have tryed with Authorization = paste("Bearer",token)... the Postman configuration of the request is that Bearer is add in Auth tab and not directly in the Header tab.
Post request, expecting 200 answer

Authenticating OAuth2.0 with R using httr

I'm trying to create authenticate into the Letterboxd API using R and the httr package. The Letterboxd docs give instructions, but I am not sure how to put everything together into a URL.
I know the url is:
url <- "https://api.letterboxd.com/api/v0/auth/token"
And then they want my username and password, presumably as JSON, but what I'll write as a named list since I'm doing this in R:
login_info <- list(
grant_type = "password",
username = "myemail#gmail.com",
password = "extremelysecurepassword"
)
I've tried various calls, using GET(), oauth2.0_token(), oauth_endpoint() functions from the httr package.
I feel like I have all the necessary information and am circling around a solution, but I can't quite nail it.
The docs contain this information:
When generating or refreshing an access token, make a form request to the /auth/token endpoint with Content-Type: application/x-www-form-urlencoded and Accept: application/json headers
(Full text is linked to above)
And I'm not sure how to add that information; in working with APIs through R, I'm used to just sending URLs with UTM parameters, but the inputs they want don't work here using ? and &.
I'm aware of this related post, but it looks like it relies on having a secret token already. And I don't seem to be able to generate a secret token inside of the GUI of Letterboxd.com, which is again what I'm used to doing with authentication. I think I need to feed it those sources of information above in login_info to the url, but I don't quite know how to connect the dots.
How do I authenticate to the Letterboxd API using R?
This runs for me but I get a 401 Unauthorized since you correctly did not supply valid credentials. It looks like there is a python library for this API https://github.com/swizzlevixen/letterboxd if you need hints how to make subsequent requests.
sign_request() is mimicking python library's api.py#L295-L304
sign_request <- function(apisecret, url, method, body = "") {
signing_bytes <- as.raw(c(charToRaw(method), 0, charToRaw(url), 0, charToRaw(body)))
# https://stackoverflow.com/a/31209556/8996878
# https://stackoverflow.com/q/54606193/8996878
digest::hmac(key = apisecret, object = signing_bytes, algo = "sha256", serialize = FALSE)
}
url <- "https://api.letterboxd.com/api/v0/auth/token"
login_info <- list(
grant_type = "password",
username = "myemail#gmail.com",
password = "extremelysecurepassword"
)
apikey <- "mytopsecretapikey"
apisecret <- "YOUR_API_SECRET"
method <- "POST"
params <- list(
apikey = apikey,
nonce = uuid::UUIDgenerate(),
timestamp = round(as.numeric(Sys.time()))
)
# now we need to sign the request
body <- paste(names(login_info), login_info, sep = "=", collapse = "&")
body <- URLencode(body)
body <- gsub("#","%40", body) # something URLencode doesn't do but post does
destination <- httr::parse_url(url)
destination$query <- params
post_url_with_params <- httr::build_url(destination)
signature <- sign_request(apikey, post_url_with_params, method, body)
token_request <- httr::POST(url, httr::add_headers(
"Accept" = "application/json",
"Authorization" = paste0("Signature ", signature)
),
query = params,
body = login_info, encode = "form", httr::verbose()
)
token_body <- httr::content(token_request, type = "application/json")
# look for the value of"access_token"

converting python POST request into R httr POST request

I have the following code in python which returns a 200 status:
> credentials = {
> 'grant_type': 'client_credentials',
> 'scope':'various scopes go here'
> }
> response = requests.post('https://sample_website.com/oauth/token',
> headers = {"Authorization": "Basic {}".format(signed_request),
> "Content-Type":"application/x-www-form-urlencoded"},
> data = credentials
> )
when i try the following code using the same signed_request I get a 401 error
credentials <- "{'grant_type': 'client_credentials','scope':'various scopes'}"
response <- POST(url = 'https://sample_website.com/oauth/token',
add_headers(
Authorization = signed_request,
'Content-Type' = 'application/x-www-form-urlencoded'
),
accept('application/json'),
body = credentials,
encode = 'json',
verbose()
)
I can't figure out what I am doing wrong but I'm assuming that I am not formatting the body correct or there might be an issue with the headers I am passing.
Your body is appropriate for the content type "application/json", but not for "x-www-form-urlencoded". The easiest solution is to pass a list, and let R take care of serialising it for you.
body <- list(
grant_type="client_credentials",
scope="scopes"
)
POST(url=*,
add_headers(Authorization=paste("Basic", signed_request)),
body=body,
encode="form"
)
Depending on what it is you're doing, you may also want to try the httr2 package (the successor to httr) and its built-in OAuth authentications options, among which is client credentials.

R: fetching pdf documents from Companies House API

I'm trying to fetch documents from the API using R. Appreciate the clarification of the process in this post. I've been following the above steps with partial success, but still fail the last step to get access to documents' content:
Find the document filing you're interested in (e.g. make a filing history request1 for the company). Parse the response for the link to the document in the field "links" : { "document_metadata" : "link URI fragment here" }.
No problem:
library(httr)
library(jsonlite)
library(openssl)
### retrieving filing history ####
company_num = 'FC013908'
key = 'my_key'
fh_path = paste0('/company/', str_to_upper(company_num), "/filing-history")
fh_url <- modify_url("https://api.companieshouse.gov.uk/", path = fh_path)
fh_test <- GET(fh_url, authenticate(key, "")) #status_code = 200
fh_parsed <- jsonlite::fromJSON(content(fh_test, "text",encoding = "utf-8"), flatten = TRUE)
docs <- fh_parsed$items
Done.
2 For a given document request the document metadata via CH Document API3. Parse the response to get the document (mime) types available and the link to the actual document data (document URI fragment).
No problems here:
md_meta_url = docs$links.document_metadata[1]
key_pass <- paste0(key,":")
decoded_auth <- paste0('Basic ', base64_encode(key_pass))
md_test <- GET(md_meta_url,
add_headers(Authorization = decoded_auth)
)
md_test #status_code = 200!
md_parsed <- jsonlite::fromJSON(content(md_test, "text",encoding = "utf-8"), flatten = TRUE)
This way I can obtain the content URL:
cont_url = md_parsed$links$document
Request the actual document9, specifying the mime type (e.g. "application/pdf").
I do it while NOT following the redirect and, as expected, I get the 302 status code with the location header:
accept = 'application/pdf'
cont_test <- GET(cont_url,
add_headers(Authorization = decoded_auth,
Accept = accept),
config(followlocation = FALSE)
)
final_url <- cont_test$headers$location
> final_url
[1] "https://s3-eu-west-1.amazonaws.com/document-api-images-prod/docs/LjBouRHeXXpIYAvqYIPWL06iXaliPz6Pucp1OXCXQhI/application-pdf?AWSAccessKeyId=ASIAJX7TVURFXZTY5DNQ&Expires=1529483765&Signature=uUQx6RTW7XBLqx4L6pYr5tOUySg%3D&x-amz-security-token=FQoDYXdzEP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDGxe7meYGe3OYhNwcSK3AwcVYJUXaUMf19oVO9s4qNPWN8AHjNNd5rrZhgE9YTkF1OmzyZSL5xHbls664kDP%2Bxd7dz9PIU5O1D%2BVxoDyoYcFiS6acDnO28KpfFE56lUZNfedf1jys%2FP0SJ8f%2F50Cbn93bfOlm0MZA9%2BQ2DYQvPfkWSvrDjMyCXHbu57gpZHjQKPNRTgzGXzUUCvFwREytGMM4eThhn4Glvvx%2FA8IiLbnsvgmEKw9iAj7KWIenhoJq3cTRytUpVeipLnQoBVLau8dFYkKdAHZaYM2Tlx0z6ObRb%2BGdm7W7eOVA1bFXuUXmUmnAHruDIwwLlgOVN2IJ9CxmJU22lY8jrEm%2BUivtrdp2oofn32PryBEJ8jJOg9cIpLbBBx%2FeOkng9zJwnZbute7Nmh%2BnaY2btsId6JjraFNsTvR%2B1qEZX9uuznUdJdqgVfTMj2gGrAmntwk0JAkILlvamzjWC%2F9vAqK7Xvt8aC6hlIMB2vdzTCU9Jf%2FrIMTClTJkk0BzBuvJ86t1l%2BXb4rF5Pab%2FegFpJ6nvZKqde%2F77wMMiTyG35EndmYx4AWqTIh9EofYwKZa9uciNvRT0E2%2BYnT5jZMo%2BdWn2QU%3D"
However, when I try to
Request this URI from Amazon again passing the content type you want again.
I get 400 error:
final_test <- GET(final_url,
add_headers(Authorization = decoded_auth,
Accept = accept
))
> final_test
Response [https://s3-eu-west-1.amazonaws.com/document-api-images-prod/docs/LjBouRHeXXpIYAvqYIPWL06iXaliPz6Pucp1OXCXQhI/application-pdf?AWSAccessKeyId=ASIAJX7TVURFXZTY5DNQ&Expires=1529483765&Signature=uUQx6RTW7XBLqx4L6pYr5tOUySg%3D&x-amz-security-token=FQoDYXdzEP%2F%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDGxe7meYGe3OYhNwcSK3AwcVYJUXaUMf19oVO9s4qNPWN8AHjNNd5rrZhgE9YTkF1OmzyZSL5xHbls664kDP%2Bxd7dz9PIU5O1D%2BVxoDyoYcFiS6acDnO28KpfFE56lUZNfedf1jys%2FP0SJ8f%2F50Cbn93bfOlm0MZA9%2BQ2DYQvPfkWSvrDjMyCXHbu57gpZHjQKPNRTgzGXzUUCvFwREytGMM4eThhn4Glvvx%2FA8IiLbnsvgmEKw9iAj7KWIenhoJq3cTRytUpVeipLnQoBVLau8dFYkKdAHZaYM2Tlx0z6ObRb%2BGdm7W7eOVA1bFXuUXmUmnAHruDIwwLlgOVN2IJ9CxmJU22lY8jrEm%2BUivtrdp2oofn32PryBEJ8jJOg9cIpLbBBx%2FeOkng9zJwnZbute7Nmh%2BnaY2btsId6JjraFNsTvR%2B1qEZX9uuznUdJdqgVfTMj2gGrAmntwk0JAkILlvamzjWC%2F9vAqK7Xvt8aC6hlIMB2vdzTCU9Jf%2FrIMTClTJkk0BzBuvJ86t1l%2BXb4rF5Pab%2FegFpJ6nvZKqde%2F77wMMiTyG35EndmYx4AWqTIh9EofYwKZa9uciNvRT0E2%2BYnT5jZMo%2BdWn2QU%3D]
Date: 2018-06-20 08:37
Status: 400
Content-Type: application/xml
Size: 523 B
<BINARY BODY>
Needless to say, executing
browseURL(final_test$url)
returns Access Denied error. I suspect it may have something to do with Amazon authorization problems similar to those described here. Any ideas how to solve this final hurdle?
Thanks!
The answer was provided by #voracityemail in response to my question on Companies House Developers Hub. Basically, the final call doesn't require the Authorization header, so if you run the following code for final_test:
final_test <- GET(final_url, add_headers(Accept = accept))
It will return 200 code
> final_test
Response [https://s3-eu-west-1.amazonaws.com/document-api-images-prod/docs/Rl1qKy2kNqdskHUIsqU9u0bGzH2goTfJfnCrNg4S0lg/application-pdf?AWSAccessKeyId=ASIAJMG7NTZHYC4NH3MA&Expires=1530093768&Signature=EteMSmwXS%2FqqdOFRmYY%2Fgf187Aw%3D&x-amz-security-token=FQoDYXdzELf%2F%2F%2F%2F%2F%2F%2F%2F%2F%2FwEaDOMKrcNPR6jb5bnzGSK3A1yzaoVZWhgAeXYCN9WJnxx8b%2BTKCEEZyZui3aR5j0WoNWIQhW9GIQ8R4xTGVkRjwQIhzgDp%2BRCfXGQ0CfPCOfseaQri5m%2BWTEWBgjfToL7%2FMdcC1IINMTFRrih1APE%2FmmTcQaW7SvyZWv3Q4bVQB%2FtOsiX5k8rWVsT7%2FecfQmnJMljcKF0%2F3vDRTtLRURTCtrdegfnIFrSqXkelLxVVypKY9UeURBgxAgngOgoP7YhYt3wD%2BEz5rBdNfMvF1Zuv91hLGDyBaKuV4fRKMRXlymDHCwNgNZl3JeyuAmnX8pexK6PJzH7MerM8QX8LoPfge1yutvqEj0%2FjRSYEShOWUebecQ2tJqWIEOZly0Ji8fc%2BMtFDO1FWZBrMl6lXgkwTMpELnTH5%2BP4ULMdFfEz30bWSnAuTGXcAxsoFWsFTIE2uO35zgkOsAUT2un4UNGnL2S8XexWbgwq%2B%2Bhtxo9ruP9WA8mTpjBkup2Qe5EpvUiNwGX9APjThi7QFTllVWWvpKgzKTSBh%2Btua9xK8RgiNAYDgEa5k%2BH%2FmWIP56WglBE6r3HGsXgbi%2Bff8Rg8z2lVFLo8f9hVv%2BCYoptXM2QU%3D]
Date: 2018-06-27 10:02
Status: 200
Content-Type: application/pdf
Size: 21.7 kB
<BINARY BODY>
and then
browseURL(final_test$url)
will open the specified document in the browser. Victory!

AWS API authentication in R : Missing Authentication Token

I need to authenticate AWS API in R. I tried using aws.signature package to do the same and I am getting 403 response with error Missing Authentication Token . It seems that I am missing some necessary parameters. Looking for assistance to debug the below code or ways to authenticate AWS API in R.
# To create aws signature for authentication for the rest API call
library(aws.signature)
library(httr)
# validate arguments and setup request URL
current <- Sys.time()
d_timestamp <- format(current, "%Y%m%dT%H%M%SZ", tz = "UTC")
hdrs <- list(`Content-Type` = "application/x-www-form-urlencoded",
Host = "jteti5wnje.execute-api.eu-central-1.amazonaws.com",
`x-amz-date` = d_timestamp)
params <- signature_v4_auth(
datetime = d_timestamp,
region = "eu-central-1",
service = "execute-api",
verb = "GET",
action = "iMetaAPI",
query_args = list(),
canonical_headers = hdrs,
request_body = "json",
key = "***************",
secret = "*****************",
session_token = NULL,
query = FALSE,
algorithm = "AWS4-HMAC-SHA256",
verbose = TRUE)
a <- GET("https://jteti5wnje.execute-api.eu-central-1.amazonaws.com/iMetaAPI",
query = params)
rawToChar(a$content)
A few things:
request_body needs to be the actual body. You're doing a GET() so it should just be NULL or "".
The response from signature_v4_auth() is a list but you don't need all of its elements - probably just the hds$Authorization <- params$SignatureHeader element.
The actual errors you're getting is because you haven't passed the headers. You need to pass the headers to GET() with something like: do.call(add_headers, hdrs) so you can do:
a <- GET("https://jteti5wnje.execute-api.eu-central-1.amazonaws.com/iMetaAPI", do.call(add_headers, headers))
That will probably work, or you'll at least get a more informative error.

Resources