Brand new to API's and in a bit over my head. Reading a bit and searching around but still confused. I was given the following code to run in curl to generate a token using oauth2 with the password grant. I want to implement this code in R and running into some issues.
curl -H "Content-type: application/x-www-form-urlencoded" -H "Authorization: Basic xxxyyy123xxx==" -d "grant_type=password&username={username}&password={password}" https://{website}.com/rest/v2/oauth/token
When I run the above command in a terminal it generates a token which I have successfully used in R using this code:
token <- {character string generated by the above curl command}
call <- "https://{website}.com/rest/v2/services/APIService/getRegistryRecordByAccessionID?id={id#}"
df <- as.data.frame(fromJSON(content(httr::GET(call, add_headers(Authorization = paste("Bearer", token))), "text"), flatten = TRUE))
All I want is to generate the token in R instead of using another program like I have been. This is where my errors come up. The basic format I believe is:
token <- oauth2.0_token(endpoint = oauth_endpoint(
authorize = NULL,
access = "https://{website}.com/rest/v2/oauth/token"),
app = oauth_app(), #not sure how to set this up
use_basic_auth = T,
add_headers(Content-type = "application/x-www-form-urlencoded",
Authorization = "Basic xxxyyy123xxx==") #not sure if i need the authorization header here, I believe I need the content-type heder
)
From MrFlick's comments above, we can integrate the curl auhtorization commands into a POST using the httr package. We then look at the content of the object this returns which will include the access token in the first item in the list.
token <- content(httr::POST("https://{website}.com/rest/v2/oauth/token", body=list(grant_type="password", "username="{username}", password="{password}"), encode="form"))[[1]]
Related
I am trying to adapt my curl request from running it in Terminal to R. For this I am using the httr package. My request is as follows and works from Terminal
curl -u user#email.com:password -H "Content-Type:application/json" -X POST "https://catalogue.onda-dias.eu/dias-catalogue/Products(48809a01-71bc-4669-8639-f0528abcdafe)/Ens.Order"
My attempt to use httr POST function is as follows:
pars <- list(username='user#email.com', password='password')
res<-POST("https://catalogue.onda-dias.eu/dias-catalogue/Products(48809a01-71bc-4669-8639-f0528abcdafe)/Ens.Order", body = pars, add_headers('Content-Type'='application/json'))
I am getting a 401 error code. Do you know how should I properly convert my request?
Try using httr:authenticate
require(httr)
headers = c('Content-Type' = 'application/json')
r <- httr::POST( url = 'https://catalogue.onda-dias.eu/dias-catalogue/Products(48809a01-71bc-4669-8639-f0528abcdafe)/Ens.Order'
, httr::add_headers(.headers=headers)
, httr::authenticate('user#email.com', 'password'))
I am trying to get data from Amazon MWS API using GetMatchingProductForId operation.
When I use Amazon MWS Scratchpad it works perfectly fine.
I am now trying to replicate the urls that are sent in the HTTP POST request but I get a Signature error message.
I need to understand how the url request should be structured.
Below is the detail of the request in Amazon MWS Scratchpad, I ANONYMIZED private identifiers but that is the only thing I changed:
HTTP POST
POST /Products/2011-10-01?AWSAccessKeyId=ANONYMIZED
&Action=GetMatchingProductForId
&SellerId=ANONYMIZED
&SignatureVersion=2
&Timestamp=2018-09-28T05%3A45%3A43Z
&Version=2011-10-01
&Signature=ANONYMIZED
&SignatureMethod=HmacSHA256
&MarketplaceId=A13V1IB3VIYZZH
&IdType=EAN
&IdList.Id.1=9781933988665 HTTP/1.1
Host: mws.amazonservices.fr
x-amazon-user-agent: AmazonJavascriptScratchpad/1.0 (Language=Javascript)
Content-Type: text/xml
String to Sign
POST
mws.amazonservices.fr
/Products/2011-10-01
AWSAccessKeyId=ANONYMIZED&Action=GetMatchingProductForId&IdList.Id.1=9781933988665&IdType=EAN&MarketplaceId=A13V1IB3VIYZZH&SellerId=ANONYMIZED&SignatureMethod=HmacSHA256&SignatureVersion=2&Timestamp=2018-09-28T05%3A45%3A43Z&Version=2011-10-01
=======
Now my question is, (and let's imagine my signature was created correctly), from the HTTP POST, what should the request look like ?
Here is my attempt:
https://mws.amazonservices.fr/Products/2011-10-01?AWSAccessKeyId=ANONYMIZED&Action=GetMatchingProductForId&SellerId=ANONYMIZED&SignatureVersion=2&Timestamp=2018-09-28T05%3A52%3A33Z&Version=2011-10-01&Signature=ANONYMIZED&SignatureMethod=HmacSHA256&MarketplaceId=A13V1IB3VIYZZH&IdType=EAN&IdList.Id.1=9781933988665
But what about '\n' escape characters that are in the scratchpad ? and what about 'HTTP/1.1' at the end, should I include that also ?
Thanks for your help.
I don't have an MWS account so I can't test the following, but this is one way you can do it:
# set this to your python2 binary; you'll need to do
# pip2 install boto
# from a command-line before using this code
Sys.setenv("RETICULATE_PYTHON"="/usr/bin/python2.7")
library(reticulate)
boto_mws_connection <- import("boto.mws.connection")
con <- boto_mws_connection$MWSConnection(
aws_access_key_id = ACCESS_KEY
aws_secret_access_key = AWS_SECRET
Merchant = MERCHANT_ID
)
con$get_matching_product_for_id(
MarketplaceId = "A13V1IB3VIYZZH",
IdType = "EAN",
IdList = c("9781933988665")
)
The HTTP/1.1 is usually created by your http client library. I am not familiar with R, but I googled and it seems there is a CURL package for R. CURL is the standard http library for a lot of languages including PHP. My PHP code to send an XML feed through curl looks like this:
$ch = curl_init();
curl_setopt($ch,CURLOPT_URL, 'https://mws.amazonservices.fr/Products/2011-10-01?.....your data and signature here...');
curl_setopt($ch,CURLOPT_POST, 1);
curl_setopt($ch,CURLOPT_RETURNTRANSFER,1);
curl_setopt($ch,CURLOPT_POSTFIELDS, $xmlcontent);
curl_setopt($ch,CURLOPT_HTTPHEADER, array(
"Content-Type: text/xml",
"Content-MD5: ".base64_encode(md5($xmlcontent,true)),
"x-amazon-user-agent: TestScript/0.01")
);
$result = curl_exec($ch);
curl_close($ch);
By looking at this, it seems to me this should be easily translatable to the R interface for CURL.
Artifactory OSS
5.4.6 rev 50406900
Trying to get access token to work.
I created token...
e.g. curl -uadmin:adminpw -X POST "myserver:8081/artifactory/api/security/token" -d "username=moehoward"
Returned msg looks like success...
{
"scope" : "member-of-groups:readers api:*",
"access_token" : <very-long-string>
"expires_in" : 3600,
"token_type" : "Bearer"
}
I can see it in the gui (admin -> Security -> Access Tokens) with "Subject" = to the user ("moehoward" in the example above) and with a "Token ID" that's a lot shorter, something like...
f2eb693a-d4ff-4618-ba52-764dc975c497
To test, I tried to ping using example in the docs...
curl -umoehoward:<very-long-string> myserver:8081/artifactory/api/system/ping
Fails with a 401 (bad credentials).
I replace the token with the "token id" I see in the gui, same result.
I replace again with the hardcoded pw of the "moehoward" user and that works (responds with "OK").
I tried the "-H"Authentication: Bearer " approach using the long string and that worked. So I guess the very long string is the token and not the "Token ID" in the gui.
Q: Any idea why this works for Bearer" and not the user by name ?
So you are right that this is supposed to work for both standard authentication and the Authentication HTTP header.
I did the test on a server with the same version Artifactory OSS 5.4.6 and the following works fine here
Inject the proper variables
export SERVER=server-artifactory
export APIKEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Create and use an access token for user moehoward
curl -u "admin:$APIKEY" -X POST "http://$SERVER/artifactory/api/security/token" -d "username=moehoward" -d "scope=member-of-groups:readers" > token.log
export TOKEN=`cat token.log | jq -r '.access_token'`
curl -u "moehoward:$TOKEN" "http://$SERVER/artifactory/api/system/ping"
curl -H "Authorization: Bearer $TOKEN" "http://$SERVER/artifactory/api/system/ping"
I get "OK" from the last two commands. Can you run exactly these commands and report back?
I have personally experienced the same problem (Bearer header working, standard user credentials not working) with an old version of curl. The obvious workaround is to use Bearer, the more complex workaround is to upgrade curl (or use another tool).
What is the version of curl you use? Any curl from 2015 or more recent should work fine.
I am able to get a API authentication token using the curl command on cygwin terminal on windows
curl -X POST -d '{"username":"usenamexxx","password":"passwordxxx"}' https://demo.website.com/api/token
output looks like
{"token":"87JrlYIOQa6g6aXciJOxNUK81","_links":{"DiscoveryPage":{"title":"Available API Resources","href":"https://demo.website.com/api"}}}
but when I try the POST command from R using the httr package I get the following error
library(httr)
tokenURL <- "https://demo.website.com/api/token"
s <- POST(tokenURL,body = list(username = "usenamexxx",password = "passwordxxx" ))
Error in function (type, msg, asError = TRUE) :
Unknown SSL protocol error in connection to demo.website.com:443
What am I doing wrong. Is there a different way to use POST?
Sorry I cannot provide a reproducible example as I cannot share my authentication details. If there is a way to provide more details please let me know.
This is a followup question to RCurl getURL with loop - link to a PDF kills looping :
I have the following getURL command:
require(RCurl)
#set a bunch of options for curl
options(RCurlOptions = list(cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")))
agent="Firefox/23.0"
curl = getCurlHandle()
curlSetOpt(
cookiejar = 'cookies.txt' ,
useragent = agent,
followlocation = TRUE ,
autoreferer = TRUE ,
httpauth = 1L, # "basic" http authorization version -- this seems to make a difference for India servers
curl = curl
)
x = getURLContent('http://timesofindia.indiatimes.com//articleshow/2933019.cms')
class(x)
#[1] "character"
attr(x, "Content-Type")
#"text/plain"
In a browser, the link above ends up redirecting to:
x = getURLContent('http://timesofindia.indiatimes.com/photo.cms?msid=2933009')
class(x)
#[1] "raw"
attr(x, "Content-Type")
#"application/pdf"
Assuming I know only the first link, how can I detect that the final location of the redirect (or redirects) is of a certain type (in this case PDF)?
Thanks!!
Maybe there's a better solution, but one way could be this:
# ...
h <- basicTextGatherer()
x = getBinaryURL('http://timesofindia.indiatimes.com//articleshow/2933019.cms',
headerfunction = h$update, curl = curl)
r <- gregexpr("Content-Type:.*?\n", h$value())
tail(regmatches(h$value(), r)[[1]], 1)
# [1] "Content-Type: application/pdf\r\n"
I have run into a similar issue trying to run getURLContent using digest authentication to get at binary data (using a non-standard mime type). I am running RCurl v1.95-4.1 on R 2.15.3.
If I run getURLContent without the binary=TRUE flag, it won't autoswitch to binary=TRUE because of the mime header for this data type, so it attempts to perform a rawToChar() and throws a 'embedded NULL in string' error. However the authentication does work.
If I add a binary=TRUE flag to the getURLContent call, it seems to cause issues with the authentication step, since I then get an 'Error: Unauthorized' response.
What finally worked was to replace getURLContent() with getBinaryURL() [as in the example above], which allowed the userpwd="u:p" authorization to work and delivered the binary data to my assigned object.
I think that the author of RCurl has made improvements to getURLContent's handling of binary data for v1.97, based on what I see in GitHub, so this may become a thing of the past...except for those of us still running older R setups.