I'm using R to make a call to the Amazon MWS API and get the following error:
The request signature we calculated does not match the signature you
provided. Check your AWS Secret Access Key and signing method. Consult
the service documentation for details.
This post helped me a lot with the Product Advertising API. However, I cannot seem to make it work on the MWS side.
Here is my code:
library(digest)
library(RCurl)
base.html.string <- "https://mws.amazonservices.com/Products/2011-10-01?"
SellerID <- 'A2UZXXXXXXXXXX'
MWSAuthToken <- 'ATVPXXXXXXXXX'
MarketplaceID <- 'ATVPXXXXXXXXX'
AWSAccessKeyId <- 'AKIAXXXXXXXXXXXXXXXX'
AWSsecretkey <- 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
ActionType <- 'GetMyPriceForASIN'
version.request = '2011-10-01'
ASINList.ASIN.1 <- 'B00XXXXXXX'
pb.txt <- Sys.time()
pb.date <- as.POSIXct(pb.txt, tz = Sys.timezone)
Timestamp = strtrim(format(pb.date, tz = "GMT", usetz = FALSE, "%Y-%m-%dT%H:%M:%SZ"), 24)
str = paste('POST\nmws.amazonservices.com\n/Products/2011-10-01\n',
'ASINList.ASIN.1=', ASINList.ASIN.1,
'&AWSAccessKeyId=', AWSAccessKeyId,
'&Action=', ActionType,
'&MWSAuthToken=', MWSAuthToken,
'&MarketplaceId=', MarketplaceID,
'&SellerId=', SellerID,
'&SignatureMethod=HmacSHA256',
'&SignatureVersion=2',
'&Timestamp=', gsub('%2E','.',gsub('%2D', '-', curlEscape(Timestamp))),
'&Version=', version.request,
sep = '')
## signature test
Signature = curlEscape(base64(hmac(enc2utf8(AWSsecretkey), enc2utf8(str), algo = 'sha256', serialize = FALSE, raw = TRUE)))
AmazonURL <- paste(base.html.string,
'ASINList.ASIN.1=', ASINList.ASIN.1,
'&AWSAccessKeyId=', AWSAccessKeyId,
'&Action=', ActionType,
'&MWSAuthToken=', MWSAuthToken,
'&MarketplaceId=', MarketplaceID,
'&SellerId=', SellerID,
'&SignatureMethod=HmacSHA256',
'&SignatureVersion=2',
'&Timestamp=', Timestamp,
'&Version=', version.request,
'&Signature=', Signature,
sep = '')
AmazonResult <- getURL(AmazonURL)
I'm using the Amazon MWS Scratchpad and made sure my string to sign matches.
My secret key does contain +'s, but I thought encoding would fix that.
Any help would be appreciated!
I figured out my problem after reading this post. I took Amazon's examples literally and used a POST instead of GET. I also had tweaked unnecessarily with my time stamp calculation, which I fixed as well. Hope this helps someone down the road.
Related
I am trying to use the recently released AWS Lambda HTTPS endpoints feature: https://aws.amazon.com/blogs/aws/announcing-aws-lambda-function-urls-built-in-https-endpoints-for-single-function-microservices
I have created an AWS Lambda function called multiply that expects an x and a y in the request body and returns the product. I created the HTTPS endpoint and set it to IAM authentication. Now, I'd like to call it from R with help of the aws.signature package.
The expected result is an HTTP response containing the Lambda function's output.
But I run into error "400 Bad Request". I suspect that the action parameter for aws.signature::signature_v4_auth must be set, but I can't figure out what it should be.
library(httr)
library(jsonlite)
function_url <- "https://<url-id>.lambda-url.eu-central-1.on.aws"
region <- "eu-central-1"
request_body <- toJSON(list(x = 2, y = 4), auto_unbox = TRUE)
headers <- list()
headers[["x-amz-date"]] <- format(Sys.time(), "%Y%m%dT%H%M%SZ", tz = "UTC")
headers[["host"]] <- function_url
headers[["content-type"]] <- "application/json"
signature <- aws.signature::signature_v4_auth(
datetime = headers[["x-amz-date"]],
region = region,
action = "",
service = "lambda",
verb = "POST",
canonical_headers = headers,
request_body = request_body,
key = "***",
secret = "***"
)
headers[["Authorization"]] <- signature[["SignatureHeader"]]
POST(
url = function_url,
body = request_body,
do.call(add_headers, headers)
)
There are two errors in the request:
The host should not include "https://". So the correct host is <url-id>.lambda-url.eu-central-1.on.aws
An empty action is not permitted. Use "/" instead.
Using the RDCOMClient package in R, I am sending a series of automated emails from R. The emails need to be sent at a specific date and time, but I cannot figure out what elements in the outMail object to manipulate to send emails with delayed delivery.
Here's a simple example of what I am doing:
library(data.table)
library(RDCOMClient)
# table of emails, names and send times (stt)
test_emails = data.table( First.Name = c("Joe", "Brit", "Anton"),
email = c("Joe#fakeemail.com",
"Brit#fakeemail.com",
"Anton#fakeemail.com" ),
stt = structure(c(1602270000, 1603270000, 1602470000),
class = c("POSIXct", "POSIXt"), tzone = ""))
# access Outook
OutApp <- COMCreate("Outlook.Application")
# loop over table of names/emails
for(i in 1:test_emails[,.N]){
#standard setup:
outMail = OutApp$CreateItem(0)
outMail$GetInspector()
signature = outMail[["HTMLBody"]]
outMail[["To"]] = test_emails[i, email]
outMail[["subject"]] = "Subject line"
# example body that prints the time and date of when the email is sent
outMail[["HTMLBody"]] =
paste0("<p>Hi ", test_emails[i, First.Name], ",</p>",
"<p>As discussed please find attached the detailed test instructions (PDF) and data set (CSV file)",
"You received this email at ",
gsub('^0', '', format(econ_test[i, stt], '%I:%M %p')), ' on ',
format(econ_test[i, stt], '%A, %B %d'), "</p>",
'<p>', signature, '</p>')
# sends right now. How to delay this?
outMail$Send()
}
I've tried looking through the MailModule object list here, but I can't find anything to use - and I'm not sure how to inspect the outMail object created above (e.g., str() doesn't work on this kind of object).
Thanks.
*** Exact Solution Based on Accepted Answer ***
# Convert the POSIXct integers to MS's time units
gmt_diff <- 4 # EDT
defer_time <- as.numeric( as.Date( test_emails[i, stt]) ) + # convert to R's date values
(as.numeric(test_emails[i, stt]) - gmt_diff*3600) %% 86400/ 86400 - # convert the POSIXct time to fraction of a day
as.numeric(as.Date("1899-12-30")) # adjust for differences in origin
# Update the DeferredDeliveryTime MailItem
outMail[['DeferredDeliveryTime']] = defer_time
Set the MailItem.DeferredDeliveryTime property - https://learn.microsoft.com/en-us/office/vba/api/outlook.mailitem.deferreddeliverytime
You don't need the MailModule object. Take a look at the live Outlook objects with OutlookSpy (I am its author) - click the Item button on the OutlookSpy ribbon.
since afaik Outlook needs to be running (ie online and connected) for you to achieve delayed delivery so therefore your computer also being powered up and running - why not time your R script accordingly with eg taskscheduleR
So I need to get Traffic estimation for some keywords via Google Adwords. I see that Google has a traffic estimator service API: https://developers.google.com/adwords/api/docs/guides/traffic-estimator-service .
I'm using R, which has quite a comprehensive RAdwords package, https://jburkhardt.github.io/RAdwords/faq/ , but one with which I failed miserably to find weather it provides access to this particular Adwords API service.
So did anyone use R to get keyword data from Google adwords via Traffic estimator? Is it possible with the RAdwords packege or does it need to be done via classic scripting ?
Thanks in advance.
you can use rgoogleads package.
Documentation: https://selesnow.github.io/rgoogleads/docs/
See Keywords Planing Data section: https://selesnow.github.io/rgoogleads/docs/reference/index.html#section-keywords-planing-data
There's a R package called RAdwordsPlus, which is built on top of RAdwords that will do just that.
devtools::install_github("adviso/RAdwordsPlus")
library(RAdwordsPlus)
google_auth <- doAuth() # requires user interaction the first time
api_version <- "v201809"
customer_id <- "xxx-xxx-xxxx"
# Build keyword request
k <- keyword(
text = c("mars cruise", "cheap cruise", "cruise"),
match.type = c("BROAD", "PHRASE", "EXACT"),
type = "Keyword"
)
# Keyword estimate request
ker <- keyword.estimate.request(
keyword = k,
max.cpc = 5000000, # 1 million == one unit (microamaounts)
is.negative = FALSE
)
# AdGroupEstimateRequest
aer <- adgroup.estimate.request(ker)
# Criteria for CampaignEstimateRequest
cer_criteria <- vector("list", length = 2)
cer_criteria[[1]] <- as.criterion(id = "2826", type = "Location") # united kingdom
cer_criteria[[2]] <- as.criterion(id = "1000", type = "Language") # english
# CampaignEstimateRequest
cer <- campaign.estimate.request(aer, campaign.id = NULL, criteria = cer_criteria)
# Build the request
request <- traffic.estimator.request(cer)
# Download data from the API
r <- get.service(request = request,
cid = customer_id,
auth = google_auth,
api.version = api_version,
user.agent = "r-adwordsplus-test",
verbose = TRUE)
I'm trying to download all the posts from a facebook page through RFacebook, but when the page has an high number of posts (over 400 or so), the script stops, returning the error
"Error in callAPI(url = url, token = token) : An unknown error has occurred." at the line where I call the getPage.
library(Rfacebook)
library(stringr)
load("fb_oauth")
token=fb_oauth
page<-getPage("bicocca", token, n = 100000, since = NULL, until = NULL, feed = TRUE)
noSpaceMsg<-str_replace_all(page$message, "[\r\n]" , "")
output<-as.data.frame(cbind(page$from_name,page$id, noSpaceMsg, page$created_time, page$type, page$link, page$likes_count, page$comments_count, page$shares_count))
colnames(output)<-c("username","msgid", "message", "created_time", "type", "link", "likes", "comments", "shares")
write.csv(output, "bicocca.csv", row.names=FALSE)
Where is the problem? How can I fix it?
It seems to be a problem with the API, not with the R package. When I try to do the query in the Graph API Explorer here, I get an error too. No idea why.
One way around this is to query month by month, wrapping the getPage function in a try command:
page <- 'bicocca'
dates <- seq(as.Date("2010/10/01"), as.Date("2015/04/20"), by="month")
n <- length(dates)-1
df <- list()
for (i in 1:n){
cat(as.character(dates[i]), " ")
try(df[[i]] <- getPage(page, token, since=dates[i], until=dates[i+1]))
cat("\n")
}
df <- do.call(rbind, df)
This will not give you all the posts, but probably most of them.
I tried to pull my location data from openpaths.cc to use it with R.
The API uses OAuth and is documented here, however, it only provides an example in Python.
After looking around how to handle OAuth (which I am barely familiar with) in R, I found ROAuth, so I used the usage example provided as a basis.
According to the API-documentation, the endpoint for all requests is https://openpaths.cc/api/1, and I have my access key and access secret, so I naively plugged them in for cKey, cSecret, reqURL, accessURL, authURL, and testURL, but only got "bad request" as a result from the credentials$handshake() line.
reqURL <- "https://openpaths.cc/api/1"
accessURL <- "https://openpaths.cc/api/1"
authURL <- "https://openpaths.cc/api/1"
cKey <- "key"
cSecret <- "secret"
testURL <- "https://openpaths.cc/api/1"
credentials <- OAuthFactory$new(consumerKey=cKey,
consumerSecret=cSecret,
requestURL=reqURL,
accessURL=accessURL,
authURL=authURL,
needsVerifier=TRUE)
credentials$handshake()
## the GET isn’t strictly necessary as that’s the default
credentials$OAuthRequest(testURL, "GET")
While I feel like I have no idea what I'm doing, I at least verified that ROAuth is capable of using the HMAC-SHA1 method, wich is required by openpaths.
EDIT: I have ROAuth version 0.9.3 installed
EDIT2: After learning about httr, I thought this might be the appropriate library for the task, however I still could not produce any usable results, since the token creation via oauth1.0_token only lead to a Bad request again.
I think my primary problem is the lack of API documentation from openpaths.cc. With all these tools, I still have no idea how to properly use them.
Here is as far as I got. I receive a "400 Not Authorized", maybe this is due to the fact that my openpaths account is not connected to foursquare, maybe something is wrong with the code. Please try it out!
Required packages:
library(RCurl)
library(digest)
library(base64)
Some functions borrowed/adapted from ROAuth:
## Get a random sequence of characters.
## Nonce - number used only once.
genNonce <- function(len = 15L + sample(1:16, 1L)) {
els <- c(letters, LETTERS, 0:9, "_")
paste(sample(els, len, replace = TRUE), collapse = "")
}
## this function is derived from utils::URLencode
## Characters not in the unreserved character set ([RFC3986] section 2.3) MUST be encoded
## unreserved = ALPHA, DIGIT, '-', '.', '_', '~'
## cf. http://oauth.net/core/1.0/#encoding_parameters
encodeURI <- function(URI, ...) {
if (!is.character(URI)) {
URI
} else {
OK <- "[^-A-Za-z0-9_.~]"
x <- strsplit(URI, "")[[1L]]
z <- grep(OK, x)
if (length(z)) {
y <- sapply(x[z], function(x) paste("%", toupper(as.character(charToRaw(x))),
sep = "", collapse = ""))
x[z] <- y
}
paste(x, collapse = "")
}
}
## we escape the values of the parameters in a special way that escapes
## the resulting % prefix in the escaped characters, e.g. %20 becomes
## %2520 as %25 is the escape for %
## cf. http://tools.ietf.org/html/rfc5849#section-3.4.1.3.2
normalizeParams <- function(params, escapeFun) {
names(params) <- sapply(names(params), escapeFun, post.amp = TRUE)
params <- sapply(params, escapeFun, post.amp = TRUE)
## If two or more parameters share the same name, they are sorted by their value.
params <- params[order(names(params), params)]
return(paste(names(params), params, sep = "=", collapse = "&"))
}
## From Ozaki Toru's code at https://gist.github.com/586468
signWithHMAC <- function(key, data) {
blockSize <- 64
hashlength <- 20
innerpad <- rawToBits(as.raw(rep(0x36, blockSize)))
outerpad <- rawToBits(as.raw(rep(0x5C, blockSize)))
zero <- rep(0 ,64)
HexdigestToDigest <- function(digest) {
as.raw(strtoi(substring(digest, (1:hashlength)*2-1,
(1:hashlength)*2), base=16))
}
mac <- function(pad, text) {
HexdigestToDigest(digest(append(packBits(xor(key, pad)), text),
algo='sha1', serialize=FALSE))
}
if(nchar(key) >= 64) {
keyDigested <- digest(key, algo="sha1", serialize=FALSE)
key <- intToUtf8(strtoi(HexdigestToDigest(keyDigested), base=16))
}
key <- rawToBits(as.raw(append(utf8ToInt(key), zero)[1:blockSize]))
base64(mac(outerpad, mac(innerpad, charToRaw(data))))[1]
}
## Sign an request made up of the URL, the parameters as a named character
## vector the consumer key and secret and the token and token secret.
signRequest <- function(uri, consumerKey, consumerSecret, params=character(),
oauthKey = "", oauthSecret = "", httpMethod = "GET",
nonce = genNonce(),
timestamp = Sys.time()) {
httpMethod <- toupper(httpMethod)
params["oauth_nonce"] <- nonce
params["oauth_timestamp"] <- as.integer(timestamp)
params["oauth_consumer_key"] <- consumerKey
params["oauth_signature_method"] <- 'HMAC-SHA1'
params["oauth_version"] <- '1.0'
if(oauthKey != "") params["oauth_token"] <- oauthKey
odat <- paste(
encodeURI(httpMethod), encodeURI(uri),
encodeURI(normalizeParams(params, encodeURI), post.amp = TRUE),
sep = "&"
)
okey <- encodeURI(consumerSecret)
if(oauthSecret != "") okey <- paste(okey, encodeURI(oauthSecret), sep = "&")
params["oauth_signature"] <- signWithHMAC(okey, odat)
return(params)
}
Now this function tries to replicate the example at the openpaths website:
openpaths <- function(
access_key=getOption("openpaths.access_key"),
secret_key=getOption("openpaths.secret_key"),
curl=getCurlHandle()) {
uri <- 'https://openpaths.cc/api/1'
params <- signRequest(uri, consumerKey=access_key, consumerSecret=secret_key)
oa_header <- paste(names(params), params, sep="=", collapse=",")
ret <- getURL(
uri,
curl=curl,
.opts=list(
header=TRUE,
verbose=TRUE,
httpheader=c(Authorization=paste("OAuth ", oa_header, sep="")),
ssl.verifypeer = TRUE,
ssl.verifyhost = TRUE,
cainfo = system.file("CurlSSL", "cacert.pem", package = "RCurl")
)
)
return(ret)
}
I've made some progress on this problem, although it's challenging due
to the flakiness of the site, and the custom OAuth process that they're
using. First you'll need to install development version of httr - this
exports some previously internal functions.
devtools::install_github("hadley/httr")
OpenPaths is unusual in that the app secret and key are the same as the
token and token secret. This means we need to write a custom auth
header:
library(httr)
app <- oauth_app("OpenPaths", "JSLEKAPZIMFVFROHBDT4KNBVSI")
#> Using secret stored in environment variable OPENPATHS_CONSUMER_SECRET
# Implement custom header for 2-leg authentication, and oauth_body_hash
auth_header <- function(url, method = "GET") {
oauth_signature(url, method, app, app$key, app$secret,
# Use sha1 of empty string since http request body is empty
body_hash = "da39a3ee5e6b4b0d3255bfef95601890afd80709")
}
Then you can use this to sign your request. This is currently failing
for me because the site seems to be down (again).
url <- "https://openpaths.cc/api/1"
r <- GET(url, oauth_header(auth_header(url)))
stop_for_status(r)
content(r)