How to properly make POST request - r

I'm trying to make my first POST request to an API. For some reason, I always get status 403 in return. I suspect it's the signature that is being incorrectly generated. The api-key and client id is for sure correct.
My code
nonce <-as.integer(Sys.time())
post_message <- paste0(nonce, data_client.id, data_key) # data_client.id = client id # data_key = key
sha.message <- toupper(digest::hmac(data_secret, object = post_message, algo = 'sha256', serialize = TRUE))
url <- 'https://www.bitstamp.net/api/v2/balance/'
body = list('API-KEY' = data_key, 'nonce' = nonce, 'signature' = sha.message)
httr::POST(url, body = body, verbose())
Output
<- HTTP/1.1 403 Authentication Failed
I'm trying to access the Bitstamp API: https://www.bitstamp.net/api/?package=Rbitcoin&version=0.9.2
All private API calls require authentication. For a successful
authentication you need to provide your API key, a signature and a
nonce parameter.
API KEY
To get an API key, go to "Account", "Security" and then "API Access".
Set permissions and click "Generate key".
NONCEN
once is a regular integer number. It must be increased with every
request you make. Read more about it here. Example: if you set nonce
to 1 in your first request, you must set it to at least 2 in your
second request. You are not required to start with 1. A common
practice is to use unix time for that parameter.
SIGNATURE
Signature is a HMAC-SHA256 encoded message containing nonce, customer
ID (can be found here) and API key. The HMAC-SHA256 code must be
generated using a secret key that was generated with your API key.
This code must be converted to it's hexadecimal representation (64
uppercase characters).

I'm not sure if your question is still standing, but based on your code, I managed to get it working. In fact, the main problem is in the body, the API documentation shows it expects 'key' instead of 'API-KEY'.
Also, serialize should be FALSE instead of TRUE.
At the moment this works (but the API may change):
nonce <-as.integer(Sys.time())
post_message <- paste0(nonce, data_client.id, data_key) # data_client.id = client id # data_key = key
sha.message <- toupper(digest::hmac(data_secret, object = post_message, algo = 'sha256', serialize = FALSE))
url <- 'https://www.bitstamp.net/api/v2/balance/'
body = list('key' = data_key, 'nonce' = nonce, 'signature' = sha.message)
httr::POST(url, body = body, verbose())

Related

Paginated API - R

With a paginated oAuth2.0 API, how do I use the response headers to view the next page?
So far I have successfully created an endpoint, app, token, and response:
# here's the key and username
API_KEY = 'my key'
API_USER = 'my username'
# set up the endpoint
api_endpoint= httr::oauth_endpoint(
authorize = "https://www.hydrovu.com/public-api/oauth/authorize",
access = "https://www.hydrovu.com/public-api/oauth/token"
)
# heres the app we will use to download data to
App = httr::oauth_app("xxx", key = API_USER, secret = API_KEY)
# heres the token we will use as a password
API.token = httr::oauth2.0_token(api_endpoint,
App,
scope = "https://www.hydrovu.com/public-api/oauth/token",
client_credentials=TRUE,
cache = FALSE,
config_init = user_agent("Testing Oauth with httr"))
# this is the response or the data
res <- httr::GET(url = "https://www.hydrovu.com/public-api/v1/locations/{id}/data?startTime=0",
user_agent("Testing Oauth with httr"),
config(token = API.token))
The response headers from res include next-page, which has a long string as the value. E.g. 6haGnbeLO09jlwe6poy7erT43qp
How to I use this next-page information to actually get the next page?
I am using the HydroVu API: https://www.hydrovu.com/public-api/docs/index.html
Let me know if you need more information!
After speaking to the excellent Hadley Wickham from httr maintenance, I have found the solution.
I was originally trying to pass x-isi-next-page to x-isi-next-page through add_headers() like so:
# original GET
res <- httr::GET(url,
user_agent("Testing Oauth with httr"),
config(token = API.token))
# write x-isi-next-page to object
nextpage = res$headers$`x-isi-next-page`
# pass to add_headers()
res <- httr::GET(url,
user_agent("Testing Oauth with httr"),
config(token = API.token),
add_headers(`x-isi-next-page` = nextpage))
When in fact I should have been passing it to x-isi-start-page
resp <- httr::GET(url,
user_agent("Testing Oauth with httr"),
config(token = API.token),
add_headers(`x-isi-start-page` = nextpage))
Thus, giving me the next page of results.
Now, to decode the json and create a compiled csv...

How do I implement my own token type for httr authentication?

I am using a web API that uses an authentication that is similar to, but not quite, OAuth 2.0. As a consequence, I would like to implement my own Token class for use by ‘httr’.
Unfortunate there are several issues:
The Token base class seems to be specific to OAuth and contains members/logic that don’t apply to me.
The documentation is incomplete and incorrect: at the very least it is missing the method for restoring a token from the cache, and it is also missing the specification for the other methods. It also specifies an incorrect return type for sign() (the calling code inside ‘httr’ expects a request S3 object).
To implement sign() correctly, I need to use an unexported function, httr:::request().
The expected usage for me would be as follows, given a valid token from a previous authorisation request:
httr::GET(endpoint_url, bearer_auth(token))
Where:
bearer_auth = function (token) {
httr::config(token = BearerToken$new(token))
}
This “works” when creating a minimal BearerToken R6 class that doesn’t inherit from httr::Token, implements only the sign() method, and cheats by accessing httr:::request():
BearerToken = R6::R6Class("BearerToken", list(
token = NULL,
initialize = function (token) {
self$token = token
},
sign = function (method, url) {
httr:::request(
method = method,
url = url,
headers = c(Authorization = paste("Bearer", self$token))
)
}
))
But that’s obviously insufficient for real usage.

The key specified is not a valid key for this encryption: Key size is not valid. Got key length of: 15

I keep getting an error message when trying to send the key through url with encodeforURL() and decodefromUrl(). The code example is below.
This is my entry page:
key = generateSecretKey(("AES"),128);
data = encrypt(serializeJSON(pg_info), key, "AES", "HEX");
location("home.cfm?str=#encodeForURL(key)#&dt=#data#", "false", "301");
This is my home page:
if ( structKeyExists(url, "str") ) {
key = DecodeFromURL(url.str);
strData = deserializeJSON(decrypt(url.dt, key, "AES", "HEX")); // This is the line where the error message is pointing
} else {
writeOutput("<p>Error! Please contact your administrator.</p>");
abort;
}
The code is very simple. When user gets to the entry page the data parameters are being encrypted and sent trhough url to home page. Once user gets to home page data is extracted from ul. I tried adding the size when creating the secret key (128) in hope that issue will be resolved. The error is still happening and it seems that might be related to something else. I though that key length is the issue, but the error message is pointing to the line of code where url string is being applied to deserializeJSON(). Is there a way to find out what is causing an error an how to fix this issue? Thank you.
BTW, I assume this code is just for testing purpose, since passing the encryption key alongside the encrypted text utterly and completely defeats the purpose of encryption ;-)
Is there a way to find out what is causing an error
With troubleshooting, location() tends to get in the way, so best to temporarily replace it with a hyperlink. Then you'll be able to output the original key generated and compare it to what's actually received on the home page.
Test Case (Single Page)
<cfscript>
// It make take a few executions to hit a failing key like `n+Py4flPF6uOwNXwpq2J4g==`.
pg_info = { "plain" : "text" };
key = "generateSecretKey(("AES"),128);
data = encrypt(serializeJSON(pg_info), key, "AES", "HEX");
writeOutput( "[key] "& key &"<br>[encoded] "& encodeForURL(key) &"<br><br>");
writeOutput( 'Test' );
if ( url.keyExists("str")) {
writeDump( var=[url.str], label="url.str (Original)" );
writeDump( var=[DecodeFromURL(url.str)], label="url.str (Decoded)" );
key = DecodeFromURL(url.str);
strData = deserializeJSON(decrypt(url.dt, key, "AES", "HEX"));
writeDump( var=strData, label="strData" );
}
</cfscript>
how to fix this issue?
CF already decodes url parameters automatically. So decoding url.str a second time alters the original key value, causing decrypt() to fail because the key is no longer valid. Notice with a failing key like n+Py4flPF6uOwNXwpq2J4g== the original url.str value differs from the decoded key?
url.str (Original) n+Py4flPF6uOwNXwpq2J4g== (has "+" char)
key (Decoded) n Py4flPF6uOwNXwpq2J4g== ("+" changes to space char)

Web2py: Sending JSON Data via a Rest API post call in Web2py scheduler

I have a form whose one field is type IS_JSON
db.define_table('vmPowerOpsTable',
Field('launchId',label=T('Launch ID'),default =datetime.datetime.now().strftime("%d%m%y%H%M%S")),
Field('launchDate',label=T('Launched On'),default=datetime.datetime.now()),
Field('launchBy',label=T('Launched By'),default = auth.user.email if auth.user else "Anonymous"),
Field('inputJson','text',label=T('Input JSON*'),
requires = [IS_NOT_EMPTY(error_message='Input JSON is required'),IS_JSON(error_message='Invalid JSON')]),
migrate=True)
When the user submits this Form, this data is also simultaneously inserted to another table.
db.opStatus.insert(launchId=vmops_launchid,launchDate=vmops_launchdate
,launchBy=vmops_launchBy,opsType=operation_type,
opsData=vmops_inputJson,
statusDetail="Pending")
db.commit()
Now from the scheduler, I am trying to retrieve this data and make a POST request.
vm_power_opStatus_row_data = vm_power_opStatus_row.opsData
Note in the above step I am able to retrieve the data. (I inserted it in a DB and saw the field exactly matches what the user has entered.
Then from the scheduler, I do a POST.
power_response = requests.post(vm_power_op_url, json=vm_power_opStatus_row_data)
The POST request is handled by a function in my controller.
Controller Function:
#request.restful()
def vmPowerOperation():
response.view = 'generic.json'
si = None
def POST(*args, **vars):
jsonBody = request.vars
print "Debug 1"+ str(jsonBody) ##-> Here it returns blank in jsonBody.
But if I do the same request from Outside(POSTMAN client or even python request ) I get the desired result.
Is anything going wrong with the data type when I am trying to fetch it from the table.
power_response = requests.post(vm_power_op_url,
json=vm_power_opStatus_row_data)
It appears that vm_power_opStatus_row_data is already a JSON-encoded string. However, the json argument to requests.post() should be a Python object, not a string (requests will automatically encode the Python object to JSON and set the content type appropriately). So, the above should be:
power_response = requests.post(vm_power_op_url,
json=json.loads(vm_power_opStatus_row_data))
Alternatively, you can use the data argument and set the content type to JSON:
power_response = requests.post(vm_power_op_url,
data=vm_power_opStatus_row_data,
headers={'Content-Type': 'application/json')
Also, note that in your REST POST function, request.vars is already passed to the function as **vars, so within the function, you can simply refer to vars rather than request.vars.

Convert string to map in lua

I am quite new to lua. I trying to convert a string of the form
{"result": "success", "data":{"shouldLoad":"true"}"}
into lua map. So that I can access it like json. e.g. someMap[data][shouldLoad] => true
I dont have any json bindings in lua. I also tried loadstring to convert string of the form {"result" = "success", "data"={"shouldLoad"="true"}"}, which is not working.
Following, is the code snippet, where I am calling getLocation hook, which in turn returns json stringified map. Now I want to access some keys from this response body and take some decisions accordingly.
access_by_lua "
local res = ngx.location.capture('/getLocation')
//res.body = {"result"= "success", "data" = {"shouldLoad" = "true"}}
local resData = loadstring('return '..res.body)()
local shoulLoad = resData['data']['shouldLoad']
"
When I try to load shouldLoad value, nginx error log reports error saying trying to index nil value.
How do I access key value with either of the string formats. Please help.
The best answer is to consider a pre-existing JSON module, as suggested by Alexey Ten. Here's the list of JSON modules from Alexey.
I also wrote a short pure-Lua json module that you are free to use however you like. It's public domain, so you can use it, modify it, sell it, and don't need to provide any credit for it. To use that module you would write code like this:
local json = require 'json' -- At the top of your script.
local jsonStr = '{"result": "success", "data":{"shouldLoad":"true"}"}'
local myTable = json.parse(jsonStr)
-- Now you can access your table in the usual ways:
if myTable.result == 'success' then
print('shouldLoad =', myTable.data.shouldLoad)
end

Resources