According to the openCPU documentation, there are some default HTTP status codes and return types for a few situations. For example, when R raises an error, openCPU returns code 400 with a response type of text/plain.
While I believe it should be possible to control those things, is it possible to customize any of those things directly from R? For example, what if I wanted to return a JSON for a specific error in my R function, with a status code 503?
You can change their R package behavior by forking opencpu or via a local copy i.e. not sure if package allows this like a functionality but the responses are configured in res.R
For e.g. this method in the link above uses 400 for error.
error <- function(msg, status=400){
setbody(msg);
finish(status);
}
I will update the answer if I can confirm this is available without changing package code.
UPDATE 17-04-2021
You can write your serving html i.e. index.html which uses opencpu.js to call the corresponding R functions from your app, the return type can be requested to be json in the opencpu.js call. And in the R function , you can tryCatch() errors to send appropriate error code as a json argument.
For e.g. in the stock example app you can see the file stock.js which calls the functions from R folder i.e.
//this function gets a list of stocks to populate the tree panel
function loadtree(){
var req = ocpu.rpc("listbyindustry", {}, function(data){
Ext.getCmp("tree-panel").getStore().setProxy({
type : "memory",
data : data,
reader : {
type: "json"
}
});
Ext.getCmp("tree-panel").getStore().load();
}).fail(function(){
alert("Failed to load stocks: " + req.responseText);
});
}
The corresponding R code being called is in listbyindustry.R, inside which you can tryCatch() and send custom json.
Related
My use case is simple: I have a Postman Collection with a few requests, one of them is:
GET www.example.com/stores?country={{country}}
Then a simple Test:
pm.test("Check number of stores", function () {
var jsonData = pm.response.json();
pm.expect(jsonData.stores.length).to.equal(pm.iterationData.get("size"));
});
So everything is nice and merry with the following Collection data used in the Collection Runner:
country,size
UK,15
US,32
However when I simply want to run this via the main Postman window, obviously the request and the Test fails. I can set a collection variable country to SE, but I have no idea how to set size in pm.iterationData just to try if my request and test script is working fine for another "country" - without running the whole collection/iterations.
Thanks in advance for all the help!
I'm not sure if you can modify iteration data variable in runtime, but a workaround for this issue is to copy your request from the original folder into a new folder, then you can run the whole folder with only one request, you dun have then to run all the requests within the collection.
I am using the Zomato API with R's httr package to get restaurant data in JSON. When I call the Zomato API using the following code block, I get some records back and the GET() call seems to work fine.
require(httr)
URL <- 'https://developers.zomato.com/api/v2.1/search?'
request <- GET(URL,
add_headers(User_key="#######_MY_API_KEY##############"),
query=list(entity_id = '94753',
entity_type = 'zone'))
content(request)
In the next code block, I make one modification to the code in order to do a text search. This API call returns 0 records, but it should return 2 - I know this because Zomato has an API testing tool at https://developers.zomato.com/documentation, which allows you to (among other things) enter your API key and some attribute values to generate API calls, and see the records returned.
I cannot understand why the following GET() returns 0 records. Suggestions?
request <- GET(URL,
add_headers(User_key="#######_MY_API_KEY##############"),
query=list(entity_id = '94753',
entity_type = 'zone',
q = 'border')) #added parameter and value
content(request)
After reloading the R environment and running the script from scratch, it appears to work fine - the Get() call in the second block retrieves the 2 records that it should. I suspect that there was something I had set in the environment that was messing it up - perhaps epi99 was right, but just about another typo.
I have a Dynamodb table with a few fields - my_id is the PrimaryKey. In the API gateway I set up a response with a method that takes in a parameter {my_id}.
Then I have an Integration Request mapping template that takes the passed in parameter and queries the table to return all the fields that match.
Then I have an Integration response mapping template that cleans up the returned items the way I want.
This all works perfect.
The thing I can't figure out how to do is if the parameter that is passed in doesn't match anything in the table, how do I get it to change from a 200 status into a 404?
From what I can tell when the passed in parameter doesn't match anything it doesn't cause an error, it just doesn't return anything.
It seems like I need to change the mapping template on the Integration response to first check if the params are empty and then somehow tell it to change the response status.
I can find info about this type of thing with people using Lambda, but I am not using Lambda - just the Dynamodb table and the API Gateway.
You can use Mapping Template to convert the response that you get from DDB and overrride the response code. You can get more details in the link https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-override-request-response-parameters.html
If you are using cloud formation, you can do this by using below snippet
IntegrationResponses:
- StatusCode: "200"
ResponseTemplates:
application/json: |
{
"payload" : {
}
},
}
IntegrationResponses:
- StatusCode: "200"
ResponseTemplates:
application/json: |
#set($inputRoot = $input.path('$'))
#if($inputRoot.toString().contains("Item"))
$input.json("$")
#set($context.responseOverride.status = 200)
#else
#set($context.responseOverride.status = 404)
#end
Api gateway currently supports mapping the status code using the status code of the integration response (Here dynamodb response code). The only workaround is to use a lambda function which outputs different error messages that can be mapped using a error regex http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-method-settings-execution-console.html.
I am attempting to return an object from a AWS Lambda function instead of a simple string.
// ...
context.fail({
"email": "Email address is too short",
"firstname": "First name is too short"
});
// ...
I have already used the errorMessage for mapping error responses to status codes and that has been great:
// ...
context.fail('That "username" has already been taken.');
// ...
Am I simply trying to do something that the AWS API Gateway does not afford?
I have also already found this article which helped: Is there a way to change the http status codes returned by Amazon API Gateway?.
Update
Since time of writing, lambda has updated the invocation signature and now passes event, context, callback.
Instead of calling context.done(err, res) you should use callback(err, res). Note that what was true for context.done still applies to the callback pattern.
Should also add that with API Gateways proxy and integration implementation this entire thread is pretty much obsolete.
I recommend reading this article if you are integrating API Gateway with Lambda: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html
Original response below
First things first, let's clear a few things up.
context.done() vs. context.fail()/context.success
context.done(error, result); is nothing but a wrapper around context.fail(error); and context.success(response);
The Lambda documentation clearly states that result is ignored if error is non null:
If the Lambda function was invoked using the RequestResponse (synchronous) invocation type, the method returns response body as follows:
If the error is null, set the response body to the string representation of result. This is similar to the context.succeed().
If the error is not null, set the response body to error.
If the function is called with a single argument of type error, the error value will be populated in the response body.
http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html
What this means is that it won't matter whether you use a combination of fail/success or done, the behaviour is exactly the same.
API Gateway and Response Code Mapping
I have tested every thinkable combination of response handling from Lambda in combination with Response code mapping in API Gateway.
The conclusion of these tests are that the "Lambda Error RegExp" is only executed against a Lambda error, i.e: you have to call context.done(error);or context.fail(error); for the RegExp to actually trigger.
Now, this presents a problem as, has already been noted, Lambda takes your error and sticks it in an object and calls toString() on whatever you supplied:
{ errorMessage: yourError.toString() }
If you supplied an error object you'll get this:
{ errorMessage: "[object Object]" }
Not very helpful at all.
The only workaround I have found thus far is to call
context.fail(JSON.stringify(error));
and then in my client do:
var errorObject = JSON.parse(error.errorMessage);
It's not very elegant but it works.
As part of my error I have a property called "code". It could look something like this:
{
code: "BadRequest",
message: "Invalid argument: parameter name"
}
When I stringify this object I get:
"{\"code\":\"BadRequest\",\"message\":\"Invalid argument: parameter name\"}"
Lambda will stick this string in the errorMessage property of the response and I can now safely grep for .*"BadRequest".* in the API Gateway response mapping.
It's very much a hack that works around two somewhat strange quirks of Lambda and API Gateway:
Why does Lambda insist on wrapping the error instead of just giving
it back as is?
Why doesn't API Gateway allow us to grep in the
Lambda result, only the error?
I am on my way to open a support case with Amazon regarding these two rather odd behaviours.
You don't have to use context.fail, use success but send different statusCode and an errorMessage, here is an example of how i format my output:
try {
// Call the callable function with the defined array parameters
// All the function called here will be catched if they throw exceptions
result.data = callable_function.apply(this, params);
result.statusCode = 200;
result.operation = operation;
result.errorMessage = ""
} catch (e) {
result.data = [];
result.statusCode = 500;
result.errorMessage = e.toString();
result.method = method;
result.resource = resource;
}
// If everything went smooth, send back the result
// If context succeed is not called AWS Lambda will fire the function
// again because it is not successfully exited
context.succeed(result);
Use the consumer logic to handle different errors case logic, don't forget that you pay for the time your function is running...
You should replace the use of your context.fail with context.done and use context.fail only for very serious Lambda function failures since it doesn't allow more than one output parameter. Integration Response is able to match mapping template by performing regex on the first parameter passed to context.done this also maps HTTP status code to the response. You can't pass this response status code directly from Lambda since it's the role of API Gateway Integration Response to abstract the HTTP protocol.
See the following:
context.done('Not Found:', <some object you can use in the model>);
and the Integration Response panel this setting:
You can replicate similar approach for any kind of error. You should also create and map the error model to your response.
For those who tried everything put on this question and couldn't make this work (like me), check the thedevkit comment on this post (saved my day):
https://forums.aws.amazon.com/thread.jspa?threadID=192918
Reproducing it entirely below:
I've had issues with this myself, and I believe that the newline
characters are the culprit.
foo.* will match occurrences of "foo" followed by any characters
EXCEPT newline. Typically this is solved by adding the '/s' flag, i.e.
"foo.*/s", but the Lambda error regex doesn't seem to respect this.
As an alternative you can use something like: foo(.|\n)*
I'm having a doozie of a time trying to get a basic http test to work with vows.
I think I've followed the async example from vows http://vowsjs.org/#-writing-asynchronous-tests and substitued the appropriate calls, but I must be missing something.
The test code looks like this:
var http = require('http'),
vows = require('vows'),
assert = require('assert');
vows.describe("homepage").addBatch({
"Get the home page": {
topic: function() {
http.get({'host': "127.0.0.1", 'port': 5000, 'path': '/'}, this.callback);
},
'should respond with 200 OK': function(res) {
assert.equal(res.statusCode, 200);
}
}
}).export(module);
I get the following error when I try to run the test for this:
/Users/<home_folder>/node_modules/vows/lib/vows.js:80
rrored', { type: 'promise', error: err.stack || err.message || JSON.stringify(
^
TypeError: Converting circular structure to JSON
at Object.stringify (native)
at EventEmitter.<anonymous> (/Users/<home_folder>/node_modules/vows/lib/vows.js:80:90)
at EventEmitter.emit (events.js:64:17)
at /Users/<home_folder>/node_modules/vows/lib/vows/context.js:31:52
at ClientRequest.<anonymous> (/Users/<home_folder>/node_modules/vows/lib/vows/context.js:46:29)
at ClientRequest.g (events.js:143:14)
at ClientRequest.emit (events.js:64:17)
at HTTPParser.onIncoming (http.js:1349:9)
at HTTPParser.onHeadersComplete (http.js:108:31)
at Socket.ondata (http.js:1226:22)
I can get a simple http example to work on it's own. I can get the vows example to work on it's own but I can't combine them for whatever reason. I'd really appreciate some help here. I've been trying to get this to work for a while now (including much googling).
UPDATE:
Apparently adding an error argument to the call back solves this problem, thanks to help from Alexis Sellier (creator of vows).
But I have no idea why. When writing out the http lib example on it's own no error argument is required. I can't find any documentation in vows to indicate why it's needed so I'm at a bit of a loss.
My new question is why is the error argument required when using the http lib in vows?
After checking vow's source code, I think I know why. Vows always ensure that when you call this.callback, the resulting receiver function's first argument is always an error object. Vows interpret the callbacks by these rules:
If the first argument of your originating callback is a boolean, use that to determine whether or not to append an error object to the receiving callback (e.g. path.exists(boolean) will emit callback(error, exists) instead)
If the first argument is an object, assume it's an error object and use that to determine whether to add the originating callback to the "error" or "success" list. The reason this list exists is to support promise based tests I guess?
While I can't confirm the above is correct, my experience is that vows' async style is made to support node-styled callbacks (e.g. err as the first arg), and 3rd party npm modules that don't conform to this standard will be hard to test.
Please don't take my answer as gospel, as this is my own experience. Another gotcha is when you have async operations inside the function that you want to test - unless you provide a callback, vows won't be able to handle it properly.
Personally, I think vows still make it hard to test async code. I wish it had some waitFor() or until() flow control functions though.
My suggestion? When dealing with async code, use Step. Don't let vows control your flow.
It is actually missing in the documentations which is still a bit short. But you can get a glimpse of it here in this page :
'when peeled *asynchronously*': {
topic: function (banana) {
banana.peel(this.callback);
},
'results in a `PeeledBanana`': function (err, result) {
assert.instanceOf (result, PeeledBanana);
}
}
As it was said by Morten Siebuhr and Ruben Tan, this is how vows works and that is why it works like that.