How to return error collection/object from AWS Lambda function and map to AWS API Gateway response code - http

I am attempting to return an object from a AWS Lambda function instead of a simple string.
// ...
context.fail({
"email": "Email address is too short",
"firstname": "First name is too short"
});
// ...
I have already used the errorMessage for mapping error responses to status codes and that has been great:
// ...
context.fail('That "username" has already been taken.');
// ...
Am I simply trying to do something that the AWS API Gateway does not afford?
I have also already found this article which helped: Is there a way to change the http status codes returned by Amazon API Gateway?.

Update
Since time of writing, lambda has updated the invocation signature and now passes event, context, callback.
Instead of calling context.done(err, res) you should use callback(err, res). Note that what was true for context.done still applies to the callback pattern.
Should also add that with API Gateways proxy and integration implementation this entire thread is pretty much obsolete.
I recommend reading this article if you are integrating API Gateway with Lambda: http://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-create-api-as-simple-proxy-for-lambda.html
Original response below
First things first, let's clear a few things up.
context.done() vs. context.fail()/context.success
context.done(error, result); is nothing but a wrapper around context.fail(error); and context.success(response);
The Lambda documentation clearly states that result is ignored if error is non null:
If the Lambda function was invoked using the RequestResponse (synchronous) invocation type, the method returns response body as follows:
If the error is null, set the response body to the string representation of result. This is similar to the context.succeed().
If the error is not null, set the response body to error.
If the function is called with a single argument of type error, the error value will be populated in the response body.
http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html
What this means is that it won't matter whether you use a combination of fail/success or done, the behaviour is exactly the same.
API Gateway and Response Code Mapping
I have tested every thinkable combination of response handling from Lambda in combination with Response code mapping in API Gateway.
The conclusion of these tests are that the "Lambda Error RegExp" is only executed against a Lambda error, i.e: you have to call context.done(error);or context.fail(error); for the RegExp to actually trigger.
Now, this presents a problem as, has already been noted, Lambda takes your error and sticks it in an object and calls toString() on whatever you supplied:
{ errorMessage: yourError.toString() }
If you supplied an error object you'll get this:
{ errorMessage: "[object Object]" }
Not very helpful at all.
The only workaround I have found thus far is to call
context.fail(JSON.stringify(error));
and then in my client do:
var errorObject = JSON.parse(error.errorMessage);
It's not very elegant but it works.
As part of my error I have a property called "code". It could look something like this:
{
code: "BadRequest",
message: "Invalid argument: parameter name"
}
When I stringify this object I get:
"{\"code\":\"BadRequest\",\"message\":\"Invalid argument: parameter name\"}"
Lambda will stick this string in the errorMessage property of the response and I can now safely grep for .*"BadRequest".* in the API Gateway response mapping.
It's very much a hack that works around two somewhat strange quirks of Lambda and API Gateway:
Why does Lambda insist on wrapping the error instead of just giving
it back as is?
Why doesn't API Gateway allow us to grep in the
Lambda result, only the error?
I am on my way to open a support case with Amazon regarding these two rather odd behaviours.

You don't have to use context.fail, use success but send different statusCode and an errorMessage, here is an example of how i format my output:
try {
// Call the callable function with the defined array parameters
// All the function called here will be catched if they throw exceptions
result.data = callable_function.apply(this, params);
result.statusCode = 200;
result.operation = operation;
result.errorMessage = ""
} catch (e) {
result.data = [];
result.statusCode = 500;
result.errorMessage = e.toString();
result.method = method;
result.resource = resource;
}
// If everything went smooth, send back the result
// If context succeed is not called AWS Lambda will fire the function
// again because it is not successfully exited
context.succeed(result);
Use the consumer logic to handle different errors case logic, don't forget that you pay for the time your function is running...

You should replace the use of your context.fail with context.done and use context.fail only for very serious Lambda function failures since it doesn't allow more than one output parameter. Integration Response is able to match mapping template by performing regex on the first parameter passed to context.done this also maps HTTP status code to the response. You can't pass this response status code directly from Lambda since it's the role of API Gateway Integration Response to abstract the HTTP protocol.
See the following:
context.done('Not Found:', <some object you can use in the model>);
and the Integration Response panel this setting:
You can replicate similar approach for any kind of error. You should also create and map the error model to your response.

For those who tried everything put on this question and couldn't make this work (like me), check the thedevkit comment on this post (saved my day):
https://forums.aws.amazon.com/thread.jspa?threadID=192918
Reproducing it entirely below:
I've had issues with this myself, and I believe that the newline
characters are the culprit.
foo.* will match occurrences of "foo" followed by any characters
EXCEPT newline. Typically this is solved by adding the '/s' flag, i.e.
"foo.*/s", but the Lambda error regex doesn't seem to respect this.
As an alternative you can use something like: foo(.|\n)*

Related

Firebase + DialogFlow - Realtime database- Cloud function return query result only after second request

I'm writing a simple Google Action which will read the Firebase Realtime Database, and return result in the response. My problem is, the query result is being passed back in response to DialogFlow only after at least 2 attempts.
Below the screenshots showing the end result in the Simulator
First query screenshot
The first line of the response is returned from the Cloud Function, and contains values passed with the "Context". There is no second line in this response.
below is the screen showing the result after sending exactly the same request second time.
Second query screenshot
First line is the same as previously, but this time I also get the second line which contains the query result data.
It looks like my code is "working" (I get the correct data from the database), but for some reason it only works if I trigger it at least 2 times in quick succession.
Below is the code snipped which handle this request:
function googleAssistantHandler(agent) {
let conv = agent.conv();
let outCommandContext = agent.getContext('outcommand');
let outCharacterContext = agent.getContext('outcharacter');
let character = outCharacterContext.parameters.character;
let command = outCommandContext.parameters.command;
agent.add('<prosody rate="140%" pitch="0.4">' + character +' '+ command +'</prosody>');
var movesRef = admin.database().ref('characters/'+character.toLowerCase()+'/moves/');
movesRef.limitToFirst(1).orderByChild("notation")
.equalTo(command.toString()).on("child_added",function(snapshot){
agent.add(`record number is ` + snapshot.key);
});
}
I've tried using once() instead of on() (as it would make more sense in my case... i don't need to listen to changes on the database, i just want to retrieve data once)- but, I couldn't get it to work.
Can you guys help me out understanding why my query returns result only after the second trigger?
Thanks!
you are using a callback method to get the data from database so there is no guaranty that it will be called before your function is returned. to solve the issue, you need to use a Promise and return that Promise in your function so the last few lines of your function will look like this
return movesRef.limitToFirst(1).orderByChild("notation")
.equalTo(command.toString()).on("child_added").then(snapshot= > {
agent.add(`record number is ` + snapshot.key);
});
You need to always use promises when working with databases. Moreover, the first response that you see might be because of the failed function which timed out. If you see your console logs in firebase, you might see the errors. Also check your default response, if it has the text that User said $name or something similar, then that is what causes the issue in the first attempt.
If you still don't get it to work, try logging the returned data and post your logs here.

AWS API Gateway - change to 404 if query returns nothing

I have a Dynamodb table with a few fields - my_id is the PrimaryKey. In the API gateway I set up a response with a method that takes in a parameter {my_id}.
Then I have an Integration Request mapping template that takes the passed in parameter and queries the table to return all the fields that match.
Then I have an Integration response mapping template that cleans up the returned items the way I want.
This all works perfect.
The thing I can't figure out how to do is if the parameter that is passed in doesn't match anything in the table, how do I get it to change from a 200 status into a 404?
From what I can tell when the passed in parameter doesn't match anything it doesn't cause an error, it just doesn't return anything.
It seems like I need to change the mapping template on the Integration response to first check if the params are empty and then somehow tell it to change the response status.
I can find info about this type of thing with people using Lambda, but I am not using Lambda - just the Dynamodb table and the API Gateway.
You can use Mapping Template to convert the response that you get from DDB and overrride the response code. You can get more details in the link https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-override-request-response-parameters.html
If you are using cloud formation, you can do this by using below snippet
IntegrationResponses:
- StatusCode: "200"
ResponseTemplates:
application/json: |
{
"payload" : {
}
},
}
IntegrationResponses:
- StatusCode: "200"
ResponseTemplates:
application/json: |
#set($inputRoot = $input.path('$'))
#if($inputRoot.toString().contains("Item"))
$input.json("$")
#set($context.responseOverride.status = 200)
#else
#set($context.responseOverride.status = 404)
#end
Api gateway currently supports mapping the status code using the status code of the integration response (Here dynamodb response code). The only workaround is to use a lambda function which outputs different error messages that can be mapped using a error regex http://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-method-settings-execution-console.html.

$http returning error response NULL on first call after launch (ionic) everytime, but after subsequent http post its ok

Whenever I launch my app, and click on login on the first few tries, the login will attempt a POST http to the server. However $http always (everytime) returns NULL on first try. sometimes after several few tries still NULL if done fast. But subsequently, its all ok.
I dont get it, why is $http returning error response NULL initially ??
Here is my login controller doing the http post
Login Controller (LoginCtrl)
https://gist.github.com/anonymous/771194bc5815e4ccdf38b57d6158853f
var req = {
method: 'POST',
url: baseURL,
data: postObject,
//timeout: 5000
};
err is NULL here:
}).error(function(err) {
I dont know if it is CORS but I'ved got this set in config.xml
<access origin="*" />
my config.xml
https://gist.github.com/anonymous/b2df3a857338d14ec3fcd6dda776e212
Any ideas ?
Im using ionic 1.7.14
on device iOS 9.3.1
UPDATE
I'ved put the problem code here. can logout first to goto login screen. enter in anything in username/password field, click login once failed, second or third try will be success.
https://github.com/axilaris/ionic_null_http_problem
some troubleshooting so far: i noticed the http post request is called twice. not sure why.
UPDATED the code using $http.post.then but still has the same effect
$http.post(baseURL, postObject).then(function successCallback(response)
response has NULL data --> Object {data: null, status: 0, config: Object, statusText: ""}
It is hard to diagnose having the above details only.
However the problem could be that your handler (login function) is triggered before digest cycle finished updating $scope.data.username and $scope.data.password and for the first tries it sends empty values for those to the server and works fine later.
You can run Safari web inspector to see what is sent to the server to prove this.
The fix may depend on how your view/template is coded. Can you please share it? Or, ideally, create a working sample at http://play.ionic.io/
Another option to fix could be to try to wrap your code related to http request into
$timeout(function() {
// your code goes here
});
or, consider using .$applyAsync() (see the docs for details)
This might help to fix the problem
You are probably getting this inconsistent behavior as you are using the 'success' promise method instead of 'then' (note that use of the success method has now been deprecated).
The key differences between these two methods are:
then() - full power of the promise API but slightly more verbose
success() - doesn't return a promise but offeres slightly more convienient syntax
as highlighted in this answer.
Hence in your scenario, instead of using 'success':
var req = {
method: 'POST',
url: baseURL + 'session/login',
data: postObject,
//timeout: 5000
};
$http(req).success(function(resp) {...
use 'then' along with angular's post shortcut method (you don't have to use this shortcut method, but I think it makes the code more succinct) e.g.:
$http.post(baseURL + 'session/login', postObject).then(function successCallback(response) {
// this callback will be called asynchronously
// when the response is available
}, function errorCallback(response) {
// called asynchronously if an error occurs
// or server returns response with an error status.
});
Using 'then' returns a promise resolved with a value returned from a callback, so it should give you a consistently valid result.
it was a timeout in app.js that caused it. was set to 1 second which gives it it arbitrary success rate.
config.timeout = 1000;

Async testing with vows using the http.get library in Node.js

I'm having a doozie of a time trying to get a basic http test to work with vows.
I think I've followed the async example from vows http://vowsjs.org/#-writing-asynchronous-tests and substitued the appropriate calls, but I must be missing something.
The test code looks like this:
var http = require('http'),
vows = require('vows'),
assert = require('assert');
vows.describe("homepage").addBatch({
"Get the home page": {
topic: function() {
http.get({'host': "127.0.0.1", 'port': 5000, 'path': '/'}, this.callback);
},
'should respond with 200 OK': function(res) {
assert.equal(res.statusCode, 200);
}
}
}).export(module);
I get the following error when I try to run the test for this:
/Users/<home_folder>/node_modules/vows/lib/vows.js:80
rrored', { type: 'promise', error: err.stack || err.message || JSON.stringify(
^
TypeError: Converting circular structure to JSON
at Object.stringify (native)
at EventEmitter.<anonymous> (/Users/<home_folder>/node_modules/vows/lib/vows.js:80:90)
at EventEmitter.emit (events.js:64:17)
at /Users/<home_folder>/node_modules/vows/lib/vows/context.js:31:52
at ClientRequest.<anonymous> (/Users/<home_folder>/node_modules/vows/lib/vows/context.js:46:29)
at ClientRequest.g (events.js:143:14)
at ClientRequest.emit (events.js:64:17)
at HTTPParser.onIncoming (http.js:1349:9)
at HTTPParser.onHeadersComplete (http.js:108:31)
at Socket.ondata (http.js:1226:22)
I can get a simple http example to work on it's own. I can get the vows example to work on it's own but I can't combine them for whatever reason. I'd really appreciate some help here. I've been trying to get this to work for a while now (including much googling).
UPDATE:
Apparently adding an error argument to the call back solves this problem, thanks to help from Alexis Sellier (creator of vows).
But I have no idea why. When writing out the http lib example on it's own no error argument is required. I can't find any documentation in vows to indicate why it's needed so I'm at a bit of a loss.
My new question is why is the error argument required when using the http lib in vows?
After checking vow's source code, I think I know why. Vows always ensure that when you call this.callback, the resulting receiver function's first argument is always an error object. Vows interpret the callbacks by these rules:
If the first argument of your originating callback is a boolean, use that to determine whether or not to append an error object to the receiving callback (e.g. path.exists(boolean) will emit callback(error, exists) instead)
If the first argument is an object, assume it's an error object and use that to determine whether to add the originating callback to the "error" or "success" list. The reason this list exists is to support promise based tests I guess?
While I can't confirm the above is correct, my experience is that vows' async style is made to support node-styled callbacks (e.g. err as the first arg), and 3rd party npm modules that don't conform to this standard will be hard to test.
Please don't take my answer as gospel, as this is my own experience. Another gotcha is when you have async operations inside the function that you want to test - unless you provide a callback, vows won't be able to handle it properly.
Personally, I think vows still make it hard to test async code. I wish it had some waitFor() or until() flow control functions though.
My suggestion? When dealing with async code, use Step. Don't let vows control your flow.
It is actually missing in the documentations which is still a bit short. But you can get a glimpse of it here in this page :
'when peeled *asynchronously*': {
topic: function (banana) {
banana.peel(this.callback);
},
'results in a `PeeledBanana`': function (err, result) {
assert.instanceOf (result, PeeledBanana);
}
}
As it was said by Morten Siebuhr and Ruben Tan, this is how vows works and that is why it works like that.

How would you handle errors when using jQuery.ajax()?

When using jQuery's ajax method to submit form data, what is the best way to handle errors?
This is an example of what a call might look like:
$.ajax({
url: "userCreation.ashx",
data: { u:userName, p:password, e:email },
type: "POST",
beforeSend: function(){disableSubmitButton();},
complete: function(){enableSubmitButton();},
error: function(xhr, statusText, errorThrown){
// Work out what the error was and display the appropriate message
},
success: function(data){
displayUserCreatedMessage();
refreshUserList();
}
});
The request might fail for a number of reasons, such as duplicate user name, duplicate email address etc, and the ashx is written to throw an exception when this happens.
My problem seems to be that by throwing an exception the ashx causes the statusText and errorThrown to be undefined.
I can get to the XMLHttpRequest.responseText which contains the HTML that makes up the standard .net error page.
I am finding the page title in the responseText and using the title to work out which error was thrown. Although I have a suspicion that this will fall apart when I enable custom error handling pages.
Should I be throwing the errors in the ashx, or should I be returning a status code as part of the data returned by the call to userCreation.ashx, then using this to decide what action to take?
How do you handle these situations?
For debugging, I usually just create an element (in the case below: <div id="error"></div>) on the page and write the XmlHttpRequest to it:
error: function (XMLHttpRequest, textStatus, errorThrown) {
$("#error").html(XMLHttpRequest.status + "\n<hr />" + XMLHttpRequest.responseText);
}
Then you can see the types of errors that are occurring and capture them correctly:
if (XMLHttpRequest.status === 404) // display some page not found error
if (XMLHttpRequest.status === 500) // display some server error
In your ashx, can you throw a new exception (e.g "Invalid User" etc.) and then just parse that out of the XMLHttpRequest.responseText? For me when I get an error the XMLHttpRequest.responseText isn't the standard Asp.Net error page, it's a JSON object containing the error like this:
{
"Message":"Index was out of range. Must be non-negative and less than the size of the collection.\r\n
Parameter name: index",
"StackTrace":" at System.ThrowHelper.ThrowArgumentOutOfRangeException(ExceptionArgument argument, ExceptionResource resource)\r\n
at etc...",
"ExceptionType":"System.ArgumentOutOfRangeException"
}
Edit: This could be because the function I'm calling is marked with these attributes:
<WebMethod()> _
<ScriptMethod()> _
Should I be throwing the errors in the
ashx, or should I be returning a
status code as part of the data
returned by the call to
userCreation.ashx, then using this to
decide what action to take? How do you
handle these situations?
Personally, if possible, I would prefer to handle this on the server side and work up a message to the user there. This works very well in a scenario where you only want to display a message to the user telling them what happened (validation message, essentially).
However, if you want to perform an action based on what happened on the server, you may want to use a status code and write some javascript to perform various actions based on that status code.
Now I have a problem as to which answer to accept.
Further thought on the problem brings me to the conclusion that I was incorrectly throwing exceptions. Duplicate user names, email addresses etc are expected issues during a sign up process and are therefore not exceptions, but simply errors. In which case I probably shouldn't be throwing exceptions, but returning error codes.
Which leads me to think that irobinson's approach should be the one to take in this case, especially since the form is only a small part of the UI being displayed. I have now implemented this solution and I am returning xml containing a status and an optional message that is to be displayed. I can then use jQuery to parse it and take the appropriate action: -
success: function(data){
var created = $("result", data).attr("success");
if (created == "OK"){
resetNewUserForm();
listUsers('');
} else {
var errorMessage = $("result", data).attr("message");
$("#newUserErrorMessage").text(errorMessage).show();
}
enableNewUserForm();
}
However travis' answer is very detailed and would be perfect during debugging or if I wanted to display an exception message to the user. I am definitely not receiving JSON back, so it is probably down to one of those attributes that travis has listed, as I don't have them in my code.
(I am going to accept irobinson's answer, but upvote travis' answer. It just feels strange to be accepting an answer that doesn't have the most votes.)

Resources