Asynchronous execution of a function App Script - asynchronous

I've been digging around, and I'm not able to find references or documentation on how I can use Asynchronous Functions in Google App Script, I found that people mention It's possible, but not mention how...
Could someone point me in the right direction or provide me with an example?
Promises, Callbacks, or something, that can help me with this.
I have this function lets call it foo that takes a while to execute (long enough that It could time out an HTTP call).
What I'm trying to do Is to refactor it, in a way that it works like this:
function doPost(e) {
// parsing and getting values from e
var returnable = foo(par1, par2, par3);
return ContentService
.createTextOutput(JSON.stringify(returnable))
.setMimeType(ContentService.MimeType.JSON);
}
function foo(par1, par2, par3) {
var returnable = something(par1, par2, par3); // get the value I need to return;
// continue in an Async way, or schedule execution for something else
// and allow the function to continue its flow
/* async bar(); */
return returnable;
}
Now I want to realize that bit in foo because It takes to long and I don't want to risk for a time out, also the logic that occurs there it's totally client Independent, so It doesn't matter, I just need the return value, that I'll be getting before.
Also, I think It's worth mentioning that this is deployed in Google Drive as a web app.
It's been long since this, but adding some context, at that moment I wanted to scheduled several things to happen on Google Drive, and It was timing out the execution, so I was looking for a way to safely schedule a job.

You want to execute functions by the asynchronous processing using Google Apps Script.
You want to run the functions with the asynchronous processing using time trigger.
If my understanding is correct, unfortunately, there are no methods and the official document for directly achieving it. But as a workaround, that can be achieved by using both Google Apps Script API and the fetchAll method which can work by asynchronous processing.
The flow of this workaround is as follows.
Deploy API executable, enable Google Apps Script API.
Using fetchAll, request the endpoint of Google Apps Script API for running function.
When several functions are requested once, those work with the asynchronous processing by fetchAll.
Note:
I think that Web Apps can be also used instead of Google Apps Script API.
In order to simply use this workaround, I have created a GAS library. I think that you can also use it.
In this workaround, you can also run the functions with the asynchronous processing using time trigger.
References:
fetchAll
Deploy the script as an API executable
scripts.run of Google Apps Script API
Benchmark: fetchAll method in UrlFetch service for Google Apps Script
GAS library for running the asynchronous processing
If I misunderstand your question, I'm sorry.

There is another way to accomplish this.
You can use time-based one-off triggers to run functions asynchronously, they take a bit of time to queue up (30-60 seconds) but it is ideal for slow-running tasks that you want to remove from the main execution of your script.
// Creates a trigger that will run a second later
ScriptApp.newTrigger("myFunction")
.timeBased()
.after(1)
.create();
There is handy script that I put together called Async.gs that will help remove the boilerplate out of this technique. You can even use it to pass arguments via the CacheService.
Here is the link:
https://gist.github.com/sdesalas/2972f8647897d5481fd8e01f03122805
// Define async function
function runSlowTask(user_id, is_active) {
console.log('runSlowTask()', { user_id: user_id, is_active: is_active });
Utilities.sleep(5000);
console.log('runSlowTask() - FINISHED!')
}
// Run function asynchronously
Async.call('runSlowTask');
// Run function asynchronously with one argument
Async.call('runSlowTask', 51291);
// Run function asynchronously with multiple argument
Async.call('runSlowTask', 51291, true);
// Run function asynchronously with an array of arguments
Async.apply('runSlowTask', [51291, true]);
// Run function in library asynchronously with one argument
Async.call('MyLibrary.runSlowTask', 51291);
// Run function in library asynchronously with an array of arguments
Async.apply('MyLibrary.runSlowTask', [51291, true]);

With the new V8 runtime, it is now possible to write async functions and use promises in your app script.
Even triggers can be declared async! For example (typescript):
async function onOpen(e: GoogleAppsScript.Events.SheetsOnOpen) {
console.log("I am inside a promise");
// do your await stuff here or make more async calls
}
To start using the new runtime, just follow this guide. In short, it all boils down to adding the following line to your appsscript.json file:
{
...
"runtimeVersion": "V8"
}

Based on Tanaike's answer, I created another version of it. My goals were:
Easy to maintain
Easy to call (simple call convention)
tasks.gs
class TasksNamespace {
constructor() {
this.webAppDevUrl = 'https://script.google.com/macros/s/<your web app's dev id>/dev';
this.accessToken = ScriptApp.getOAuthToken();
}
// send all requests
all(requests) {
return requests
.map(r => ({
muteHttpExceptions: true,
url: this.webAppDevUrl,
method: 'POST',
contentType: 'application/json',
payload: {
functionName: r.first(),
arguments: r.removeFirst()
}.toJson(),
headers: {
Authorization: 'Bearer ' + this.accessToken
}
}), this)
.fetchAll()
.map(r => r.getContentText().toObject())
}
// send all responses
process(request) {
return ContentService
.createTextOutput(
request
.postData
.contents
.toObject()
.using(This => ({
...This,
result: (() => {
try {
return eval(This.functionName).apply(eval(This.functionName.splitOffLast()), This.arguments) // this could cause an error
}
catch(error) {
return error;
}
})()
}))
.toJson()
)
.setMimeType(ContentService.MimeType.JSON)
}
}
helpers.gs
// array prototype
Array.prototype.fetchAll = function() {
return UrlFetchApp.fetchAll(this);
}
Array.prototype.first = function() {
return this[0];
}
Array.prototype.removeFirst = function() {
this.shift();
return this;
}
Array.prototype.removeLast = function() {
this.pop();
return this;
}
// string prototype
String.prototype.blankToUndefined = function(search) {
return this.isBlank() ? undefined : this;
};
String.prototype.isBlank = function() {
return this.trim().length == 0;
}
String.prototype.splitOffLast = function(delimiter = '.') {
return this.split(delimiter).removeLast().join(delimiter).blankToUndefined();
}
// To Object - if string is Json
String.prototype.toObject = function() {
if(this.isBlank())
return {};
return JSON.parse(this, App.Strings.parseDate);
}
// object prototype
Object.prototype.toJson = function() {
return JSON.stringify(this);
}
Object.prototype.using = function(func) {
return func.call(this, this);
}
http.handler.gs
function doPost(request) {
return new TasksNamespace.process(request);
}
calling convention
Just make arrays with the full function name and the rest are the function's arguments. It will return when everything is done, so it's like Promise.all()
var a = new TasksNamespace.all([
["App.Data.Firebase.Properties.getById",'T006DB4'],
["App.Data.External.CISC.Properties.getById",'T00A21F', true, 12],
["App.Maps.geoCode",'T022D62', false]
])
return preview
[ { functionName: 'App.Data.Firebase.Properties.getById',
arguments: [ 'T006DB4' ],
result:
{ Id: '',
Listings: [Object],
Pages: [Object],
TempId: 'T006DB4',
Workflow: [Object] } },
...
]
Notes
it can handle any static method, any method off a root object's tree, or any root (global) function.
it can handle 0 or more (any number) of arguments of any kind
it handles errors by returning the error from any post

// First create a trigger which will run after some time
ScriptApp.newTrigger("createAsyncJob").timeBased().after(6000).create();
/* The trigger will execute and first delete trigger itself using deleteTrigger method and trigger unique id. (Reason: There are limits on trigger which you can create therefore it safe bet to delete it.)
Then it will call the function which you want to execute.
*/
function createAsyncJob(e) {
deleteTrigger(e.triggerUid);
createJobsTrigger();
}
/* This function will get all trigger from project and search the specific trigger UID and delete it.
*/
function deleteTrigger(triggerUid) {
let triggers = ScriptApp.getProjectTriggers();
triggers.forEach(trigger => {
if (trigger.getUniqueId() == triggerUid) {
ScriptApp.deleteTrigger(trigger);
}
});
}

While this isn't quite an answer to your question, this could lead to an answer if implemented.
I have submitted a feature request to Google to modify the implementation of doGet() and doPost() to instead accept a completion block in the functions' parameters that we would call with our response object, allowing additional slow-running logic to be executed after the response has been "returned".
If you'd like this functionality, please star the issue here: https://issuetracker.google.com/issues/231411987?pli=1

Related

Notify browser UI about SignalR requests to the server

I ma using SignalR in the browser. Some request (calling function on the server) are long and I would like to show spinner/loading-bar.
Can I somehow hook for an event when this function is started and when it returns back.
I'm trying to figure out what you mean - I think basically you want some way to hook into the start of a call and the end of a call (to load and unload a spinner)?
I've done this in two different ways - firstly as a one-off (first example), and then more systematically (the second example). Hopefully one of these will be what you need.
$.connection.myHub.server.hubMethod().done(function () {
//called on success
}).fail(function (e) {
//called on failure - I don't recommend reading e
}).always(function() {
//called regardless
spinner.close();
});
spinner.open(); // must be triggerd AFTER call incase exception thrown (due to connection not being up yet)
If you don't like that - perhaps because you call hub methods in hundreds of different sections of codes, then there are other tricks which are a bit more complicated. Lets see:
function SetupSpinnerOnCallToSignalrMethod(hubServer, method, spinnerStartCallback, spinnerEndCallback) {
var prevFunc = hubServer[method];
hubServer[method] = function () {
var ret = prevFunc.apply(this, arguments);
spinnerStartCallback(); // must be triggerd AFTER call incase exception thrown (due to connection not being up yet)
ret.always(function() {
spinnerEndCallback();
});
return ret;
};
}
//then call this for each method
SetupSpinnerOnCallToSignalrMethod($.connection.myHub.server,
"hubMethod",
function() { spinner.open(); },
function() { spinner.close(); }
);
//the server call should then work exactly as before, but the spinner open and close calls are invoked each time.

Test an async PipeTransform

Context
I have a basic PipeTransform, expect the fact that it is async. Why? because I have my own i18n service (because of parsing, pluralization and other constraints, I did my own) and it returns a Promise<string>:
#Pipe({
name: "i18n",
pure: false
})
export class I18nPipe implements PipeTransform {
private done = false;
constructor(private i18n:I18n) {
}
value:string;
transform(value:string, args:I18nPipeArgs):string {
if(this.done){
return this.value;
}
if (args.plural) {
this.i18n.getPlural(args.key, args.plural, value, args.variables, args.domain).then((res) => {
this.value = res;
this.done = true;
});
}
this.i18n.get(args.key, value, args.variables, args.domain).then((res) => {
this.done = true;
this.value = res;
});
return this.value;
}
}
This pipe works well, because the only delayed call is the very first one (the I18nService uses lazy loading, it loads JSON data only if the key is not found, so basically, the first call will be delayed, the other ones are instant but still async).
Problem
I can't figure out how to test this pipe using Jasmine, since it is working inside a component I know it works, but the goal here is to get this fully tested using jasmine, this way I can add it to a CI routine.
The above test:
describe("Pipe test", () => {
it("can call I18n.get.", async(inject([I18n], (i18n:I18n) => {
let pipe = new I18nPipe(i18n);
expect(pipe.transform("nope", {key: 'test', domain: 'test domain'})).toBe("test value");
})));
});
Fails because since the result given by the I18nService is async, the returned value is undefined in a sync logic.
I18n Pipe test can call I18n.get. FAILED
Expected undefined to be 'test value'.
EDIT: One way to do it would be to use setTimeout but I want a prettier solution, to avoid adding setTimeout(myAssertion, 100) everywhere.
Use fakeAsync from #angular/core/testing. It allows you to call tick(), which will wait for all currently queued asynchronous tasks to complete before continuing. This gives the illusion of the actions being synchronous. Right after the call to tick() we can write our expectations.
import { fakeAsync, tick } from '#angular/core/testing';
it("can call I18n.get.", fakeAsync(inject([I18n], (i18n:I18n) => {
let pipe = new I18nPipe(i18n);
let result = pipe.transform("nope", {key: 'test', domain: 'test domain'});
tick();
expect(result).toBe("test value");
})));
So when should we use fakeAsync and when should we use async? This is the rule of thumb that I go by (most of the time). When we are making asynchronous calls inside the test, this is when we should use async. async allows to test to continue until all asynchronous calls are complete. For example
it('..', async(() => {
let service = new Servce();
service.doSomething().then(result => {
expect(result).toBe('hello');
});
});
In a non async test, the expectation would never occur, as the test would complete before the asynchronous resolution of the promise. With the call to async, the test gets wrapped in a zone, which keeps track of all asynchronous tasks, and waits for them to complete.
Use fakeAsync when the asynchronous behavior is outside the control of the test (like in your case is going on in the pipe). Here we can force/wait for it to complete with the call to tick(). tick can also be passed a millisecond delay to allow more time to pass if needed.
Another option is to mock the service and make it synchronous, as mentioned in this post. When unit testing, if your components in test are dependent on heavy logic in the service, then the component in test is at the mercy of that service working correctly, which kinda defeats the purpose of a "unit" test. Mocking makes sense in a lot of cases.

How to use wrapAsync in Meteor

I am having trouble figuring out how to use wrapAsync properly using Meteor. I am currently working with node-apac and the Amazon Product Advertising API.
If am trying to run the following code, how do I run it asynchronously:
opHelper.execute('ItemSearch', {
'SearchIndex': 'Books',
'Keywords': 'harry potter',
'ResponseGroup': 'ItemAttributes, Offers'
}, function(err, results) {
console.log(results);
});
I have tried to watch several videos, but am having trouble
Meteor.wrapAsync takes an asynchronous method like opHelper.execute and makes it synchronous. It can do this so long as the last param taken by the method returns a callback where the first param is the error and second the result. This is just like your method!
You make a new method that is synchronous:
var opExecuteSynchronous = Meteor.wrapAsync(opHelper.Execute, opHelper);
The first param opHelper.Execute is the method you want to make asynchronous, and the second param is the context of the method (opHelper).
You can use this synchronously now:
var results = opExecuteSynchronous('ItemSearch', {
'SearchIndex': 'Books',
'Keywords': 'harry potter',
'ResponseGroup': 'ItemAttributes, Offers'
})
This will throw an error if err is called instead of results in the callback.

How to 'transform' data returned via a Meteor.publish?

Meteor Collections have a transform ability that allows behavior to be attached to the objects returned from mongo.
We want to have autopublish turned off so the client does not have access to the database collections, but we still want the transform functionality.
We are sending data to the client with a more explicit Meteor.publish/Meteor.subscribe or the RPC mechanism ( Meteor.call()/Meteor.methods() )
How can we have the Meteor client automatically apply a transform like it will when retrieving data directly with the Meteor.Collection methods?
While you can't directly use transforms, there is a way to transform the result of a database query before publishing it. This is what the "publish the current size of a collection" example describes here.
It took me a while to figure out a really simple application of that, so maybe my code will help you, too:
Meteor.publish("publicationsWithHTML", function (data) {
var self = this;
Publications
.find()
.forEach(function(entry) {
addSomeHTML(entry); // this function changes the content of entry
self.added("publications", entry._id, entry);
});
self.ready();
});
On the client you subscribe to this:
Meteor.subscribe("publicationsWithHTML");
But your model still need to create a collection (on both sides) that is called 'publications':
Publications = new Meteor.Collection('publications');
Mind you, this is not a very good example, as it doesn't maintain the reactivity. But I found the count example a bit confusing at first, so maybe you'll find it helpful.
(Meteor 0.7.0.1) - meteor does allow behavior to be attached to the objects returned via the pub/sub.
This is from a pull request I submitted to the meteor project.
Todos = new Meteor.Collection('todos', {
// transform allows behavior to be attached to the objects returned via the pub/sub communication.
transform : function(todo) {
todo.update = function(change) {
Meteor.call('Todos_update', this._id, change);
},
todo.remove = function() {
Meteor.call('Todos_remove', this._id);
}
return todo;
}
});
todosHandle = Meteor.subscribe('todos');
Any objects returned via the 'todos' topic will have the update() and the remove() function - which is exactly what I want: I now attach behavior to the returned data.
Try:
let transformTodo = (fields) => {
fields._pubType = 'todos';
return fields;
};
Meteor.publish('todos', function() {
let subHandle = Todos
.find()
.observeChanges({
added: (id, fields) => {
fields = transformTodo(fields);
this.added('todos', id, fields);
},
changed: (id, fields) => {
fields = transformTodo(fields);
this.changed('todos', id, fields);
},
removed: (id) => {
this.removed('todos', id);
}
});
this.ready();
this.onStop(() => {
subHandle.stop();
});
});
Currently, you can't apply transforms on the server to published collections. See this question for more details. That leaves you with either transforming the data on the client, or using a meteor method. In a method, you can have the server do whatever you want to the data.
In one of my projects, we perform our most expensive query (it joins several collections, denormalizes the documents, and trims unnecessary fields) via a method call. It isn't reactive, but it greatly simplifies our code because all of the transformation happens on the server.
To extend #Christian Fritz answer, with Reactive Solution using peerlibrary:reactive-publish
Meteor.publish("todos", function() {
const self = this;
return this.autorun(function(computation) {
// Loop over each document in collection
todo.find().forEach(function(entry) {
// Add function to transform / modify each document here
self.added("todos", entry._id, entry);
});
});
});

What is the best control flow module for node.js?

I've used caolan's async module which is very good, however tracking errors and the varying way of passing data through for control flow causes development to sometimes be very difficult.
I would like to know if there are any better options, or what is currently being used in production environments.
Thanks for reading.
I use async as well. To help tracking errors it's recommended you name your functions, instead of having loads of anonymous functions:
async.series([
function doSomething() {...},
function doSomethingElse() {...},
function finish() {...}
]);
This way you'll get more helpful information in stack traces.
...however tracking errors and the varying way of passing data through for control flow causes development to sometimes be very difficult.
I've recently created a simple abstraction named "wait.for" to call async functions in sync mode (based on Fibers): https://github.com/luciotato/waitfor
Using wait.for, you can use 'try/catch' while still calling async functions, and you keep function scope (no closures needed). Example:
function inAFiber(param){
try{
var data= wait.for(fs.readFile,'someFile'); //async function
var result = wait.for(doSomethingElse,data,param); //another async function
otherFunction(result);
}
catch(e) {
//here you catch if some of the "waited.for"
// async functions returned "err" in callback
// or if otherFunction throws
};
see the examples at https://github.com/luciotato/waitfor
Sometimes it is hard to put all the functions in an array. When you have an array of objects and want to do something for each object, I use something like the example below.
read more in: http://coppieters.blogspot.be/2013/03/iterator-for-async-nodejs-operations.html
var list = [1, 2, 3, 4, 5];
var sum = 0;
Application.each(list, function forEachNumber(done) {
sum += this;
// next statement most often called as callback in an async operation
// file, network or database stuff
done(); // pass an error if something went wrong and automatically end here
}, function whenDone(err) {
if (err)
console.log("error: " + err);
else
console.log("sum = " + sum);
});
I name the functions, because it is easier to debug (and easier to read)

Resources