I am trying to run certain insert statements after a collection has been updated. For example, if a user adds an embedded document Location to their User document, I would like to also insert that embedded document into a separate Location collection. Is there a way to do this on the server side so that the operation is guaranteed to run?
If you're willing to use some code I wrote (https://gist.github.com/matb33/5258260), you can hook in like so:
EDIT: code is now part of a project at https://github.com/matb33/meteor-collection-hooks
var test = new Meteor.Collection("test");
if (Meteor.isServer) {
test.before("insert", function (userId, doc) {
doc.created = doc.created || Date.now();
});
test.before("update", function (userId, selector, modifier, options) {
if (!modifier.$set) modifier.$set = {};
modifier.$set.modified = Date.now();
});
test.after("update", function (userId, selector, modifier, options, previous) {
doSomething();
});
}
you would need to do it in a method.. you can keep latency compensation by implementing a client-side Method stub:
Calling methods on the client defines stub functions associated with
server methods of the same name. You don't have to define a stub for
your method if you don't want to. In that case, method calls are just
like remote procedure calls in other systems, and you'll have to wait
for the results from the server.
If you do define a stub, when a client invokes a server method it will
also run its stub in parallel. On the client, the return value of a
stub is ignored. Stubs are run for their side-effects: they are
intended to simulate the result of what the server's method will do,
but without waiting for the round trip delay. If a stub throws an
exception it will be logged to the console.
see my Meteor stub example here: https://stackoverflow.com/a/13145432/1029644
Related
I've been digging around, and I'm not able to find references or documentation on how I can use Asynchronous Functions in Google App Script, I found that people mention It's possible, but not mention how...
Could someone point me in the right direction or provide me with an example?
Promises, Callbacks, or something, that can help me with this.
I have this function lets call it foo that takes a while to execute (long enough that It could time out an HTTP call).
What I'm trying to do Is to refactor it, in a way that it works like this:
function doPost(e) {
// parsing and getting values from e
var returnable = foo(par1, par2, par3);
return ContentService
.createTextOutput(JSON.stringify(returnable))
.setMimeType(ContentService.MimeType.JSON);
}
function foo(par1, par2, par3) {
var returnable = something(par1, par2, par3); // get the value I need to return;
// continue in an Async way, or schedule execution for something else
// and allow the function to continue its flow
/* async bar(); */
return returnable;
}
Now I want to realize that bit in foo because It takes to long and I don't want to risk for a time out, also the logic that occurs there it's totally client Independent, so It doesn't matter, I just need the return value, that I'll be getting before.
Also, I think It's worth mentioning that this is deployed in Google Drive as a web app.
It's been long since this, but adding some context, at that moment I wanted to scheduled several things to happen on Google Drive, and It was timing out the execution, so I was looking for a way to safely schedule a job.
You want to execute functions by the asynchronous processing using Google Apps Script.
You want to run the functions with the asynchronous processing using time trigger.
If my understanding is correct, unfortunately, there are no methods and the official document for directly achieving it. But as a workaround, that can be achieved by using both Google Apps Script API and the fetchAll method which can work by asynchronous processing.
The flow of this workaround is as follows.
Deploy API executable, enable Google Apps Script API.
Using fetchAll, request the endpoint of Google Apps Script API for running function.
When several functions are requested once, those work with the asynchronous processing by fetchAll.
Note:
I think that Web Apps can be also used instead of Google Apps Script API.
In order to simply use this workaround, I have created a GAS library. I think that you can also use it.
In this workaround, you can also run the functions with the asynchronous processing using time trigger.
References:
fetchAll
Deploy the script as an API executable
scripts.run of Google Apps Script API
Benchmark: fetchAll method in UrlFetch service for Google Apps Script
GAS library for running the asynchronous processing
If I misunderstand your question, I'm sorry.
There is another way to accomplish this.
You can use time-based one-off triggers to run functions asynchronously, they take a bit of time to queue up (30-60 seconds) but it is ideal for slow-running tasks that you want to remove from the main execution of your script.
// Creates a trigger that will run a second later
ScriptApp.newTrigger("myFunction")
.timeBased()
.after(1)
.create();
There is handy script that I put together called Async.gs that will help remove the boilerplate out of this technique. You can even use it to pass arguments via the CacheService.
Here is the link:
https://gist.github.com/sdesalas/2972f8647897d5481fd8e01f03122805
// Define async function
function runSlowTask(user_id, is_active) {
console.log('runSlowTask()', { user_id: user_id, is_active: is_active });
Utilities.sleep(5000);
console.log('runSlowTask() - FINISHED!')
}
// Run function asynchronously
Async.call('runSlowTask');
// Run function asynchronously with one argument
Async.call('runSlowTask', 51291);
// Run function asynchronously with multiple argument
Async.call('runSlowTask', 51291, true);
// Run function asynchronously with an array of arguments
Async.apply('runSlowTask', [51291, true]);
// Run function in library asynchronously with one argument
Async.call('MyLibrary.runSlowTask', 51291);
// Run function in library asynchronously with an array of arguments
Async.apply('MyLibrary.runSlowTask', [51291, true]);
With the new V8 runtime, it is now possible to write async functions and use promises in your app script.
Even triggers can be declared async! For example (typescript):
async function onOpen(e: GoogleAppsScript.Events.SheetsOnOpen) {
console.log("I am inside a promise");
// do your await stuff here or make more async calls
}
To start using the new runtime, just follow this guide. In short, it all boils down to adding the following line to your appsscript.json file:
{
...
"runtimeVersion": "V8"
}
Based on Tanaike's answer, I created another version of it. My goals were:
Easy to maintain
Easy to call (simple call convention)
tasks.gs
class TasksNamespace {
constructor() {
this.webAppDevUrl = 'https://script.google.com/macros/s/<your web app's dev id>/dev';
this.accessToken = ScriptApp.getOAuthToken();
}
// send all requests
all(requests) {
return requests
.map(r => ({
muteHttpExceptions: true,
url: this.webAppDevUrl,
method: 'POST',
contentType: 'application/json',
payload: {
functionName: r.first(),
arguments: r.removeFirst()
}.toJson(),
headers: {
Authorization: 'Bearer ' + this.accessToken
}
}), this)
.fetchAll()
.map(r => r.getContentText().toObject())
}
// send all responses
process(request) {
return ContentService
.createTextOutput(
request
.postData
.contents
.toObject()
.using(This => ({
...This,
result: (() => {
try {
return eval(This.functionName).apply(eval(This.functionName.splitOffLast()), This.arguments) // this could cause an error
}
catch(error) {
return error;
}
})()
}))
.toJson()
)
.setMimeType(ContentService.MimeType.JSON)
}
}
helpers.gs
// array prototype
Array.prototype.fetchAll = function() {
return UrlFetchApp.fetchAll(this);
}
Array.prototype.first = function() {
return this[0];
}
Array.prototype.removeFirst = function() {
this.shift();
return this;
}
Array.prototype.removeLast = function() {
this.pop();
return this;
}
// string prototype
String.prototype.blankToUndefined = function(search) {
return this.isBlank() ? undefined : this;
};
String.prototype.isBlank = function() {
return this.trim().length == 0;
}
String.prototype.splitOffLast = function(delimiter = '.') {
return this.split(delimiter).removeLast().join(delimiter).blankToUndefined();
}
// To Object - if string is Json
String.prototype.toObject = function() {
if(this.isBlank())
return {};
return JSON.parse(this, App.Strings.parseDate);
}
// object prototype
Object.prototype.toJson = function() {
return JSON.stringify(this);
}
Object.prototype.using = function(func) {
return func.call(this, this);
}
http.handler.gs
function doPost(request) {
return new TasksNamespace.process(request);
}
calling convention
Just make arrays with the full function name and the rest are the function's arguments. It will return when everything is done, so it's like Promise.all()
var a = new TasksNamespace.all([
["App.Data.Firebase.Properties.getById",'T006DB4'],
["App.Data.External.CISC.Properties.getById",'T00A21F', true, 12],
["App.Maps.geoCode",'T022D62', false]
])
return preview
[ { functionName: 'App.Data.Firebase.Properties.getById',
arguments: [ 'T006DB4' ],
result:
{ Id: '',
Listings: [Object],
Pages: [Object],
TempId: 'T006DB4',
Workflow: [Object] } },
...
]
Notes
it can handle any static method, any method off a root object's tree, or any root (global) function.
it can handle 0 or more (any number) of arguments of any kind
it handles errors by returning the error from any post
// First create a trigger which will run after some time
ScriptApp.newTrigger("createAsyncJob").timeBased().after(6000).create();
/* The trigger will execute and first delete trigger itself using deleteTrigger method and trigger unique id. (Reason: There are limits on trigger which you can create therefore it safe bet to delete it.)
Then it will call the function which you want to execute.
*/
function createAsyncJob(e) {
deleteTrigger(e.triggerUid);
createJobsTrigger();
}
/* This function will get all trigger from project and search the specific trigger UID and delete it.
*/
function deleteTrigger(triggerUid) {
let triggers = ScriptApp.getProjectTriggers();
triggers.forEach(trigger => {
if (trigger.getUniqueId() == triggerUid) {
ScriptApp.deleteTrigger(trigger);
}
});
}
While this isn't quite an answer to your question, this could lead to an answer if implemented.
I have submitted a feature request to Google to modify the implementation of doGet() and doPost() to instead accept a completion block in the functions' parameters that we would call with our response object, allowing additional slow-running logic to be executed after the response has been "returned".
If you'd like this functionality, please star the issue here: https://issuetracker.google.com/issues/231411987?pli=1
I understand that Meteor methods let you do a client to server call, but what's the best approach to call another function or method from a Meteor method, i.e. a server to server call.
Right now if I do a regular JS function call it only works if the JS file is in the lib folder. But I need it to be in the server folder.
Here is the code
I have a topics collection which sits in the collection folder and has the following
I have the following which is a collection
Meteor.methods({
topicPost: function(topicAttributes) {
var user = Meteor.user(),
topicWithSameTitle = Topics.findOne({title: topicAttributes.title});
// ensure the user is logged in
if (!user)
throw new Meteor.Error(401, "You need to login to add a new topic");
Meteor.call('checkUser');
}
});
I then have the following method which sits in the server folder
Meteor.methods({
checkUser: function () {
alert('aaaa');
}
});
This works, but it's not a great solution. My method for handling this is to have all of my functions outside the Meteor.methods, and simply relay to the proper functions when necessary.
// Client
Meteor.call('foo');
And:
// Server
Meteor.methods({
foo: function() {
foo();
}
});
foo = function() {
foo = bar;
};
The advantage is that the foo fn can be called from anywhere on the server without a Meteor.call. Meanwhile, Meteor.methods only exposes what is absolutely necessary to the client.
[EDIT] There is some ambiguity as to which 'foo' you're talking about; obviously the server knows you mean the one outside the methods call. But if you're feeling confused, you can always rename one or the other. The advantage to this is that there is minimal refactoring involved.
Just to clarify for readers who don't notice that the OP's code actually contains the answer, you just do
Meteor.call('checkUser');
on the server. Per the meteor docs (https://docs.meteor.com/api/methods.html#Meteor-call), on the server, if you use Meteor.call() without a callback argument, the call runs synchronously and waits for the result. For example, if 'checkUser' was written to provide a userId value, you'd just do
let userId = Meteor.call('checkUser');
On the client, though, you have to provide a callback function as an argument to Meteor.call(), and the userId would be provided asynchronously to your callback function.
I have a fairly typical need, to ensure an insert / update has finished before I run a find. The code goes something like this:
//Update my collections
Messages.insert({author:_id,text:text});
Authors.update({_id:_id},{$inc:{messages:1}});
//Wait until the update / insert has finished
//Perform some actions on the collections just updated
var author = Authors.findOne({task:taskId},{sort:{'messages':1}});
//Do some more complex stuff...
While in most cases this would be fine as asynchronous calls, with the dom updating as and when things complete, in my case it is essential that the insert and update have completed before I run the function call.
Do I need to perform the insert and update as a server side call with a callback function, or is there some way I could do this on the client side?
At the moment I have something like:
Meteor.call("recordMessage", _id, text,
function(err, out){postMessage(_id)}
);
which works - but I'd like to know if I could do this on the client side.
Isn't that what the optional callback arguments are for?
var author;
Messages.insert({author:_id, text:text},
function(err, result) {
Authors.update({_id: result},
{$inc: {messages:1}},
function(err, result) {
author = Authors.findOne({task:taskId},
{sort:{'messages':1}});
}
);
});
I am relating to java as to how to do thread/async. I use new Thread(target).start() where target is Runnable as one way to do threading in java. New concurrent api has alternatives but we know that on specific call new threads are creating and passed in tasks are executed.
Similarly how is async done in Dart ?
I read on send/receivport, completer/future, spawnFunction. To me only spawnFunction is convincing statement that will create new thread. can one explain how completer/future help. i know they take callbacks but is there some implicit logic/rule in javascript/dart that callbacks always be execute in different thread.
below is dart snippet/pseudo code:
void callback() {
print("callback called");
}
costlyQuery(sql, void f()) {
executeSql(sql);
f();
}
costlyQuery("select * from dual", callback);
I hope my costlyQuery signature to take function as 2nd parameter is correct. so now I do not think that f() after executeSql(sql) is going to be async. may be taking above example add completer/future if that can make async to help me understand.
tl;dr: There is no implicit rule that a callback will not block.
Javascript event queue
In Javascript, there are no threads (except WebWorkers, but that's different). This means that if any part of your code blocks, the whole application is blocked. There is nothing magic about callbacks, they are just functions:
function longLoop(cb) {
var i = 1000000;
while (i--) ;
cb();
}
function callback() {
console.log("Hello world");
}
function fun() {
longLoop(callback);
console.log("Called after Hello World is printed");
}
To make something asynchronous, the callback must be put on the event queue, which only happens through some API calls:
user initiated event handlers- clicks, keyboard, mouse
API event handlers- XmlHTTPRequest callbacks, WebWorker communication
timing functions- setTimeout, setInterval
When the event is triggered, the callback is placed on the event queue to be executed when all other callbacks have finished executing. This means that no two lines of your code will ever be executing at the same time.
Futures
I assume you have at least stumbled on this post about futures. If not, it's a good read and comes straight from the horses mouth.
In Javascript, you have to do some work to make sense of asynchronous code. There's even a Javascript library called futures for doing common things, such as asynchronous loops and sequences (full disclosure, I have personally worked with the author of this library).
The notion of a future is a promise made by the function creating the future that the future will be completed at some point down the road. If you are going to make some asynchronous call, create a future, return it, and fulfill the promise when the asynchronous call finishes. From the blog post:
Future<Results> costlyQuery() {
var completer = new Completer();
database.query("SELECT * FROM giant_table", (results) {
// when complete
completer.complete(results);
});
// this returns essentially immediately,
// before query is finished
return completer.future;
}
If database.query is a blocking call, the future will be completed immediately. This example assumes that database.query is a non-blocking call (async), so the future will be fulfilled after this function exits. completer.complete will call whatever function is passed to completer.then() with the arguments specified.
Your example, modified to be idiomatic and asynchronous:
void callback() {
print("callback called");
}
costlyQuery(sql) {
var completer = new Completer();
executeSql(sql, () => completer.complete());
return completer.future;
}
costlyQuery("select * from dual").then(callback);
This is asynchronous and uses futures correctly. You could pass your own callback function into costlyQuery as you have done and call that in the callback to executeSql to achieve the same thing, but that is the Javascript way of doing it, not the Dart way.
Isolates
Isolates are similar to Java's threads, except that isolates are a concurrency model and threads are a parallelism model (see this SO question for more information about concurrency vs parallelism).
Dart is special in that the specification does not mandate that everything be run in a single thread. It only mandates that different executing contexts do not have access to the same data. Each isolate has it's own memory and can only communicate (e.g. send data) with other isolates through send/receive ports.
These ports work similarly to channels in Go if you're familiar.
An isolate is an isolated execution context that runs independently of other code. In Javascript terms, this means that it has it's own event queue. See the Dart tour for a more complete introduction to isolates.
Lets say your executeSql is a blocking function. If we don't want to wait for it to finish, we could load it into an isolate:
void callback() {
print("callback called");
}
void costlyQuery() {
port.receive((sql, reply) {
executeSql(sql);
reply.send();
});
}
main() {
var sendPort = spawnFunction(costlyQuery);
// .call does all the magic needed to communicate both directions
sendPort.call("select * from dual").then(callback);
print("Called before executeSql finishes");
}
This code creates an isolate, sends data to it then registers a callback for when it's done. Even if executeSql blocks, main() will not necessarily block.
I would like to use the GetLocalizedItems method in the Anguilla Framework.
I don't know how to create a new filter and set the conditions or what to use for success and failure.
In the GUI:
tridion.Web.UI.ContentManager.WhereUsed.GetListUsedItems(id, filter.conditions,
filter.columns, success, failure);
Are the methods in this namespace intended to be used by our extensions?
Building a filter
Here is an example of how to build a filter
var filter = new Tridion.ContentManager.ListFilter();
filter.conditions.ItemTypes = 16 | 2; // folders and components
filter.conditions.Recursive = true;
filter.conditions.BasedOnSchema = "tcm:1-23-8,tcm:1-32-8".split(",");
filter.columns = Tridion.Constants.ColumnFilter.DEFAULT;
Or this extremely simple case from General.js:
var templateFilter = new Tridion.ContentManager.ListFilter({
conditions: { ItemTypes: [ itemType ] }
});
Calling a WCF method
The second part of your question was really already covered in https://stackoverflow.com/a/9385975/209103, although I'll make it a bit more concrete here.
WCF/AJAX calls such as this are executed asynchronously, since they may take some time to complete. While you would normally simple handle the result of the call on the line after the closing parenthesis, you can't do that in AJAX calls since that line will be execute before the function has completed. Instead you have to pass in one or more callback functions that get called once the function completes.
I typically just pass in two functions that break into the JavaScript debugger of my browser when I first start figuring out such a method:
Tridion.Web.UI.ContentManager.WhereUsed.GetListUsedItems(
"tcm:1-23",
filter.conditions,
filter.columns,
new function() { console.log(arguments); debugger; },
new function() { console.log(arguments); debugger; }
);
So the first (anonymous) function is called when the (asynchronous) HTTP call to the TCM server succeeded, while the second is called when the call failed. In the answer I linked above, we called them onSuccess and onFailure to make their nature more explicit.
Both functions in this case simply write the implicit arguments parameter that is always passed in JavaScript. They then break into the JavaScript debugger of your browser, so you can inspect the arguments further.