$http.get upon previous returned response - asynchronous

my brain drained... Help please.
Background is about Github v3 API get all organizations more than 100 I am continue working using the answer.
Pass http://host.com/api/v3/organizations?page=1&per_page=100 the response header contains a link of <http://host.otpp.com/api/v3/organizations?page=1&per_page=100&since=318>; rel="next", <http://host.com/api/v3/organizations{?since}>; rel="first"
so now I continue to extract 318 and use http://host.com/api/v3/organizations?page=1&per_page=100&since=318 and now the response header contains link of "<http://host.com/api/v3/organizations?page=1&per_page=100&since=560>; rel="next", <http://github.host.com/api/v3/organizations{?since}>; rel="first""
Extract 560 and Use "http://github.host.com/api/v3/organizations?page=1&per_page=100&since=560". The response header is "<http://github.host.com/api/v3/organizations{?since}>; rel="first"
Since step 3 does not have "next", I know that's the last page.
Here is my code:
function loadAll($scope, $http) {
var data = [];
$http.get(HostgitAPIURL + "organizations?page=1&per_page=100").then(
function(res) {
data.push(res.data);
var link = parse_link_header(res.headers()['link']);
// links[name]=url
while ("next" in link) {
$http.get(link[name]).then(
function(response) {
data.push(response.data);
link = parse_link_header(response.headers()['link']);
}
);
}
$.each(data, function(i, item) {
// process functions
});
$scope.abc = abc;
return abc;
}
);
}
Giving me async errors. I have referred to Recursive $http.get in for loop but seems like different case. Seems like I need 1 promise resolved before calling to next one. Any idea for this recursive call please?
Thank you very much.

Related

Firebase reCAPTCHA has already been rendered in this element

Authenticate with Firebase with a Phone Number (JS) requires a mandatory reCAPTCHA verifier, it takes the ID of the container. For the ID of the container, I am generating a random one -
firebase_recaptcha_container: "recaptcha-container",
firebase_recaptcha_reset: function() {
if (typeof appVerifier != "undefined") {
appVerifier.reset()
appVerifier.clear()
}
let id = loadJS.firebase_recaptcha_container
let newID = loadJS.randomString(10)
$("#"+id).contents().remove()
$("#"+id).prop("id", newID)
loadJS.firebase_recaptcha_container = newID
return newID
}
then requesting for the RecaptchaVerifier and upon receiving I set this as a global variable window.appVerifier .
firebase_recaptcha: function(name_r="default") {
let promiseD = new firebase.auth.RecaptchaVerifier(name_r, {
'size': 'invisible',
'callback': function(response) {
resolve(response)
},
'expired-callback': function(r) {
console.log("expired", r)
},
'isolated' : false
});
return promiseD
},
_____________________
let container_recaptcha = $utils.firebase_recaptcha_reset()
window.appVerifier = await $utils.firebase_recaptcha(container_recaptcha)
It works totally fine for the very first time. But its a honest mistake for users not to use correct phone number. So for next time, I am doing the same thing again and getting error while generating the RecaptchaVerifier -
reCAPTCHA has already been rendered in this element
Which sadly does not make sense as the new element is totally different and also clear, reset methods were called following the documentation. I am neither using any other reCaptcha on this page. Refreshing the page might be a possible solution but that I really hate. Any insight would be helpful.
Thanks!
Finally found the solution, looks like it was a stupid mistake!
Invoking firebase.auth.RecaptchaVerifier adds new recaptcha scripts, every time! Hence all needed to be done is, calling it once, it does the rest on its own.
This won't get fixed just by implementing recaptchaVerifier.clear() method.
In the callback where you are passing this appVerifier, you'll have to implement the above clear method and add that "recaptcha-container" using ref
The below would be the element in render method:
<div ref={recaptchaWrapperRef}>
<div id="recaptcha-container"></div>
</div>
GenerateCaptcha function:
const generateRecaptcha = () => {
appVerifier = new RecaptchaVerifier(
"recaptcha-container",
{
size: "invisible",
},
authentication
);
Inside submit Callback:
if (appVerifier && recaptchaWrapperRef.current) {
appVerifier.clear();
recaptchaWrapperRef.current.innerHTML = `<div id="recaptcha-container"></div>`;
}
// Initialize new reCaptcha verifier
generateRecaptcha();

Asynchronous execution of a function App Script

I've been digging around, and I'm not able to find references or documentation on how I can use Asynchronous Functions in Google App Script, I found that people mention It's possible, but not mention how...
Could someone point me in the right direction or provide me with an example?
Promises, Callbacks, or something, that can help me with this.
I have this function lets call it foo that takes a while to execute (long enough that It could time out an HTTP call).
What I'm trying to do Is to refactor it, in a way that it works like this:
function doPost(e) {
// parsing and getting values from e
var returnable = foo(par1, par2, par3);
return ContentService
.createTextOutput(JSON.stringify(returnable))
.setMimeType(ContentService.MimeType.JSON);
}
function foo(par1, par2, par3) {
var returnable = something(par1, par2, par3); // get the value I need to return;
// continue in an Async way, or schedule execution for something else
// and allow the function to continue its flow
/* async bar(); */
return returnable;
}
Now I want to realize that bit in foo because It takes to long and I don't want to risk for a time out, also the logic that occurs there it's totally client Independent, so It doesn't matter, I just need the return value, that I'll be getting before.
Also, I think It's worth mentioning that this is deployed in Google Drive as a web app.
It's been long since this, but adding some context, at that moment I wanted to scheduled several things to happen on Google Drive, and It was timing out the execution, so I was looking for a way to safely schedule a job.
You want to execute functions by the asynchronous processing using Google Apps Script.
You want to run the functions with the asynchronous processing using time trigger.
If my understanding is correct, unfortunately, there are no methods and the official document for directly achieving it. But as a workaround, that can be achieved by using both Google Apps Script API and the fetchAll method which can work by asynchronous processing.
The flow of this workaround is as follows.
Deploy API executable, enable Google Apps Script API.
Using fetchAll, request the endpoint of Google Apps Script API for running function.
When several functions are requested once, those work with the asynchronous processing by fetchAll.
Note:
I think that Web Apps can be also used instead of Google Apps Script API.
In order to simply use this workaround, I have created a GAS library. I think that you can also use it.
In this workaround, you can also run the functions with the asynchronous processing using time trigger.
References:
fetchAll
Deploy the script as an API executable
scripts.run of Google Apps Script API
Benchmark: fetchAll method in UrlFetch service for Google Apps Script
GAS library for running the asynchronous processing
If I misunderstand your question, I'm sorry.
There is another way to accomplish this.
You can use time-based one-off triggers to run functions asynchronously, they take a bit of time to queue up (30-60 seconds) but it is ideal for slow-running tasks that you want to remove from the main execution of your script.
// Creates a trigger that will run a second later
ScriptApp.newTrigger("myFunction")
.timeBased()
.after(1)
.create();
There is handy script that I put together called Async.gs that will help remove the boilerplate out of this technique. You can even use it to pass arguments via the CacheService.
Here is the link:
https://gist.github.com/sdesalas/2972f8647897d5481fd8e01f03122805
// Define async function
function runSlowTask(user_id, is_active) {
console.log('runSlowTask()', { user_id: user_id, is_active: is_active });
Utilities.sleep(5000);
console.log('runSlowTask() - FINISHED!')
}
// Run function asynchronously
Async.call('runSlowTask');
// Run function asynchronously with one argument
Async.call('runSlowTask', 51291);
// Run function asynchronously with multiple argument
Async.call('runSlowTask', 51291, true);
// Run function asynchronously with an array of arguments
Async.apply('runSlowTask', [51291, true]);
// Run function in library asynchronously with one argument
Async.call('MyLibrary.runSlowTask', 51291);
// Run function in library asynchronously with an array of arguments
Async.apply('MyLibrary.runSlowTask', [51291, true]);
With the new V8 runtime, it is now possible to write async functions and use promises in your app script.
Even triggers can be declared async! For example (typescript):
async function onOpen(e: GoogleAppsScript.Events.SheetsOnOpen) {
console.log("I am inside a promise");
// do your await stuff here or make more async calls
}
To start using the new runtime, just follow this guide. In short, it all boils down to adding the following line to your appsscript.json file:
{
...
"runtimeVersion": "V8"
}
Based on Tanaike's answer, I created another version of it. My goals were:
Easy to maintain
Easy to call (simple call convention)
tasks.gs
class TasksNamespace {
constructor() {
this.webAppDevUrl = 'https://script.google.com/macros/s/<your web app's dev id>/dev';
this.accessToken = ScriptApp.getOAuthToken();
}
// send all requests
all(requests) {
return requests
.map(r => ({
muteHttpExceptions: true,
url: this.webAppDevUrl,
method: 'POST',
contentType: 'application/json',
payload: {
functionName: r.first(),
arguments: r.removeFirst()
}.toJson(),
headers: {
Authorization: 'Bearer ' + this.accessToken
}
}), this)
.fetchAll()
.map(r => r.getContentText().toObject())
}
// send all responses
process(request) {
return ContentService
.createTextOutput(
request
.postData
.contents
.toObject()
.using(This => ({
...This,
result: (() => {
try {
return eval(This.functionName).apply(eval(This.functionName.splitOffLast()), This.arguments) // this could cause an error
}
catch(error) {
return error;
}
})()
}))
.toJson()
)
.setMimeType(ContentService.MimeType.JSON)
}
}
helpers.gs
// array prototype
Array.prototype.fetchAll = function() {
return UrlFetchApp.fetchAll(this);
}
Array.prototype.first = function() {
return this[0];
}
Array.prototype.removeFirst = function() {
this.shift();
return this;
}
Array.prototype.removeLast = function() {
this.pop();
return this;
}
// string prototype
String.prototype.blankToUndefined = function(search) {
return this.isBlank() ? undefined : this;
};
String.prototype.isBlank = function() {
return this.trim().length == 0;
}
String.prototype.splitOffLast = function(delimiter = '.') {
return this.split(delimiter).removeLast().join(delimiter).blankToUndefined();
}
// To Object - if string is Json
String.prototype.toObject = function() {
if(this.isBlank())
return {};
return JSON.parse(this, App.Strings.parseDate);
}
// object prototype
Object.prototype.toJson = function() {
return JSON.stringify(this);
}
Object.prototype.using = function(func) {
return func.call(this, this);
}
http.handler.gs
function doPost(request) {
return new TasksNamespace.process(request);
}
calling convention
Just make arrays with the full function name and the rest are the function's arguments. It will return when everything is done, so it's like Promise.all()
var a = new TasksNamespace.all([
["App.Data.Firebase.Properties.getById",'T006DB4'],
["App.Data.External.CISC.Properties.getById",'T00A21F', true, 12],
["App.Maps.geoCode",'T022D62', false]
])
return preview
[ { functionName: 'App.Data.Firebase.Properties.getById',
arguments: [ 'T006DB4' ],
result:
{ Id: '',
Listings: [Object],
Pages: [Object],
TempId: 'T006DB4',
Workflow: [Object] } },
...
]
Notes
it can handle any static method, any method off a root object's tree, or any root (global) function.
it can handle 0 or more (any number) of arguments of any kind
it handles errors by returning the error from any post
// First create a trigger which will run after some time
ScriptApp.newTrigger("createAsyncJob").timeBased().after(6000).create();
/* The trigger will execute and first delete trigger itself using deleteTrigger method and trigger unique id. (Reason: There are limits on trigger which you can create therefore it safe bet to delete it.)
Then it will call the function which you want to execute.
*/
function createAsyncJob(e) {
deleteTrigger(e.triggerUid);
createJobsTrigger();
}
/* This function will get all trigger from project and search the specific trigger UID and delete it.
*/
function deleteTrigger(triggerUid) {
let triggers = ScriptApp.getProjectTriggers();
triggers.forEach(trigger => {
if (trigger.getUniqueId() == triggerUid) {
ScriptApp.deleteTrigger(trigger);
}
});
}
While this isn't quite an answer to your question, this could lead to an answer if implemented.
I have submitted a feature request to Google to modify the implementation of doGet() and doPost() to instead accept a completion block in the functions' parameters that we would call with our response object, allowing additional slow-running logic to be executed after the response has been "returned".
If you'd like this functionality, please star the issue here: https://issuetracker.google.com/issues/231411987?pli=1

Meteor-angular autocomplete from huge data

I have angular-meteor app that needs Material md-autocomplete from a collection with 53,296 documents with angularUtils.directives.dirPagination but this amount of data make my browser hang.
I'm publishing the collection with:
Meteor.publish('city', function (options, searchString) {
var where = {
'city_name': {
'$regex': '.*' + (searchString || '') + '.*' ,
'$options': 'i'
}
};
return City.find(where, options);
});
I subscribe with:
subscriptions: function () {
Meteor.subscribe('city');
this.register('city', Meteor.subscribe('city'));
}
and have pagination on controller :
$scope.currentPage = 1;
$scope.pageSize = 100;
$scope.sort = {city_name_sort : 1};
$scope.orderProperty = '1';
$scope.helpers({
city: function(){
return City.find({});
}
});
but it takes a long time to load and its make chrome stop working.
You already have most of the server-side searching done because your search is running inside a subscription. You should make sure that the city_name field is indexed in mongo! You should only return that field to minimize data transfer. You can also simplify your regex.
Meteor.publish('city', function (searchString) {
const re = new RegExp(searchString,'i');
const where = { city_name: { $regex: re }};
return City.find(where, {sort: {city_name: 1}, fields: {city_name: 1}});
});
What I've found helps with server-side auto-complete is:
Don't start searching until the user has typed 3 or 4 characters. This drastically narrows down the search results.
Throttle the search to only run every 500ms so that you're not sending every character to the server because then it has to keep re-executing the search. If the person is typing fast the search might only run every 2 or 3 characters.
Run the same .find() on the client that you're running on the server (instead of just querying for {}). That's just good practice since the client-side collection is the union of all subscriptions on that collection, there might be documents there that you don't want to list.
Lastly I don't know why you're subscribing twice here:
subscriptions: function () {
Meteor.subscribe('city');
this.register('city', Meteor.subscribe('city'));
}
only one of those Meteor.subscribe('city') calls is necessary.

How to get a published collection's total count, regardless of a specified limit, on the client?

I'm using the meteor-paginated-subscription package in my app. On the server, my publication looks like this:
Meteor.publish("posts", function(limit) {
return Posts.find({}, {
limit: limit
});
});
And on the client:
this.subscriptionHandle = Meteor.subscribeWithPagination("posts", 10);
Template.post_list.events = {
'click #load_more': function(event, template) {
template.subscriptionHandle.loadNextPage();
}
};
This works well, but I'd like to hide the #load_more button if all the data is loaded on the client, using a helper like this:
Template.post_list.allPostsLoaded = function () {
allPostsLoaded = Posts.find().count() <= this.subscriptionHandle.loaded();
Session.set('allPostsLoaded', allPostsLoaded);
return allPostsLoaded;
};
The problem is that Posts.find().count() is returning the number of documents loaded on the client, not the number available on the server.
I've looked through the Telescope project, which also uses the meteor-paginated-subscription package, and I see code that does what I want to do:
allPostsLoaded: function(){
allPostsLoaded = this.fetch().length < this.loaded();
Session.set('allPostsLoaded', allPostsLoaded);
return allPostsLoaded;
}
But I'm not sure if it's actually working. Porting their code into mine does not work.
Finally, it does look like Mongo supports what I want to do. The docs say that, by default, cursor.count() ignores the effects of limit.
Seems like all the pieces are there, but I'm having trouble putting them together.
None of the answers do what you really want becase none provide solution that is reactive.
This package does exactly what you want and also reactive.
publish-counts
I think you can see the demo: counts-by-room in meteor doc
It can help you publish the counts of your posts at server and get it at client
You can simply write this:
// server: publish the current size of your post collection
Meteor.publish("counts-by-room", function () {
var self = this;
var count = 0;
var initializing = true;
var handle = Posts.find().observeChanges({
added: function (id) {
count++;
if (!initializing)
self.changed("counts", 'postCounts', {count: count});
},
removed: function (id) {
count--;
self.changed("counts", postCounts, {count: count});
}
});
initializing = false;
self.added("counts", 'postCounts', {count: count});
self.ready();
self.onStop(function () {
handle.stop();
});
});
// client: declare collection to hold count object
Counts = new Mongo.Collection("counts");
// client: subscribe to the count for posts
Tracker.autorun(function () {
Meteor.subscribe("postCounts");
});
// client: simply use findOne, you can get the count object
Counts.findOne()
The idea of sub.loaded() is to help you with exactly this problem.
Posts.count() isn't going to return the right thing because, as you've guessed, on the client, Meteor has no way of knowing the real number of posts that live on the server. But what the client knows is how many posts it's tried to load. That's what that .loaded() tells you, and is why the line this.fetch().length < this.loaded() will tell you if there are more posts on the server or not.
What I would do is write a Meteor server side method that retrieves the count like so:
Meteor.methods({
getPostsCount: function () {
return Posts.find().count();
}
});
Then call it on the client, in observe to make it reactive:
function updatePostCount() {
Meteor.call('getPostsCount', function (err, count) {
Session.set('postCount', count);
});
}
Posts.find().observe({
added: updatePostCount,
removed: updatePostCount
});
Although this question is old, I thought I would provide an answer that ended up working for me. I did not create the solution, I found the basis for it here (so credit where credit is due): Discover Meteor
Anyway, in my case I was trying to get "size" of the database from client side, so I can determine when to hide the "load more" -button. I was using template level subscriptions. Oh and for this solution to work, you need to add reactive-var -package. Here is my (in short):
/*on the server we define the method which returns
the number of posts in total in the database*/
if(Meteor.isServer){
Meteor.methods({
postsTotal: function() {
return PostsCollection.find().count();
}
});
}
/*In the client side we first create the reactive variable*/
if(Meteor.isClient){
Template.Posts.onCreated(function() {
var self = this;
self.totalPosts = new ReactiveVar();
});
/*then in my case, when the user clicks the load more -button,
we call the postsTotal-method and set the returned value as
the value of the totalPosts-reactive variable*/
Template.Posts.events({
'click .load-more': function (event, instance){
Meteor.call('postsTotal', function(error, result){
instance.totalPosts.set(result);
});
}
});
}
Hope this helps someone (I recommend checking the link first). For template level subscriptions, I used this as my guide Discover Meteor - template level subscriptions. This was my first stacked-post and I am just learning Meteor, so please have mercy...:D
Ouch this post is old, anyway maybe it will help someone.
I had exactly the same issue. I managed to solve it with 2 simple lines...
Remember the :
handle = Meteor.subscribeWithPagination('posts', 10);
Well I used in client handle.loaded() and Posts.find().count(). Because when they are different it means that all the posts are loaded. So here is my code :
"click #nextPosts":function(event){
event.preventDefault();
handle.loadNextPage();
if(handle.loaded()!=Posts.find().count()){
$("#nextPosts").fadeOut();
}
}
I had the same problem, and using the publish-counts package didn't work with the subs-manager package. I created a package that can set a reactive server-to-client session, and keep the document count in this session. You can find an example here:
https://github.com/auweb/server-session/#getting-document-count-on-the-client-before-limit-is-applied
I'm doing something like this:
On cliente
Template.postCount.posts = function() {
return Posts.find();
};
Then you create a template:
<template name="postCount">
{{posts.count}}
</template>
Then, whatever you want to show the counter: {{> postCount}}
Much easier than any solution i have seen.

Sporadic behaviour with Session variables in Meteor

I've been scratching my head as to why this code will work some of the time, but not all (or at least most of the time). I've found that it actually does run displaying the correct content in the browser some of the time, but strangely there will be days when I'll come back to the same code, run the server (as per normal) and upon loading the page will receive an error in the console: TypeError: 'undefined' is not an object (evaluating 'Session.get('x').html')
(When I receive that error there will be times where the next line in the console will read Error - referring to the err object, and other times when it will read Object - referring the data object!?).
I'm obviously missing something about Session variables in Meteor and must be misusing them? I'm hoping someone with experience can point me in the right direction.
Thanks, in advance for any help!
Here's my dummy code:
/client/del.html
<head>
<title>del</title>
</head>
<body>
{{> hello}}
</body>
<template name="hello">
Hello World!
<div class="helloButton">{{{greeting}}}</div>
</template>
My client-side javascript file is:
/client/del.js
Meteor.call('foo', 300, function(err, data) {
err ? console.log(err) : console.log(data);
Session.set('x', data);
});
Template.hello.events = {
'click div.helloButton' : function(evt) {
if ( Session.get('x').answer.toString() === evt.target.innerHTML ) {
console.log('yay!');
}
}
};
Template.hello.greeting = function() {
return Session.get('x').html;
};
And my server-side javascript is:
/server/svr.js
Meteor.methods({
doubled: function(num) {
return num * 2;
},
foo: function(lmt) {
var count = lmt,
result = {};
for ( var i = 0; i < lmt; i++ ) {
count++;
}
count = Meteor.call('doubled', count);
result.html = "<em>" + count + "</em>";
result.answer = count;
return result;
}
});
I think it's just that the session variable won't be set yet when the client first starts up. So Session.get('x') will return undefined until your method call (foo) returns, which almost certainly won't happen before the template first draws.
However after that it will be in the session, so things will probably behave right once you refresh.
The answer is to just check if it's undefined before trying to access the variable. For example:
Template.hello.greeting = function() {
if (Session.get('x')) return Session.get('x').html;
};
One of the seven principles of Meteor is:
Latency Compensation. On the client, use prefetching and model simulation to make it look like you have a zero-latency connection to the database.
Because there is latency, your client will first attempt to draw the lay-out according to the data it has at the moment your client connects. Then it will do the call and then it will update according to the call. Sometimes the call might be able to respond fast enough to be drawn at the same time.
As now there is a chance for the variable to not be set, it would throw an exception in that occasion and thus break down execution (as the functions in the call stack will not continue to run).
There are two possible solutions to this:
Check that the variable is set when using it.
return Session.get('x') ? Session.get('x').html : '';
Make sure the variable has an initial value by setting it at the top of the script.
Session.set('x', { html = '', answer = ''});
Another approach would be to add the templates once the call responds.
Meteor.call('foo', 300, function(err, data) {
Session.set('x', data);
$('#page').html(Meteor.ui.render(function() {
return Template.someName();
}));
});

Resources