How to delay between each axios call - http

I need to put some delays between each axios POST call so POST calls are in sequence -- wait one finishes before issue the next POST.
The delay code I put in seems to delay -- console log shows "Delaying" and pauses there for some seconds, but on the server side, POST calls are still concurrent.
import * as Axios from "axios";
delay(delayTime:number) {
var tNow = Date.now();
var dateDiff = 0;
do {
dateDiff = Date.now() - tNow;
} while (dateDiff < delayTime); //milliseconds
}
// the code below is in a for loop
let axiosConfig = {
url: myurl,
method: ‘POST’,
timeout: 5,
headers: {
'Authorization' : AuthStr,
'Accept' : 'application/json',
'Content-Type' : 'application/json'
},
data: objContent
}
console.log(">>>>>>>>>>>>>>>>>>> Delaying”);
delay(10000);
let request = Axios.create().request(axiosConfig).
catch((rejection:any) => {
// some code
});

In order to wait one HTTP request finishes before issue the next request, you need to queue the HTTP request (queue async operation). The steps are:
When HTTP request is needed, add it to the queue.
Check whether there are any elements in the queue. If any, pick up one and execute it.
After that HTTP request is finished, goto step 2.
Example code would look like below:
const axios = require('axios');
let queue = [];
function sendRequest(callback) {
axios.get('http://example.com')
.then(function() {
callback();
}).catch(function () {
callback();
});
}
function addRequestToQueue() {
let id = (Math.random()*100000).toFixed(0);
if (queue.length === 0) {
sendRequest(function() {
queue.splice(queue.indexOf(id), 1);
consumeQueue();
});
}
queue.push(id);
}
function consumeQueue() {
if (!queue.length) {
return;
}
let id = queue[0];
sendRequest(function() {
queue.splice(queue.indexOf(id), 1);
consumeQueue();
});
}
addRequestToQueue();
addRequestToQueue();
addRequestToQueue();
addRequestToQueue();
addRequestToQueue();

Related

Deno connection close event

How to detect in Deno that remote has closed (aborted) the TCP/IP (HTTP) connection?
const server = Deno.listen({ port: 8080 });
for await (const conn of server) {
conn.on('abort', () => { // <-- API I expect but doesn't exist
// ...
});
const httpConn = Deno.serveHttp(conn);
for await (const requestEvent of httpConn) {
//
}
}
While Deno does not provide an API to know when a connection was closed, the most reliable way to detect a connection closure is to attempt to write to it, which will throw an error if it's closed.
The following snippet that tries to perform a zero-length write periodically will solve your issue:
const server = Deno.listen({ port: 8080 });
for await (const conn of server) {
const httpConn = Deno.serveHttp(conn);
for await (const requestEvent of httpConn) {
let interval;
const stream = new ReadableStream({
start(controller) {
interval = setInterval(() =>
// attempt to write a 0 length buffer, it will fail if
// connection is closed
controller.enqueue(new Uint8Array(0)),
500); // tune interval depending on your needs
},
async pull(controller) {
/*
const result = await someComputation();
// in case you want to return some response
controller.enqueue(result);
// cleanup
clearInterval(interval);
controller.close();
*/
},
});
requestEvent.respondWith(new Response(stream))
.catch((err) => {
clearInterval(interval);
// check for <connection closed> error
if (err.message.includes('connection closed before message completed')) {
// stop your operation
console.log('connection closed');
}
});
}
}
The error logic can also be added to ReadableStreamDefaultController cancel method:
const stream = new ReadableStream({
start(controller) {
// ..
},
async pull(controller) {
// ...
},
cancel(reason) {
clearInterval(interval);
if (reason && reason.message.includes('connection closed before message completed')) {
// stop your operation
console.log('connection closed');
}
}
});
AFAIK there's not an event-oriented API, but when the connection's ReadableStream closes, you'll know that the connection has closed. This will also be reflected in Deno's internal resource map. Consider the following self-contained example:
A TCP listener is started, and is closed after 500ms. While it is open, three connections are created and closed (once every 100ms). When each connection is established:
The current TCP entries from Deno's resource map are printed to the console.
A reader is acquired on the connection's readable stream and a read is performed. Because the connection is closed from the client without any data being written, the first read is the final read (reflected in the read result's done property being true).
The reader's lock on the stream is released. The stream is closed.
The current TCP entries from Deno's resource map are printed to the console. Note that none appear at this point.
so-74228364.ts:
import { delay } from "https://deno.land/std#0.161.0/async/delay.ts";
function getTCPConnectionResources() {
return Object.fromEntries(
Object.entries(Deno.resources()).filter(([, type]) => type === "tcpStream"),
);
}
async function startServer(options: Deno.ListenOptions, signal: AbortSignal) {
const listener = Deno.listen(options);
signal.addEventListener("abort", () => listener.close());
for await (const conn of listener) {
console.log("Resources after open:", getTCPConnectionResources());
const reader = conn.readable.getReader();
reader.read()
.then(({ done }) => console.log({ done }))
.then(() => {
reader.releaseLock();
console.log("Resources after final read:", getTCPConnectionResources());
});
}
}
const controller = new AbortController();
delay(500).then(() => controller.abort());
const options: Deno.ListenOptions = {
hostname: "localhost",
port: 8080,
};
startServer(options, controller.signal);
for (let i = 0; i < 3; i += 1) {
await delay(100);
(await Deno.connect(options)).close();
}
% deno --version
deno 1.27.0 (release, x86_64-apple-darwin)
v8 10.8.168.4
typescript 4.8.3
% deno run --allow-net=localhost so-74228364.ts
Resources after open: { "7": "tcpStream" }
{ done: true }
Resources after final read: {}
Resources after open: { "10": "tcpStream" }
{ done: true }
Resources after final read: {}
Resources after open: { "13": "tcpStream" }
{ done: true }
Resources after final read: {}

SendGrid with Firebase Cloud Functions error: "socket hang up"

I have a Cloud Function triggered by a pub/sub event. I use sendgrid nodejs api. The main idea is sending my clients a weekly stats email. sendEmail() function run for each client (80 times). But when I check function logs I see that 25-30 of client emails are sent with success but the remaining it gives that error: "socket hang up"
I shortened the whole code to show the main part related sending email. Here is the last part.
// I shortened the whole function as it is a very long function.
// The main and the last part is as below
// I have nearly 80 clients and sendEmail function run for each client.
function calcData(i, data) {
return admin.database().ref('clientUrlClicks/' + data.key)
.orderByChild('date')
.startAt(dateStartEpox)
.endAt(dateEndEpox)
.once('value', urlClickSnap => {
clients[i].clickTotalWeek = urlClickSnap.numChildren();
clients[i].listTotalWeek = 0;
admin.database().ref('clientImpressions/' + data.key)
.orderByKey()
.startAt(dateStart)
.endAt(dateEnd)
.once('value', snap => {
snap.forEach(function(impressionSnap) {
clients[i].listTotalWeek += impressionSnap.val();
})
}).then(resp => {
return sendEmail(i, clients[i]);
}).catch(err => {
console.log(err);
});
}).catch(err => {
clients[i].clickTotalWeek = 0;
console.log(err);
});
}
function sendEmail(i, data) {
var options = {
method: 'POST',
url: 'https://api.sendgrid.com/v3/mail/send',
headers:
{
'content-type': 'application/json',
authorization: 'Bearer ' + sgApiKey
},
body:
{
personalizations:
[{
to: [{ email: data.email, name: data.name }],
dynamic_template_data:
{
dateStart: xxx,
dateEnd: xxx,
}
}],
from: { email: 'info#xxx.com', name: 'xxx' },
reply_to: { email: 'info#xxx.com', name: 'xxx' },
template_id: 'd-f44eeexxxxxxxxxxxxx'
},
json: true
};
request(options, function (error, response, body) {
if (error) {
console.log("err: " + error);
return;
}
return;
});
}
Edit:
In addition to answers below related to "chaining the promises correctly", I also added all emails and personalizations to "personalizations" array as an object on "sendEmail" function. So, instead making a request for each email I make one request. No problem now.
You are not chaining the promises correctly and therefore not returning a final promise at the end of the chaining, which is mandatory for a Cloud Function.
The following set of modifications is a first attempt to solve this problem.
Also, it is not crystal clear how do you call Sendgrid and return the Promise returned by the Sendgrid call. I would suggest that you use the send() method, which returns a Promise, as explained in the doc of the Sendgrid v3 Web API for Node.js, see https://github.com/sendgrid/sendgrid-nodejs/tree/master/packages/mail.
function calcData(i, data) {
//Declare clients aray here
return admin.database().ref('clientUrlClicks/' + data.key)
.orderByChild('date')
.startAt(dateStartEpox)
.endAt(dateEndEpox)
.once('value')
.then(urlClickSnap => {
clients[i].clickTotalWeek = urlClickSnap.numChildren();
clients[i].listTotalWeek = 0;
return admin.database().ref('clientImpressions/' + data.key) //Here you didn't return the promise
.orderByKey()
.startAt(dateStart)
.endAt(dateEnd)
.once('value');
.then(snap => {
snap.forEach(function(impressionSnap) {
clients[i].listTotalWeek += impressionSnap.val();
})
return sendEmail(i, clients[i]);
}).catch(err => {
clients[i].clickTotalWeek = 0;
console.log(err);
return null;
});
}
I see two issues with your code related to promise chaining, which may be causing this problem.
First is that you are using request with callback in your sendEmail function. This will simply not wait for your network call to finish and returns the function. Now this will build up the calls in parallel and before you hit your 80 clients counts the execution of your cloud function will finish. The solution would be to use request-promise-native (https://github.com/request/request-promise-native) library with your request library. So your sendEmail Function will now become
sendEmail (i, data) {
.
.
.
return rpn(options).then((d)=>{return d}).catch((e)=>{return console.log(e)})
}
Other solution is to use sendgrid client for nodejs which will simply return the promise and you don't need to use request. https://github.com/sendgrid/sendgrid-nodejs/tree/master/packages/mail
Second issue is in you call for data read from firebase where you are also using callbacks instead of promises. Correct solution will be:
function calcData(i, data) {
return admin.database().ref('clientUrlClicks/' + data.key)
.orderByChild('date')
.startAt(dateStartEpox)
.endAt(dateEndEpox)
.once('value').then( urlClickSnap => {
clients[i].clickTotalWeek = urlClickSnap.numChildren();
clients[i].listTotalWeek = 0;
return admin.database().ref('clientImpressions/' + data.key)
.orderByKey()
.startAt(dateStart)
.endAt(dateEnd)
.once('value').then( snap => {
snap.forEach(function(impressionSnap) {
clients[i].listTotalWeek += impressionSnap.val();
})
return sendEmail(i, clients[i]);
})
.catch(err => {
console.log(err);
});
}).catch(err => {
clients[i].clickTotalWeek = 0;
console.log(err);
});
}
This will make sure that you function calcData returns after finishing the execution of all the promises chained.
One more thing if you are calling calcData in a loop, then make sure that you store all the promises in an array and after loop call Promise.all(promisesArray), so that you function waits for all the executions to finish.

How to refresh data after refresh token refreshes jwt

I've been trying to get my refresh token to work for a while now, and I hope I'm close. My token refreshes and triggers a subsequent 200 call to whatever call caused the 401, but my the data on my page doesn't refresh.
When an access token expires, the following happens:
After the 401, the GetListofCompanyNames returns 200 with a list of names using the correct updated access token. However, my dropdown does not refresh.
My interceptor:
app.factory('authInterceptorService',['$q', '$location', 'localStorageService', '$injector', function($q, $location, localStorageService, $injector) {
return {
request: function(config) {
config.headers = config.headers || {};
var authData = localStorageService.get('authorizationData');
if (authData) {
config.headers.Authorization = 'Bearer ' + authData.token;
}
return config;
},
responseError: function(rejection) {
//var promise = $q.reject(rejection);
var authService = $injector.get('authService');
if (rejection.status === 401) {
// refresh the token
authService.refreshToken().then(function() {
// retry the request
var $http = $injector.get('$http');
return $http(rejection.config);
});
}
if (rejection.status === 400) {
authService.logOut();
$location.path('/login');
}
return $q.reject(rejection);
}
};
}
]);
My return statement on the 401 rejection looks suspect here, but I'm not sure what to replace it with. Thereby my question is: How can I get my page to refresh it's data when I make the new call?
Update:
This gets me past when the 200 returns and I can get a dropdown to refresh, but I lose any state on the page (ex. selected dropdown) with the below.
authService.refreshToken().then(function() {
var $state = $injector.get('$state');
$state.reload();
});
Back to the drawing board!
Try putting up your retry call in $timeout, it should work.
Here's the updated code:
app.factory('authInterceptorService',['$q', '$location', 'localStorageService', '$injector', function($q, $location, localStorageService, $injector) {
return {
request: function(config) {
config.headers = config.headers || {};
var authData = localStorageService.get('authorizationData');
if (authData) {
config.headers.Authorization = 'Bearer ' + authData.token;
}
return config;
},
responseError: function(rejection) {
//var promise = $q.reject(rejection);
var authService = $injector.get('authService');
if (rejection.status === 401) {
// refresh the token
authService.refreshToken().then(function() {
// retry the request
return $timeout(function() {
var $http = $injector.get('$http');
return $http(rejection.config);
}});
}
if (rejection.status === 400) {
authService.logOut();
$location.path('/login');
}
return $q.reject(rejection);
}
};
}
]);
$timeout returns a promise that is completed with what is returned
from the function parameter, so we can conveniently just return the
$http call wrapped in $timeout.
Thanks.
I think you may want to change up how you go about this. One way to go about this would be to inject the $rootScope into your authInterceptorService and then once you successfully refresh the token, call something like $rootScope.broadcast('tokenRefreshed').
I don't quite know how you have set up the view and controller that handles your dropdown, but I would set up a listener for that 'tokenRefreshed' event. From here, you can do another call to GetListofCompanyNames. If you do it this way you can easily control and ensure that the model gets updated.
My final solution:
app.factory('authInterceptorService', ['$q', '$location', 'localStorageService', '$injector', function($q, $location, localStorageService, $injector) {
var $http;
var retryHttpRequest = function(config, deferred) {
$http = $http || $injector.get('$http');
$http(config).then(function(response) {
deferred.resolve(response);
},
function(response) {
deferred.reject(response);
});
}
return {
request: function(config) {
config.headers = config.headers || {};
var authData = localStorageService.get('authorizationData');
if (authData) {
config.headers.Authorization = 'Bearer ' + authData.token;
}
return config;
},
responseError: function(rejection) {
var deferred = $q.defer();
if (rejection.status === 401) {
var authService = $injector.get('authService');
authService.refreshToken().then(function() {
retryHttpRequest(rejection.config, deferred);
},
function () {
authService.logOut();
$location.path('/login');
deferred.reject(rejection);
});
} else {
deferred.reject(rejection);
}
return deferred.promise;
}
};
}
]);
Copied almost 1 for 1 from https://github.com/tjoudeh/AngularJSAuthentication/blob/master/AngularJSAuthentication.Web/app/services/authInterceptorService.js .
This one transparently handles all requests and refreshes them when necessary. It logs out users when the refresh token is expired and passes errors along to the controllers by properly rejecting them. However, it doesn't seem to work with multiple in flight requests, I'll look into that when I get a use case for it in my system.

RxJS wait for second observable then retry original observable on error - TypeScript/Angular 2

I am fairly new to Angular 2, TypeScript and RxJS and I am creating a simple application that leverages the Salesforce Ajax Toolkit connections library.
I am trying to write a handler to catch when a token has expired any time a method from the connections library is called. I have created a service that essentially wraps the connections library to use observables. For example if we look at the insert function I have created my own wrapper function:
public insert(object: sforce.SObject): Observable<any> {
return new Observable(observer => {
// successfully inserted the record
let insertSuccess = (result) => {
observer.next(result);
observer.complete();
}
// An error occured inserting the record
let insertError = (result) => {
// This does not work yet
if (result.faultcode.indexOf('INVALID_SESSION_ID') != -1) {
this.refreshToken();
}
else {
observer.error(result);
}
}
let callback = { onSuccess: insertSuccess, onFailure: insertError };
sforce.connection.create([object], callback);
});
}
I have another function that refreshes the access token:
public refreshToken(): void {
this.loginService.login().subscribe(
response => {
Globals.SESSION_TOKEN = response.access_token;
//initialize the salesforce connection
this.init(Globals.SESSION_TOKEN, this.loginService.AuthParams.SOAP_URL);
},
error => {
}
);
}
I essentially want the original insert function to wait for refreshToken to complete. If it is successful I want to retry the same insert again, otherwise I want the original insert observable to call observer.error.
I've looked into retry and retryWhen, however I haven't been able to figure out how to implement it to wait for the refreshToken() function to complete. Any guidance or advice on this matter would be greatly appreciated. Thank you in advance.
The catch operator accepts a function which processes an error and the source Observable. This means that if you catch an error you can determine whether you want to resubscribe to the original source in the catch block:
public insert(object: sforce.SObject): Observable<any> {
return new Observable(observer => {
// successfully inserted the record
let insertSuccess = (result) => {
observer.next(result);
observer.complete();
}
// An error occured inserting the record
let insertError = (result) => observer.error(result);
let callback = { onSuccess: insertSuccess, onFailure: insertError };
sforce.connection.create([object], callback);
}).catch((err, source) => {
if (err.faultcode.indexOf('INVALID_SESSION_ID') != -1) {
//This waits for the refresh to complete and then resubscribes
//to the source
//If the refresh errors then it will skip the resubscribe
return this.refreshToken().flatMapTo(source);
}
//Non-authentication error
return Observable.throw(err);
});
}
Then make your refreshToken function into something like so:
public refreshToken(): Observable<any> {
return this.loginService.login()
.tap(response => {
Globals.SESSION_TOKEN = response.access_token;
//initialize the salesforce connection
this.init(Globals.SESSION_TOKEN, this.loginService.AuthParams.SOAP_URL);
});
}

Extracting data out of http call [duplicate]

I'm using Meteor for first time and i'm trying to have a simple http call within a method so i can call this method from the client.
The problem is that this async call it's keep running even if i put it within a wrapper.
Client side:
Meteor.call('getToken', function(error, results) {
console.log('entered');
if(error) {
console.log(error);
} else {
console.log(results);
}
});
Server Side
Meteor.methods({
getToken: function(){
// App url
var appUrl = 'myAppUrl';
// Key credentials
var apiKey = 'mykey';
var apiSecret = 'mySecret';
function asyncCall(){
Meteor.http.call(
'POST',
appUrl,
{
data: {
key: apiKey,
secret: apiSecret
}
}, function (err, res) {
if(err){
return err;
} else {
return res;
}
}
);
}
var syncCall = Meteor.wrapAsync(asyncCall);
// now you can return the result to client.
return syncCall;
}
});
I'm always getting an undefined return.
If i log the response within the http.post call i'm geting the correct response.
If i try to log the syncCall i get nothing.
I would very appreciate any help on this.
You should use the synchronous version of HTTP.post in this case. Give something like this a try:
Meteor.methods({
getToken: function() {
var appUrl = 'myAppUrl';
var data = {apiKey: 'mykey', apiSecret: 'mySecret'};
try {
var result = HTTP.post(appUrl, {data: data});
return result;
} catch (err) {
return err;
}
}
});
Instead of returning the err I'd recommend determining what kind of error was thrown and then just throw new Meteor.Error(...) so the client can see the error as its first callback argument.

Resources