I'm using node.js as a backend server for sending push notification from the Firebase Cloud Messaging service. The notifications are working fine with local server but on live server, I get this error:
Error while making request: socket hang up. Error code: ECONNRESET
Things to consider are that...
Number of users are in the thousands on live server
Firebase version is firebase-admin#6.5.1
Previously unregistered tokens are still there. But now registered tokens are being stored.
This is my code for sending notifications:
for (let c = 0; c < tokens.length; c++)
{
let notifyTo = tokens[c];
const platform = platforms[c];
let payload;
if (platform === "ios") {
payload = {
notification: {
title: "title",
subtitle :"messgae",
sound: "default",
badge: "1"
},
data: {
sendFrom: "",
notificationType: "",
flag: "true"
}
};
} else if (platform === "android") {
payload = {
data: {
title: "",
message : "",
flag: "true"
}
};
}
const registrationtoken = notifyTo;
await admin.messaging().sendToDevice(registrationtoken, payload)
.then(function (response) {
console.log("Successfully sent message:");
})
.catch(function (error) {
console.log("Error sending message: ");
});
}
Your issue is caused by your function taking too long to respond to the client (more than 60 seconds) and is caused by the following line:
await admin.messaging().sendToDevice(registrationtoken, payload)
Because you are waiting for each call of sendToDevice() individually, you are running your for-loop in synchronous sequential order, rather than asynchronously in parallel.
To avoid this, you want to make use of array mapping and Promise.all() which will allow you to build a queue of sendToDevice() requests. As in your current code, any failed messages will be silently ignored, but we will also count them.
Your current code makes use of two arrays, tokens and platforms, so in the code below I use a callback for Array.prototype.map() that takes two arguments - the current mapped value (from tokens) and it's index (your for-loop's c value). The index is then used to get the correct platform entry.
let fcmPromisesArray = tokens.map((token, idx) => {
let platform = platforms[idx];
if (platform === "ios") {
payload = {
notification: {
title: "title",
subtitle :"messgae",
sound: "default",
badge: "1"
},
data: {
sendFrom: "",
notificationType: "",
flag: "true"
}
};
} else if (platform === "android") {
payload = {
data: {
title: "",
message : "",
flag: "true"
}
};
}
return admin.messaging().sendToDevice(token, payload) // note: 'await' was changed to 'return' here
.then(function (response) {
return true; // success
})
.catch(function (error) {
console.log("Error sending message to ", token);
return false; // failed
});
});
let results = await Promise.all(fcmPromisesArray); // wait here for all sendToDevice() requests to finish or fail
let successCount = results.reduce((acc, v) => v ? acc + 1 : acc, 0); // this minified line just counts the number of successful results
console.log(`Successfully sent messages to ${successCount}/${results.length} devices.`);
After this snippet has run, don't forget to send a result back to the client using res.send(...) or similar.
Related
I'm setting up a new web project using Deno and oak.
I've passed an AbortSignal into the listen call and I'm listening for a SIGTERM from the OS and calling abort, in case this is not built-in behaviour.
Similar to setups described here: Deno and Docker how to listen for the SIGTERM signal and close the server
Question: Upon abort, will the await listen(...) call return immediately or after all remaining requests have completed?
If not then I guess I will need to accurately count concurrent requests using Atomics and wait until that counter drops to zero before ending the process.
Rather than rely on second hand information from someone else (which might not be correct), why not just do a test and find out for yourself (or review the source code)?
Here's a reproducible example which indicates that — when using Deno#1.28.2 with Oak#11.1.0 — the server gracefully shuts down: it still responds to a pending request even after the AbortSignal is aborted:
so-74600368.ts:
import {
Application,
type Context,
} from "https://deno.land/x/oak#v11.1.0/mod.ts";
import { delay } from "https://deno.land/std#0.166.0/async/delay.ts";
async function sendRequestAndLogResponseText(): Promise<void> {
try {
const response = await fetch("http://localhost:8000/");
if (!response.ok) {
throw new Error(`Response not OK (Status code: ${response.status})`);
}
const text = await response.text();
console.log(performance.now(), text);
} catch (ex) {
console.error(ex);
}
}
async function sendSquentialRequsets(numOfRequests: number): Promise<void> {
for (let i = 0; i < numOfRequests; i += 1) {
await sendRequestAndLogResponseText();
}
}
function printStartupMessage({ hostname, port, secure }: {
hostname: string;
port: number;
secure?: boolean;
}): void {
if (!hostname || hostname === "0.0.0.0") hostname = "localhost";
const address =
new URL(`http${secure ? "s" : ""}://${hostname}:${port}/`).href;
console.log(`Listening at ${address}`);
console.log("Use ctrl+c to stop");
}
async function main() {
const log = new Map<Context, boolean>();
const controller = new AbortController();
controller.signal.addEventListener("abort", () => {
console.log(performance.now(), "Abort method invoked");
});
const app = new Application();
app.use(async (ctx) => {
log.set(ctx, false);
if (log.size > 2) {
console.log(performance.now(), "Aborting");
controller.abort(new Error("Received third request. Aborting now."));
}
// A bit of artificial delay, to ensure that no unaccounted for latency
// might cause a non-deterministic/unexpected result:
await delay(300);
ctx.response.body = `Response OK: (#${log.size})`;
log.set(ctx, true);
});
app.addEventListener("listen", (ev) => {
console.log(performance.now(), "Server starting");
printStartupMessage(ev);
});
const listenerPromise = app.listen({
hostname: "localhost",
port: 8000,
signal: controller.signal,
})
.then(() => {
console.log(performance.now(), "Server stopped");
return { type: "server", ok: true };
})
.catch((reason) => ({ type: "server", ok: false, reason }));
const requestsPromise = sendSquentialRequsets(3)
.then(() => {
console.log(performance.now(), "All responses OK");
return { type: "requests", ok: true };
})
.catch((reason) => ({ type: "requests", ok: false, reason }));
const results = await Promise.allSettled([listenerPromise, requestsPromise]);
for (const result of results) console.log(result);
const allResponsesSent = [...log.values()].every(Boolean);
console.log({ allResponsesSent });
}
if (import.meta.main) main();
% deno --version
deno 1.28.2 (release, x86_64-apple-darwin)
v8 10.9.194.1
typescript 4.8.3
% deno run --allow-net=localhost so-74600368.ts
62 Server starting
Listening at http://127.0.0.1:8000/
Use ctrl+c to stop
378 Response OK: (#1)
682 Response OK: (#2)
682 Aborting
682 Abort method invoked
990 Server stopped
992 Response OK: (#3)
992 All responses OK
{ status: "fulfilled", value: { type: "server", ok: true } }
{ status: "fulfilled", value: { type: "requests", ok: true } }
{ allResponsesSent: true }
How to detect in Deno that remote has closed (aborted) the TCP/IP (HTTP) connection?
const server = Deno.listen({ port: 8080 });
for await (const conn of server) {
conn.on('abort', () => { // <-- API I expect but doesn't exist
// ...
});
const httpConn = Deno.serveHttp(conn);
for await (const requestEvent of httpConn) {
//
}
}
While Deno does not provide an API to know when a connection was closed, the most reliable way to detect a connection closure is to attempt to write to it, which will throw an error if it's closed.
The following snippet that tries to perform a zero-length write periodically will solve your issue:
const server = Deno.listen({ port: 8080 });
for await (const conn of server) {
const httpConn = Deno.serveHttp(conn);
for await (const requestEvent of httpConn) {
let interval;
const stream = new ReadableStream({
start(controller) {
interval = setInterval(() =>
// attempt to write a 0 length buffer, it will fail if
// connection is closed
controller.enqueue(new Uint8Array(0)),
500); // tune interval depending on your needs
},
async pull(controller) {
/*
const result = await someComputation();
// in case you want to return some response
controller.enqueue(result);
// cleanup
clearInterval(interval);
controller.close();
*/
},
});
requestEvent.respondWith(new Response(stream))
.catch((err) => {
clearInterval(interval);
// check for <connection closed> error
if (err.message.includes('connection closed before message completed')) {
// stop your operation
console.log('connection closed');
}
});
}
}
The error logic can also be added to ReadableStreamDefaultController cancel method:
const stream = new ReadableStream({
start(controller) {
// ..
},
async pull(controller) {
// ...
},
cancel(reason) {
clearInterval(interval);
if (reason && reason.message.includes('connection closed before message completed')) {
// stop your operation
console.log('connection closed');
}
}
});
AFAIK there's not an event-oriented API, but when the connection's ReadableStream closes, you'll know that the connection has closed. This will also be reflected in Deno's internal resource map. Consider the following self-contained example:
A TCP listener is started, and is closed after 500ms. While it is open, three connections are created and closed (once every 100ms). When each connection is established:
The current TCP entries from Deno's resource map are printed to the console.
A reader is acquired on the connection's readable stream and a read is performed. Because the connection is closed from the client without any data being written, the first read is the final read (reflected in the read result's done property being true).
The reader's lock on the stream is released. The stream is closed.
The current TCP entries from Deno's resource map are printed to the console. Note that none appear at this point.
so-74228364.ts:
import { delay } from "https://deno.land/std#0.161.0/async/delay.ts";
function getTCPConnectionResources() {
return Object.fromEntries(
Object.entries(Deno.resources()).filter(([, type]) => type === "tcpStream"),
);
}
async function startServer(options: Deno.ListenOptions, signal: AbortSignal) {
const listener = Deno.listen(options);
signal.addEventListener("abort", () => listener.close());
for await (const conn of listener) {
console.log("Resources after open:", getTCPConnectionResources());
const reader = conn.readable.getReader();
reader.read()
.then(({ done }) => console.log({ done }))
.then(() => {
reader.releaseLock();
console.log("Resources after final read:", getTCPConnectionResources());
});
}
}
const controller = new AbortController();
delay(500).then(() => controller.abort());
const options: Deno.ListenOptions = {
hostname: "localhost",
port: 8080,
};
startServer(options, controller.signal);
for (let i = 0; i < 3; i += 1) {
await delay(100);
(await Deno.connect(options)).close();
}
% deno --version
deno 1.27.0 (release, x86_64-apple-darwin)
v8 10.8.168.4
typescript 4.8.3
% deno run --allow-net=localhost so-74228364.ts
Resources after open: { "7": "tcpStream" }
{ done: true }
Resources after final read: {}
Resources after open: { "10": "tcpStream" }
{ done: true }
Resources after final read: {}
Resources after open: { "13": "tcpStream" }
{ done: true }
Resources after final read: {}
This is my first question ever here:-)
I need to list users in our Cognito use pool. Seems this can be done only using the aws-sdk CognitoIdentityServiceProvider APIs. I got the below code to work perfectly from my local node.js. It lists all users as expected.
However, the same code behaves differently when put into an AWS lambda function. It still runs, but it never waits for the Cognito listUsers() call to return. It just simply completes, like the "await" is not waiting at all. None of the console.log() is invoked from the client.listUsers(params, function(err, data)..._ block.
I tested this inside Lambda directly as well as from AWS API gateway. The same null is return. The call itself is successful, just no data is returned.
See log at the end.
BTW, I did create a role and add a policy giving the role full access to the cognito user pool.
What did I miss? Appreciate your help!
Martin C.
-----------------code------------------------
async function getUserList() {
console.log("enter LAMDA function**********");
var aws = require('aws-sdk');
aws.config.update({accessKeyId: 'xxxxxxxx', secretAccessKey: 'xxxxxxxxxxx'});
var CognitoIdentityServiceProvider = aws.CognitoIdentityServiceProvider;
var client = new CognitoIdentityServiceProvider({ apiVersion: '2016-04-19', region: 'us-east-2' });
var params = {
UserPoolId: 'us-east-xxxxxxxx', /* required */
AttributesToGet: [
'given_name','family_name','phone_number','email','profile'
],
Filter: null,
Limit: 0,
PaginationToken: null
};
console.log("Right before call the listUser method");
let result = await client.listUsers(params, function(err, data) {
console.log("call back reached!");
if (err) {
console.log(err, err.stack); // an error occurred
const response = {
statusCode: 500,
body: JSON.stringify('An error occurred.'),
}
return response;
}
else {
console.log(data);
var count = data.Users.length;
// successful response
const response = {
statusCode: 200,
body: JSON.stringify("sucessful list users! User count="+count)
}
return response;
}
});
console.log("no waiting here. async!!!")
}
getUserList();
***************Lambda log*****************
**************Log when called from node.js****************
getUserList is your lambda function? I don't know why you call it by your self getUserList().
I see, you are using lambda runtime is nodejs version > 8, you use await keyword with a callback function(fail) => you not wait anything.
When a function call by Lambda, the function (async function) will finish when get a return or run to end of function (without return), in your case the function finish when console.log("no waiting here. async!!!") has been executed. In local environment, the funciton finishs when callstack has been clear (do not have any callback function in callstack).
Right way, you have use promise version of aws-sdk then use await syntax to get a result. Relate to How to use Async and Await with AWS SDK Javascript
async function getUserList() {
console.log("enter LAMDA function**********");
var aws = require('aws-sdk');
aws.config.update({ accessKeyId: 'xxxxxxxx', secretAccessKey: 'xxxxxxxxxxx' });
var CognitoIdentityServiceProvider = aws.CognitoIdentityServiceProvider;
var client = new CognitoIdentityServiceProvider({ apiVersion: '2016-04-19', region: 'us-east-2' });
var params = {
UserPoolId: 'us-east-xxxxxxxx', /* required */
AttributesToGet: [
'given_name', 'family_name', 'phone_number', 'email', 'profile'
],
Filter: null,
Limit: 0,
PaginationToken: null
};
console.log("Right before call the listUser method");
try {
let result = await client.listUsers(params).promise(); // use Promise style
console.log(data);
var count = data.Users.length;
// successful response
const response = {
statusCode: 200,
body: JSON.stringify("sucessful list users! User count=" + count)
}
return response; // return to finish function
} catch (err) {
console.log(err, err.stack); // an error occurred
const response = {
statusCode: 500,
body: JSON.stringify('An error occurred.'),
}
return response;
}
}
getUserList(); // remove this line when deploy funtion to Lambda.
I have a Cloud Function triggered by a pub/sub event. I use sendgrid nodejs api. The main idea is sending my clients a weekly stats email. sendEmail() function run for each client (80 times). But when I check function logs I see that 25-30 of client emails are sent with success but the remaining it gives that error: "socket hang up"
I shortened the whole code to show the main part related sending email. Here is the last part.
// I shortened the whole function as it is a very long function.
// The main and the last part is as below
// I have nearly 80 clients and sendEmail function run for each client.
function calcData(i, data) {
return admin.database().ref('clientUrlClicks/' + data.key)
.orderByChild('date')
.startAt(dateStartEpox)
.endAt(dateEndEpox)
.once('value', urlClickSnap => {
clients[i].clickTotalWeek = urlClickSnap.numChildren();
clients[i].listTotalWeek = 0;
admin.database().ref('clientImpressions/' + data.key)
.orderByKey()
.startAt(dateStart)
.endAt(dateEnd)
.once('value', snap => {
snap.forEach(function(impressionSnap) {
clients[i].listTotalWeek += impressionSnap.val();
})
}).then(resp => {
return sendEmail(i, clients[i]);
}).catch(err => {
console.log(err);
});
}).catch(err => {
clients[i].clickTotalWeek = 0;
console.log(err);
});
}
function sendEmail(i, data) {
var options = {
method: 'POST',
url: 'https://api.sendgrid.com/v3/mail/send',
headers:
{
'content-type': 'application/json',
authorization: 'Bearer ' + sgApiKey
},
body:
{
personalizations:
[{
to: [{ email: data.email, name: data.name }],
dynamic_template_data:
{
dateStart: xxx,
dateEnd: xxx,
}
}],
from: { email: 'info#xxx.com', name: 'xxx' },
reply_to: { email: 'info#xxx.com', name: 'xxx' },
template_id: 'd-f44eeexxxxxxxxxxxxx'
},
json: true
};
request(options, function (error, response, body) {
if (error) {
console.log("err: " + error);
return;
}
return;
});
}
Edit:
In addition to answers below related to "chaining the promises correctly", I also added all emails and personalizations to "personalizations" array as an object on "sendEmail" function. So, instead making a request for each email I make one request. No problem now.
You are not chaining the promises correctly and therefore not returning a final promise at the end of the chaining, which is mandatory for a Cloud Function.
The following set of modifications is a first attempt to solve this problem.
Also, it is not crystal clear how do you call Sendgrid and return the Promise returned by the Sendgrid call. I would suggest that you use the send() method, which returns a Promise, as explained in the doc of the Sendgrid v3 Web API for Node.js, see https://github.com/sendgrid/sendgrid-nodejs/tree/master/packages/mail.
function calcData(i, data) {
//Declare clients aray here
return admin.database().ref('clientUrlClicks/' + data.key)
.orderByChild('date')
.startAt(dateStartEpox)
.endAt(dateEndEpox)
.once('value')
.then(urlClickSnap => {
clients[i].clickTotalWeek = urlClickSnap.numChildren();
clients[i].listTotalWeek = 0;
return admin.database().ref('clientImpressions/' + data.key) //Here you didn't return the promise
.orderByKey()
.startAt(dateStart)
.endAt(dateEnd)
.once('value');
.then(snap => {
snap.forEach(function(impressionSnap) {
clients[i].listTotalWeek += impressionSnap.val();
})
return sendEmail(i, clients[i]);
}).catch(err => {
clients[i].clickTotalWeek = 0;
console.log(err);
return null;
});
}
I see two issues with your code related to promise chaining, which may be causing this problem.
First is that you are using request with callback in your sendEmail function. This will simply not wait for your network call to finish and returns the function. Now this will build up the calls in parallel and before you hit your 80 clients counts the execution of your cloud function will finish. The solution would be to use request-promise-native (https://github.com/request/request-promise-native) library with your request library. So your sendEmail Function will now become
sendEmail (i, data) {
.
.
.
return rpn(options).then((d)=>{return d}).catch((e)=>{return console.log(e)})
}
Other solution is to use sendgrid client for nodejs which will simply return the promise and you don't need to use request. https://github.com/sendgrid/sendgrid-nodejs/tree/master/packages/mail
Second issue is in you call for data read from firebase where you are also using callbacks instead of promises. Correct solution will be:
function calcData(i, data) {
return admin.database().ref('clientUrlClicks/' + data.key)
.orderByChild('date')
.startAt(dateStartEpox)
.endAt(dateEndEpox)
.once('value').then( urlClickSnap => {
clients[i].clickTotalWeek = urlClickSnap.numChildren();
clients[i].listTotalWeek = 0;
return admin.database().ref('clientImpressions/' + data.key)
.orderByKey()
.startAt(dateStart)
.endAt(dateEnd)
.once('value').then( snap => {
snap.forEach(function(impressionSnap) {
clients[i].listTotalWeek += impressionSnap.val();
})
return sendEmail(i, clients[i]);
})
.catch(err => {
console.log(err);
});
}).catch(err => {
clients[i].clickTotalWeek = 0;
console.log(err);
});
}
This will make sure that you function calcData returns after finishing the execution of all the promises chained.
One more thing if you are calling calcData in a loop, then make sure that you store all the promises in an array and after loop call Promise.all(promisesArray), so that you function waits for all the executions to finish.
I am trying to use the DDP Realtime API to initiate a LiveChat conversation but I am facing issues.
https://rocket.chat/docs/developer-guides/realtime-api/livechat-api
I am doing all the steps as per the documentation. In the first API outcome, you can see that it saus numAgents: 2 and online: true. However when I try to send a message to the same department, it says: "Sorry, no online agents".
Is there a way to find out the problem?
Result of livechat:getInitialData
{ enabled: true,
title: 'xyz.com',
color: '#C1272D',
registrationForm: false,
room: null,
triggers: [],
departments:
[ { _id: 'CxCTgXL4csw3TcW6S',
enabled: true,
name: 'Support',
description: '',
numAgents: 2,
showOnRegistration: true,
_updatedAt: 2017-09-24T06:46:39.657Z } ],
allowSwitchingDepartments: true,
online: true,
offlineColor: '#666666',
offlineMessage: 'We are not online right now. Please leave us a message:',
offlineSuccessMessage: '',
offlineUnavailableMessage: '',
displayOfflineForm: true,
videoCall: true,
offlineTitle: 'Leave a message',
language: '',
transcript: false,
transcriptMessage: 'Would you like a copy of this chat emailed?' }
Result of livechat:registerGuest
{ userId: 'j65Cp5peeLJLYhWQi',
token: 'J8IpnpB1yN1AYtO0e0EzLhuaRhe0zaZkjHBAamsehSO' }
Result of Login
{ id: 'j65Cp5peeLJLYhWQi',
token: 'J8IpnpB1yN1AYtO0e0EzLhuaRhe0zaZkjHBAamsehSO',
tokenExpires: 2017-12-23T07:45:01.928Z }
Result of sendMessageLivechat
{ isClientSafe: true,
error: 'no-agent-online',
reason: 'Sorry, no online agents',
message: 'Sorry, no online agents [no-agent-online]',
errorType: 'Meteor.Error' }
These are the parameters I am sending to sendMessageLiveChat.
"_id" : "j65Cp5peeLJLYhWQi"
"rid" : "a_random_string"
"msg": "Hello"
"token" : "J8IpnpB1yN1AYtO0e0EzLhuaRhe0zaZkjHBAamsehSO"
Could someone help me?
This is how I called registerGuest.
ddpClient.call("livechat:registerGuest",[{"token":authToken,"name":"test1","email":"test2#gmail.com","department":department._id},25],function(err, info){
});
the token passed by me here is the admin's authToken
The ddpClient object is obtained using the DDP npm package.
I solved this by a combination of
setting the bot as livechat agent & manager at the same time (I've read that tip somewhere it might be nonsense)
in Admin -> Omnichannel -> routing I've set 'accept even when no agents are online' (since my bot was never online, bould it was replying when DMessaged) + 'assign bot agents to new conversations'
I've setup myself a livechat-manager + livechat-agent role, but stayed in a different department, that way I can takeover
The rocket chat live api docs are quite out of date, just got stream-room-messages working because of a random forum post. Generally, registerGuest works with very minimal parameters as well, namely a random, self generated token + a name.
Here's my code for the complete setup
async subscribeToLiveRoom(message){
var _self = this
// let initial = await this.api
// .call("livechat:getInitialData",[token])
// register
const token = this.randomString()
var guestUser = await this.api
.call(
'livechat:registerGuest',
[{
token: token,
name: _self.$auth.user.name
}]
)
.catch(console.error)
console.log('guest', guestUser.visitor.token)
this.setActiveGuest(guestUser)
var roomId = this.randomString()
this.setActiveRoom(roomId)
let msg = await this.api
.call(
'sendMessageLivechat',
[{
_id: _self.randomString(),
rid: roomId,
msg: message,
token: guestUser.visitor.token
}])
.catch(console.error)
try {
let liveStream = await this.$subscribe("stream-livechat-room",[
roomId,
{
"useCollection": true,
"args":[
{
"visitorToken": guestUser.visitor.token
}
]
}
])
this.msgLive = await this.find('stream-livechat-room')
} catch (e) {
console.log(e)
}
//
try {
var roomStream = await this.$subscribe("stream-room-messages",[
roomId,
{
"useCollection": true,
"args":[
{
"visitorToken": guestUser.visitor.token
}
]
}
])
console.log('roomstream')
var update = this.find('stream-room-messages')
} catch (e) {
console.log('an error occured', e)
}
console.log( this.msg)
},
async sendToLiveRoom(message, rId){
var _self = this
// let initial = await this.api
// .call("livechat:getInitialData",[token])
// register
let msg = await this.api
.call(
'sendMessageLivechat',
[{
_id: _self.randomString(),
rid: rId,
msg: message,
token: _self.guest.visitor.token
}])
.catch(console.error)
},
By the way, since it's not well documented, you will get room-messages in livechat rooms via subscribing to stream-room-messages while you get room status changes (like switched to another agent) by subscribing to stream-livechat-room