I am using firebase functions with pubsub. The functions are invoked every time there is an incoming message (push)
I am looking at incorporating retry mechanism for the functions. The retry is not working. The sample function is shown below.
export const helloWorldWithRetry = functions
.runWith({
timeoutSeconds: 300,
memory: "512MB",
vpcConnector: cloudFunctionOptions.connectors["abcd_data_connector"],
vpcConnectorEgressSettings: "ALL_TRAFFIC",
maxInstances: 1,
failurePolicy: {
retry: {},
}
})
.region("asia-south1")
.pubsub.topic("helloWorldPubSubWithRetry")
.onPublish(async (message, context) => {
console.log("----------- ATTEMPTING -------------")
const eventAgeMs = Date.now() - Date.parse(context.timestamp);
console.log(`The event timestamp is ${Date.parse(context.timestamp)} and eventAge is ${eventAgeMs}`)
const eventMaxAgeMs = 60 * 1000;
if (eventAgeMs > eventMaxAgeMs) {
console.log(`Dropping event with age[ms]: ${eventAgeMs}`);
return;
}
throw new Error("Error, Retry")
});
Is there a possibility to use retry in these cases? The documentation says that the message is ack immediately on function invocation
Is putting the message back on queue or pull method the only logical way to achieve retries ?
The functions can be configured to retry the failures - refer docs.
And do this in the code -
functions.runWith({failurePolicy: true}).foo.onBar(myHandler);
So all you need to do is identify what kind of exceptions are re-triable and throw them from your function.
Related
In the case of network connectivity loss, the following code just loops endlessly and keeps making API calls. Is there a way to cancel with a timeout (for example, 5000 ms) using Firebase API? Or would I have to make my own Coroutine to handle this?
fun updateUserFieldInDB(
collectionPath: String,
strArr: ArrayList<String>,
onSuccess: (() -> Unit),
onFail: (() -> Unit)
) {
val fbUser = Firebase.auth.currentUser
if (fbUser == null) {
Log.i(TAG, "user is null....")
return
}
val db = Firebase.firestore
when (strArr.size) {
2 -> {
db.collection(collectionPath).document(fbUser.uid).update(strArr[0], strArr[1])
.addOnSuccessListener {
onSuccess()
}
.addOnFailureListener {
onFail()
}
}
}
}
The onSuccess ad onFail completion handlers for Firestore only fire once the write operation has been committed or rejected on the server. You should only use them if you're interested in detecting that situation, in which case the looping is to be expected.
If you only care whether the write operation was recorded by the Firestore client (in its local cache), the best way to detect that is when the update(strArr[0], strArr[1]) call completes.
So pretty much: when the next line of code executes, the write has been recorded locally; when the completion listeners fire, the write has been handled on the server.
Inside the Firebase Console, under the Cloud Messaging view, users are able to create test notifications. This functionality also allows you to schedule the time at which the notification will send to a device or set of devices.
Is it possible to create and send scheduled FCM notifications to specific devices by using firebase cloud functions and the Firebase Admin SDK? Is there an alternative way to solving this?
The current way that I send scheduled messages to users is like so:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const schedule = require('node-schedule');
admin.initializeApp();
exports.setScheduledNotification = functions.https.onRequest(async (req, res) => {
const key = req.query.notification_key;
const message = {
notification: {
title: 'Test Notification',
body: 'Test Notification body.'
}
};
var currentDate = new Date();
var laterDate = new Date(currentDate.getTime() + (1 * 60000));
var job = schedule.scheduleJob(key, laterDate, () => {
const snapshot = admin.messaging().sendToDevice(key, message);
});
return res.status(200).send(`Message has been scheduled.`);
});
First of all, I am unsure how node-schedule interacts with firebase cloud functions. The logs appear that the function terminates very quickly which I would think would be correct. The longer the operation runs the more costly it is in our firebase bills. The notification does still run on a scheduled time though. I'm confused on how that all is working behind the scenes.
Secondly, I am having issues canceling these scheduled notifications. The notifications will most likely be on a 2hr timed schedule from the point it gets created. Before the 2hrs is up, I'd like the have the ability to cancel/overwrite the notification with an updated scheduled time.
I tried this to cancel the notification and it failed to find the previously created notification. Here is the code for that:
exports.cancelScheduledNotification = functions.https.onRequest(async (req, res) => {
const key = req.query.notification_key;
var job = schedule.scheduledJobs[key];
job.cancel();
return res.status(200).send(`Message has been canceled.`);
});
Is it possible to tap into the scheduling functionality of firebase cloud messaging outside of the firebase console? Or am I stuck with hacking my way around this issue?
A Cloud Function can run for a maximum of 9 minutes. So unless you're using node-schedule for periods shorter than that, your current approach won't work. Even if it would work, or if you are scheduling for less than 9 minutes in advance, using this approach is very uneconomic as you'll be paying for the Cloud Functions for all this time while it's waiting.
A more common approach is to store information about what message you want to be delivered to whom at what time in a database, and then use regular scheduled functions to periodically check what messages to send. For more on this, see these previous questions:
Firebase scheduled notification in android
How to schedule push notifcations for react native expo?
Schedule jobs in Firebase
Ionic: Is it possible to delay incoming push FCM push notification from showing to my device until a specific time
Cloud Functions for Firebase trigger on time?
How to create cron jobs dynamically in firebase
A recent improvement on this is to use the Cloud Tasks API to programmatically schedule Cloud Functions to be called at a specific time with a specific payload, and then use that to send the message through FCM. Doug Stevenson wrote a great blog post about this here: How to schedule a Cloud Function to run in the future with Cloud Tasks (to build a Firestore document TTL). While the post is about deleting documents at a certain time, you can combine it with the previous approach to schedule FCM messages too.
Scheduling of tasks is now also described in the documentation on enqueueing functions with Cloud Tasks
A final option, and one I'd actually recommend nowadays, is to separate the delivery of the message from the display of the notification.
Display of data messages (unlike notification messages) is never handled by the system, and always left to your application. So you can deliver the FCM data message straight away that then contains the time to display the message, and then wake the device up to display the message (often called a local notification) at that time.
To make Frank's answer more tangible, I am including some sample code below for scheduled cloud functions, that can help you achieve the 'scheduled FCM notifications'.
You should store the information required to send your notification(s) in Firestore (e.g. the when-to-notify parameter and the FCM token(s) of the users you want to send the notification to) and run a cloud function every minute to evaluate if there is any notification that needs to be delivered.
The function checks what Firestore documents have a WhenToNofity parameter that is due, and send the notifications to the receiver tokens immediately. Once sent, the function sets the boolean 'notificationSent' to true, to avoid that the users receive the same notification again on the next iteration.
The code below achieves just that:
const admin = require('firebase-admin');
admin.initializeApp();
const database = admin.firestore();
exports.sendNotification = functions.pubsub.schedule('* * * * *').onRun(async (context) => {
//check whether notification should be sent
//send it if yes
const query = await database.collection("experiences")
.where("whenToNotify", '<=', admin.firestore.Timestamp.now())
.where("notificationSent", "==", false).get();
query.forEach(async snapshot => {
sendNotification(snapshot.data().tokens);
await database.doc('experiences/' + snapshot.id).update({
"notificationSent": true,
});
});
function sendNotification(tokens) {
let title = "INSERT YOUR TITLE";
let body = "INSERT YOUR BODY";
const message = {
notification: { title: title, body: body},
tokens: tokens,
android: {
notification: {
sound: "default"
}
},
apns: {
payload: {
aps: {
sound: "default"
}
}
}
};
admin.messaging().sendMulticast(message).then(response => {
return console.log("Successful Message Sent");
}).catch(error => {
console.log(error);
return console.log("Error Sending Message");
});
}
return console.log('End Of Function');
});
If you're unfamiliar with setting up cloud functions, you can check how to set them up here. Cloud functions require a billing account, but you get 1M cloud invocations per month for free, which is more than enough to cover the costs of this approach.
Once done, you can insert your function in the index.js file.
This trigger is used for detecting sequence in schedule has been updated, and help to update the schedule's overview status and finished time.
But it didn't always work when an internal error was occurred as below:
Error: 13 INTERNAL: An internal error occurred. at Object.exports.createStatusError
(/srv/node_modules/grpc/src/common.js:91:15) at Object.onReceiveStatus
(/srv/node_modules/grpc/src/client_interceptors.js:1204:28) at InterceptingListener._callNext
(/srv/node_modules/grpc/src/client_interceptors.js:568:42) at InterceptingListener.onReceiveStatus
(/srv/node_modules/grpc/src/client_interceptors.js:618:8) at callback
(/srv/node_modules/grpc/src/client_interceptors.js:845:24)
Here is my code:
export const calc_status = function.firestore.document("users/{userid}/schedule/{scheduledid}").onUpdate(async (change, context) => {
// before error occurred ...
const data = change.after.data();
let curStatus = data.status;
...
...
// after getting occurred ...
if(data.status !== curStatus ) {
data.status = curStatus;
if(curStatus === 'finished') {
data.finish_time = new Date().toISOString();
}
if(curStatus !== 'expired'){
data.update_time = data.expired_time;
data.finish_time = data.expired_time;
} else {
data.update_time = new Date().toISOString();
}
await change.after.ref.update(data);
return Status.SUCCEEDED;
}
return Status.SUCCEEDED;
}
I'm very confused why the error occurred because this function works fine at most time.
Does anyone met the same problem as mine?
Why the error happened? And what's your solution?
Thank you.
This appears to be long standing framework bug github.com/firebase/firebase-functions/issues/536 with no resolution as of yet.
Though you can't get around the error which anecdotally and very intermittently happens on a cold start you can work around it by enabling retries for the function via the full console see Retry Cloud Functions for Firebase until it succeeds for instructions.
This assumes you handle internal errors in your code very well as it will retry for any failure but in my case the functions onCreate handler was just queuing up some later processing via PubSub so any failure meant it should retry.
Oct 2020 Update
Since v3.11 of firebase-functions you can now set the retry mode in your function code by setting failurePolicy to true
module.exports = functions.runWith({ failurePolicy: true }).firestore.document('collection/doc').onWrite(async (change, context) => {
//do function stuff
});
I've got a Google Cloud app with several cloud functions that call an API, then save the response in Firebase. I have this scheduled function set up to retry the API on error and it works great. But I want to retry the call if the data I need isn't there yet. Calling again right away could work if I throw an error, but it's highly unlikely that the missing data will be available seconds later, so I'd rather check once an hour after that until I have valid data.
Below is a shortened version of my function. I can imagine adding a setTimeout and having the function call itself again, but I'm not sure I would do that or if it's a good idea since it would keep the function alive for a long time. Is there a way to automatically retry this scheduled function on an arbitrary time interval?
exports.fetchData= functions.pubsub
.schedule('every Tuesday 6:00')
.timeZone('America/New_York')
.onRun(async context => {
const response = fetch(...)
.then(res => {
if (res.status < 400) {
return res;
} else {
throw new Error(`Network response was not ok. ${res}`);
}
})
.then(res => res.json());
const resObj = await response;
resObj.map(x => {
// check response for valid data
})
if (// data is valid) {
// save to Firebase
} else {
// retry in 1 hour
}
});
});
Scheduled functions only run on the schedule you specify. There is no "arbitrary" scheduling. If you think that the function might frequently fail, consider just increasing the frequency of the schedule, and bail out of function invocations that don't need to run because of recent success.
If you enable retries, and the function generates an error by throwing an exception, returning a rejected promise, or timing out, then Cloud Functions will automatically retry the function on a schedule that you can't control.
setTimeout is not a feasible option to keep a function alive for longer than its configured timeout. Cloud Functions will terminate the function and all of its ongoing work after the timeout expires (and you would be paying for the time the function sits idle, which is kind of a waste).
I'm trying to unit test a DialogflowApp locally by using the firebase shell environment. (in a cli do firebase experimental:functions:shell and then call my methods)
I have followed this guide by google https://firebase.google.com/docs/functions/local-emulator but they don't use the DialogflowApp where the invoked function tries to bind a request object containing intents and parameters like this ->
exports.myFunction = functions.https.onRequest((request, response) => {
const app = new App({ request, response });
function myMethod(app) {
let myArgument = app.getArgument(MY_ARGUMENT);
app.tell('Here we are responding');
}
let actionMap = new Map();
actionMap.set(MYMETHOD_ACTION, myMethod);
app.handleRequest(actionMap);
});
Regardless of what request object I send in the CLI, like this myFunction(require("../test/testdata.json")), the request body object is empty, like this body: {} which means I can't do app.handleRequest() or app.getArgument(). The error message I get is
RESPONSE RECEIVED FROM FUNCTION: 400, Action Error: no matching intent
handler for: null
I thought that if I populated testdata.json with the json request data shown in Actions on Google -> console.actions.google.com -> Simulator it would be valid data but no.
My question is, how can i mock my request data so that I can start unit testing my fullfillment methods locally?
EDIT 1:
firebase > myMethod.post("/").form(require("../test/testdata.json"))
Sent request to function.
firebase > info: User function triggered, starting execution
info: Function crashed
info: TypeError: Cannot destructure property `parameters` of 'undefined' or 'null'.
if we look in dialogflow_app.js we can see this code for fetching an argument value
getArgument (argName) {
debug('getArgument: argName=%s', argName);
if (!argName) {
error('Invalid argument name');
return null;
}
const { parameters } = this.body_.result;
if (parameters && parameters[argName]) {
return parameters[argName];
}
return this.getArgumentCommon(argName);
}
this.body_ is always just empty {}, regardless of how and what I send into the method when running locally.
EDIT 3
firebase > myMethod({method: "post",json: true, body: require("../test/testdata.json")})
Sent request to function.
firebase > info: User function triggered, starting execution
info: Function crashed
info: TypeError: Cannot destructure property parameters of 'undefined' or 'null'.
Invoking a Firebase HTTPS function using the shell requires a different form. It takes the parameters that the request module does, so in order to emulate a webhook, it will be something like this:
myfunction({
method: 'POST',
json: true,
body: require("../test/testdata.json")
});
These three parameters are important:
You need to specify that this is a POST operation
You need to indicate that the body will be JSON. This will send the correct header and won't try to send the body as x-www-form-urlencoded
You need to include the body. As an object is ok because you've set the json parameter to true.