I want to see Cloud Firestores logs like I can see for Cloud Functions for Firebase. There is a logs tab in Functions tab of the Firebase console. Similarly I want to see this for Cloud Firestore. How?
I wrote a firebase cloud function to log all activity to Firestore:
/**
* Logs all database activity
*
* #type {CloudFunction<Change<DocumentSnapshot>>}
*/
exports.dbLogger = functions.firestore
.document('{collection}/{id}')
.onWrite(async (change, context) => {
const {collection, id} = context.params;
if (collection !== 'firestore_log') {
const event = context.eventType;
const data = change.after.data();
const created_at = Date.now();
admin.firestore().collection('firestore_log').add({collection, id, event, data, created_at});
}
});
Here's an example of the logged data:
Note that this is a quick dev only version that logs everything; in production we have Firestore rules to deny any read / write to the table (the admin on firebase functions can still access them):
// firestore.rules
match /firestore_log/{entryId} {
allow read: if false;
allow write: if false;
}
and filter the logged collections to avoid persisting sensitive data.
Note that if you prefer to persist this in the logs instead of Firestore you can use console.log({....}). I prefer Firestore because its designed to handle larger data sets.
There are currently no log entries for Cloud Firestore exposed to developers, so for the time being we recommend logging any statistics you are about separately.
Related
In short: Is there some kind of cold start when connecting to Firestore directly from Client SDK
Hey. I'm using Firestore client sdk in Andoid and IOS application through #react-native-firebase.
Everything works perfectly but I have noticed weird behavior I haven't found explanation.
I have made logging to see how long it takes from user login to retrieve uid corresponding data from Firestore and this time has been ~0.4-0.6s. This is basically the whole onAuthStateChanged workflow.
let userLoggedIn: Date;
let userDataReceived: Date;
auth().onAuthStateChanged(async (user) => {
userLoggedIn = new Date();
const eventsRetrieved = async (data: UserInformation) => {
userDataReceived = new Date();
getDataDuration = `Get data duration: ${(
(userDataReceived.getTime() - userLoggedIn.getTime()) /
1000
).toString()}s`;
console.log(getDataDuration)
// function to check user role and to advance timing logs
onUserDataReceived(data);
};
const errorRetrieved = () => {
signOut();
authStateChanged(false);
};
let unSub: (() => void) | undefined;
if (user && user.uid) {
const userListener = () => {
return firestore()
.collection('Users')
.doc(user.uid)
.onSnapshot((querySnapshot) => {
if (querySnapshot && querySnapshot.exists) {
const data = querySnapshot.data() as UserInformation;
data.id = querySnapshot.id;
eventsRetrieved(data);
} else errorRetrieved();
});
};
unSub = userListener();
} else {
if (typeof unSub === 'function') unSub();
authStateChanged(false);
}
});
Now the problem. When I open the application ~30-50 minutes after last open the time to retrieve uid corresponding data from Firestore will be ~3-9s. What is this time and why does it happen? And after I open the application right after this time will be low again ~0.4-0-6s.
I have been experiencing this behavior for weeks. It is hard to debug as it happens only on build application (not in local environments) and only between +30min interval.
Points to notice
The listener query (which I'm using in this case, I have used also simple getDoc function) is really simple and focused on single document and all project configuration works well. Only in this time interval, which seems just like cold start, the long data retrieval duration occurs.
Firestore Rules should not be slowing the query as subsequent request are fast. Rules for 'Users' collection are as follows in pseudo code:
function checkCustomer(){
let data =
get(/databases/$(database)/documents/Users/$(request.auth.uid)).data;
return (resource.data.customerID == data.customerID);
}
match /Users/{id}{
allow read:if
checkUserRole() // Checks user is logged in and has certain customClaim
&& idComparison(request.auth.uid, id) // Checks user uid is same as document id
&& checkCustomer() // User can read user data only if data is under same customer
}
Device cache doesn't seem to affect the issue as application's cache can be cleaned and the "cold start" still occurs
Firestore can be called from another environment or just another mobile device and this "cold start" will occur to devices individually (meaning that it doesn't help if another device opened the application just before). Unlike if using Cloud Run with min instances, and if fired from any environment the next calls right after will be fast regardless the environment (web or mobile).
EDIT
I have tested this also by changing listener to simple getDoc call. Same behavior still happens on a build application. Replacing listener with:
await firestore()
.collection('Users')
.doc(user.uid)
.get()
.then(async document => {
if (document.exists) {
const data = document.data() as UserInformation;
if (data) data.id = document.id;
eventsRetrieved(data);
}
});
EDIT2
Testing further there has been now 3-15s "cold start" on first Firestore getDoc. Also in some cases the timing between app open has been only 10 minutes so the minimum 30 min benchmark does not apply anymore. I'm going to send dm to Firebase bug report team to see things further.
Since you're using React Native, I assume that the documents in the snapshot are being stored in the local cache by the Firestore SDK (as the local cache is enabled by default on native clients). And since you use an onSnapshot listener it will actually re-retrieve the results from the server if the same listener is still active after 30 minutes. From the documentation on :
If offline persistence is enabled and the listener is disconnected for more than 30 minutes (for example, if the user goes offline), you will be charged for reads as if you had issued a brand-new query.
The wording here is slightly different, but given the 30m mark you mention, I do expect that this is what you're affected by.
In the end I didn't find straight answer why this cold start appeared. I ended up changing native Client SDK to web Client SDK which works correctly first data fetch time being ~0.6s (always 0.5-1s). Package change fixed the issue for me while functions to fetch data are almost completely identical.
Inside the Firebase Console, under the Cloud Messaging view, users are able to create test notifications. This functionality also allows you to schedule the time at which the notification will send to a device or set of devices.
Is it possible to create and send scheduled FCM notifications to specific devices by using firebase cloud functions and the Firebase Admin SDK? Is there an alternative way to solving this?
The current way that I send scheduled messages to users is like so:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const schedule = require('node-schedule');
admin.initializeApp();
exports.setScheduledNotification = functions.https.onRequest(async (req, res) => {
const key = req.query.notification_key;
const message = {
notification: {
title: 'Test Notification',
body: 'Test Notification body.'
}
};
var currentDate = new Date();
var laterDate = new Date(currentDate.getTime() + (1 * 60000));
var job = schedule.scheduleJob(key, laterDate, () => {
const snapshot = admin.messaging().sendToDevice(key, message);
});
return res.status(200).send(`Message has been scheduled.`);
});
First of all, I am unsure how node-schedule interacts with firebase cloud functions. The logs appear that the function terminates very quickly which I would think would be correct. The longer the operation runs the more costly it is in our firebase bills. The notification does still run on a scheduled time though. I'm confused on how that all is working behind the scenes.
Secondly, I am having issues canceling these scheduled notifications. The notifications will most likely be on a 2hr timed schedule from the point it gets created. Before the 2hrs is up, I'd like the have the ability to cancel/overwrite the notification with an updated scheduled time.
I tried this to cancel the notification and it failed to find the previously created notification. Here is the code for that:
exports.cancelScheduledNotification = functions.https.onRequest(async (req, res) => {
const key = req.query.notification_key;
var job = schedule.scheduledJobs[key];
job.cancel();
return res.status(200).send(`Message has been canceled.`);
});
Is it possible to tap into the scheduling functionality of firebase cloud messaging outside of the firebase console? Or am I stuck with hacking my way around this issue?
A Cloud Function can run for a maximum of 9 minutes. So unless you're using node-schedule for periods shorter than that, your current approach won't work. Even if it would work, or if you are scheduling for less than 9 minutes in advance, using this approach is very uneconomic as you'll be paying for the Cloud Functions for all this time while it's waiting.
A more common approach is to store information about what message you want to be delivered to whom at what time in a database, and then use regular scheduled functions to periodically check what messages to send. For more on this, see these previous questions:
Firebase scheduled notification in android
How to schedule push notifcations for react native expo?
Schedule jobs in Firebase
Ionic: Is it possible to delay incoming push FCM push notification from showing to my device until a specific time
Cloud Functions for Firebase trigger on time?
How to create cron jobs dynamically in firebase
A recent improvement on this is to use the Cloud Tasks API to programmatically schedule Cloud Functions to be called at a specific time with a specific payload, and then use that to send the message through FCM. Doug Stevenson wrote a great blog post about this here: How to schedule a Cloud Function to run in the future with Cloud Tasks (to build a Firestore document TTL). While the post is about deleting documents at a certain time, you can combine it with the previous approach to schedule FCM messages too.
Scheduling of tasks is now also described in the documentation on enqueueing functions with Cloud Tasks
A final option, and one I'd actually recommend nowadays, is to separate the delivery of the message from the display of the notification.
Display of data messages (unlike notification messages) is never handled by the system, and always left to your application. So you can deliver the FCM data message straight away that then contains the time to display the message, and then wake the device up to display the message (often called a local notification) at that time.
To make Frank's answer more tangible, I am including some sample code below for scheduled cloud functions, that can help you achieve the 'scheduled FCM notifications'.
You should store the information required to send your notification(s) in Firestore (e.g. the when-to-notify parameter and the FCM token(s) of the users you want to send the notification to) and run a cloud function every minute to evaluate if there is any notification that needs to be delivered.
The function checks what Firestore documents have a WhenToNofity parameter that is due, and send the notifications to the receiver tokens immediately. Once sent, the function sets the boolean 'notificationSent' to true, to avoid that the users receive the same notification again on the next iteration.
The code below achieves just that:
const admin = require('firebase-admin');
admin.initializeApp();
const database = admin.firestore();
exports.sendNotification = functions.pubsub.schedule('* * * * *').onRun(async (context) => {
//check whether notification should be sent
//send it if yes
const query = await database.collection("experiences")
.where("whenToNotify", '<=', admin.firestore.Timestamp.now())
.where("notificationSent", "==", false).get();
query.forEach(async snapshot => {
sendNotification(snapshot.data().tokens);
await database.doc('experiences/' + snapshot.id).update({
"notificationSent": true,
});
});
function sendNotification(tokens) {
let title = "INSERT YOUR TITLE";
let body = "INSERT YOUR BODY";
const message = {
notification: { title: title, body: body},
tokens: tokens,
android: {
notification: {
sound: "default"
}
},
apns: {
payload: {
aps: {
sound: "default"
}
}
}
};
admin.messaging().sendMulticast(message).then(response => {
return console.log("Successful Message Sent");
}).catch(error => {
console.log(error);
return console.log("Error Sending Message");
});
}
return console.log('End Of Function');
});
If you're unfamiliar with setting up cloud functions, you can check how to set them up here. Cloud functions require a billing account, but you get 1M cloud invocations per month for free, which is more than enough to cover the costs of this approach.
Once done, you can insert your function in the index.js file.
I am trying to send a one to one device specific notification using FCM and Firebase CLI. For this I am sending the token from android to Firebase realtime database and trying to capture this token in CLI using an onwrite event. Here is the structure of the realtime database:
Here is the code where I am trying to capture the onwrite event and the token:
exports.sendNotification = functions.database.ref('/Notification/{notification_id}').onWrite((data, context) => {
const notification_id = context.params.notification_id;
const receiver_token = data.ref.parent.child(notification_id).child("token");
})
But I get the following error in the log:
sendNotification
TypeError: Cannot read property 'parent' of undefined at exports.sendNotification.functions.database.ref.onWrite
I am writing CLI code for the first time and hence any help would be appreciated
The onWrite trigger gets a change parameter, which contains the snapshots before and after the change that triggered the code.
So you'll need to get the change.before or change.after to get the actual data.
exports.sendNotification = functions.database.ref('/Notification/{notification_id}').onWrite((change, context) => {
const notification_id = context.params.notification_id;
const receiver_token = change.after.ref.parent.child(notification_id).child("token");
})
See the Firebase documentation on onWrite triggers.
Note that it's much more common to use onCreate for this scenario, as you're typically deleting the notification after you've handled it.
I setup my cloud functions with firebase-admin like :
const admin = require('firebase-admin');
const fn = require('firebase-functions');
admin.initializeApp(fn.config().firebase);
However I am getting permission denied upon writing on the database. What's weird is it only happens to some collection but not to all of them. Some works, some won't.
My understanding of admin.initializeApp(fn.config().firebase); is that this will allow my cloud functions to have an absolute power over the database regardless of the security rules.
Here's the error:
EDIT
I write the data like this.
exports.foo = fn.database.ref('some-path').onWrite(e => {
// some handling
const ref = e.data.ref;
return ref.child('bar').set('some-data').then( // ).catch( // );
})
To obtain full access to the database from the DeltaSnapshot provided in an event, use adminRef:
Returns a Reference to the Database location where the triggering
write occurred. Similar to ref, but with full read and write access
instead of end-user access
exports.foo = fn.database.ref('some-path').onWrite(e => {
// some handling
const ref = e.data.adminRef; // <== CHANGED
return ref.child('bar').set('some-data').then( // ).catch( // );
})
I will be implementing an elastic search index alongside my firebase application so that it can better support ad-hoc full text searches and geo searches. Thus, I need to sync firebase data to the elastic search index and all the examples require a server process that listens for firebase events.
e.g. https://github.com/firebase/flashlight
However, it would be great if I can just have a google cloud function triggered by an insert in a firebase node. I see that google cloud functions has various triggers: pub sub, storage and direct... can any of these bridge to a firebase node event without an intermediate server?
firebaser here
We just released Cloud Functions for Firebase. This allows you to run JavaScript functions on Google's servers in response to Firebase events (such as database changes, users signing in and much more).
I believe Cloud Functions for Firebase are what you are looking for.
Here are a few links:
Official Documentation
Intro video
Google Cloud Functions and Firebase (Google Cloud Next '17)
yes, you can trigger a Google Cloud Functions via firebase event without a server.
As per documents,Firebase allows for example you can send notifications using a cloud function when a user write into firebase database.
For that, I had to write a javascript as below
'use strict';
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.sendNotification = functions.database.ref('/articles/{articleId}')
.onWrite(event => {
// Grab the current value of what was written to the Realtime Database.
var eventSnapshot = event.data;
var str1 = "Author is ";
var str = str1.concat(eventSnapshot.child("author").val());
console.log(str);
var topic = "android";
var payload = {
data: {
title: eventSnapshot.child("title").val(),
author: eventSnapshot.child("author").val()
}
};
// Send a message to devices subscribed to the provided topic.
return admin.messaging().sendToTopic(topic, payload)
.then(function (response) {
// See the MessagingTopicResponse reference documentation for the
// contents of response.
console.log("Successfully sent message:", response);
})
.catch(function (error) {
console.log("Error sending message:", error);
});
});