I have a marketplace web app running on Firebase Hosting, with backend on Firestore, and images stored on Cloud Storage. The load time for this is a really poor approx. 9 secs. The website is pinged within a couple of seconds and key data starts loading from Realtime based on which data from Firestore and then Cloud storage is loaded. But firestore and Cloud storage data takes about 6 to 8 seconds to load??!! even when there is no cloud storage involved it gets worse
I have read about making the entire Google Cloud bucket public makes the images load in 1/3 the time, but that seems like a potential security issue for all data? I am sure I am doing something incorrectly as the platform is used by so many developers and if you cant load a website in less than 3 secs, then users generally leave.... (Mobile is worse at about 17 secs fully loaded). Performance is not much different even if I am in the same city as the hosting of the cloud
https://app.latom.in/co/markusconsulting - with images
below is the link to the gtmetrix performance to see the waterfall
[https://gtmetrix.com/reports/app.latom.in/54P1tHJE/][2]
Any guidance on what can be done better?
CODE:
Initializing:
const app = initializeApp(firebaseConfig);
const db = getFirestore (app)
const realTimeDB = getDatabase()
Query:
export const getMyCompanyServiceData = (companyId, firebaseUserID, pageLimit = 10)=>new Promise( async function(successCallback, errorCallback) {
const myDocRef = collection(db, "UserServices/"+companyId+"/Services/")
const q = query(myDocRef,
where("companyID", "==", parseInt(companyId)),
where("serviceProviderId."+firebaseUserID, "==", true),
where("status", "==", SERVICE_STATUS[0]),
limit(pageLimit)
// orderBy("serviceType")
)
try {
const docSnap = await getDocs(q)
const _data = []
docSnap.forEach((doc) => {
_data.push(doc.data())
});
successCallback(_data)
} catch (err) {
errorCallback(err)
}
});
Once the object is obtained it is rendered to get images and display data
Related
In short: Is there some kind of cold start when connecting to Firestore directly from Client SDK
Hey. I'm using Firestore client sdk in Andoid and IOS application through #react-native-firebase.
Everything works perfectly but I have noticed weird behavior I haven't found explanation.
I have made logging to see how long it takes from user login to retrieve uid corresponding data from Firestore and this time has been ~0.4-0.6s. This is basically the whole onAuthStateChanged workflow.
let userLoggedIn: Date;
let userDataReceived: Date;
auth().onAuthStateChanged(async (user) => {
userLoggedIn = new Date();
const eventsRetrieved = async (data: UserInformation) => {
userDataReceived = new Date();
getDataDuration = `Get data duration: ${(
(userDataReceived.getTime() - userLoggedIn.getTime()) /
1000
).toString()}s`;
console.log(getDataDuration)
// function to check user role and to advance timing logs
onUserDataReceived(data);
};
const errorRetrieved = () => {
signOut();
authStateChanged(false);
};
let unSub: (() => void) | undefined;
if (user && user.uid) {
const userListener = () => {
return firestore()
.collection('Users')
.doc(user.uid)
.onSnapshot((querySnapshot) => {
if (querySnapshot && querySnapshot.exists) {
const data = querySnapshot.data() as UserInformation;
data.id = querySnapshot.id;
eventsRetrieved(data);
} else errorRetrieved();
});
};
unSub = userListener();
} else {
if (typeof unSub === 'function') unSub();
authStateChanged(false);
}
});
Now the problem. When I open the application ~30-50 minutes after last open the time to retrieve uid corresponding data from Firestore will be ~3-9s. What is this time and why does it happen? And after I open the application right after this time will be low again ~0.4-0-6s.
I have been experiencing this behavior for weeks. It is hard to debug as it happens only on build application (not in local environments) and only between +30min interval.
Points to notice
The listener query (which I'm using in this case, I have used also simple getDoc function) is really simple and focused on single document and all project configuration works well. Only in this time interval, which seems just like cold start, the long data retrieval duration occurs.
Firestore Rules should not be slowing the query as subsequent request are fast. Rules for 'Users' collection are as follows in pseudo code:
function checkCustomer(){
let data =
get(/databases/$(database)/documents/Users/$(request.auth.uid)).data;
return (resource.data.customerID == data.customerID);
}
match /Users/{id}{
allow read:if
checkUserRole() // Checks user is logged in and has certain customClaim
&& idComparison(request.auth.uid, id) // Checks user uid is same as document id
&& checkCustomer() // User can read user data only if data is under same customer
}
Device cache doesn't seem to affect the issue as application's cache can be cleaned and the "cold start" still occurs
Firestore can be called from another environment or just another mobile device and this "cold start" will occur to devices individually (meaning that it doesn't help if another device opened the application just before). Unlike if using Cloud Run with min instances, and if fired from any environment the next calls right after will be fast regardless the environment (web or mobile).
EDIT
I have tested this also by changing listener to simple getDoc call. Same behavior still happens on a build application. Replacing listener with:
await firestore()
.collection('Users')
.doc(user.uid)
.get()
.then(async document => {
if (document.exists) {
const data = document.data() as UserInformation;
if (data) data.id = document.id;
eventsRetrieved(data);
}
});
EDIT2
Testing further there has been now 3-15s "cold start" on first Firestore getDoc. Also in some cases the timing between app open has been only 10 minutes so the minimum 30 min benchmark does not apply anymore. I'm going to send dm to Firebase bug report team to see things further.
Since you're using React Native, I assume that the documents in the snapshot are being stored in the local cache by the Firestore SDK (as the local cache is enabled by default on native clients). And since you use an onSnapshot listener it will actually re-retrieve the results from the server if the same listener is still active after 30 minutes. From the documentation on :
If offline persistence is enabled and the listener is disconnected for more than 30 minutes (for example, if the user goes offline), you will be charged for reads as if you had issued a brand-new query.
The wording here is slightly different, but given the 30m mark you mention, I do expect that this is what you're affected by.
In the end I didn't find straight answer why this cold start appeared. I ended up changing native Client SDK to web Client SDK which works correctly first data fetch time being ~0.6s (always 0.5-1s). Package change fixed the issue for me while functions to fetch data are almost completely identical.
Inside the Firebase Console, under the Cloud Messaging view, users are able to create test notifications. This functionality also allows you to schedule the time at which the notification will send to a device or set of devices.
Is it possible to create and send scheduled FCM notifications to specific devices by using firebase cloud functions and the Firebase Admin SDK? Is there an alternative way to solving this?
The current way that I send scheduled messages to users is like so:
const functions = require('firebase-functions');
const admin = require('firebase-admin');
const schedule = require('node-schedule');
admin.initializeApp();
exports.setScheduledNotification = functions.https.onRequest(async (req, res) => {
const key = req.query.notification_key;
const message = {
notification: {
title: 'Test Notification',
body: 'Test Notification body.'
}
};
var currentDate = new Date();
var laterDate = new Date(currentDate.getTime() + (1 * 60000));
var job = schedule.scheduleJob(key, laterDate, () => {
const snapshot = admin.messaging().sendToDevice(key, message);
});
return res.status(200).send(`Message has been scheduled.`);
});
First of all, I am unsure how node-schedule interacts with firebase cloud functions. The logs appear that the function terminates very quickly which I would think would be correct. The longer the operation runs the more costly it is in our firebase bills. The notification does still run on a scheduled time though. I'm confused on how that all is working behind the scenes.
Secondly, I am having issues canceling these scheduled notifications. The notifications will most likely be on a 2hr timed schedule from the point it gets created. Before the 2hrs is up, I'd like the have the ability to cancel/overwrite the notification with an updated scheduled time.
I tried this to cancel the notification and it failed to find the previously created notification. Here is the code for that:
exports.cancelScheduledNotification = functions.https.onRequest(async (req, res) => {
const key = req.query.notification_key;
var job = schedule.scheduledJobs[key];
job.cancel();
return res.status(200).send(`Message has been canceled.`);
});
Is it possible to tap into the scheduling functionality of firebase cloud messaging outside of the firebase console? Or am I stuck with hacking my way around this issue?
A Cloud Function can run for a maximum of 9 minutes. So unless you're using node-schedule for periods shorter than that, your current approach won't work. Even if it would work, or if you are scheduling for less than 9 minutes in advance, using this approach is very uneconomic as you'll be paying for the Cloud Functions for all this time while it's waiting.
A more common approach is to store information about what message you want to be delivered to whom at what time in a database, and then use regular scheduled functions to periodically check what messages to send. For more on this, see these previous questions:
Firebase scheduled notification in android
How to schedule push notifcations for react native expo?
Schedule jobs in Firebase
Ionic: Is it possible to delay incoming push FCM push notification from showing to my device until a specific time
Cloud Functions for Firebase trigger on time?
How to create cron jobs dynamically in firebase
A recent improvement on this is to use the Cloud Tasks API to programmatically schedule Cloud Functions to be called at a specific time with a specific payload, and then use that to send the message through FCM. Doug Stevenson wrote a great blog post about this here: How to schedule a Cloud Function to run in the future with Cloud Tasks (to build a Firestore document TTL). While the post is about deleting documents at a certain time, you can combine it with the previous approach to schedule FCM messages too.
Scheduling of tasks is now also described in the documentation on enqueueing functions with Cloud Tasks
A final option, and one I'd actually recommend nowadays, is to separate the delivery of the message from the display of the notification.
Display of data messages (unlike notification messages) is never handled by the system, and always left to your application. So you can deliver the FCM data message straight away that then contains the time to display the message, and then wake the device up to display the message (often called a local notification) at that time.
To make Frank's answer more tangible, I am including some sample code below for scheduled cloud functions, that can help you achieve the 'scheduled FCM notifications'.
You should store the information required to send your notification(s) in Firestore (e.g. the when-to-notify parameter and the FCM token(s) of the users you want to send the notification to) and run a cloud function every minute to evaluate if there is any notification that needs to be delivered.
The function checks what Firestore documents have a WhenToNofity parameter that is due, and send the notifications to the receiver tokens immediately. Once sent, the function sets the boolean 'notificationSent' to true, to avoid that the users receive the same notification again on the next iteration.
The code below achieves just that:
const admin = require('firebase-admin');
admin.initializeApp();
const database = admin.firestore();
exports.sendNotification = functions.pubsub.schedule('* * * * *').onRun(async (context) => {
//check whether notification should be sent
//send it if yes
const query = await database.collection("experiences")
.where("whenToNotify", '<=', admin.firestore.Timestamp.now())
.where("notificationSent", "==", false).get();
query.forEach(async snapshot => {
sendNotification(snapshot.data().tokens);
await database.doc('experiences/' + snapshot.id).update({
"notificationSent": true,
});
});
function sendNotification(tokens) {
let title = "INSERT YOUR TITLE";
let body = "INSERT YOUR BODY";
const message = {
notification: { title: title, body: body},
tokens: tokens,
android: {
notification: {
sound: "default"
}
},
apns: {
payload: {
aps: {
sound: "default"
}
}
}
};
admin.messaging().sendMulticast(message).then(response => {
return console.log("Successful Message Sent");
}).catch(error => {
console.log(error);
return console.log("Error Sending Message");
});
}
return console.log('End Of Function');
});
If you're unfamiliar with setting up cloud functions, you can check how to set them up here. Cloud functions require a billing account, but you get 1M cloud invocations per month for free, which is more than enough to cover the costs of this approach.
Once done, you can insert your function in the index.js file.
I want to see Cloud Firestores logs like I can see for Cloud Functions for Firebase. There is a logs tab in Functions tab of the Firebase console. Similarly I want to see this for Cloud Firestore. How?
I wrote a firebase cloud function to log all activity to Firestore:
/**
* Logs all database activity
*
* #type {CloudFunction<Change<DocumentSnapshot>>}
*/
exports.dbLogger = functions.firestore
.document('{collection}/{id}')
.onWrite(async (change, context) => {
const {collection, id} = context.params;
if (collection !== 'firestore_log') {
const event = context.eventType;
const data = change.after.data();
const created_at = Date.now();
admin.firestore().collection('firestore_log').add({collection, id, event, data, created_at});
}
});
Here's an example of the logged data:
Note that this is a quick dev only version that logs everything; in production we have Firestore rules to deny any read / write to the table (the admin on firebase functions can still access them):
// firestore.rules
match /firestore_log/{entryId} {
allow read: if false;
allow write: if false;
}
and filter the logged collections to avoid persisting sensitive data.
Note that if you prefer to persist this in the logs instead of Firestore you can use console.log({....}). I prefer Firestore because its designed to handle larger data sets.
There are currently no log entries for Cloud Firestore exposed to developers, so for the time being we recommend logging any statistics you are about separately.
I'd like to integrate my firebase project with some 3rd party API's, like the twitter API.
3rd party API
The following code will listen to new tweets containing the certain text 'little rocket man':
var Twitter = require('twitter');
// setup new twitter client
var client = new Twitter({
consumer_key: '',
consumer_secret: '',
access_token_key: '',
access_token_secret: ''
});
// open new twitter stream
let stream = this.client.stream('statuses/filter', { track: 'little rocket man' });
stream.on('data', (event: any) => {
let tweetText = event && event.text; // this should be written to the firebase db
});
Firebase Cloud Functions
The following firebase cloud functions listens to incoming HTTP GET requests on a specific route and saves data back to the firebase db:
import * as functions from 'firebase-functions';
import * as admin from 'firebase-admin'; // Firebase Admin SDK
admin.initializeApp(functions.config().firebase);
// this example uses a HTTP trigger, but how can I trigger it whenever new tweets containint 'little rocket man' are detected??
exports.addMessage = functions.https.onRequest((req, res) => {
const original = req.query.text;
admin.database().ref('/messages').push({original: original}).then(snapshot => {
res.redirect(303, snapshot.ref);
});
});
Question
How can I write the tweets I'm recieving from the twitter client back to the firebase db? If possible, I'd like to run all the code on firebase cloud functions.
Disclaimer:
I'm new to firebase and although googling around for a few hours I wasn't able to find the solution to my problem on the net. I'd like to apologize in advance, should I have overseen it.
You can't use streaming APIs like this in Cloud Functions. A function may only respond to some distinct event, such as an HTTP request, or some change in your database. Functions can't run indefinitely.
If you want to collect tweets that match some query into your database, you can use IFTTT to periodically send them to a function as they become available. I recently finished a small project that does exactly that.
I will be implementing an elastic search index alongside my firebase application so that it can better support ad-hoc full text searches and geo searches. Thus, I need to sync firebase data to the elastic search index and all the examples require a server process that listens for firebase events.
e.g. https://github.com/firebase/flashlight
However, it would be great if I can just have a google cloud function triggered by an insert in a firebase node. I see that google cloud functions has various triggers: pub sub, storage and direct... can any of these bridge to a firebase node event without an intermediate server?
firebaser here
We just released Cloud Functions for Firebase. This allows you to run JavaScript functions on Google's servers in response to Firebase events (such as database changes, users signing in and much more).
I believe Cloud Functions for Firebase are what you are looking for.
Here are a few links:
Official Documentation
Intro video
Google Cloud Functions and Firebase (Google Cloud Next '17)
yes, you can trigger a Google Cloud Functions via firebase event without a server.
As per documents,Firebase allows for example you can send notifications using a cloud function when a user write into firebase database.
For that, I had to write a javascript as below
'use strict';
const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp(functions.config().firebase);
exports.sendNotification = functions.database.ref('/articles/{articleId}')
.onWrite(event => {
// Grab the current value of what was written to the Realtime Database.
var eventSnapshot = event.data;
var str1 = "Author is ";
var str = str1.concat(eventSnapshot.child("author").val());
console.log(str);
var topic = "android";
var payload = {
data: {
title: eventSnapshot.child("title").val(),
author: eventSnapshot.child("author").val()
}
};
// Send a message to devices subscribed to the provided topic.
return admin.messaging().sendToTopic(topic, payload)
.then(function (response) {
// See the MessagingTopicResponse reference documentation for the
// contents of response.
console.log("Successfully sent message:", response);
})
.catch(function (error) {
console.log("Error sending message:", error);
});
});