I have created a simple firebase webapp, which is subscribed to listen for changes within a collection in firestore. As there are only 3 items in that collection, I was wondering how it was possible to reach the quota limit of 50k reads a day. I did leave the app running on localhost while I was away from my computer, which I am assuming is the reason that it reached the daily quota limit. But how was it possible to do that?
Basically, I have one page on my webapp that is subscribed to my collection in firestore. I then use that data retrieved to display it on the page.
As I said, there are only 3 items in the collection I am subscribed to. How does the subscription work? Is it constantly reading the database to watch for changes, or does it only do a read request when the collection changes (a doc being added or removed, for example)?
PS I have not even published this app to the web, so I don't think it's even possible for read requests to be coming from another computer.
You got a loop there
useEffect is triggered whenever products changes, and you are calling setProducts in the useEffect itself.
So i'm thinking that this has consumed all your daily quota.
Related
According to Firestore's Best Practices docs (below), one should avoid adding and removing snapshot listeners in quick succession. The docs state, "snapshot listeners should have a lifetime of 30 seconds or longer." However, because a snapshot subscribe and subsequent unsubscribe is controlled by the user's actions (e.g. navigation to and away from a particular page), it may not be possible to always to keep a connection open for more than 30 seconds.
As an example, my app has an Account Details page. The page has one listener the subscribes to the main details (i.e. Account Name, Primary Address, Primary Contact, etc.). The page also has several tables (e.g. Locations, Inventory, Purchase Orders, etc.) each of which has their own listener.
That being said, would it be problematic if my users navigate between several Account Details pages very quickly (since each page will be opening and closing its own set of 3-5 listeners)? If it is problematic, what type of issues will this create for my app? For instance, would Firestore simply slow down temporarily? Or could there be bigger issues with data consistency (i.e. where the snapshot temporarily shows old snapshot data while waiting for the new snapshot to prime)?
Here's what is stated in Firestore's Best Practices documentation:
Avoid frequently churning listeners, especially when your database is under significant write load.
Ideally, your application should set up all the required snapshot listeners soon after opening a connection to Cloud Firestore. After setting up your initial snapshot listeners, you should avoid quickly adding or removing snapshot listeners in the same connection.
To ensure data consistency, Cloud Firestore needs to prime each new snapshot listener from its source data and then catch up to new changes. Depending on your database's write rate, this can be an expensive operation.
Your snapshot listeners can experience increased latency if you frequently add or remove snapshot listeners to references. In general, a constantly-attached listener performs better than attaching and detaching a listener at that location for the same amount of data. For best performance, snapshot listeners should have a lifetime of 30 seconds or longer. If you encounter listener performance issues in your app, try tracking your app's listens and unlistens to determine if they may be happening too frequently.
https://firebase.google.com/docs/firestore/best-practices#realtime_updates
Rapidly adding and removing listeners due to such user action won't create technical problems, but it just means you'll be using more resources than you'd ideally like.
If you imagine that many users may follow this same click path that you describe, consider running a single query that gets you the data for all those screens in one go. That might mean you need a different/additional data structure, but on the other hand also means you have less churn in setting up/tearing down listeners.
But again, this is not a technical limit in any way. It is merely an observation of patterns that the writers of that documentation have seen get the best value out of Firestore.
I have a Flutter & Firebase app that provides users with price updates on the dashboard on the main screen. It does this with a Firestore Snapshot Listener, but my read count is flying through the roof (I'm at 203 reads and just 1 device is connected)... This info will only change once a month, and it really doesn't need to keep checking for updates at all times.
Is there a way to setup the listener so that it checks less frequently for updates? Or perhaps, can I setup a Firebase In-App Messaging system with which I send the price out and the app just saves and displays it?
Or is there any efficient solutions?
Since the Listener (which you do not show, so we can't even help you there) ONLY triggers when there is an update to the documents (NOT on a schedule) something is updating the collection you are listening to. 203 is not particularly high over a few days - how long has your app been running? Have you been restarting the app frequently? Most Listeners will give you a result at least once when you initialize them - i.e. when you start the app. There is no "long-term global memory" like that.
A listener will always receive updates as soon as they becomes available. This can't be configured. If you want a different update frequency, you should instead schedule that yourself within the app, and use get() to poll the database as often as you need. You can also use FCM on your backend to push updates to the app, if that works for your case.
I am developing an Android app which basically does this: On the landing(home) page it shows a couple of words. These words need to be updated on daily basis. Secondly, there is an 'experiences' tab in which a list of user experiences (around 500) shows up with their profile pic, description,etc.
This basic app is expected to get around 1 million users daily who will open the app daily at least once to see those couple of words. Many may occasionally open up the experiences section.
Thirdly, the app needs to have a push notification feature.
I am planning to purchase a managed wordpress hosting, set up a website, and add a post each day with those couple of words, use the JSON-API to extract those words and display them on app's home page. Similarly for the experiences, I will add each as a wordpress post and extract them from the Wordpress database. The reason I am choosing wordpress is that it has ready made interfaces for data entry which will save my time and effort.
But I am stuck on this: will the wordpress DB be able to handle such large amount of queries ? With such a large userbase and spiky traffic, I suspect I might cross the max. concurrent connections limit.
What's the best strategy in my case ? Should I use WP, or use firebase or any other service ? I need to make sure the scheme is cost effective also.
My app is basically very similar to this one:
https://play.google.com/store/apps/details?id=com.ekaum.ekaum
For push notifications, I am planning to use third party services.
Kindly suggest the best strategy I should go with for designing the back end of this app.
Thanks to everyone out there in advance who are willing to help me in this.
I have never used Wordpress, so I don't know if or how it could handle that load.
You can still use WP for data entry, and write a scheduled function that would use WP's JSON API to copy that data into Firebase.
RTDB-vs-Firestore scalability states that RTDB can handle 200 thousand concurrent connections and Firestore 1 million concurrent connections.
However, if I get it right, your app doesn't need connections to be active (i.e. receive real-time updates). You can get your data once, then close the connection.
For RTDB, Enabling Offline Capabilities on Android states that
On Android, Firebase automatically manages connection state to reduce bandwidth and battery usage. When a client has no active listeners, no pending write or onDisconnect operations, and is not explicitly disconnected by the goOffline method, Firebase closes the connection after 60 seconds of inactivity.
So the connection should close by itself after 1 minute, if you remove your listeners, or you can force close it earlier using goOffline.
For Firestore, I don't know if it happens automatically, but you can do it manually.
In Firebase Pricing you can see that 100K Firestore document reads is $0.06. 1M reads (for the two words) should cost $0.6 plus some network traffic. In RTDB, the cost has to do with data bulk, so it requires some calculations, but it shouldn't be much. I am not familiar with the pricing small details, so you should do some more research.
In the app you mentioned, the experiences don't seem to change very often. You might want to try to build your own caching manually, and add the required versioning info in the daily data.
Edit:
It would possibly be more efficient and less costly if you used Firebase Hosting, instead of RTDB/Firestore directly. See Serve dynamic content and host microservices with Cloud Functions and Manage cache behavior.
In short, you create a HTTP function that reads your database and returns the data you need. You configure hosting to call that function, and configure the cache such that subsequent requests are served the cached result via hosting (without extra function invocations).
I have a use case where I log each activity open count in a firestore document (activity_name->field, and count->value)... So I wanted to know, when the user is offline and each of his activity navigation is stored in firestore cache, as soon as the user gets online and firebase sdk syncs the changes to the main database, does firestore record the synced changes as a single write or it sees the various individual field changes since the last change and record as multiple writes?
The writes are queued up in the client and delivered individually, so there will be a cost of one write for each document that was written offline.
The important issue here is not so much the billing as it is the evaluation of security rules. It's entirely possible a series of 5 writes might actually only result in 4 successful writes and 1 failure due to the violation of a security rule. If those writes were actually compressed into a single write, that would potentially cause everything to fail, which would be undesirable (you'd likely want as many writes to succeed as possible).
Hi I'm creating an app in React-Native and Invertase RN-Firebase. I'm using firestore for the first time for my new project. I notice the realtime update 'Snapshot' the behavior seems to listen too fast. I'm worried that this will kill my budget for the billing.
Form my previous experience with Firebase Realtime-Database the update/listening from snapshot was not so fast and it check ever other 4 to 10 seconds which in FIrestore it listening per millisecond.
let me know if there is a way to slow down the Listening of snapshot for Cloud firestore.
thanks,
There is no way to limit or configure how often the triggered listeners receive data - in theory any changes should be sent to your app almost immediately, but in practice there's some latency in the process... resulting in the 4-10 seconds you observed elsewhere. Firestore is apparently better optimized since you are not seeing that much delay.
If you are concerned about budget or billing, the best advice I can give you is to review the official Optimization Suggestions. Those are for the RTDB, not Firestore, but they're still useful suggestions.
Note that it doesn't count as a billable "read" unless the data is changing, and unless your data is changing near-constantly, you shouldn't have to worry about billing... Check out the Firestore pricing page which includes this explanatory paragraph regarding billing for listeners:
Listening to query results
Cloud Firestore allows you to listen to the
results of a query and get realtime updates when the query results
change.
When you listen to the results of a query, you are charged for a read
each time a document in the result set is added or updated. You are
also charged for a read when a document is removed from the result set
because the document has changed. (In contrast, when a document is
deleted, you are not charged for a read.)
Also, if the listener is disconnected for more than 30 minutes (for
example, if the user goes offline), you will be charged for reads as
if you had issued a brand-new query.