Firestore costs are based on document operations and on size of stored data.
In the Firebase console, we can easily track number of document operations but I don't find any place where I can track size of stored data.
I have only found in Google Cloud Console (in App Engine > Quotas) a metric corresponding to the amount of stored data in gigabyte stored the current day, but not the total amount of stored data.
Is there a means of monitoring total size of stored data (ideally with indexes included) ?
It seems that the only available option at this moment is to calculate the storage size for Cloud Firestore in Native mode manually.
I have submitted a feature request asking to implement a solution that would display the size. I'd recommend you to star that request to be notified once there is an update in the thread.
Related
I am using a collection in Firebase Firestore to log some activities but I don't want this log collection to grow forever. Is there a way to set a limit to the number of documents in a collection or a size limit for the whole collection or get a notification if it passes a limit?
OR is there a way to automatically delete old documents in a collection just by settings and not writing some cron job or scheduled function?
Alternatively, what options are there to create a rotational logging system for client activities in Firebase?
I don't want this log collection to grow forever.
Why not? There are no downsides. In Firestore the performance depends on the number of documents you request and not on the number of documents you search. So it doesn't really matter if you search 10 documents in a collection of 100 documents or in a collection of 100 MIL documents, the response time will always be the same. As you can see, the number of documents within a collection is irrelevant.
Is there a way to set a limit to the number of documents in a collection or a size limit for the whole collection or get a notification if it passes a limit?
There is no built-in mechanism for that. However, you can create one mechanism yourself in a very simple way. Meaning, that you can create a document in which you can increment/decrement a numeric value, each time a document is added or deleted from the collection. Once you hit the limit, you can restrict the addition of documents in that particular collection.
OR is there a way to automatically delete old documents in a collection just by settings and not writing some cron job or scheduled function?
There is also no automatic operation that can help you achieve that. You can either use the solution above and once you hit the limit + 1, you can delete the oldest document. Or you can use a Cloud Function for Firebase to achieve the same thing. I cannot see any reason why you should use a cron job. You can use a Cloud Scheduler to perform some operation at a specific time, but as I understand you want it to happen automatically when you hit the limit.
Alternatively, what options are there to create a rotational logging system for client activities in Firebase?
If you still don't want to have larger collections, maybe you can export the data into a file and add that file to Cloud Storage for Firebase.
I'm trying to figure out what the overhead of my data will be given the automatic indexing that happens with Firestore.
First I understand that you are charged for the amount of data that you store in Cloud Firestore, including storage overhead.
I've also read through the Firebase documentation on calculating document size for Firestore in native mode. I used that to calculate a rough estimate of a document, the full collection, and the size of the automatic indexes.
I am looking for a way to check that my calculations are correct. The closest thing I've found is Googles App Engine "Cloud Firestore Stored Data" quota. However, based on my calculations and testing, it seems to me that this only includes user added data and not the automatically created indexes or other metadata. Is that correct? and if so, is there any other way to figure out how much data I have in my Firestore DB besides calculating it by hand?
Update : Previous answer was for Firestore in Datastore mode but OP's question is about Firestore in Native Mode. The answer has been updated to reflect Firestore in Native Mode
I can't find any direct answer but I would infer it does by looking at the following documentations
Quotas documentation breaks down the different parts of the quota and it includes a section on indexes
Storage Size documentation also has an entry for calculating the size of the index which leads me to conclude it affects your storage size which in turn is part of your quota
I want my app to fetch data from my server only once in every one hour. so if the timestamp is over one hour, then I use Source.SERVER to retrieve data from firestore server, otherwise, if it is below range of one hour, then just get previous data from cache using Source.CACHE
so my question is, if I get the data from Source.CACHE, will it be counted as Read operation in Firestore ?
currently I am investigating the number of read operation in firestore, because in my project, the number of reads is way above the expected number. so I suspect this cache is the problem, because I now assume the data from cache will also be counted as read operation. am I right ?
I have no other idea why the number of read is so high in my project.
Documents read from cache are not billed as reads.
Understand firestore charge based on read / write operation.
But I notice that the firestore read from server per app launch, it will cause a big read count if many user open the app quite frequent.
Q1 Can I just limit user read from server for first time login. After that it just read for those update document per app launch?
For example there's a chat app group.
100 users
100 message
100 app launch / user / day
It will become 1,000,000 read count per day?
Which is ridiculous high.
Q2 Read is count per document, doesn't matter is root collection / sub collection, right?
For example, I read from a root collection that contain 10 subcollection and each of them having 10 documents, which will result 100 read count, am i right?
Thanks.
That’s correct, Cloud Firestore cares less about the amount of downloaded data and more about the number of performed operations.
As Cloud Firestore’s pricing depends on the number of reads, writes, and deletes that you perform, it means that if you had 100 users communicating within one chat room, each of the users would get an update once someone sends a message in that chat, therefore, increasing the number of read operations.
Since the number of read operations would be very much affected by the number of people in the same chatroom, Cloud Firestore suits best (price-wise) for a person-to-person chat app.
However, you could structure your app to have more chat rooms in order to decrease the volume of reads. Here you can see how to store different chat rooms, while the following link will guide you to the best practices on how to optimize your Cloud Firestore realtime updates.
Please keep in mind that Cloud Firestore itself does not have any rate limiting by default. However, Google Cloud Platform, has configurable billing alerts that apply to your entire project.
You can also limit the billing to $25/month by using the Flame plan, and if there is anything unclear in your bill, you can always contact Firebase support for help.
Regarding your second question, a read occurs any time a client gets data from a document. Remember, only the documents that are retrieved are counted - Cloud Firestore does searching through indexes, not the documents themselves.
By using subcollections, you can still retrieve data from a single document, which will count only as 1 read, or you can use a collection group query that will retrieve all the documents within the subcollection, counting into multiple reads depending on the amount of documents (in the example you put, it would be 10x10 = 100).
I was wondering, how does Firestore handle real-time syncing of deeply nested objects? Specifically, does it only sync the diff?
For example, I have a state of the app which is just an array of 3 values and this state is synced between devices. If I then change one of the values will the whole new array be synced (transmitted on the network) or only the diff? What if my state is the nested object?
I'm asking because I want to sync the whole state which is an object with multiple fields but I don't wont to sync the whole object when I only change single field.
Like Realtime Database, Cloud Firestore uses data synchronization to update data on any connected device. However, it's also designed to make simple, one-time fetch queries efficiently.
Queries are indexed by default: Query performance is proportional to the size of your result set, not your data set.
Cloud Firestore will only send your device only the difference of the document.
Tips:
Add queries to limit the data that your listen operations return and use listeners that only download updates to data.
Place your listeners as far down the path as you can to limit the amount of data they sync. Your listeners should be close to the data you want them to get. Don't listen at the database root, as that results in downloads of your entire database.
Hope it helps!