I have a two-part question. After reading through Firestore pricing, it says that you are charged based on the number of documents you read and write and the operations you perform such as deleting and some other factors. With that being said, I am using a StreamBuilder that continually refreshes itself whenever the list is scrolled whether there is new data in the database or not. Right now, the builder is fetching from a collection with very little data but that collect can grow to be bigger. With that being said, my questions are:
Each time the StreamBuilder refreshes itself to show new data, is it actually fetching all the documents again from the Firestore collection or is it only looking for changes and updating them? If it is fetching the documents again from Firestore, does Firestore consider this as downloading multiple documents each time it refreshes or does it count it only once and if there are updates to any new document fetched, those are counted separately?
If it fetches all the documents over and over again every 2 seconds or even less as in the current behavior, is there a way to limit this say to every 30 seconds or to when every a RefreshIndicator is used so as to avoid multiple unnecessary reads? I tried using a StreamController but the stream still refreshes every time the list is touched or every second.
Well i guess it depends a bit on your code. I think there are methods to listen to firestore changes constantly.
However if you use the most common queries then this should not be the case. Here my reasoning why, according to my understanding:
Streambuilder: The builder function is triggered everytime data hits the sink of a stream.
Sink is the input channel for any data. Streams immediately return data which is put in the sink.
Firestore: If you execute a firestore "query" it will read document by document and return it once it is read. Once all documents are read the connection will be closed.
If you now assign the firestore query as stream to your builder, example below. The builder is triggered when a document is read. In the builder you then probably build a widget which is displayed.
Once the firestore query has read all documents no new data will be pushed into the sink and therefore the builder will not be triggered anymore. This query will then be completed and no longer listen to changes as the connection will be closed.
Therefore the documents are usually only read once during the lifetime of a streambuilder.
StreamBuilder<QuerySnapshot>(
stream: Firestore.instance.collection('your collection').snapshots(),
builder: (BuildContext context,
AsyncSnapshot<QuerySnapshot> snapshot) {
//Your own code to handle the data
})
I recently build an app where I read tasks from firestore and process the documents via StreamBuilder. An easy way to test how often a document is read is by simply printing the document to your console in the builder section.
What I observed is that documents are only read once as long as the Widget tree in which the Streambuilder resides is not rebuild.
So to answer your question:
My understanding is that if the StreamBuilder refreshes or is initialized again then it triggers again the query and reads the data. According to the firestore documentation each read of the document is counting towards your limits and costs. So I would say yes it counts for all documents included in your query.
I am not sure how you constantly refresh or initialize the streambuilder, therefore I can't give you a clear answer. If you just use the code similar to above once during the build of the widget tree then it should be read only once...
Without some more details, I cannot provide more information.
Each time the StreamBuilder refreshes, it will query Firestore for the documents in the collection again. Firestore will count this as a read operation for each document retrieved. However, Firestore does not count it as multiple reads for the same document if it is fetched multiple times. Firestore only charges for each document read once.
Firestore provides real-time updates through the use of listeners. When you listen to a Firestore collection using a StreamBuilder, Firestore will automatically send updates whenever there are changes to the documents in the collection. This means that the StreamBuilder will only fetch the documents that have changed, rather than the entire collection. However, if the entire collection changes (e.g. if a new document is added), then Firestore will fetch the entire collection again.
Each time the StreamBuilder refreshes, it creates a new query to Firestore, which counts as a read operation. If you refresh the StreamBuilder frequently, this can lead to a large number of read operations. However, Firestore does not charge you for reading the same document multiple times in a short period of time. For example, if you refresh the StreamBuilder every second and the same document is retrieved each time, Firestore will only count this as one read operation for that document.
To limit the number of reads, you can consider using a caching mechanism to store the data on the client-side and avoid making unnecessary network requests. You can also adjust the frequency of fetching new data by using a timer or a debounce mechanism to avoid refreshing too frequently.
Related
If I understand correctly, in the initialization phase of addSnapshotListener you get a list of all the documents (even if it is 500 trillion documents) from the QuerySnapshot if you call the getDocuments function.
Then, every time you modify or add a document to a collection, you get from QuerySnapshot all the documents that have been modified by calling the getDocumentChanges function or all the existing documents by calling getDocuments.
That means both at the initialization stage and after every change, I always get a list of all the documents. That's logical? Assuming I have 500 trillion documents under the same collection (just for the sake of exaggeration), at every change and initialization of the app will I get them all?
Is that really the case?
Or is it some kind of lazy instantiation or something?
Because if so, when I would like to question the whole collection, no matter what I get at first the whole list?
The QuerySnapshot always contains all documents that match the query (or collection reference). Even when there's an update and only a subset of the documents matched by the query is changed, the QuerySnapshot still contains all of the documents, even though in its communication between the SDK and the backend serves, Firestore only synchronizes the modified documents. If you only want to process the changes, you can process just the changes between the snapshots.
I'm currently developing a Flutter web application using Firestore for data persistence. The app is not live in production, so I'm the only one accessing this backend. There is only one collection that holds a single document, with many nested fields (6 levels deep). My understanding from looking at https://firebase.google.com/docs/firestore/pricing, is that reads are counted per doc, so every time I reload my app it should count as one read, yet in the last 4 hours since I started working today I already hit 1.7K reads (as reported in the usage tab). I know I haven't reloaded the app that many times, and there's also no hidden loop that calls the collection multiple times.
This is the Flutter code that calls Firestore:
final sourceRef=FirebaseFirestore.instance.collection("source");
var data=await sourceRef.doc("stats").get();
What am I missing please?
According to Firebase pricing, writes are defined as:
You are charged for each document read, write, and delete that you perform with Cloud Firestore.
Charges for writes and deletes are straightforward. For writes, each set or update operation counts as a single write.
Meaning that one document created is one write. If the same document is updated later, then Firebase counts it as one more write.
Here is a more detailed table that you can use for billing, and an example.
It is recommended to view individual product usage in the "Usage" tab for many products in the Firebase console, as this can narrow the product that is causing the elevated usage that you are seeing.
I would highly recommend adding write and view logs to your application; that way, you can monitor how many writes and reads you have.
When using firestore snapshot(), and set a listener, Cloud Firestore sends your listener an initial snapshot of the data, and then another snapshot each time the documents change.
However if I close the app, and reopen it, does firestore make a read on all the data it already has queried or is there an internal sync system (for example if they store documents metadata, like updatedAt they could only read documents that haven't been updated since x) ?
In other words. if I use onSnapshot() listener, I will make x documents read initially, then 1 document each time a document changes. My question is: If I close the app and a document changes, then when I open the app, is 1 read made or x + 1 ?
It is important for me because I have a bunch of initial calls and I'm wondering how that'd affect the cost($).
It's also important to know for data modeling and how it affects the cost.
Every time you perform a new query against the server (this is the default), it will cost a read, and the documents will have to be transferred. It will not use the cache unless there is no connection, or your specifically target the cache for the query. Quitting and returning to the app doesn't change this behavior at all.
I suggest reading this: https://medium.com/firebase-developers/firestore-clients-to-cache-or-not-to-cache-or-both-8f66a239c329
It depends on the type of listener
OnChange() will read only when data changes
addListenerForSingleValueEvent will check just once, and if it is the onCreate section, it will be executed immediately
addValueEventListener will keep checking constantly, but will log as a read only if the data changes
Is there any way to check the read or write count depending on the call made by the code. For instance, let's say I have
final ref = Firestore.instance.document('users/uid');
ref.get();
Now that this is 1 read operation, and I can use
ref.delete()
or
ref.setData(data);
etc.
This is easy for single operations, how about I am listening for a Stream, how do I know how many read/write operations are taking place by my function calls?
It doesn't matter if it's a stream, or iterating an array, or any other form of consumption. Once the query hits the server, you are charged for the number of documents returned by that query, regardless of how you choose to iterate them. They show up in the client app all at the same time, and you get to choose how to deal with them.
For realtime listeners, you are charged a read for each document that gets delivered to your snapshot listener. Again, it doesn't matter what you do with the documents after that - the cost is already paid.
I know that whenever I turn on a listener to a QUERY, the minimum charge is one read, even if this query returns no documents (because there are no documents that fit the query, or even if there are but there were no changes since last time this listener was on).
According to documentation:
Minimum charge for queries
There is a minimum charge of one document read for each query that you perform, even if the query returns no results.
Stream<QuerySnapshot> stream = Firestore().collection("someCollection").where("someField", isGreaterThan: 42).snapshots();
stream.listen((querySnapshot) => doSomething());
Also, I know that I'm not charged if I use a simple get to read some document, as long as this document already exists in the offline cache,
and was not changed in Firestore (According to Doug Stevenson).
DocumentSnapshot doc = await Firestore().collection("someCollection").document("documentID").get();
My question is:
Suppose a document already exists in my offline cache, and was not changed in Firestore since.
If instead of a get I turn on a listener to this single document
(DocumentSnapshot, NOT a QuerySnapshot), like this:
Stream<DocumentSnapshot> stream = Firestore().collection("someCollection").document("documentID").snapshots();
stream.listen((documentSnapshot) => doSomething());
Will I be charged for one read when this listener is turned on?
Will I be charged for one read when this listener is turned on?
No. If you turn on a listener that is listening to a single document, you are not charged at all since the document already exists in the cache. Remember, you are charged with one document read when a query does not return any results but in your case, you aren't performing any query, you are just listening to a document change. It makes no sense to be charged with one document read each time you start listening for changes or each time you open the application. That's the beauty of Firestore, you have a local copy of your database so you cannot be charged for cached documents, unless the documents are changed in the back-end.