I have an app that has 50k - 60k document reads a day and I can't afford another plan now, so I'm looking for a way to optimize reads, getting less reads possible I can.
The database has 4 collections with some subcollections, I think I might have around 1000 documents in the whole server.
It is there anyway: I can read documents from cloud Firestore (first time the user opens the app) and store on the Firestore cache, after that, make the app only load data from Firestore cache, and if there is a new document(s), it reads these documents only, store to the cache, and keep reading from the cache?
The app could only load data from cache (using the parameter "source") and never directly from Firestore, if possible.
Firestore.instance.collection("images").getDocuments(source: Source.cache);
Is is there anyway: I can read documents from cloud Firestore (first time the user opens the app) and store on the Firestore cache
That's the default behavior. According to the official documentation regarding Firestore offline persistence:
For Android and iOS, offline persistence is enabled by default. To disable persistence, set the PersistenceEnabled option to false.
For the web, offline persistence is disabled by default. To enable persistence, call the enablePersistence method. Cloud Firestore's cache isn't automatically cleared between sessions. Consequently, if your web app handles sensitive information, make sure to ask the user if they're on a trusted device before enabling persistence.
So there is nothing special that you need to do. Once you open a stream on one of your collections, the data is added to the cache.
after that, make the app only load data from Firestore cache, and if there is a new document(s), it reads these documents only, stores to the cache, and keeps reading from the cache?
That's again the default behavior, but this work as long as the documents in your database are not changed. If a document in the database is changed, you'll be charged with a document read for each document. The mechanism is the same.
Besides that, you can tell Firestore to read data only from the cache if needed, as explained in the answer from the following post:
How to cache Firebase data in Flutter?
Related
I have a clarifying question about Cloud Firestore and Flutter:
I am making an app that users can create log entries of sort, which will be saved on Firebase. But they might make up to 30 entries offline, before they have internet connection again.
And I know that Firestore has an offline feature, with which any created documents can be viewed offline because it is saved in the order of logging, and then synced with the database when internet connection is gained. But it is absolutely crucial that these logs cannot be lost in my app before having a chance of uploading it. Is there a way to ensure that my app will not lose this data before connecting to Firebase again, or should I create a Sembast database on the device, and save a copy of everything, and then check that once in a while against the database?
Does Firestore have offline contingency for if the phone's battery dies before it could sync with Firestore?
Or is there another solution I am unaware of?
For Android and iOS, offline persistence is enabled by default.
You don't have to do anything in your code.
Note there is a default cache size of 100 MB. This can be changed though, eg
let settings = Firestore.firestore().settings
settings.cacheSizeBytes = FirestoreCacheSizeUnlimited
Firestore.firestore().settings = settings
Does Firestore have offline contingency for if the phone's battery dies before it could sync with Firestore?
When Firestore's offline persistence is enabled, it stores data that your app has recently read, as well as any pending writes, to a database on the local device or browser. This disk based cache will survive restarts of the app/reloads of the page.
For new version of use below code -
FirebaseFirestore.instance.settings =
Settings(cacheSizeBytes: Settings.CACHE_SIZE_UNLIMITED);
I have a single page web application and I have NOT enabled offline persistence.
A component in this application has subscribed to a document using the onSnapshot function and recieves updates.
An other component in the same application instance reads the same document using the get function. Will firebase provide the same instance of the document, which is already loaded in the subscription? Or will it do a new request to the server?
A third component in the same application instance subscribes to the same document. Will this trigger new requests to the server?
What about reading and subscribing collections?
What about subscribing to a collection which is already subscribed, but the second subscription is called after narrowing results with the where function? Will Firestore reuse the already loaded complete collection and filter in my client instance? Or will it make server requests?
As a rule, Firestore does maintain a local cache, mostly for latency/response, not for cost savings. That said, if a document can be read from the cache, it will be, saving the read. I wouldn't try to "design" it in, however; memory limits and other considerations mean you don't really know what's in the cache or not, and shouldn't know.
IF this performance/cost is important to you, it's better to combine Firestore with something like Redux (which I do) to maintain local "sets" of documents to reduce round-trip calls, and Listeners to keep Redux current.
yes. Firebase stores documents and collections in cache memory. when you are offline you can access cache copy of data. also you can manage cache copy of documents and collections.
read this offical docs for more info
https://firebase.google.com/docs/firestore/manage-data/enable-offline
https://firebase.google.com/docs/firestore says:
Like Realtime Database, Cloud Firestore uses data synchronization to update data on any connected device.
I'm trying to figure out what this "data synchronization" consists of.
https://firebase.google.com/docs/firestore/manage-data/enable-offline says:
When persistence is enabled, Cloud Firestore caches every document received from the backend for offline access.
Does Cloud Firestore update that cache whenever changes happen in the back end?
For example, if I have a certain document in my cache, does the document
update (a) automatically when that document is changed in the back end or
(b) whenever I do a query that uses that document, and I happen to be online
at the time?
UPDATE
I'm going to restate my last paragraph, because it's been misunderstood by
at least one person.
Imagine the following scenario:
I have a smartphone. To make things easy, assume the smartphone
is online all the time.
Installed on this smartphone is a mobile app that contains the Firestore SDK.
The app has enabled the Firestore cache.
The cache contains, say, 1000 documents.
Now my question is: if one of these documents changes in the cloud (not on
the smartphone), under what circumstances will the Firestore SDK update the
document in the cache? Will it (a) update the document automatically, soon
after the change on the cloud, or (b) update the document the next time the
mobile app does a query that uses the document?
Both of the highlighted paragraphs are in my opinion straightforward explanations, and to answer your questions:
Does Firestore update that cache whenever changes happen in the back end?
Yes, if you are offline and some documents are changed, once you are back online, you get all changed documents and vice versa, if you change some documents locally (in your cache), when you're back online, all changed documents are sent to the Firebase servers.
If you have a query that returns 10 documents, it means that you are in sync with 10 documents, which basically means that if of those 10 documents is changed, your listener is triggered only for that change.
does the document update (a) automatically when that document is changed in the back end
Yes, first of all in your cache and once you are back online, also on Firebase servers.
whenever I do a query that uses that document, and I happen to be online at the time?
Whenever you are online and a document is changed, you are notified in realtime.
Edit:
if one of these documents changes in the cloud (not on the smartphone), under what circumstances will the Firestore SDK update the document in the cache? Will it (a) update the document automatically, soon after the change on the cloud, or (b) update the document the next time the mobile app does a query that uses the document?
As long as you are in sync with those documents, it happens in the second moment the document is changed. What I mean through in sync is, that you are using in your code a get() call to get those documents or if you are listening in realtime for documents changes. Simply creating a reference to a document or a query without using a listener, you aren't notified in any way.
update the document the next time the mobile app does a query that uses the document?
Is not correct since you are online and in sync with the documents, the listener is triggered instantly.
Edit2:
In short, a listener is called everytime a change in a document occurs. "Next time the app does the query" is not correct sentence because when you create a query and attach a listener, you are always in sync with the database and when a document is changed you are immediately notified. It is not like, the documents are chainging in the database and when I query (at some point of time), I get the changes, no, you are always notified as the changes occur. That's the beauty of a realtime database, to be notified as the changes are happening :)
If you have the local cache enabled, Firestore automatically stores any documents it receives in that local cache.
This only happens when your app is requesting the documents by calling get() or onSnapshot. Firestore does not automatically request the documents on your behalf. So if your app doesn't request the document, the cache will not be updates.
My iOS app's firestore cache seems to have got out of sync with Firestore. As a result I've had to disable persistence. Is there anyway to reset the cache? And is there a way to make sure it's constantly in sync? All I did was delete documents from the database!
There is now a feature in the API for clearing persistence. It is not recommended for anything but tests, but you can use
firebase.firestore().clearPersistence().catch(error => {
console.error('Could not enable persistence:', error.code);
})
It must run before the Firestore database is used.
There's no API for manipulating the local cache in any way. The Firestore SDK chooses what to store based on queries that you perform.
On Android, users can manually clear the local data for an app without uninstalling it. This will remove all the data locally stored by the app.
If you have a specific use case, please feel free to file a feature request.
It seems odd to me that Firestore would charge me for read queries to locally cached data, but I can't find any clarification to the contrary in the Firestore Pricing document. If I force Firebase into offline mode and then perform reads on my locally cached data, am I still charged for each individual entity that I retrieve?
Second, offline users in my app write many small updates to a single entity. I want the changes to persist locally each time (in case they quit the app), but I only need eventually consistent saves to the cloud. When a user reconnects to the internet and Firestore flushes the local changes, will I be charged a single write request for the entity or one per update call that I made while offline?
Firestore could potentially fit my use case very well, but if offline reads and writes are charged at the same rate as online ones it would not be an affordable option.
As the offical documentation says,
Cloud Firestore supports offline data persistence. This feature caches a copy of the Cloud Firestore data that your app is actively using, so your app can access the data when the device is offline. You can write, read, listen to, and query the cached data. When the device comes back online, Cloud Firestore synchronizes any local changes made by your app to the data stored remotely in Cloud Firestore.
So, every client that is using a Firestore database and sets PersistenceEnabled to true, maintains it's own internal (local) version of the database. When data is inserted/updated, it is first written to this local version of the database. As a result, all writes to the database are added to a queue. This means that all the operations that where stored there will be commited on Firebase servers once you are back online. This also means that those operations will be seen as independent operations and not as a whole.
But remeber, don't use Firestore as an offline-only database. It is really designed as an online database that came work for short to intermediate periods of being disconnected. While offline it will keep queue of write operations. As this queue grows, local operations and app startup will slow down. Nothing major, but over time these may add up.
If Google Cloud Firestore priceing model does not fit your use case very well then use Firebase Realtime Database. As mentioned also in this post from the Firebase offical blog, one the reasons you still might want to use the Realtime Database is:
As we noted above, Cloud Firestore's pricing model means that applications that perform very large numbers of small reads and writes per second per client could be significantly more expensive than a similarly performing app in the Realtime Database.
So it's up to you which option you choose.
According to this If you want to work completely offline with Cloud Firestore you can disable network by :
FirebaseFirestore.getInstance().disableNetwork()
but firestore will cause client offline error for first user get request, that you must consider this error as empty response.