I am wondering if IndexedDB is the only cache Firestore uses.
To be clear, I am not talking about persisting data with enableIndexedDbPersistence(). I am talking about an internal store for the sole purpose of optimistic updates when the app is still in state. Is there something similar to redux, svelte store, or InMemoryCache which is run in the background when a subscription is open?
It seems that when I use onSnapShot() for a list of items, and I update an item in the list with setDoc elsewhere, the UI gets updated immediately, optimistically.
Perhaps Firestore is just that quick where the data is sent to the server, changed, and sent back to the client with the UI being updated that quickly, but it seems to me it is an optimistic update.
Does Firestore use any other caching techniques or state management techniques when the app is still running besides IndexedDB?
J
References (maybe releated):
Does Firebase cache the data?
As long as you have an active onSnapShot listener, the Firestore SDK will have a copy of the latest query snapshot for that listener in memory. If you attach another listener to the same (or partially overlapping) data, that listener may get (part of) its data from the existing listener.
So when you perform an update in the same client as where you have a listener, the SDK immediately applies that update to its local copy of the data and fires an event (so that is almost instantly). It then sends the update to the server, which executes it on the backend storage layer. If that update gets rejected (a relatively rare occurrence), the client will revert the change it make locally and fire another event with the corrected state.
An easy way to see this in practice is to perform a write operation that is rejected by your security rules. You'll briefly see the invalid state on the client, before it reverts to the correct state. This invalid state only happens on the client that performs the invalid write, so it's typically fine to ignore it there.
Related
I came across a wonderful feature of Firebase offline feature. I integrated that in my app just by writing one line of code in my main.dart file after initializing Firebase await FirebaseDatabase.instance.setPersistenceEnabled(true);
Question 1 :
I couldn't able to understand the database.keepSynced(true) function because without using this line of code, my app is persisting old as well as fetching new updated data, so what this exactly does ?
Question 2 :
How could I prevent the write operations when a user is offline, because I read that after setting persistence enabled, it makes a queues of write operations and update them when user gets online, so how could I stop this ?
Question 3 :
Is this persistence feature going to work in IOS device as well or need some permission settings first ?
Thanks
When you call FirebaseDatabase.instance.setPersistenceEnabled(true) you're allowing Firebase to create a local file on the device where it persists any data it's recently read, and all writes that are pending while the device is offline.
When you call keepSynced(true) on a node, you are telling the SDK to always keep that node synchronized. It essentially creates a onValue listener on the node without any handler code, so you're purely doing this to keep the data synchronized for when the device does go offline.
By combining keepSynced(true) with setPersistenceEnabled(true), you're specifying that you want the app to continue working when it's offline across restarts, and which data is needed for that.
If you call keepSynced(true) on the root of your database, you're telling the SDK to synchronize all data in the database to the disk cache. While this may initially be a quick way to get offline mode for your app working, it typically won't scale when you more people start using your app.
If you only want to allow write operations while the client has a connection to the database backend, you can register a local listener to the .info/connected node, which is a true value when there is a connection and false otherwise.
Note that Firebase doesn't require this, as it queues the pending writes and executes them when the connection is restored. In general, I'd recommend working with the system here instead of against it, and also trying to make your app work gracefully in the offline scenario. In many cases there is no need to disable functionality while the app is offline.
Offline disk persistence is available on Android and iOS, but not on web.
If I perform an update operation on a document in Firestore, and I know for certain that the document exists, what possible reason, outside of my control, could an error be generated?
And if an error is generated, is the update automatically retried?
Or is this update only automatically retried if there is no connection and Firestore's offline capabilities take over?
If I perform an update-data operation on a document in Firestore, and I know for certain that the document exists, what possible reason, outside of my control, could an error be generated?
A common error that can arise, is when the Firebase servers reject an operation due to improper security rules. Meaning that you are not allowed to do that particular operation.
And if an error is generated, is the update automatically retried?
No, that will not happen while online.
Or is this update only automatically retried if there is no connection and Firestore's offline capabilities take over?
Yes, that will indeed happen. All operations that take place while offline, are added to a queue. Behind the scenes, Firestore SDK tries to reconnect until the devices regain connectivity. Once the user regains the connection, every change that is made while offline will be updated on Firebase servers. In other words, all update operations will be committed on the server, as long as you have proper rules.
I have a single page web application and I have NOT enabled offline persistence.
A component in this application has subscribed to a document using the onSnapshot function and recieves updates.
An other component in the same application instance reads the same document using the get function. Will firebase provide the same instance of the document, which is already loaded in the subscription? Or will it do a new request to the server?
A third component in the same application instance subscribes to the same document. Will this trigger new requests to the server?
What about reading and subscribing collections?
What about subscribing to a collection which is already subscribed, but the second subscription is called after narrowing results with the where function? Will Firestore reuse the already loaded complete collection and filter in my client instance? Or will it make server requests?
As a rule, Firestore does maintain a local cache, mostly for latency/response, not for cost savings. That said, if a document can be read from the cache, it will be, saving the read. I wouldn't try to "design" it in, however; memory limits and other considerations mean you don't really know what's in the cache or not, and shouldn't know.
IF this performance/cost is important to you, it's better to combine Firestore with something like Redux (which I do) to maintain local "sets" of documents to reduce round-trip calls, and Listeners to keep Redux current.
yes. Firebase stores documents and collections in cache memory. when you are offline you can access cache copy of data. also you can manage cache copy of documents and collections.
read this offical docs for more info
https://firebase.google.com/docs/firestore/manage-data/enable-offline
https://firebase.google.com/docs/firestore says:
Like Realtime Database, Cloud Firestore uses data synchronization to update data on any connected device.
I'm trying to figure out what this "data synchronization" consists of.
https://firebase.google.com/docs/firestore/manage-data/enable-offline says:
When persistence is enabled, Cloud Firestore caches every document received from the backend for offline access.
Does Cloud Firestore update that cache whenever changes happen in the back end?
For example, if I have a certain document in my cache, does the document
update (a) automatically when that document is changed in the back end or
(b) whenever I do a query that uses that document, and I happen to be online
at the time?
UPDATE
I'm going to restate my last paragraph, because it's been misunderstood by
at least one person.
Imagine the following scenario:
I have a smartphone. To make things easy, assume the smartphone
is online all the time.
Installed on this smartphone is a mobile app that contains the Firestore SDK.
The app has enabled the Firestore cache.
The cache contains, say, 1000 documents.
Now my question is: if one of these documents changes in the cloud (not on
the smartphone), under what circumstances will the Firestore SDK update the
document in the cache? Will it (a) update the document automatically, soon
after the change on the cloud, or (b) update the document the next time the
mobile app does a query that uses the document?
Both of the highlighted paragraphs are in my opinion straightforward explanations, and to answer your questions:
Does Firestore update that cache whenever changes happen in the back end?
Yes, if you are offline and some documents are changed, once you are back online, you get all changed documents and vice versa, if you change some documents locally (in your cache), when you're back online, all changed documents are sent to the Firebase servers.
If you have a query that returns 10 documents, it means that you are in sync with 10 documents, which basically means that if of those 10 documents is changed, your listener is triggered only for that change.
does the document update (a) automatically when that document is changed in the back end
Yes, first of all in your cache and once you are back online, also on Firebase servers.
whenever I do a query that uses that document, and I happen to be online at the time?
Whenever you are online and a document is changed, you are notified in realtime.
Edit:
if one of these documents changes in the cloud (not on the smartphone), under what circumstances will the Firestore SDK update the document in the cache? Will it (a) update the document automatically, soon after the change on the cloud, or (b) update the document the next time the mobile app does a query that uses the document?
As long as you are in sync with those documents, it happens in the second moment the document is changed. What I mean through in sync is, that you are using in your code a get() call to get those documents or if you are listening in realtime for documents changes. Simply creating a reference to a document or a query without using a listener, you aren't notified in any way.
update the document the next time the mobile app does a query that uses the document?
Is not correct since you are online and in sync with the documents, the listener is triggered instantly.
Edit2:
In short, a listener is called everytime a change in a document occurs. "Next time the app does the query" is not correct sentence because when you create a query and attach a listener, you are always in sync with the database and when a document is changed you are immediately notified. It is not like, the documents are chainging in the database and when I query (at some point of time), I get the changes, no, you are always notified as the changes occur. That's the beauty of a realtime database, to be notified as the changes are happening :)
If you have the local cache enabled, Firestore automatically stores any documents it receives in that local cache.
This only happens when your app is requesting the documents by calling get() or onSnapshot. Firestore does not automatically request the documents on your behalf. So if your app doesn't request the document, the cache will not be updates.
According to firestore documentation:
a transaction is a set of read and write operations on one or more documents.
Also:
Transactions will fail when the client is offline.
Now the limitation in firestore is that:
In Cloud Firestore, you can only update a single document about once per second, which might be too low for some high-traffic applications.
So using cloud functions and running transactions to increment/decrement counters when the traffic is high will fail.
So they have discussed to use the approach of distributed counters.
According to the algorithm of distrbuted counter:
create shards
choose a shard randomly
run a transaction to increment/decrement the counter
get all the shards and aggregate the
result to show the value of a counter
Scenerio:
consider you have a counter which is to be updated when a document is added and that counter is being displayed in the UI. Now for good UX, I cannot block the UI when network is offline. So I must allow creation/updation of documents even when client is offline and sync these changes once client is online so that everyone else listening to these changes receive the correct value of the counter.
Now transactions fail when the client is offline.
So my question for best user experience (even when offline) is:
Do you really require a transaction to increment a counter? I know
transactions ensure that writes are atomic and are either
successful/unsuccessful and prevent partial writes. But what's the
point when they fail offline? I was thinking maybe write them to local cache and sync it once the network is back online.
Should this be done via client sdks of via cloud functions?
Do you really require a transaction to increment a counter?
Definitely yes! Because we are creating apps that can be used in a multi user environment, transactions are mandatory, so we can provide consistent data.
But what's the point when they fail offline?
When there is a loss of network connectivity (there is no network connection on user device), transactions are not supported for offline use. This is because a transaction absolutely requires round trip communications with server in order to ensure that the code inside the transaction completes successfully. So, transactions can only execute when you are online.
Should this be done via client sdks of via cloud functions?
Please note, that the Firestore SDK for Android has a local cache that's enabled by default. According to the official documentation regarding Firestore offline persistence:
For Android and iOS, offline persistence is enabled by default. To disable persistence, set the PersistenceEnabled option to false.
So all read operations will come from cache if there are no updates on the server. So Firestore provides this feature for handle offline data.
You can also write a function in Cloud Function that will increment the counter while a new document is added or to decrement the conter while a document is deleted.
I also recommend you to take a look:
How to count the number of documents under a collection in Firestore?
So you may also consider using Firebase realtime database for that. Cloud Firestore and Firebase realtime database work very well together.
Edit:
It allows one to upvote the answer even when the device is offline. After the network is online, it syncs to the server and the counter is updated. Is there a way i can do this in firestore when the device is offline.
This is also happening by default. So if the user tries to add/delete documents while offline, every operation is added to a queue. Once the user regains the connection, every change that is made while offline, will be updated on Firebase servers. With other words, all queries will be commited on the server.
Cloud fnctions are triggered only when the change is received and that can only happen when the device is online.
Yes, that correct. Once the device regains the network connection, the document is added/deleted from the database, moment in which the function fires and increases/decreases the counter.
Edit2:
Suppose I have made around 100 operations offline, will that not put a load on the cloud functions when the device comes online? What's your thought on this?
When offline, pending writes that have not yet been synced to the server are held in a queue. If you do too many write operations without going online to sync them, that queue will grow fast and it will not slow down only the write operations it will also slow down your read operations. So I suggest use this database for its online capabilities.
Regarding Cloud Functions for those 100 offline operations, there will be no issues. Firebase servers work very well with concurent operations.