I am new to Flutter and mobile development in general and currently trying to wrap my head around certain database principles. I am building an note-taking app which uses Firestore for storing note data to the cloud. Users first need to register an account and log in, after which they are able to view and store notes to the cloud. The way I initially designed this was by using StreamProvider connected to the Firestore instance to update the user's list of notes after they add a note to their list.
After some reflection and worries that my app would be 'read' intensive, I realised that what I am trying to build does not require the notes to be constantly fetched from the server, as the data is private to them and should only be fetched by the user that creates it. My solution is instead of creating notes and listening to changes on the Firestore server to update the list, when a user creates a new note, a function should run and update the Firestore server as well as a local nosql database, removing the need for a read operation on the server after writing to it. This would allow me to have a local duplicate copy of the Firestore server and to periodically update it with write operations when needed. The only read operation would be on the app's startup to fetch a copy of the Firestore server.
My questions are:
Is storing a local version of the Firestore server a viable solution to minimise the number of read operations in my app, or am I overcomplicating things?
The StreamProvider allowed me to easily access data relating to the user, notes, etc throughout the app using Provider.of<Model>(Context). How can I easily access this data regardless of where I am in the app?
Given that my Firestore database structure for accessing notes is users/{user.id}/notes/{note.id}/note , how can I get a snapshot of the current logged in user ID and all the child fields in a single request (i.e. all the user's notes in users/{user.id}/notes and user data under users/{user.id}/)
Related
Here is our use case:
We have way more than 200,000 clients need to connect to firebase realtime db. So we created multiple database with same data and load blance the connections.
Here is the problem:
If we update one database, we will have to initiate connection and udpate the rest of the database as well. I would like to check if there is a way to auto sync up data between multiple databases.
Docs I have went through:
https://firebase.google.com/docs/database/usage/limits
https://firebase.google.com/docs/database/usage/sharding
Also I checked rules, and it seems that rules is not meant to be used to sync data.
Thanks
firebaser here
There is nothing built into Firebase to automatically synchronize data between multiple database instances. A common way to implement this when writing through a server-side process, is to simply write to each database in turn there.
If the data you want to write comes from a client-side SDK, I'd have the client write it to a staging area (just a temporary node in the database), and then use Cloud Functions to write the data the permanent location in all database instances.
I am working on the following topic right now:
about 8000 stores/objects(uniqueID, name, adress, latitude, longitude, geohash, amount of productX, amount of productY.)
in the future there will be new stores and also some of the stores will be deleted
I am planning to add a version attribute to each data: for example I start with version1. If I receive an update or a new store then it will have version2
it is not needed to receive realtime changes. This data will be changed only a couple of times each month
Flutter app with a feature: find n stores at this location
because the data is not changed frequently I was planning to put an updated local sqlite database to each build
I will use Realtime Database or Firestore to receive updated data within my app to avoid to publish a new version only because database was changed
Firestore: I will add only a new attribute to my document and I will request all data that is newer than my local database version and I will also have a collection that has all deleted items
Realtime database: here I would have something like this
-updates
version1
newData: List of unique IDs
deletedData: List of unique IDs
I read a lot about geo queries on firestore and I also found a working flutter plugin to do this filtering on serverside, but because my data is not changed that frequently I am very confused about the best approach.
Questions:
Is my approach a good idea or should I forget about the local sqlite DB and use only Firebase/Firestore?
If I will use the local database and will only observe updates from the backend, which of the services should I use in terms of pricing etc.? (Advantage Firestore: If I use Firestore I can easily switch to it by using the flutter plugin and forget about my local database)
If you have only a couple of thousand objects, and the data hardly every changes, I'd typically consider querying the data locally on the device. You can likely very quickly iterate over the 8,000 items to check their lat/lon against the user location.
You could then use something like Firebase Storage or even Firebase Hosting to distribute updated data sets to the users. Of course you could use Firestore or Realtime Database for that too, but if you're not using their querying or realtime capabilities, there are cheaper options to distribute a static file.
my app is fetching data from a Firestore database but this data should only be accessible to users who are currently subscribed to In-App-Purchases.
What is the best way to protect this data?
My first thought was routing everything through a Firebase Function, so the user has to send their subscription ID to the function, the function checks it and returns the data, but this would require way too many function calls for just a few users.
If you are not using any server client libraries then you can create a rule on firestore to restrict the reads on the table.
firestore rules docs..
You could simply check if the requesting user has subscribed to In-App-Purchases (assuming there is a flag in the user's data to confirm the same).
It seems odd to me that Firestore would charge me for read queries to locally cached data, but I can't find any clarification to the contrary in the Firestore Pricing document. If I force Firebase into offline mode and then perform reads on my locally cached data, am I still charged for each individual entity that I retrieve?
Second, offline users in my app write many small updates to a single entity. I want the changes to persist locally each time (in case they quit the app), but I only need eventually consistent saves to the cloud. When a user reconnects to the internet and Firestore flushes the local changes, will I be charged a single write request for the entity or one per update call that I made while offline?
Firestore could potentially fit my use case very well, but if offline reads and writes are charged at the same rate as online ones it would not be an affordable option.
As the offical documentation says,
Cloud Firestore supports offline data persistence. This feature caches a copy of the Cloud Firestore data that your app is actively using, so your app can access the data when the device is offline. You can write, read, listen to, and query the cached data. When the device comes back online, Cloud Firestore synchronizes any local changes made by your app to the data stored remotely in Cloud Firestore.
So, every client that is using a Firestore database and sets PersistenceEnabled to true, maintains it's own internal (local) version of the database. When data is inserted/updated, it is first written to this local version of the database. As a result, all writes to the database are added to a queue. This means that all the operations that where stored there will be commited on Firebase servers once you are back online. This also means that those operations will be seen as independent operations and not as a whole.
But remeber, don't use Firestore as an offline-only database. It is really designed as an online database that came work for short to intermediate periods of being disconnected. While offline it will keep queue of write operations. As this queue grows, local operations and app startup will slow down. Nothing major, but over time these may add up.
If Google Cloud Firestore priceing model does not fit your use case very well then use Firebase Realtime Database. As mentioned also in this post from the Firebase offical blog, one the reasons you still might want to use the Realtime Database is:
As we noted above, Cloud Firestore's pricing model means that applications that perform very large numbers of small reads and writes per second per client could be significantly more expensive than a similarly performing app in the Realtime Database.
So it's up to you which option you choose.
According to this If you want to work completely offline with Cloud Firestore you can disable network by :
FirebaseFirestore.getInstance().disableNetwork()
but firestore will cause client offline error for first user get request, that you must consider this error as empty response.
I'm thinking moving an Android app's persistent data to Firebase Database. Currently I use Sqlite with a Python HTTP REST service.
In the app, I have a big list of cities, called citiesList. Since the list is quite big and hardly updated, I don't want the app fetch the list whenever it goes online.
My current strategy is provide a citiesListVersion. When the app goes online, it checks citiesListVersion. If the server's citiesListVersion is newer, the app will fetch citiesList from server. Otherwise, the app continues working with the cached data.
My question is: can I keep my current strategy when moving to Firebase Database? As I understand, Firebase tries to sync data whenever the app goes online.