I am wondering whether it is a sound strategy to use the firebase offline capabilities as a "free" cache.
Let's assume that I am in activity A, I fetch some data from firebase, and then I move to activity B, which needs the same data. If the app is configured with setPersistenceEnabled(true) and, if necessary, also with keepSynced(true), can I just re-query the same data in activity B, rather that passing it around?
I understand that there is a difference between the two approaches regarding reading-from-memory and reading-from-disk (firebase offline cache). But do I really get rid of all the network overhead by using firebase offline?
Relevant links:
Firebase Offline Capabilities and addListenerForSingleValueEvent
https://groups.google.com/forum/#!msg/firebase-talk/ptTtEyBDKls/XbNKD_K8CQAJ
Yes, you can easily re-query your Firebase Database in each activity instead of passing data around. If you enable disk persistence, this will be a local read operation. But since you attach a listener (or keep it attached through keepSynced()), it will cause network traffic.
But don't use Firebase as an offline-only database. It is really designed as an online database that can work for short to intermediate periods of being disconnected. While offline it will keep queue of write operations. As this queue grows, local operations and app startup will slow down. Nothing major, but over time these may add up.
Related
We are building an app for our teams out in the field that they collect their daily information using Firebase. However one of our concerns is poor connectivity. We are looking to build an Online/Offline button they can click to essentially work offline for when things slow down. We've built a workflow in which we query all the relevant information from Firestore.
I wanted to know if there was a way to tell Firestore to work directly on the cache only and not try to hit the servers directly. I don't want Firestore attempting to make server calls until they enable online again.
You shouldn't need to do this. If you use realtime listeners, they will already first return the data from the local cache, and only then reach out to the server to check for updates.
If you are performing one-time reads, the SDK will by default try to reach the server first (since it has only one chance to give you a value). If you want it to only check the local cache, you can pass an argument to the get call to do so.
You can also disable the network completely, in which case the client will never call on the network and only serve from the local cache. I recommend reading about that and more in the documentation on using Firestore offline.
I am developing an Android app which basically does this: On the landing(home) page it shows a couple of words. These words need to be updated on daily basis. Secondly, there is an 'experiences' tab in which a list of user experiences (around 500) shows up with their profile pic, description,etc.
This basic app is expected to get around 1 million users daily who will open the app daily at least once to see those couple of words. Many may occasionally open up the experiences section.
Thirdly, the app needs to have a push notification feature.
I am planning to purchase a managed wordpress hosting, set up a website, and add a post each day with those couple of words, use the JSON-API to extract those words and display them on app's home page. Similarly for the experiences, I will add each as a wordpress post and extract them from the Wordpress database. The reason I am choosing wordpress is that it has ready made interfaces for data entry which will save my time and effort.
But I am stuck on this: will the wordpress DB be able to handle such large amount of queries ? With such a large userbase and spiky traffic, I suspect I might cross the max. concurrent connections limit.
What's the best strategy in my case ? Should I use WP, or use firebase or any other service ? I need to make sure the scheme is cost effective also.
My app is basically very similar to this one:
https://play.google.com/store/apps/details?id=com.ekaum.ekaum
For push notifications, I am planning to use third party services.
Kindly suggest the best strategy I should go with for designing the back end of this app.
Thanks to everyone out there in advance who are willing to help me in this.
I have never used Wordpress, so I don't know if or how it could handle that load.
You can still use WP for data entry, and write a scheduled function that would use WP's JSON API to copy that data into Firebase.
RTDB-vs-Firestore scalability states that RTDB can handle 200 thousand concurrent connections and Firestore 1 million concurrent connections.
However, if I get it right, your app doesn't need connections to be active (i.e. receive real-time updates). You can get your data once, then close the connection.
For RTDB, Enabling Offline Capabilities on Android states that
On Android, Firebase automatically manages connection state to reduce bandwidth and battery usage. When a client has no active listeners, no pending write or onDisconnect operations, and is not explicitly disconnected by the goOffline method, Firebase closes the connection after 60 seconds of inactivity.
So the connection should close by itself after 1 minute, if you remove your listeners, or you can force close it earlier using goOffline.
For Firestore, I don't know if it happens automatically, but you can do it manually.
In Firebase Pricing you can see that 100K Firestore document reads is $0.06. 1M reads (for the two words) should cost $0.6 plus some network traffic. In RTDB, the cost has to do with data bulk, so it requires some calculations, but it shouldn't be much. I am not familiar with the pricing small details, so you should do some more research.
In the app you mentioned, the experiences don't seem to change very often. You might want to try to build your own caching manually, and add the required versioning info in the daily data.
Edit:
It would possibly be more efficient and less costly if you used Firebase Hosting, instead of RTDB/Firestore directly. See Serve dynamic content and host microservices with Cloud Functions and Manage cache behavior.
In short, you create a HTTP function that reads your database and returns the data you need. You configure hosting to call that function, and configure the cache such that subsequent requests are served the cached result via hosting (without extra function invocations).
https://firebase.google.com/docs/firestore/manage-data/enable-offline
How does Firestore work with offline data?
How are writes merged by many clients editing the same data offline, that then come online at the same time?
How long is offline data persisted? If my user uses my app for 5 years offline, then comes back online, will this be an issue? Do offline changes persist after device restarts?
Does query performance of offline data degrade as the data set gets larger?
Im specifically interested in the web Firestore client.
Do all the language clients implement the above in the same manner?
Thanks.
How are writes merged by many clients editing the same data offline, that then come online at the same time?
The write operations that will take place on Firebase servers, will be in the order in which that series of operations happened. The last operation (the most recent one) will be the one that will be available in the database by the time the synchronization occurs.
How long is offline data persisted? If my user uses my app for 5 years offline, then comes back online, will this be an issue?
The problem is not about how long is about how many operations do you make while the device is offline. While offline, Firestore will keep in queue all the write operations. As this queue grows, local operations and app startup will slow down. Nothing major, but over time these may add up. The major problem in this case is that the end result of this will be that the data on the server stays unmodified. Then what is the purpose of a realtime database? Firestore is really designed as an online database that came work for short to intermediate periods of being disconnected, not to stay offline for 5 years. Beside that, in 5 years it might be a problem of compatibility and not of the number of writes.
Do offline changes persist after device restarts?
The offline persistence is also called disk persistence. This type of persistence is enabled by default in Cloud Firestore and it means that recently listened data (as well as any pending writes from the app to the database) are persisted to disk. The data in this cache survives app restarts and device reboots.
Does query performance of offline data degrade as the data set gets larger?
Yes it does, like explained above.
Do all the language clients implement the above in the same manner?
No. For iOS and Android, the offline feature works fine while for web, this feature is still experimental.
I have an application that needs to send data to a cloud database (DynamoDb).
The app runs on a computer that can lose internet connectivity or be switched off at any time, but I must ensure that all data eventually gets to the cloud database.
I can assume the application will eventually be switched on, and will eventually get internet access back.
The app is written in VB .NET
What are some schemes for achieving this, and are there any ready-made products that already achieve this?
You could implement a write-through cache using a local DynamoDB instance (or even using SQLite). But without getting specific details about what kind of data you'd be storing into the database, and what data should be made available "offline" it's hard to say exactly how you should structure your application. You'll definitely want to not keep everything local, unless the volume of data is really small overall.
Then there is the problem of resolving conflicts that may occur during network partitions (ie. a client goes offline and makes some database modifications, while other clients also make modifications to the database; these need to be reconciled and it's up to you, and your users to determine how)
It's not a simple problem to solve.
From Firebase's FAQ:
What happens to my app if I lose my network connection?
Firebase transparently reconnects to the Firebase servers as soon as
you regain connectivity. In the meantime, all Firebase operations done
locally by your app will immediately fire events (...). Once
connectivity is reestablished, you’ll receive the appropriate set of
events so that your client “catches up” with the current server state
Then what happens if I go offline and keep modifying my local data, then come back online and other clients have performed different changes? Which one will ultimately be saved?
If the data on the server gets overridden, does it mean older data can replace newer one?
If the newer data added online is kept, do I know that the data submitted while offline has been discarded?
When your client comes back online, after an offline period and writing data, the behavior of those changes will be determined by which method you used to write them:
The set(), setWithPriority(), remove(), and push() methods are last-write-wins. This means that if offline client A makes a change at t=0, and online client B makes a change at t=10, that offline client A's changes will overwrite client B's changes when upon reconnection. Note that this specifically applies to the changes that were made (i.e. set /a/b/c to 1), not the entire Firebase.
The transaction() method, however, is built specifically for handling conflicts. When offline client A reconnects, your transaction update function will re-run and apply the new change to your Firebase data.
In most applications, users are appending data to lists or modifying individual user state, but not modifying the same piece of data. In the event that multiple users are modifying the same piece of data, you'll want to use transaction() whether you're offline or not.
Generally speaking, Firebase has been built to handle going offline and online automatically and so you shouldn't have to write application code to detect and handle that case.