I would like to get all the data from my Realtime Database (200GB) to migrate it to a different database (PostgreSQL).
I considered:
creating a script with the Admin SDK and request all the entries, but this would be really slow and would make the database reach the peek (that means slow down all the database).
using the "Export JSON" function of the console but it is not possible because the size (200GB) is too large.
I just made a manual backup on the console and hopefully I'm going to receive a huge ZIP file that then I will need to manage with a script.
Any other smarter solution?
Related
I'm creating an android app which uses Firebase Firestore database to store data. I store about 3600 questions and about 400 images. I know that firestore support offline. But I'm not sure whether the database download itself or not every time I request data from it. And under which circumstance does the firestore database get downloaded from the cloud? Does the database get only updated when the cloud database change?
Every query performed while the app is online will download all the necessary data from the database to satisfy that query.
Any data cached locally as a result of a prior query is only used when the app is offline. As suggested by the documentation, the idea is for your app to be usable when internet connectivity is interrupted. The assumption is that the connectivity will eventually come back, and queries will revert to using the online database as the primary source of data.
Here is our use case:
We have way more than 200,000 clients need to connect to firebase realtime db. So we created multiple database with same data and load blance the connections.
Here is the problem:
If we update one database, we will have to initiate connection and udpate the rest of the database as well. I would like to check if there is a way to auto sync up data between multiple databases.
Docs I have went through:
https://firebase.google.com/docs/database/usage/limits
https://firebase.google.com/docs/database/usage/sharding
Also I checked rules, and it seems that rules is not meant to be used to sync data.
Thanks
firebaser here
There is nothing built into Firebase to automatically synchronize data between multiple database instances. A common way to implement this when writing through a server-side process, is to simply write to each database in turn there.
If the data you want to write comes from a client-side SDK, I'd have the client write it to a staging area (just a temporary node in the database), and then use Cloud Functions to write the data the permanent location in all database instances.
I am using Firestore NoSQL for iOS application. While debugging, I occasionally executed setData instead of updateData, what led to lost of all the user data (one user).
Is there a way to reverse back the changes?
How to do Versioning for Firestore, for similar cases, so there's a way to cancel/ backup. I don't quite know, but I've read that it supposed to be Versioning.
There is no option to undo changes in Firestore. I'd recommend creating a new database (project) for testing purposes so you don't accidentally write on the main database. For now if it's just a single user then I'd recommend manually add that back.
Unlike realtime database, you can only have one Firestore instance per project at the moment so you may have to create a new project but I think that'll be safe to prevent such accidental writes :)
"You can use the Cloud Firestore managed export and import service to recover from accidental deletion of data and to export data for offline processing."
That's what I've found for now: https://firebase.google.com/docs/firestore/manage-data/export-import
I'm thinking moving an Android app's persistent data to Firebase Database. Currently I use Sqlite with a Python HTTP REST service.
In the app, I have a big list of cities, called citiesList. Since the list is quite big and hardly updated, I don't want the app fetch the list whenever it goes online.
My current strategy is provide a citiesListVersion. When the app goes online, it checks citiesListVersion. If the server's citiesListVersion is newer, the app will fetch citiesList from server. Otherwise, the app continues working with the cached data.
My question is: can I keep my current strategy when moving to Firebase Database? As I understand, Firebase tries to sync data whenever the app goes online.
I'm new with Firebase technology and I would like to optimize Firebase database size (including for decrease cost).
What are the different ways to decrease Firebase database size?
Can I simply use node names as short as possible, for example instead of having a node "user", rename this node "u"? (relevant if this node is very present)
Do there are other tips?
Here's the approach we take from one of our mobile apps:
We have a mobile app, web service, Firebase Database and Firebase Storage. We sometimes have a small SQL database as well.
We have the mobile app display data from Firebase but write data to Firebase via the web service, never directly.
We started with using Firebase Database as our storage, then changed to a hybrid Firebase Database + Firebase Storage mix.
We now store the "view data" is Firebase Storage and only store a "stub/pointer" in the Firebase Database (it reduces data size and it reduces traffic).
We end doing an extra read from Firebase Storage every time the value of a "stub/pointer" changes in Firebase Database, but that works for our scenario. We also don't do it for every situation, so we peek and chose where it makes sense to use this approach.
We ended up reducing cost - that was our main reason to search for a solution and it looks like that's your motivation as well.
Other than that, using short names for the key names may help as well.