How can I download firestore collections to my local pc to make a process then upload it again to firestore.
I downloaded the collection from gcloud console but the collection did not open and the format is different than what is in firestore.
My guess is that you downloaded a backup of the data, which is not really in a format meant for editing.
I'd probably use the API to download the documents, store them in a format that works for you, and then reupload them through the API again.
Related
still new to flutter and firebase. I understand how to store and retrieve images and display it on the app.
But how do I go about synchronizing files from the app's local asset folder to an asset folder stored in firebase storage? My intention is to check the cloud folder if a new image like an icon is recently uploaded, and download it to the app's local folder. If a file is removed in the cloud storage, it should also remove it from the local assets folder, mirroring it.
I need a way to compare local AssetManifest.json to the one on firebase storage. I Just need a little direction/algorithm to start with. Thanks for the help!
There is nothing specific built into Cloud Storage's Firebase SDK for this, so you'll have to build it in your own application code.
Using only the Cloud Storage for Firebase API
If you're just using Cloud Storage, you'll have to:
List all files that you're interested in from Storage.
Loop over the files.
Get the metadata for each file and check then the files was last updated
Compare that to when you wrote the local file.
Download the file if it is new or modified.
This approach will work, but it requires quite some calls to the Storage API, because there's no specific API to give you files that were modified since a specific date/time.
Storing metadata in a cloud database
You could also consider storing the metadata in a cloud database, like Firebase's Realtime Database or Cloud Firestore, and then use the query capabilities of that database to retrieve only files that were modified since your device last synchronized.
The recipe then becomes:
Determine when we last synchronized, which is a value you'll want to store in local storage/shared preferences.
Execute a query to the database to determine which files were added/modified since then.
Loop over the query results and...
Download each file that was modified.
In here, only step 2 and 4 make calls to the external APIs, so it is likely to be faster and cheaper to execute (but more work for you to write initially).
I would like to sync all of the files in my Google Cloud Storage bucket with the exported files in my Firebase Storage Emulator.
I downloaded all of my cloud files using gsutil to my local machine.
I used BeyondCompare to move all of the new files to the '../storage_export/blobs/ directory.
How do I update/create the JSON metadata in '../storage_export/metadata' to reflect these new files and make them available when I run the emulator and import them in?
Edit:
The gsutil docs mention the following:
when you download data from the cloud, it ends up in a file with no associated metadata, unless you have some way to keep or re-create that metadata.
How would one "keep" or "re-create" that metadata during a gsutil cp download?
You can use gsutil or the SDK to get each object's metadata and then write it down to a JSON file however, there's currently no native way to import Google Cloud Storage data in the Storage Emulator. But as I stated in my answer to this post, you can study how the emulator register the object by uploading sample files within the emulator and then running the export, you will see that the emulator will require 1 object and 1 JSON file that contains it's metadata.
Lastly, you can add the option --export-on-exit when starting the emulator, Downloaded all data from the real Firebase project, uploaded everything with the Emulator, then kill the emulator; as stated in this post.
Note: This is not a documented feature! Firebase doesn't expose the concept of download tokens in its public SDKs or APIs, so manipulating tokens this way feels a bit "hacky". For your further reference, check this post.
I'm using the Firebase Emulator to run all Firebase services. I have managed to run the emulator with a backup of my Firestore data by running:
firebase emulators:start --import ./my-directory
... but I can't find a way to do the same with my Storage data.
Firestore has an option to do import and export, while Firebase storage don't have this feature yet (storage uses upload and download). Currently, there's no native way to import Google Cloud Storage data in the Storage Emulator.
Additionally, you can study how the emulator register the object by uploading sample files within the emulator and then running the export, you will see that the emulator will require 1 object and 1 JSON file that contains it's metadata. For now, it'll be up to you to download the objects from your production bucket along with a separate JSON file containing its metadata and then structure it for import.
There's also an opened issue for this in github that you can monitor as well.
Although #RJC's answer is correct, I want to share what I did.
You can add the option --export-on-exit when starting the emulator, and it will export the state of the local Firebase instance to a folder of your choice. So I downloaded all my data from the real Firebase project, uploaded everything with the Emulator, then killed the emulator, everything (including storage) got exported and now I start it with the --import option using the same folder where I exported everything.
This was not very awful since I didn't that much stuff in my storage.
I'm very new to using Firebase Cloud Storage. I'm currently using it to store data generated by some in-house tablet apps. We want users to be able to easily access this data in the form of Dropbox, however don't want to integrate the Dropbox API into our tablet applications as we're already using Firebase for all login management, etc and don't want users to have to login twice.
Is it possible to setup of a mirror between Firebase Cloud Storage and a Dropbox account such that any file added to Firebase Cloud Storage is immediately copied to the Dropbox directory?
You can simply use cloud functions which are being triggered by cloud storage service (it will triggered on any uploading, updating, deleting files or folders) on your firebase project and then write your cloud functions in a way to use dropbox api and then make the same change in the dropbox directory structure.
Cost and performance wise I don't think it would an efficient thing to do, unless it is what really you want.
I have build an app using ionic framework, and the data uses sqlite database and stores in app /documents folder.
Now I want to know is it possible to sync the database into iCloud, so the user can retrieve the data automatically if they reinstall the app or use a new device.
I did search for some solution, some suggest sync to remote database using such as pouchDB or parse. But as for our company policy, we cannot store app data into any other remote place exclude iCloud.
Any ideas? Thank you so much.
As far as I know, iCloud automatically backs up data on the user's local storage. You don't need to do anything extra.
I might be wrong, but soome people seem to have problems with it doing that
Check out this page
Relevant:
BackupWebStorage(string)
Default: cloud
Allowed values: none, local, cloud.
Set to cloud to allow web storage data to backup via iCloud. Set to local to allow only local backups via iTunes sync. Set to none prevent web storage backups.