It's been a little while since we were warned of the migration from the Datastore to Firestore (in Datastore mode).
When we first created our app we didn't have access to regions (early adopters) - and I was hoping we would have the opportunity to change the region of the app when the migration happens. Does anyone know if we can choose the region of the servers before the migration begins? Or should we just bite the bullet and migrate manually?
Thanks in advance,
Guy
The location of your Firestore should be the same as your Project's location.
The documentation says the following:
The location setting for your Google Cloud project applies to both
Firestore in Datastore mode and App Engine. When you select a location
in either product, you set the location for your entire Google Cloud
project.
Important: After you select the location of your project, you cannot change it.
You can verify your project's location by running gcloud app describe.
If you can't see your project's location because your app is used since the time there was no access to regions, I'd double-check that with the GCP Support.
The migration to Cloud Firestore in Datastore mode will not change the location of your database.
As of today (2021-06-07) if you want to change the location of your Cloud Datastore or Cloud Firestore database, you'll need to do so manually.
Related
I am using firebase CLI to initialize a local firebase project directory from an existing firebase project. This existing project does have firestore with 100% certainty (there is tonnes of data in it). It is definitely firestore and not real time db.
I am running..
firebase init firestore
Then I pick the existing project from the list, and I get...
Error: It looks like you haven't used Cloud Firestore in this project before. Go to https://console.firebase.google.com/project/some-project/firestore to create your Cloud Firestore database.
Is there potentially a permission issue here?
I came here looking for the answer to this question and whilst your issue may be different, I found that I had no default resource location set for my firebase project.
In the firebase console go to cog icon > project settings > general. the fourth option down is 'Default GCP resource location'. This was not set, so editing it (pencil icon) and clicking done (There was only one option for me) fixed the issue.
Discussion around this topic can be found here: https://github.com/firebase/firebase-tools/issues/4254#issuecomment-1310619259
I am trying to create firestore database but got the message "This project is set up to use Cloud Firestore in Datastore mode. This mode can only be accessed from Google Cloud Platform".
This is setup using GCP. How can I do that? Also not able to switch to native mode. Please help me.
This message indicates that your database was created in Datastore mode, when you created your project. You should be able to change it by accessing the Datastore Console using this link here and clicking in the available button to switch the mode - check here. However, as clarified in this other post from the Community here, you can only do it once and if you have not performed any changes in your database.
In case you don't fit in these above scenarios, you will need to create a new project and configure Firestore from the beginning.
Let me know if the information helped you!
I face the same problem on creating a fresh project.
I think it's default behavior with latest changes.
Just go to Datastore and switch to NATIVE MODE.
Go over to Firestore Database
Click on the link provided
Change to Native Mode
I have a Firebase project which basically have two environments: Staging and Production. The way I organized them is by creating different Firebase projects. Each of my projects uses Firebase Cloud Functions and Firestore. Except for that, I have each of the projects associated with a specific GIT branch. Both of the branches are integrated into CI/CD pipeline in Google Cloud Build.
So, in order to make it absolutely clear, I will share a simple diagram:
As you can see, I have the source code for the cloud functions under source control and there's nothing to worry about there. The issue comes in when I have the following situation:
A Firestore schema change is present on Staging
Cloud function (on Staging) is adjusted to the new schema.
Merge staging branch into production.
Due to the old Firestore schema on production, the new functions there won't work as expected.
In order to work around it, I need to manually go to the production Firestore instance and adjust the schema there (there's a risk to mess up production data).
In the perfect case, I would have that operation automated and existing project data would be adjusted to a new schema which comes in dynamically after merge.
Is that possible somehow? Something like migrations in .NET Core.
Cloud Firestore is schema-less - documents have no enforced schema. Code is able to write whatever fields it wants at any time that it wants. (For web and mobile clients, this is gated by security rules, but for backend code, there are no restrictions.) As such, there is no such thing as a formal migration in Cloud Firestore.
The "schema" of your documents is effectively defined by your code that reads and writes those documents. This means that migrating a data to a new format means that you're going to have to write code to perform the required changes. There is really no easy way around this. All you can really do is design your updates so that they are not disruptive to existing code when it comes time to move them to another environment. This means your code should be resilient to breaking changes, or simply do not perform breaking changes until after all code has been updated to deal with those changes.
You have to use Google Cloud to download an archive of the Firestore data. Run a migration script yourself on the archive, and then upload the archive to restore your Firestore database.
https://cloud.google.com/firestore/docs/manage-data/export-import
Google Cloud gives you a lot of command line access for managing your Firestore service.
// manage indexes
gcloud firestore indexes
// export all data to a bucket
gcloud firestore export gs://[BUCKET_NAME]
// import data from a bucket
gcloud firestore import gs://[BUCKET_NAME]/[filename]
// manage admin "functions" currently running (i.e. kill long processes)
gcloud firestore operations
To download/upload your JSON archive from Google Cloud buckets
// list files in a bucket
gsutil ls gs://[BUCKET_NAME]
// download
gsutil cp gs://[BUCKET_NAME]/[filename] .
// upload
gsutil cp [filename] gs://[BUCKET_NAME]/[filename]
Once you setup Google Cloud to be accessible from your build scripts. It's possible to automate data migration scripts to download, transform and then upload data.
It's recommended to maintain a "migrations" document in your Firestore so you can track which reversion of the migration needs to be done.
To avoid heavy migration tasks try adding a "version" property to documents, and then use the converter callbacks on the query builder to mutate data to the latest schema on "client side". While this won't help you handle changes with Firestore rules or functions. It's often easier to make tiny changes that are mostly cosmetic.
Context, I have a project with datastore which already has information loaded, currently we wanted to use cloud firestore (native mode), but we realized that migration is not possible, what alternatives do I have to use cloud firestore (native mode)?
Update June 16, 2021:
You can now do gcloud datastore export in your first project, followed by gcloud firestore import in your new project. The longer more involved migration below is no longer needed.
Just keep in mind that the Datastore export goes to a Cloud Storage bucket. Make sure that the account running the Firestore import has access to that bucket.
Original answer from 2019
I just migrated from Datastore to Firestore (native mode) for one of my web apps. Here is what I needed to do:
Create a new GCP project, as Firestore (native mode) and Datastore
can't co-exist in the same project.
Migrate the data from Datastore in my old project to Firestore (native mode) in my new project. As of this writing, there are no tools to do that in an automatic way. I wrote Python scripts that read all records from Datastore and wrote them to Firestore in the new project. These scripts ran locally on my machine, using service account keys downloaded from the Cloud Console.
(Side note: You might be tempted to use gcloud datastore export followed by gcloud firestore import. It seems to work and no error messages pop up when you do. But doc IDs and JSON properties don't translate well. This was a big time-sink for me. Don't go down this road.)
Rewrite the data access layer in your app. Firestore (native mode) has a different API than Datastore.
This was a fair amount of work, but it was worth it in my case:
I was able to retire a lot of server-side code because the clients can access the database directly.
I was able to retire a lot of client-side code for supporting offline mode because the Firestore client library implements it already.
Best of luck!
Unfortunately, you'll need to create your Cloud Firestore database in a new project that allow your existing service accounts to access that new database.
I have my android app running firebase cloud message using uat#gmail.com. and now, i need to change the administration to production account bbb#mycompany.com
what i have done, using bbb#mycompany.com to create new project and generate a new google-services.json and replace existing google-services.json i created with aaa#gmail.com
using android studio connect to firebase and show
Firebase: A config file "D:\xxx\xxx\google-services.json" was found, but it indicates a project ID "xxx-1vv" that you dont appear to have access to. Request access or delete the file in order to proceed with the connection process.
anyone could help??
You can transfer ownership of a Cloud (Firebase) product without having to change the way your users user the project. Please read this for details. Note that all Firebase projects are also Cloud projects, so anything you do in the Cloud console applies to Firebase.