I have a single page application which I want to host in different gcp projects, i.e. different firebase projects.
The deployment should be done continuously to my firebase hosting, I want to seperate Continuous Build and Continuous Deployment steps and therefore I don't want to deploy from my local machine to all my firebase projects manually.
My question is:
Is there any best practice around for deploying to a firebase hosting project when I don't want to deploy from my local machine?
In my naive way, I am going to setup the following:
I need to build a zip file that contains all the static html and js files plus my firebase.json and .firebaserc file which I upload to my storage bucket.
With those files the firebase CLI tool is able to execute firebase deploy --only hosting:mytarget. A google cloud builder could run this command on a git push (from github.com for example) that knows about all my gcp projects I want to deploy the single page app to...
However this is a lot of work and maybe there is a nicer solution for this.
I tried to put a gs:// link directly to the firebase.json hosting/public section, however firebase CLI does not recognize this (it needs a local folder) :-(
{
"hosting": [
{
"target": "mySPAApp",
"public" : "gs://mystorage-containing-files-for-hosting/",
"rewrites": [
{
"source": "apps/myapp/**",
"destination": "myapp/index.html"
}]
}
]
}
My guts feelings say that there must be an easier way, maybe even with gcloud CLI that just deploys a bunch of files to firebase hosting.
Internally in firebase the functionality should be implemented similarly to this, because in the firebase console the user can simply rollback to a previous version of the hosted application, however there seems to be no documentation about this.
Ok, I found out that there is an official REST API for firebase hosting which is a bit cumbersome to use.
https://firebase.google.com/docs/hosting/reference/rest/v1beta1/sites.versions/create
The format of the configuration is different to what is provided normally in firestore.json and the files needed to be uploaded in a state-machine manner in a Version before a Release is created.
Still, not a solution to opt for in my opinion.
Related
The problem
I have configured a project that currently is possible to deploy to Firebase Hosting multi sites by using the Firebase CLI.
The problem is that currently I am doing this manually. And I want to programatically do this, and if possible, using NodeJS.
The steps of my system:
User uploads a .zip file with a index.html
I extract and check the .zip file with NodeJS
I deploy the files to Firebase Hosting
Currently I am doing like this:
First create the site page with, where game-project-id-unique should be unique:
firebase hosting:sites:create game-project-id-unique
Then, add the target/project ID linked to the game, where game-id is defined by us:
firebase target:apply hosting game-id game-project-id-unique
Add to the firebase.json the configuration of the file (array):
{
"target": "game-id",
"public": "games/game-id",
"ignore": ["firebase.json", "**/.*", "**/node_modules/**"]
}
Then, to publish:
firebase deploy --only hosting
Possible solutions?
I have researched here, I have found out that is possible to use the API-DEPLOY or the CLOUD BUILD, but this is not enough to guide me into the programatically approach, without the need of a human in-between.
Is there a way of doing this? Is it even possible?
You can certainly just write a script that invokes the CLI. This is very common.
You can also use the Firebase Hosting REST API, but in my opinion that is going to be a lot more work.
I've given the service account for the functions the necessary permissions ('Secret Manager Secret Accessor') and when deployed, the firebase functions are able to access the secrets without any problems.
However, when using firebase serve or firebase emulators:start --only functions in local development, I'm getting the following error
Unhandled error Error: 7 PERMISSION_DENIED: Permission 'secretmanager.versions.access' denied for resource
I've found in the documentation that setting export GOOGLE_APPLICATION_CREDENTIALS=pathtoserviceaccount.json is needed to be entered in the terminal, though this did also not work for me.
I would be thankful for all pointers. Cheers.
I've found the answer myself:
When the functions are emulated locally, they do not get run by the App Engine default service account per default, this needs to be enabled as well.
So I had to follow this tutorial https://firebase.google.com/docs/functions/local-shell
The App Engine default service account needs a key which can be created in the Service Accounts settings in the Google Cloud, and then
I had to enter
export GOOGLE_APPLICATION_CREDENTIALS="path/to/key.json"
in the terminal. By running then firebase emulators:start they also got permission to access the Secret Manager.
So while I was on the right track, I was exporting the wrong Service Account key, and not the one that was allowed to run access the Secret Manager.
In order to access Secret Manager from your Firebase application running with local emulator you need to add role of:
"Secret Manager Secret Accessor" to YOUR account used to authenticate with Firebase
You can verify it by running: firebase login in local CLI.
If you're already logged in, it should respond with Already logged in as [email address].
This email address is the Principal account you need to add the role to.
As you've mentioned in your question the "firebase-adminsdk" service account permissions are used on the production deployment, but not on local, unless you specify it with: export GOOGLE_APPLICATION_CREDENTIALS="path/to/key.json"
"[...] you can override secrets values by setting up a .secret.local file. This makes it easy for you to test your functions locally, especially if you don't have access to the secret value."
https://firebase.google.com/docs/functions/config-env#secrets_and_credentials_in_the_emulator
Step by step:
make sure you have the latest version of firebase-tools installed, as this feature is relatively new.
Create a file named secret.local in the root of your firebase project (along side the .firebaserc and firebase.json
add your secrets to the file, formatted the same as way as a regular .env file. e.g.
MY_SECRET_1=foo
MY_SECRET_2=bar
run the emulator firebase emulators:start
within your firebase functions, access the secrets on the process object. e.g. process.env.MY_SECRET_1
note, as far as I can tell, the secrets are only available inside the block scope of a function handler. you can't access them in the root scope of your functions JS code (if somebody finds a way to do that, please comment here as I'd love to know too)
I had the same problem and tried the solution from pureth's answer to adding local overrides, but it didn't work. What did work for me was to create the .secret.local file in the functions directory, not in the project root.
My project structure is as follows:
/
|- .firebaserc
|- firebase.json
|- package.json
|- /* ... */
|- functions/
|- .secret.local
|- package.json
|- /* ... */
So the .secret.local file needs to be placed in the directory where your functions reside and not where the .firebaserc file is.
Also, please note that the file name starts with a dot.
I have two projects for dev and prod. I want to be able to run a script to copy dev config to prod.
Firebase Remote Config has an API for programatically updating Remote Config. But as far as I can tell, you need to init admin with a project-specific service account. It seems like I would need two admin instances, but I'm not sure that's possible?
I'm wondering if someone has done this before and has an example script. Thanks!
See docs:
https://firebase.google.com/docs/remote-config/automate-rc
There is no Firebase Admin SDK for Flutter, so you'll have to implement this on a different platform that is supported. For a list of these platforms and instructions on setting it up, see the documentation on adding Firebase to a server.
For these platforms that the Firebase Admin SDK targets, you can create multiple instances of the FirebaseApp class, and initialize each of them with different credentials and project configuration. For examples of how to do this, see the documentation on initializing multiple apps.
I am building a web application with Firebase. Currently I can say that I do have two stages - development, the firebase serve which runs the localhost and firebase deploy --only hosting which uploads the web application on Firebase hosting.
Everything is fine with that, but I do not see this as a professional solution. The problem that I see is that, my local environment and the live web application share the same database. I did quite some research on the topic and I understood that there is no way to have two databases per one project on Firebase. The solution that is offered out there, is to create two projects on Firebase, one for development and one for production. Or even if you want to, one for staging.
This solution seemed completely fine with me. It's a good idea for sure. Couple of projects, for couple of environments, separate databases, just perfect. Then just before implementing this solution another problem bumped in my head. If I say, let's create a staging project, in order to serve me as a staging environment, and I decide to deploy my web application, the staging web application will be publicly available, so it will also get indexed by Google and so on.
So, what could you advice me in this situation? How can I make sure that my staging web application (hosted on the staging Firebase app) will not be available for others and will not be indexed by search engines. I thought about white-listing IPs or VPC, but I have no clue how to proceed in a way that is free and reliable.
In case anyone has this question, there's an article on the Firebase Blog about this.
Note: This Firebase article assumes that you have already created a second Firebase project for this new environment (i.e. project-dev), and have copied the config details into your working env (i.e. project-dev). Master and dev are two different env, so it makes sense to have two different Firebase configs.
The article states:
Fortunately for us, the Firebase CLI makes it simple to setup and deploy to multiple environments.
Adding and switching between environments with the Firebase CLI is as
simple as one command: firebase use.
$ firebase use --add
This command prompts you to choose from one of your existing projects
Select the project you want to use for a different environment, and then give it an alias. The alias can really be whatever you want, but it’s common to use aliases like “development”, “staging”, or “production”.
Once you’ve created a new alias, it will be set as the current
environment for deployment. Running firebase deploy will deploy your
app to that environment.
Switching environments
If you want to switch to another environment, just provide the alias in the use command.
$ firebase use default # sets environment to the default alias
$ firebase use staging # sets environment to the staging alias
For a single command, you can also specify the environment using the -P flag:
$ firebase deploy -P staging # deploy to staging alias
Hope that helps!
Edit: The following solution is for Firebase "Realtime Database". It does not apply to "Firestore". Read the difference here.
1. Firebase Realtime Databases Sharding
Now (2018 March), Firebase Realtime Database allows you to create multiple instance.
Official Document: Scale with Multiple Databases
Go to your Firebase Project
In the Firebase console, go to the Data tab in the Develop > Database section.
Select Create new database from the menu in the Databases section (upper right corner).
Customize your Database reference and Security rules, then click Got it.
(Optional) Modify the Security rule and Backup option of the new instance.
2. Usage
// Get the default database instance for an app
var database = firebase.database();
// Get a secondary database instance by URL
var database = firebase.database('https://testapp-1234.firebaseio.com');
3. Example Usage: Different Environment
firebase-config.js
const BUILD_LEVEL = "dev";
// const BUILD_LEVEL = 'stage'
// const BUILD_LEVEL = 'prod'
let config = {
apiKey: "your_apiKey",
authDomain: "your_authDomain",
projectId: "your_projectId",
storageBucket: "your_storageBucket",
messagingSenderId: "your_messagingSenderId"
};
if (BUILD_LEVEL === "dev") {
config.databaseURL = "https://your-project-dev.firebaseio.com/";
} else if (BUILD_LEVEL === "stage") {
config.databaseURL = "https://your-project-stage.firebaseio.com";
} else if (BUILD_LEVEL === "prod") {
config.databaseURL = "https://your-project-dev.firebaseio.com";
}
firebase.initializeApp(config);
Now to change the Firebase Database instance, you only need to change the BUILD_LEVEL variable.
Combine this feature with Git/Github/Gitlab workflow, Git hook, webpack, CI/CD tool, and you have a very flexible solution.
I am considering using Firebase as MBaaS, however I couldn't find any reliable solution to the following problem:
I would like to set up two separate Firebase environments, one for development and one for production, but I don't want to do a manual copy of features (eg. remote configuration setup, notification rules, etc.) between the development and production environment.
Is there any tool or method I can rely on? Setting up remote configuration or notification rules from scratch can be a daunting task and too risky.
Any suggestions? Is there a better approach than having two separate environments?
Before you post another answer to the question which explains how to set up separate Firebase accounts: it is not the question, read it again. The question is: how to TRANSFER changes between separate dev and prod accounts or any better solution than manually copy between them.
If you are using firebase-tools there is a command firebase use which lets you set up which project you are using for firebase deploy
firebase use --add will bring up a list of your projects, select one and it will ask you for an alias. From there you can firebase use alias and firebase deploy will push to that project.
In my personal use, I have my-app and my-app-dev as projects in the Firebase console.
As everyone has pointed out - you need more than one project/database.
But to answer your question regarding the need to be able to copy settings/data etc from development to production. I had the exact same need. A few months in development and testing, I didn't want to manually copy the data.
My result was to backup the data to a storage bucket, and then restore it from there into the other database. It's a pretty crude way to do it - and I did a whole database backup/restore - but you might be able to look in that direction for a more controlled way. I haven't used it - it's very new - but this might be a solution: NPM Module firestore-export-import
Edit: Firestore backup/export/import info here Cloud Firestore Exporting and Importing Data
If you're using Firebase RTDB, and not Firestore - this documentation might help:
Firebase Automated Backups
You will need to set the permissions correctly to allow your production database access to the same storage bucket as your development.
Good luck.
I'm not currently using Firebase, but considering it like yourself. Looks like the way to go is to create a completely separate project on the console. There was a blogpost up recommending this on the old Firebase site, looks to be removed now though. https://web.archive.org/web/20160310115701/https://www.firebase.com/blog/2015-10-29-managing-development-environments.html
Also this discussion recommending same:
https://groups.google.com/forum/#!msg/firebase-talk/L7ajIJoHPcA/7dsNUTDlyRYJ
The way I did it:
I had 2 projects on firebase- one for DEV other for PROD
Locally my app also had 2 branches - one named DEV, the other named PROD
In my DEV branch I always have JSON file of DEV firebase project & likewise for PROD
This way I am not required to maintain my JSONs.
You will need to manage different build types
Follow this
First, create a new project at Firebase console, name id as YOURAPPNAME-DEV
Click "Add android app" button and create a new app. Name it com.yourapp.debug, for example. New google-services.json file will
be downloaded automatically
Under your project src directory create new directory with name "debug" and copy new google-services.json file here
In your module level build.gradle add this
debug {
applicationIdSuffix ".debug"
}
Now when you build a debug build google-services.json from "debug" folder will be used and when you will build in release mode google-services.json from module root directory will be considered.
I'm updating this answer based on information I just found.
Step 1
In firebase.google.com, create your multiple environments (i.e.; dev, staging, prod)
mysite-dev
mysite-staging
mysite-prod
Step 2
a. Move to the directly you want to be your default (i.e.; dev)
b. Run firebase deploy
c. Once deployed, run firebase use --add
d. An option will come up to select from the different projects you currently have.
Scroll to the project you want to add: mysite-staging, and select it.
e. You'll then be asked for an alias for that project. Enter staging.
Run items a-e again for prod and dev, so that each environment will have an alias
Know which environment you're in
Run firebase use
default (mysite-dev)
* dev (mysite-dev)
staging (mysite-staging)
prod (mysite-dev)
(one of the environments will have an asterisk to the left of it. That's the one you're currently in. It will also be highlighted in blue)
Switch between environments
Run firebase use staging or firebase use prod to move between them.
Once you're in the environment you want, run firebase deploy and your project will deploy there.
Here's a couple helpful links...
CLI Reference
Deploying to multiple environments
Hope this helps.
We chose to fire up instances of the new Firebase emulator on a local dev server for Test and UAT, leaving GCP out of the picture altogether. It's designed exactly for this use-case.
https://firebase.google.com/docs/emulator-suite
This blogpost describes a very simple approach with a debug and release build type.
In a nutshell:
Create a new App on Firebase for each build type using different application id suffix.
Configure your Android project with the latest JSON file.
Using applicationIdSuffix, change the Application Id to match the different Apps on Firebase depending on the build type.
=> see the blogpost for a detailed description.
If you want to use different build flavors, read this extensive blogpost from the official firebase blog. It contains a lot of valuable information.
Hope that helps!
To solve this for my situation I created three Firebase projects, each with the same Android project (i.e. same applicationId without using the applicationIdSuffix suggested by others). This resulted in three google-services.json files which I stored in my Continuous Integration (CI) server as custom environment variables. For each stage of the build (dev/staging/prod), I used the corresponding google-services.json file.
For the Firebase project associated with dev, in its Android project, I added the debug SHA certificate fingerprint. But for staging and prod I just have CI sign the APK.
Here is a stripped-down .gitlab-ci.yml that worked for this setup:
# This is a Gitlab Continuous Integration (CI) Pipeline definition
# Environment variables:
# - variables prefixed CI_ are Gitlab predefined environment variables (https://docs.gitlab.com/ee/ci/variables/predefined_variables.html)
# - variables prefixed GNDR_CI are Gitlab custom environment variables (https://docs.gitlab.com/ee/ci/variables/#creating-a-custom-environment-variable)
#
# We have three Firebase projects (dev, staging, prod) where the same package name is used across all of them but the
# debug signing certificate is only provided for the dev one (later if there are other developers, they can have their
# own Firebase project that's equivalent to the dev one). The staging and prod Firebase projects use real certificate
# signing so we don't need to enter a Debug signing certificate for them. We don't check the google-services.json into
# the repository. Instead it's provided at build time either on the developer's machine or by the Gitlab CI server
# which injects it via custom environment variables. That way the google-services.json can reside in the default
# location, the projects's app directory. The .gitlab-ci.yml is configured to copy the dev, staging, and prod equivalents
# of the google-servies.json file into that default location.
#
# References:
# https://firebase.googleblog.com/2016/08/organizing-your-firebase-enabled-android-app-builds.html
# https://stackoverflow.com/questions/57129588/how-to-setup-firebase-for-multi-stage-release
stages:
- stg_build_dev
- stg_build_staging
- stg_build_prod
jb_build_dev:
stage: stg_build_dev
image: jangrewe/gitlab-ci-android
cache:
key: ${CI_PROJECT_ID}-android
paths:
- .gradle/
script:
- cp ${GNDR_CI_GOOGLE_SERVICES_JSON_DEV_FILE} app/google-services.json
- ./gradlew :app:assembleDebug
artifacts:
paths:
- app/build/outputs/apk/
jb_build_staging:
stage: stg_build_staging
image: jangrewe/gitlab-ci-android
cache:
key: ${CI_PROJECT_ID}-android
paths:
- .gradle/
dependencies: []
script:
- cp ${GNDR_CI_GOOGLE_SERVICES_JSON_STAGING_FILE} app/google-services.json
- ./gradlew :app:assembleDebug
artifacts:
paths:
- app/build/outputs/apk/
jb_build_prod:
stage: stg_build_prod
image: jangrewe/gitlab-ci-android
cache:
key: ${CI_PROJECT_ID}-android
paths:
- .gradle/
dependencies: []
script:
- cp ${GNDR_CI_GOOGLE_SERVICES_JSON_PROD_FILE} app/google-services.json
# GNDR_CI_KEYSTORE_FILE_BASE64_ENCODED created on Mac via:
# base64 --input ~/Desktop/gendr.keystore --output ~/Desktop/keystore_base64_encoded.txt
# Then the contents of keystore_base64_encoded.txt were copied and pasted as a Gitlab custom environment variable
# For more info see http://android.jlelse.eu/android-gitlab-ci-cd-sign-deploy-3ad66a8f24bf
- cat ${GNDR_CI_KEYSTORE_FILE_BASE64_ENCODED} | base64 --decode > gendr.keystore
- ./gradlew :app:assembleRelease
-Pandroid.injected.signing.store.file=$(pwd)/gendr.keystore
-Pandroid.injected.signing.store.password=${GNDR_CI_KEYSTORE_PASSWORD}
-Pandroid.injected.signing.key.alias=${GNDR_CI_KEY_ALIAS}
-Pandroid.injected.signing.key.password=${GNDR_CI_KEY_PASSWORD}
artifacts:
paths:
- app/build/outputs/apk/
I'm happy with this solution because it doesn't rely on build.gradle tricks which I believe are too opaque and thus hard to maintain. For example, when I tried the approaches using applicationIdSuffix and different buildTypes I found that I couldn't get instrumented tests to run or even compile when I tried to switch build types using testBuildType. Android seemed to give special properties to the debug buildType which I couldn't inspect to understand.
Virtuously, CI scrips though are quite transparent and easy to maintain, in my experience. Indeed, the approach I've described worked: When I ran each of the APKs generated by CI on an emulator, the Firebase console's "Run your app to verify installation" step went from
Checking if the app has communicated with our servers. You may need to uninstall and reinstall your app.
to:
Congratulations, you've successfully added Firebase to your app!
for all three apps as I started them one by one in an emulator.
Firebase has a page on this which goes through how to set it up for dev and prod
https://firebase.google.com/docs/functions/config-env
Set environment configuration for your project To store environment
data, you can use the firebase functions:config:set command in the
Firebase CLI. Each key can be namespaced using periods to group
related configuration together. Keep in mind that only lowercase
characters are accepted in keys; uppercase characters are not allowed.
For instance, to store the Client ID and API key for "Some Service",
you might run:
firebase functions:config:set someservice.key="THE API KEY" someservice.id="THE CLIENT ID"
Retrieve current environment configuration To inspect what's currently
stored in environment config for your project, you can use firebase
functions:config:get. It will output JSON something like this:
{
"someservice": {
"key":"THE API KEY",
"id":"THE CLIENT ID"
}
}
Create the Tow project with Dev and production Environment on the firebase
Download the json file from thre
and setup the SDK as per : https://firebase.google.com/docs/android/setup Or for Crashlytics: https://firebase.google.com/docs/crashlytics/get-started?platform=android
First, place the respective google_services.json for each buildType in the following locations:
app/src/debug/google_services.json
app/src/test/google_services.json
app/google_services.json
Note: Root app/google_services.json This file should be there according to the build variants copy the json code in the root json file
Now, let’s whip up some gradle tasks in your: app’s build.gradle to automate moving the appropriate google_services.json to app/google_services.json
copy this in the app/Gradle file
task switchToDebug(type: Copy) {
description = 'Switches to DEBUG google-services.json'
from "src/debug"
include "google-services.json"
into "."
}
task switchToRelease(type: Copy) {
description = 'Switches to RELEASE google-services.json'
from "src/release"
include "google-services.json"
into "."
}
Great — but having to manually run these tasks before you build your app is cumbersome. We would want the appropriate copy task above run sometime before: assembleDebug or :assembleRelease is run. Let’s see what happens when :assembleRelease is run: copy this one in the /gradlew file
Zaks-MBP:my_awesome_application zak$ ./gradlew assembleRelease
Parallel execution is an incubating feature.
.... (other tasks)
:app:processReleaseGoogleServices
....
:app:assembleRelease
Notice the :app:processReleaseGoogleServices task. This task is responsible for processing the root google_services.json file. We want the correct google_services.json to be processed, so we must run our copy task immediately beforehand.
Add this to your build.gradle. Note the afterEvaluate enclosing.
copy this in the app/Gradle file
afterEvaluate {
processDebugGoogleServices.dependsOn switchToDebug
processReleaseGoogleServices.dependsOn switchToRelease
}
Now, anytime :app:processReleaseGoogleServices is called, our newly defined :app:switchToRelease will be called beforehand. Same logic for the debug buildType. You can run :app:assembleRelease and the release version google_services.json will be automatically copied to your app module’s root folder.
The way we are doing it is by creating different json key files for different environments. We have used service account feature as recommended by google and have one development file and another for production