I'm developing a backend microservice app with node.js along with #google-cloud/firestore and i'm trying to access my firebase project locally with gcloud-sdk. I've run gcloud init so i can log in and chose the project i need to connect to in order to access the db.
The problem is that, i keep getting 7 PERMISSION_DENIED: Missing or insufficient permissions when i try to make any type of interaction with the db even though i already have all permissions in the project. At the beginning i thought that maybe i was having that problem because i was using a different account that did not have permissions to access these projects, but that didn't make sense because i do not see that account in my list of accounts in the sdk
But, when i gave permissions to the other account that i thought maybe logged in in my sdk, i could access the db, so it was weird, it is like it's stuck with the other account that maybe i added when i first install the sdk.
So, i've tried everything to correct this, i've deleted all accounts from my gcloud sdk, i've deleted the list of configurations, i've uninstalled (twice) the sdk and installed it again (since people that i work with told me that they did this and worked for them, since they had same issue), i've run g cloud init to log in again and all that stuff, and still, when i try to connect to my firebase db, it still says that i still do not have permissions, it's like the account that i'm logging in is not the one that is being saved/used to access my project.
What can i do to make this work ?
EDIT #1 -> How i'm connecting to firebase and sdk commands to connect to the project
Connecting to my project on firestore
import 'reflect-metadata';
import { Firestore } from '#google-cloud/firestore';
import { GCP_PROJECT } from '#util';
export const firestore = new Firestore({ projectId: GCP_PROJECT });
Commands to connect to my project by sdk
-> gcloud init
Welcome! This command will take you through the configuration of gcloud.
Settings from your current configuration [coordinadora-work] are:
core:
account: diego.cifuentes#coordinadora.com
disable_usage_reporting: 'True'
project: cm-reparto-dev
Pick configuration to use:
[1] Re-initialize this configuration [coordinadora-work] with new settings
[2] Create a new configuration
Please enter your numeric choice: 1
Your current configuration has been set to: [coordinadora-work]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).
Choose the account you would like to use to perform operations for this configuration:
[1] diego.cifuentes#coordinadora.com
[2] Log in with a new account
-> Please enter your numeric choice: 1
You are logged in as: [...my account that is having problems...].
Pick cloud project to use:
...
-> Please enter numeric choice or text value (must exactly match list item): 21
Your current project has been set to: [...project that i'm having problems with...].
-> Do you want to configure a default Compute Region and Zone? (Y/n)? n
Your Google Cloud SDK is configured and ready to use!
The Google Cloud Client libraries use the Application Default Credentials, not the current credentials setup using gcloud auth login.
To setup your Application Default Credentials for local development, simple execute gcloud auth application-default login. This will open a browser window and allow you to select the account to use as default credentials.
Another method is setting the environment variable GOOGLE_APPLICATION_CREDENTIALS to point to a service account key file.
Read more here : https://cloud.google.com/docs/authentication/application-default-credentials
When running in GCP, your credentials will get picked up from the environment, i.e. the service account your Cloud Function is running with.
https://cloud.google.com/docs/authentication/client-libraries
Related
I have GCP project A and Firebase project B (in a separate GCP project). I'm trying to use Cloud Build in A to build a web app and deploy it to Firebase Hosting in B.
In B's IAM page, I have granted A's <id>#cloudbuild.gserviceaccount.com service account the API Keys Admin, Firebase Admin, and Service Account User roles as described in e.g. this question.
The final step in the Cloud Build config used by A is the following:
- id: firebase_deploy
name: gcr.io/$PROJECT_ID/firebase
entrypoint: sh
args:
- '-c'
- |
firebase use $_FIREBASE_PROJECT_ID
firebase target:apply hosting prod $_FIREBASE_HOSTING_TARGET
firebase deploy --project=$_FIREBASE_PROJECT_ID --only=hosting,firestore:rules
I set the _FIREBASE_PROJECT_ID substitution variable to B and the _FIREBASE_HOSTING_TARGET variable to a Hosting alias that I use for the site.
When I trigger a build, it fails with the following error:
...
Step #3 - "firebase_deploy": Error: Invalid project selection, please verify project B exists and you have access.
Step #3 - "firebase_deploy":
Step #3 - "firebase_deploy": Error: Must have an active project to set deploy targets. Try firebase use --add
Step #3 - "firebase_deploy":
Step #3 - "firebase_deploy": Error: Failed to get Firebase project B. Please make sure the project exists and your account has permission to access it.
Finished Step #3 - "firebase_deploy"
I suspect that the problem may be that I'm not running the Firebase CLI's extra login step first. To do that, it seems that I would need to run firebase login:ci locally to generate a token and then pass it via the FIREBASE_TOKEN environment variable as described in the docs, but the permissions associated with the token appear to be much broader than needed:
The build process should only have access to Firebase project B, rather than "all my Firebase data and settings" and "my Google Cloud data".
Is there any way to avoid needing to run firebase login here? It seems like the service account should already have sufficient access to deploy to Firebase Hosting.
If I need to run firebase login, is there any way to create a token with a limited scope (assuming that my understanding of the default scope is correct)?
(I've also given B's service account the Cloud Functions Developer role in A and am able to successfully run gcloud --project=$_FIREBASE_PROJECT_ID functions deploy ... in a different build config. I'm also using a Cloud Build config similar to the one described above to deploy to Firebase Hosting in the same GCP project, so I suspect that firebase login isn't necessary in all cases.)
I found an approach that lets project A's Cloud Build service account deploy to B without needing excessive permissions.
First, I created a service account named deploy under B and granted it the Firebase Hosting Admin, Firebase Rules Admin, and Cloud Datastore Index Admin roles. (I'm not sure whether the Datastore role is needed, but the console showed it as being used recently so I left it.)
Next, I generated a JSON key for the new service account, pasted it (including newlines and double-quotes) as a substitution variable named _DEPLOY_CREDENTIALS, and updated the build step to copy it to the environment:
- id: firebase_deploy
name: gcr.io/$PROJECT_ID/firebase
entrypoint: bash
args: ['-e', '--', 'build/deploy_hosting.sh']
env:
- DEPLOY_CREDENTIALS=$_DEPLOY_CREDENTIALS
- FIREBASE_PROJECT_ID=$_FIREBASE_PROJECT_ID
- FIREBASE_HOSTING_TARGET=$_FIREBASE_HOSTING_TARGET
In deploy_hosting.sh, I write the credentials to a temporary file and then pass them to the firebase command via the GOOGLE_APPLICATION_CREDENTIALS environment variable:
#!/bin/bash
set -e
CREDS=$(mktemp -t creds.json.XXXXXXXXXX)
printenv DEPLOY_CREDENTIALS >"$CREDS"
export GOOGLE_APPLICATION_CREDENTIALS=$CREDS
firebase --debug use "$FIREBASE_PROJECT_ID"
firebase target:apply hosting prod "$FIREBASE_HOSTING_TARGET"
firebase deploy --project="$FIREBASE_PROJECT_ID" --only=hosting,firestore:rules
I created a separate shell script for the step since I ran into problems with quotes being stripped from the credentials when writing them directly from the build step. It would likely be possible to store the credentials in Secret Manager instead, but that felt like overkill for my use case.
I'm still curious about whether there's a way to let A's service account deploy to B without using a service account in B while running the firebase executable.
I've got a function which uses #google-cloud/language.
The function works when deployed live but when running in the local function emulator it complains
I've done firebase login with my Google account but get the following when running the function:
Error: 7 PERMISSION_DENIED: Cloud Natural Language API has not been used in project XXXXX before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/language.googleapis.com/overview?project=XXXXX then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.
XXXXX is a project Id I don't recognise - where has this come from and how can I check if I'm mistaken?
The account I'm authed with firebase locally is already in the Owner role for the project, so either the project ID is wrong, or I need to grant permissions to the user?
What do I need to do?
The solution is to set the environment variable GOOGLE_APPLICATION_CREDENTIALS for authentication:
https://cloud.google.com/docs/authentication/getting-started#windows
And restart all the things due to my ENV variable not being updated.
I am running scripts to interact with Firestore (e.g. creating a doc). I've provided a service account key file to do so (tried with key files from Firebase and GCloud) - scripts run the same.
I am aware that Firebase is now built / closely linked to GCloud so I am expecting the key files to be similar.
However, the docs recommend initializing the sdk with a key from Firebase console when adding Firebase admin to your server and from GCloud Console for unit testing.
So here are my questions:
Why recommend 2 different ways to generate them? Is one more appropriate than the other?
Do service account key files generated in Firebase (1) and GCloud (2) consoles differ in terms of scope? Do they allow the same operations? Are they granted equivalent permissions?
Firebase Console > Project Settings > Service Accounts > Firebase Admin SDK > Create new private key
https://console.firebase.google.com/project/[my-project-id]/settings/serviceaccounts/adminsdk
Google Cloud Console > IAM & Admin > Service Accounts > App Engine default service account > Create key
https://console.cloud.google.com/iam-admin/serviceaccounts?project=[my-project-id]
The service account and credentials provided by the Firebase console are no different than one that you'd create in the Cloud console. Firebase is just making it easier to get started, so you don't have to learn the Cloud console. If you want to use the one provided by Firebase, fine. If you want to use one you create and configure in the Cloud console, fine. The Firebase service account should be visible in the Cloud console just like any other. When it comes to assigning permissions to individual products, you can still choose either one to work with. It's up to you.
I am always grateful for your help.
I want to write code admin.initializeApp(); both locally and in production.
When I deploy functions to production with no auguments, it works.
But locally, it requires me to write it like below:
const serviceAccount = require("/home/yhirochick/development/ServiceAccountKey.json");
admin.initializeApp({
credential: admin.credential.cert(serviceAccount),
databaseURL: "https://xxxx.firebaseio.com/"
});
In the official documentation it says that configuration is applied automatically when you initialize the Firebase Admin SDK with no arguments
But when I execute the command firebase serve --only functions locally and some calls some requests by postman produce the error below:
[2019-07-22T06:45:26.227Z] #firebase/database: FIREBASE WARNING: Provided
authentication credentials for the app named "[DEFAULT]" are invalid. This
usually indicates your app was not initialized correctly. Make sure the
"credential" property provided to initializeApp() is authorized to access the
specified "databaseURL" and is from the correct project.
I want to know How can I "admin.initializeApp();" no arguments locally.
I have grappled with this also and I don't think the local testing scenario currently is explained very well in the official documentation. But here is a solution:
For your local environment you need to download the firebase projects firebase service account json file (found in firebase console under project settings -> service account) and set an environment variable GOOGLE_APPLICATION_CREDENTIALS to point to the file:
# Linux/MACOS version
export GOOGLE_APPLICATION_CREDENTIALS="[PATH_TO_YOUR_SERVICE_ACCOUNT_FILE]"
Read more here, also on how to do this on Windows
Now you will be able to use admin.initializeApp() (with no arguments) locally.
A possible downside of this approach is that you have to set the environment variable each time you fire up a terminal before you start the firebase emulator, because the variable gets deleted when you end the session.
Automate it...
You could automate the export ... command by bundling it together with the command that fires up the emulator. You could do this by adding an entry to the scripts section of your package.json, e.g.:
"local": "export GOOGLE_APPLICATION_CREDENTIALS='[PATH_TO_YOUR_SERVICE_ACCOUNT_FILE]' && firebase emulators:start --only functions"
Then, in this example, you would only need to type npm run local.
Alternative: provide explicit credentials in local environment only
Look at this example: https://stackoverflow.com/a/47517466/1269280.
It basically use a runtime node environment variable to separate between local and production and then use the explicit way of providing credentials in the local environment only.
This is my preferred way of doing things, as I think it is more portable. It allows me to put the service account file inside my codebase and not deal with its absolute file path.
If you do something like this then remember to to exclude the service account file from your repo! (it contains sensitive info).
Background: difference between production and local service account discovery
The reason that admin.initializeApp() (with no arguments) works out-of-the-box in production is that when you deploy to production, i.e. Firebase Functions, the code ends up in a 'Google managed environment'. In Google managed environments like Cloud Functions, Cloud Run, App Engine.. etc, the admin SDK has access to your applications default service account (the one you downloaded above) and will use that when no credentials are specified.
This is part of Google Clouds Application Default Credentials (ADC) strategy which also applies to firebase functions.
Now, your local environment is not a 'google managed environment' so it doesn't have access to the default service account credentials. To google cloud, your local box is just an external server trying to access your private Firebase ressources. So you need to provide your service account credentials in one of the ways described above.
Before I knew this, I thought that because I was already logged in to firebase via my terminal i.e. firebase login and were able to deploy code to firebase, the firebase emulator would also have the necessary credentials for the firebase admin sdk, but this is not the case.
I am trying to run instrumented tests using the glcoud CLI as a service account in CircleCi. When I run:
gcloud config set project project-name-12345
gcloud auth activate-service-account firebase-testlab-serviceuser#project-name-12345.iam.gserviceaccount.com --key-file ${HOME}/client-secret.json
gcloud firebase test android run --type instrumentation --app debug-app.apk --test debug-test.apk --device model=Nexus6P,version=27,locale=en,orientation=portrait --environment-variables coverage=true,coverageFile=/sdcard/tmp/code-coverage/connected/coverage.ec --directories-to-pull=/sdcard/tmp --timeout 20m
I get:
ERROR: (gcloud.firebase.test.android.run) Could not copy [debug-app.apk] to [gs://test-lab-xxxxxxxx-yyyyyyyy/2018-01-18_17:14:09.964449_zPAw/] ResponseError 403: firebase-testlab-serviceuser#project-name-12345.iam.gserviceaccount.com does not have storage.objects.create access to bucket test-lab-xxxxxxxx-yyyyyyyy..
Using the API Console (https://console.cloud.google.com/iam-admin/iam/project) I've given my service user all the permission I can think would be relevant:
Firebase Crash Symbol Uploader
Firebase Test Lab Admin
Storage Admin
Storage Object Admin
Storage Object Creator
Storage Object Viewer
Firebase Rules System
Any help would be greatly appreciate. Thanks.
You should be able to use a service account created in the Google Cloud Console. Did your service account have the required project Editor role? (as noted in this doc: https://firebase.google.com/docs/test-lab/continuous)
After lots of clicking through the Firebase console and the Google Cloud Console, reading SO, asking for help on Slack, and more trial and error than I care to admit, I discovered that the Firebase console has a service account page:
https://console.firebase.google.com/u/0/project/project-name-12345/settings/serviceaccounts/adminsdk
That is different from the service accounts page in the Google Cloud Console
https://console.cloud.google.com/iam-admin/serviceaccounts/project?project-name-12345
It turns out you want the Firebase service account, you can not create one via the cloud console. Super, super annoying.
The steps I took to create the key is as follow:
1. Firebase Console https://console.firebase.google.com/
2. Project Settings
3. "Service Accounts" tab
4. Inside "Service Accounts" panel, Firebase Admin SDK
5. At the bottom of "Firebase Admin SDK" panel, "Generate new private key"
This is what Etherton answered
https://stackoverflow.com/a/48327579/2353939
Even after that, I still had some errors. So, I added a bunch of roles as follows.
Firebase Test Lab Admin
Firebase Service Management Service Agent
Firebase Admin SDK Administrator Service Agent
Service Account Token Creator
Storage Object Creator
That also didn't fix. So, finally, I applied P. Davis answer by adding Editor role to the service account.
https://stackoverflow.com/a/48331465/2353939
Steps to add Editor role is as follows
1. Go to google cloud console https://console.cloud.google.com/iam-admin/iam
2. Go into "IAM"
3. Use "client_email" from the json file downloaded from firebase console to find the service account you need to edit
4. Click the "Edit" icon on the right
5. Scroll down and "Add Another Role"
6. Click the input field and type in "Editor" to search
7. Choose the one with subtitle "Edit access to all resource"
8. Save
9. Now you should be able to use it
This is the list of the roles that I put in to my service account :
Firebase - Firebase admin ( I think this is overkill. I might update it later )
Project - Editor
Storage - Storage Object Creator
It does not matter whether you create the service account from Firebase or google cloud console. As long as you have these roles in your service account then you should be able run the Firebase test lab.
For people who stumble upon this and don't want to use the all powerful Project Editor role, here are the roles I'm using for my service account:
Firebase Test Lab Roles
I think the Firebase Analytics Viewer role is not necessary, because it mostly just execute the tests. To view the result we use the developer accounts instead but haven't tried removing it.
We ran into the same permissions issue with storage.objects.create. We have added all the roles that were mentioned here, except for the Editor role which we wanted to avoid, but it still failed. We were using a Service Account and it definitely had the proper permissions.
In the end our workaround was to setup a cloud storage bucket manually and then use it in the --results-bucket argument for gcloud. See the documentation here. That finally fixed it for us.