I have a script in a VM that write data in a bucket in another project.
I want to schedule this script with Airflow but I have IAM access problem when the script need to write data:
AccessDeniedException: 403 148758369895-compute#developer.gserviceaccount.com does not have storage.objects.list access to ******
To launch the script I use the following command :
bash_command=' gcloud config set project project2 && gcloud compute --project "project1" ssh --zone "europe-west1-c" "VMname" --command="python script.py"',
If I want to launch the script with Google Cloud Shell, I need to use gcloud auth login but how can I do this with Airflow/Composer ??
I tried
bash_command='gcloud auth login && gcloud config set project project2 && gcloud compute --project "project1" ssh --zone "europe-west1-c" "VMname" --command="python script.py"',
without success
The recommended way to share resources across GCP projects is to grant the service account associated with your Cloud Composer environment the minimum required IAM permissions in the target project.
In your case, you would grant the service account user 148758369895-compute#developer.gserviceaccount.com the objectViewer role on either the target project or the target bucket.
Be careful, though! The 148758369895-compute#developer.gserviceaccount.com appears to be the default Google Compute Engine service account. Granting additional permissions may also give other VMs in your Composer project the same permissions. To avoid this: create a custom service account and associate it with the Composer environment when you create the environment.
Related
I have GCP project A and Firebase project B (in a separate GCP project). I'm trying to use Cloud Build in A to build a web app and deploy it to Firebase Hosting in B.
In B's IAM page, I have granted A's <id>#cloudbuild.gserviceaccount.com service account the API Keys Admin, Firebase Admin, and Service Account User roles as described in e.g. this question.
The final step in the Cloud Build config used by A is the following:
- id: firebase_deploy
name: gcr.io/$PROJECT_ID/firebase
entrypoint: sh
args:
- '-c'
- |
firebase use $_FIREBASE_PROJECT_ID
firebase target:apply hosting prod $_FIREBASE_HOSTING_TARGET
firebase deploy --project=$_FIREBASE_PROJECT_ID --only=hosting,firestore:rules
I set the _FIREBASE_PROJECT_ID substitution variable to B and the _FIREBASE_HOSTING_TARGET variable to a Hosting alias that I use for the site.
When I trigger a build, it fails with the following error:
...
Step #3 - "firebase_deploy": Error: Invalid project selection, please verify project B exists and you have access.
Step #3 - "firebase_deploy":
Step #3 - "firebase_deploy": Error: Must have an active project to set deploy targets. Try firebase use --add
Step #3 - "firebase_deploy":
Step #3 - "firebase_deploy": Error: Failed to get Firebase project B. Please make sure the project exists and your account has permission to access it.
Finished Step #3 - "firebase_deploy"
I suspect that the problem may be that I'm not running the Firebase CLI's extra login step first. To do that, it seems that I would need to run firebase login:ci locally to generate a token and then pass it via the FIREBASE_TOKEN environment variable as described in the docs, but the permissions associated with the token appear to be much broader than needed:
The build process should only have access to Firebase project B, rather than "all my Firebase data and settings" and "my Google Cloud data".
Is there any way to avoid needing to run firebase login here? It seems like the service account should already have sufficient access to deploy to Firebase Hosting.
If I need to run firebase login, is there any way to create a token with a limited scope (assuming that my understanding of the default scope is correct)?
(I've also given B's service account the Cloud Functions Developer role in A and am able to successfully run gcloud --project=$_FIREBASE_PROJECT_ID functions deploy ... in a different build config. I'm also using a Cloud Build config similar to the one described above to deploy to Firebase Hosting in the same GCP project, so I suspect that firebase login isn't necessary in all cases.)
I found an approach that lets project A's Cloud Build service account deploy to B without needing excessive permissions.
First, I created a service account named deploy under B and granted it the Firebase Hosting Admin, Firebase Rules Admin, and Cloud Datastore Index Admin roles. (I'm not sure whether the Datastore role is needed, but the console showed it as being used recently so I left it.)
Next, I generated a JSON key for the new service account, pasted it (including newlines and double-quotes) as a substitution variable named _DEPLOY_CREDENTIALS, and updated the build step to copy it to the environment:
- id: firebase_deploy
name: gcr.io/$PROJECT_ID/firebase
entrypoint: bash
args: ['-e', '--', 'build/deploy_hosting.sh']
env:
- DEPLOY_CREDENTIALS=$_DEPLOY_CREDENTIALS
- FIREBASE_PROJECT_ID=$_FIREBASE_PROJECT_ID
- FIREBASE_HOSTING_TARGET=$_FIREBASE_HOSTING_TARGET
In deploy_hosting.sh, I write the credentials to a temporary file and then pass them to the firebase command via the GOOGLE_APPLICATION_CREDENTIALS environment variable:
#!/bin/bash
set -e
CREDS=$(mktemp -t creds.json.XXXXXXXXXX)
printenv DEPLOY_CREDENTIALS >"$CREDS"
export GOOGLE_APPLICATION_CREDENTIALS=$CREDS
firebase --debug use "$FIREBASE_PROJECT_ID"
firebase target:apply hosting prod "$FIREBASE_HOSTING_TARGET"
firebase deploy --project="$FIREBASE_PROJECT_ID" --only=hosting,firestore:rules
I created a separate shell script for the step since I ran into problems with quotes being stripped from the credentials when writing them directly from the build step. It would likely be possible to store the credentials in Secret Manager instead, but that felt like overkill for my use case.
I'm still curious about whether there's a way to let A's service account deploy to B without using a service account in B while running the firebase executable.
so what every i am doing in the gcp giving it roles iam roles to configure the policies , i am not able to do to my firebase project. i am able to log into it using my terminal with firebase login and then then firebare init is there and i put the function initilaize but it says i don't have permission to do that i need to give it permission so i open the file
/iam-admin/iam?folder=&organizationId=&project=clone-6de06 because this is my firebase project but i dont' have access to it in the gcp so i am not able to change the permission to change the policy or
need to run the role command
gcloud projects add-iam-policy-binding ourcafe-mucqxq
--member=serviceAccount:service-1044193269753#gcf-admin-robot.iam.gserviceaccount.com
--role=roles/cloudfunctions.serviceAgent as that says it will take care of following error: Deployment of your Cloud Function failed:
Missing necessary permission resourcemanager.projects.getIamPolicy for
service-1044193269753#gcf-admin-robot.iam.gserviceaccount.com on
resource projects/ourcafe-mucqxq. Please grant
service-1044193269753#gcf-admin-robot.iam.gserviceaccount.com the
Cloud Functions Service Agent
any help will be appreciated thank for the help.
Did you try to log in to firebase with a proper account?
If you're not sure that your account with proper permission, you can go to Firebase project >> Project settings >> Users and permission to check that your account is on the list with the right permission.
Also, you can see all of the accounts on the IAM of GCP console.
I have a Firebase project under a GCP organization. I have NodeJS 10 callable functions which work fine with Firebase local emulator and in in GCP functions test page, but when attempting to call them in production using httpsCallable in a webapp, I get 401 Unauthorized - Your client does not have permission to the requested URL
There is practically nothing my test function - it just returns a hardcoded string. There is nothing in the logs after the call attempts.
The function call attempt is done after passing Firebase authentication in the webapp, and in development I see that the user is indeed passed to the context parameter.
I use Firebase Tools 8.0.2 to deploy.
In GCP console function details I see "Ingress settings - Allow all traffic", and as I wrote above, it runs successfully from the test tab there.
I tried to make the function public but I can't set permissions in the console or gcloud. When I try to run the following command with gcloud:
gcloud functions add-iam-policy-binding my-function-name \
--member="allUsers" \
--role="roles/cloudfunctions.invoker"
I get:
ERROR: (gcloud.functions.add-iam-policy-binding) ResponseError: status=[403], code=[Forbidden], message=[Permission 'cloudfunctions.functions.setIamPolicy' denied on resource 'projects/my-project-name/locations/us-central1/functions/my-function-name' (or resource may not exist).]
even though my account has the following roles: Project Owner, Cloud Functions Admin, Security Admin, IAP Policy Admin, Organization Administrator
Any guidance will be appreciated.
Per John's instructions, it turned out gcloud was set to an incorrect active account, so I set it to the correct account that had the required permissions using gcloud config set account my-email#my-domain.com and then I was able to make the function publicly invokable using
gcloud functions add-iam-policy-binding my-function-name \
--member="allUsers" \
--role="roles/cloudfunctions.invoker"
I have deployed a JVM application to Google cloud run that uses the Firebase Admin SDK to send notifications on Firebase Cloud Messaging.
Everything works fine locally using GOOGLE_APPLICATION_CREDENTIALS. The deployed app, however, throws errors as follows:
java.lang.IllegalArgumentException: Project ID is required to access messaging service. Use a service account credential or set the project ID explicitly via FirebaseOptions. Alternatively you can also set the project ID via the GOOGLE_CLOUD_PROJECT environment variable.
I create the FirebaseMessaging instance as:
FirebaseMessaging.getInstance(
FirebaseApp.initializeApp(
FirebaseOptions.Builder().setCredentials(
GoogleCredentials.getApplicationDefault()
).build()
I have deployed the Cloud Run instance with a service account which has admin permissions for Cloud Run.
My understanding is that an application deployed to any GCP service acquires application credentials automatically. Is there a difference for the combination of Cloud Run and Firebase Admin SDK?
Any help would be much appreciated.
You have two options:
Query the cloud metadata service to find the project ID where your Cloud Run instance is deployed to. This can only be done within Cloud Run - it won't work if you're testing locally.
Make the project ID available to Cloud Run in some way during deployment.
An easy way to implement #2 is to put the project ID in the environment for the Admin SDK to automatically pick up. For example, a shell script:
service_id="your-service-id"
project_id="your-project-id"
gcloud run deploy "$service_id" \
--project "$project_id" \
--image "..." \
--platform managed \
--update-env-vars "GOOGLE_CLOUD_PROJECT=$project_id"
Note the --update-env-vars which populates GOOGLE_CLOUD_PROJECT.
I have functions deployed to gcloud functions and i want to configure CI/CD for deploying this functions from gitlab.
To do any operations from gitlab i need to get firebase auth token with
firebase login:ci
command.
The problem is that i need to get this token using gcloud service account, which is not displayed in browser, when i run
firebase login:ci
I have this service account data (project_id, private_key, private_key_id, etc.)
How should i authorize using this acc?
If you set an environment variable called GOOGLE_APPLICATION_CREDENTIALS as a path pointing to your service account JSON file, the Firebase CLI will automatically pick that up and use it to authorize commands. You don't need to be logged in or provide a --token argument when this is the case.
If anyone finds this and is wondering how to do it on CircleCI, this worked for me.
Generate a json file key for you service account in the GCP Console
Set the json to a CircleCI environment variable at the org level or the project level
We use one at the org level called GSA_KEY
In your workflow config, before you run the firebase command, run this command:
echo echo $GSA_KEY > "$HOME"/gcloud.json
Then run your firebase deploy command, first setting the path to GOOGLE_APPLICATION_CREDENTIALS
The deployment run looks like:
steps:
- checkout
- run:
name: Create SA key JSON
command: echo $GSA_KEY > "$HOME"/gcloud.json
- run:
name: Deploy to Firebase
command:
GOOGLE_APPLICATION_CREDENTIALS="$HOME"/gcloud.json firebase deploy [project specific stuff]
Use command:
gcloud auth activate-service-account xyz#project-id.iam.gserviceaccount.com --key-file=/path/to/file.json --project=project-id
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/file.json"
in Bash
run the following commands 1 and 2 in order
export GOOGLE_APPLICATION_CREDENTIALS="/path/to/file.json"
"/path/to/file.json" -- the location of the file where the service account json file is saved.
npx firebase-tools deploy --json
Do not forget to use the right project when deploying like
firebase use dev or
firebase use qa
This is what worked for me:
Added the environment variable GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
firebase deploy --non-interactive # needed to bypass any prompts that might stall out a CI script.
You can also set the value of the Environment variable to be the JSON in the key file.