Google Cloud Task not calling http requests - firebase

Question

I have created a google cloud task and the queue keeps retrying and the function is not getting invoked as well.
This is the log from the cloud console.
attemptResponseLog: {
attemptDuration: "0.133874s"
dispatchCount: "19"
maxAttempts: 0
responseCount: "0"
retryTime: "2020-06-21T21:20:18.518655Z"
scheduleTime: "2020-06-21T21:20:15.718098Z"
status: "UNAVAILABLE"
targetAddress: "POST some url"
targetType: "HTTP"
}

Answer 1

I ran into this same error, and I must say that the documentation is not clear enough.
WARNING : I feel there's a bit of latency for the roles to be taking into account, especially with the ServiceAccountUser one.
I made multiple test, and tried to keep the lowest rights possible, so I did try to remove some... do some test, it works... great, it's not necessary to have this right... came back later, and the thing is broken.
Here is my setup :
I use Cloud Scheduler to trigger a Cloud Function every 15 minutes by posting a message on a queue.
The CloudFunction build a list of tasks to compute stats on MySQL and create the tasks
Another Cloud Function run SQL query to get stats and store the results in FireStore.
I use cloud task so that the load on MySQL is not too heavy.
Below, I use functional names to make it easy to understand.
TaskCreatorCloudFunction running with TaskCreatorServiceAccount
TaskCreatorServiceAccount requires
the "Cloud Task Enqueuer" role #1
be a ServiceAccountUser on the CloudTaskComputeStatsServiceAccount (see after) #2
The Roles needed to do the job(read SQL to get the list of tasks to create, write logs, access secret manager, listen to pubsub as it's triggered by Cloud Scheduler via pubsub)
TaskImplementationCloudFunction (http) running with TaskImplementationServiceAccount
TaskImplementationServiceAccount has no specific role for CloudTasks, only the one needed to do the job (read SQL, write logs, access secret manager, firestore write)
The TaskQueue is named "compute-stats-on-mysql".
I've created a dedicated ServiceAccount named CloudTaskComputeStatsServiceAccount #3
CloudTaskComputeStatsServiceAccount has the specifics rights for the whole thing to work.
Cloud Function Invoker #4
Add CloudTaskComputeStatsServiceAccount as ServiceAccountUser on TaskImplementationServiceAccount #5
To do the last one in the console (script version below), you need to
go to IAM->Service Account
check the TaskImplementationServiceAccount
In the upper right corner, click "Show Info Panel" if it's not already displayed
click the Add Member
Paste the full name of the CloudTaskComputeStatsServiceAccount
Choose Service Account User as role
Save
You can edit this in the console, but it's better to script it.
gcloud tasks queues create compute-stats-on-mysql \
--max-dispatches-per-second=10 \
--max-concurrent-dispatches=15 \
--max-attempts=2 \
--min-backoff=1s
#3
gcloud iam service-accounts create CloudTaskComputeStatsServiceAccount --description="Service Account for the cloud task compute-stats-on-mysql" --display-name="Service Account for the cloud task compute-stats-on-mysql"
#4
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member serviceAccount:CloudTaskComputeStatsServiceAccount#${PROJECT_ID}.iam.gserviceaccount.com --role "roles/cloudfunctions.invoker"
#1
gcloud projects add-iam-policy-binding ${PROJECT_ID} --member serviceAccount:TaskCreatorServiceAccount#${PROJECT_ID}.iam.gserviceaccount.com --role "roles/cloudtasks.enqueuer"
#5
gcloud iam service-accounts add-iam-policy-binding TaskImplementationServiceAccount#${PROJECT_ID}.iam.gserviceaccount.com --member="serviceAccount:CloudTaskComputeStatsServiceAccount#${PROJECT_ID}.iam.gserviceaccount.com" --role "roles/iam.serviceAccountUser"
#2
gcloud iam service-accounts add-iam-policy-binding CloudTaskComputeStatsServiceAccount#${PROJECT_ID}.iam.gserviceaccount.com --member="serviceAccount:TaskCreatorServiceAccount#${PROJECT_ID}.iam.gserviceaccount.com" --role=roles/iam.serviceAccountUser
when Creating the Task, you use the CloudTaskComputeStatsServiceAccount in the oidcToken
const body = Buffer.from(JSON.stringify(data)).toString('base64');
const task = {
httpRequest: {
httpMethod: 'POST',
url,
oidcToken: {
serviceAccountEmail: "CloudTaskComputeStatsServiceAccount#${PROJECT_ID}.iam.gserviceaccount.com",
},
headers: {
'Content-Type': 'application/json',
},
body,
},
};
My understanding is that when you run the
const [response] = await cloudTasksClient.createTask({parent, task});
The Cloud Function (Task Creator) need to Create Task, and act as the "CloudTaskComputeStatsServiceAccount"
And "CloudTaskComputeStatsServiceAccount" need to have the cloud function invoker and act as the target cloud function.

Answer 2

Indeed it's not a service account issue. It's the OIDC token audience missing. Seems that for cloud functions this is needed. I found two references... you can recreate this problem with the OIDC token in the cli by omitting this argument to gcloud tasks create-http-task
--oidc-token-audience=OIDC_TOKEN_AUDIENCE
The audience to be used when generating an OpenID Connect token to be
included in the request sent to the target when executing the task.
If not specified, the URI specified in the target will be used
The second reference that popped up, in ruby shows audience
https://googleapis.dev/ruby/google-cloud-tasks/latest/Google/Cloud/Tasks/V2/OidcToken.html
Code using google-cloud-tasks 1.5.0, the tasks object looks like this, where url_oidc has just the url to the cloud function (i.e. the trigger url... no url parameters)
# Construct the request body.
task = {
'http_request': { # Specify the type of request.
'http_method': 'GET',
'url': url_final, # The full url path that the task will be sent to.
'oidc_token': {
'service_account_email': service_account_email,
'audience': url_oidc
}
}
}