gcp cloud function queue - firebase

Question

I have written a GCP Cloud Function with a http trigger ("REST API") that invokes a Cloud Function with a Firebase Realtime Database trigger (onCreate).
The Cloud Function with a Firebase Realtime Database trigger performs REST calls to other services based on received data from the REST API.
I have noticed that the called services sometimes returns http 429 (too many calls) since my REST API does not have a limit to how many calls can be received.
The REST API has security measures in place to prevent unauthorised calls to invoke the Cloud Function with a Firebase Realtime Database trigger (onCreate). I do not wish to limit the amount of calls to my API, but rather place all requests in a queue and process them in sequence.
It is important that all calls are processed as promptly as possible. I do not wish to process transactions in 60 second intervals.
In my current solution all calls to the GCP HTTP http REST API immediately triggers the GCP Cloud Function via Firebase Realtime Database insert (onCreate event).
What I would like is to maybe have a queue in between my REST API and the Firebase Realtime Database insert (onCreate event) to ensure that only one GCP Cloud Function can execute simultaneously.
What is the best way to achieve this functionality?
Kind regards /K
EDIT:
Might Maximum instances be a solution here?
https://cloud.google.com/functions/docs/configuring/max-instances

Answer 1

You can Trigger Cloud Functions using Cloud Tasks. Here's an example I use whereby emails are placed inside of the Cloud Task queue I created and they are then processed by the task runner one-after-another:
import { CloudTasksClient } from '#google-cloud/tasks';
// define and use the following for your task:
// CLOUD_FUNCTION_URL
// TASK_HTTP_METHOD
// TASK_CONTENT_TYPE
// SERVICE_ACCOUNT_EMAIL
// PROJECT_ID
// REGION
// QUEUE_NAME
const client = new CloudTasksClient({ projectId: PROJECT_ID });
/**
* Build the Cloud Task
* In the below example we take a POST body and
* stringify it before converting to base64.
*/
const convertedPayload = JSON.stringify(payload);
const body = Buffer.from(convertedPayload).toString('base64');
const task: TaskT = {
httpRequest: {
url: CLOUD_FUNCTION_URL,
httpMethod: TASK_HTTP_METHOD,
body,
headers: {
'Content-Type': TASK_CONTENT_TYPE,
},
oidcToken: {
serviceAccountEmail: SERVICE_ACCOUNT_EMAIL,
},
},
scheduleTime: {
seconds: Date.now() / 1000, // <--- start the task now (in unix time)
},
};
return client.createTask({
parent: client.queuePath(PROJECT_ID, REGION, QUEUE_NAME),
task,
});
You'll also need to configure some IAM permissions for your development and app IAM users like Cloud Functions Invoker and Cloud Functions Viewer.