does anyone know how to enable the Message Ordering property on a subscription when working with Firebase and NodeJs?
If I navigate to the GCP console and open the queue's subscription detail, I can change the parameter manually but I'd rather have it done when my functions deploy.
This link (https://cloud.google.com/pubsub/docs/ordering#enabling_message_ordering) also talks about a parameter that can be provided when creating a subscription but that doesn't seem to be available the way I usually create/deploy my functions.
GCP Documentation
Own Code
It sounds like you're asking if pubsub functions deployed with the Firebase CLI (and not gcloud) can request ordering. Currently, this is not possible. When you deploy a pubsub function with the Firebase CLI, it will automatically create the pubsub topic for you. However, there is no option to set the ordering setting.
If you would like see this implemented, you should file a feature request on the Firebase CLI GitHub and/or contact Firebase support directly.
Alternatively, you can switch to use gcloud to establish the trigger and create its pubsub topic with your customizations.
You should note that enabling ordering does not necessarily ensure that function invocations will be perfectly serialized with respect to incoming messages. Cloud Functions is designed for incoming events to be handled in parallel as fast as possible by using multiple server instances as needed. There is no guarantee that pubsub messages will be processed fully in order. The ordering only applies to pubsub topic consumer code that runs in a single process. You could try to set the max instances of a function to 1, but that is not a hard guarantee.
If you need fully ordered serialized execution of some code in response to pubsub events, maybe Cloud Functions isn't the product you want to use. Maybe a single App Engine or Compute Engine instance would work better.
Related
I currently working on a sync third party data project, that I decided to implement queue to ensure the message has received to the worker, but the worker need to do only one task at the time, which means the previous task should finished and acknowledged before executing the next one.
so the question is, how can I config the firebase pub/sub trigger to execute task one by one ?.
If I misunderstand on the concept of google pub/sub feel free to tell me :)
What you're trying to do isn't really compatible with the way pubsub functions work. There are other Google Cloud products that can help you limit the rate of invocation of some code, but it won't be implemented with a pubsub type function.
GCP offers a product called Cloud Tasks, which lets you set up a queue that can be configured with a rate limit. The queue can dispatch a task invocation to App Engine, or to an HTTP type Cloud Function (beta). So, if you've deployed an HTTP function, you could use its endpoint URL to configure a Cloud Tasks queue that invokes that function only once at a time.
You can control the number of instances of Cloud Functions with the MAX-INSTANCES flag of gcloud command line tool. See the documentation on Using max instances for full details.
The Firebase SDK for Cloud Functions doesn't wrap this option (yet). But you can deploy the function using the Firebase CLI, and then change the setting in the Cloud console, by following the steps shown in Clearing max instances limits.
I am working on a project that, currently, is 100% Firebase. Ideally, given I'm fully Firebase, I'd like to stay with Firebase for a next task which is updating some of the records based on external API calls once per day.
I'm currently using Firebase Functions for triggered events, not using it for API calls, everything that happens in the functions is after a user does something, and doesn't respond back to any clients (only responds back to the database for updates).
Is Firestore Cloud Functions a good place to run something like this that could call an external API and then update as necessary? I saw the scheduled functions that require the Blaze plan, have considered it but not sure if there's another approach that's better built for this task.
Cloud Functions that trigger on Firestore events probably aren't what you're looking for. Firestore triggers only fire when something in your Cloud Firestore database has changed. That means you need something that's writing to some document in the database in order to get the code to run. Which means you need a way to schedule that operation.
No matter what kind of trigger you write, you will need to be on a billing plan in order to make external requests anyway. So even if you somehow managed to put together a solution that uses Firestore triggers, your project would still need to be on a billing plan.
This approach is perfectly okay - in fact, I am using the exact same approach in my project which has 100% Firebase back-end. The overall (Firebase) Cloud Functions gives flexibility in terms of invocation i.e. they can be invoked based on trigger (e.g. storage or database event) or can be called with the HTTP end-point. So, depending on your need you can either use Firestore trigger or database trigger or call an end-point.
Switching to Blaze plan is perfectly fine since otherwise we can't call an external end-point. I switched to Blaze plan just a few months back and didn't pay anything for that as my usage is within the free limit.
I am new on firebase cloud functions. I would like to ask a question about always running or self-triggering functions. How can we handle these things? How can we implement always running functions or self-triggering?
Google Cloud Functions are snippets of code that run in response to events that happen somewhere else. Some example events:
a HTTPS URL is accessed, either from application code, or in some other way
a User account is created on Firebase Authentication
a node is written in the Firebase Realtime Database
a message is sent to a Cloud PubSub topic
There is no concept on a forever-running function on Cloud Functions (nor on other, similar Functions-as-a-Service offerings), although it's definitely possible to create a function that gets triggered every minute or so (like a cron job).
So I know that Cloud Functions for Firebase don't natively support cron jobs, and one have to depend on an external source to trigger an HTTP cloud function. But I don't think this will work in my case.
I have an app in which users can create competitions that end in 24 hours, thus each time a competition is created, I need to set up a timer that triggers one time after 24 hours.
Do you guys have any idea on how this problem can be approached?
At this moment the solution that the Firebase guys are providing for solving the cron issue is to use Google App Engine to deploy a server and trigger the functions via Pub/Sub hooks. Here you can find the tutorial.
Basically you have to:
Create a Cloud Function that responds to a PubSub hook.
Deploy an App Engine that will publish an event depending on the cron setup you set.
If you don't want to setup an App Engine you can use an external resource like cron-job.org that can call your HTTP Cloud Function.
I have written some server-side code to use firebase-queue](https://github.com/firebase/firebase-queue) for scalability, however, with the release of Cloud Functions for Firebase (and its promise of automatic scalability), I am wondering if there is any need for Queue...Has anyone out there combined these two technologies to a greater purpose? Specifically to Firebase developers like #Frank van Puffelen, will Functions replace firebase-queue?
firebaser here
I'm not sure if firebase-queue is not obsolete. Time will have to tell.
But we definitely now use Cloud Functions for Firebase in a lot of scenarios where we'd previously have used firebase-queue and a node worker process. No longer needing to bring our own Node.js process gives an increase in development speed. The auto-scaling of Cloud Functions has proven to be invaluable already.
Combining Cloud Functions with firebase-queue seems illogical. If you simply append nodes to the database and consume them in your function, you have the same behavior without needing the extra library.
Update: one of our database engineers just gave a scenario where Functions can't replace the queue. When making backups, the workers need to mount and unmount remote disks. That task may be possible in a Cloud Function, it is much easier to do in a standalone, self-managed Node process.
I don't think Firebase cloud functions should render firebase-queue obsolete, and I don't think it's illogical at all to want to combine firebase-queue with cloud functions as ##Frank van Puffelen has said.
Firebase-queue provides much more than a way for tasks to listen to firebase and launch tasks. It provides a protocol for communication between parties assigning and and responding to task requests, coordinating retries of failed tasks, reporting progress, state and additional metadata. One of the things this allows is task chaining.
I think it would be useful for Firebase or a third party to develop a firebase-functions-queue package that extends the firebase-functions and allows you to write a firebase cloud function with the same signature as firebase-queue. Something like this:
const functions = require('firebase-functions');
const functions-queue = require('firebase-functions-queue'); //extends firebase-functions with onQueue function
admin.initializeApp(functions.config().firebase);
functions.database.ref('/queue').onQueue(options,function(data,progress,resolve,reject){
...
})
This new package would work just like firebase-queue and use the same metadata and options, except it wouldn't need to deal with managing multiple worker processes since cloud functions already do this automatically and seamlessly.
This would have the following advantages:
Would allow firebase developers to use a standard way of assigning jobs to queues, monitor progress, failure, etc.
An app could assign a job to a queue and not care if it was being handled by a cloud function or a different environment.
A developer could take an existing queue and move it from their node server to cloud functions without changing the client app.
It could even allow task chaining where one task could be handled by a cloud function and another by a different server within the same job.
I don't think it's as simple as saying it replaces firebase-queue. Ideally, you would use Functions for most of the common use cases for firebase-queue.
However, there are probably also some legitimate reasons to use firebase-queue anyway.
It's a pretty robust queuing system that allows you to use multi-state tasks and allows you to set the number of workers.
It runs in an environment you specify instead of App Engine (you could install binaries on your server that aren't available in GAE).
Could be cheaper depending on specifics of bandwidth vs invocations.
It can contact third party endpoints for free, while Functions requires a paid account for these.
Functions are still in Beta and don't offer an SLA, they also currently have some startup latencies that wouldn't be experienced in firebase-queue (this point should become invalid around GA release time).