I have written some server-side code to use firebase-queue](https://github.com/firebase/firebase-queue) for scalability, however, with the release of Cloud Functions for Firebase (and its promise of automatic scalability), I am wondering if there is any need for Queue...Has anyone out there combined these two technologies to a greater purpose? Specifically to Firebase developers like #Frank van Puffelen, will Functions replace firebase-queue?
firebaser here
I'm not sure if firebase-queue is not obsolete. Time will have to tell.
But we definitely now use Cloud Functions for Firebase in a lot of scenarios where we'd previously have used firebase-queue and a node worker process. No longer needing to bring our own Node.js process gives an increase in development speed. The auto-scaling of Cloud Functions has proven to be invaluable already.
Combining Cloud Functions with firebase-queue seems illogical. If you simply append nodes to the database and consume them in your function, you have the same behavior without needing the extra library.
Update: one of our database engineers just gave a scenario where Functions can't replace the queue. When making backups, the workers need to mount and unmount remote disks. That task may be possible in a Cloud Function, it is much easier to do in a standalone, self-managed Node process.
I don't think Firebase cloud functions should render firebase-queue obsolete, and I don't think it's illogical at all to want to combine firebase-queue with cloud functions as ##Frank van Puffelen has said.
Firebase-queue provides much more than a way for tasks to listen to firebase and launch tasks. It provides a protocol for communication between parties assigning and and responding to task requests, coordinating retries of failed tasks, reporting progress, state and additional metadata. One of the things this allows is task chaining.
I think it would be useful for Firebase or a third party to develop a firebase-functions-queue package that extends the firebase-functions and allows you to write a firebase cloud function with the same signature as firebase-queue. Something like this:
const functions = require('firebase-functions');
const functions-queue = require('firebase-functions-queue'); //extends firebase-functions with onQueue function
admin.initializeApp(functions.config().firebase);
functions.database.ref('/queue').onQueue(options,function(data,progress,resolve,reject){
...
})
This new package would work just like firebase-queue and use the same metadata and options, except it wouldn't need to deal with managing multiple worker processes since cloud functions already do this automatically and seamlessly.
This would have the following advantages:
Would allow firebase developers to use a standard way of assigning jobs to queues, monitor progress, failure, etc.
An app could assign a job to a queue and not care if it was being handled by a cloud function or a different environment.
A developer could take an existing queue and move it from their node server to cloud functions without changing the client app.
It could even allow task chaining where one task could be handled by a cloud function and another by a different server within the same job.
I don't think it's as simple as saying it replaces firebase-queue. Ideally, you would use Functions for most of the common use cases for firebase-queue.
However, there are probably also some legitimate reasons to use firebase-queue anyway.
It's a pretty robust queuing system that allows you to use multi-state tasks and allows you to set the number of workers.
It runs in an environment you specify instead of App Engine (you could install binaries on your server that aren't available in GAE).
Could be cheaper depending on specifics of bandwidth vs invocations.
It can contact third party endpoints for free, while Functions requires a paid account for these.
Functions are still in Beta and don't offer an SLA, they also currently have some startup latencies that wouldn't be experienced in firebase-queue (this point should become invalid around GA release time).
Related
From this link https://code.markedmondson.me/googleCloudRunner/articles/usecase-r-event-driven-pubsub.html, it's possible to trigger an R function from pub/sub.
But does Google offer possibility to trigger an R function when a new file is uploaded to a bucket?
Thanks.
You should be able to do this using EventArc and Cloud Run, see Create a trigger for Cloud Run.
NOTE Cloud Run supports any runtime (that you can containerize) and -- although opinionated (constrained set of runtime versions), Cloud Functions next generation (gen2) creates Cloud Run services.
One other way to do this is less practical: Cloud Storage triggers with Cloud Functions. R isn't a supported runtime for Cloud Functions and so, while not impossible, I don't recommend this approach.
There are probably others, GCP's eventing is confusing and my take is that Google's supporting different approaches for customer continuity reasons and doesn't provide clear guidance on the best solutions for "green field" applications.
I think (!?) Eventarc is the strategic solution and recommend using that if you can.
does anyone know how to enable the Message Ordering property on a subscription when working with Firebase and NodeJs?
If I navigate to the GCP console and open the queue's subscription detail, I can change the parameter manually but I'd rather have it done when my functions deploy.
This link (https://cloud.google.com/pubsub/docs/ordering#enabling_message_ordering) also talks about a parameter that can be provided when creating a subscription but that doesn't seem to be available the way I usually create/deploy my functions.
GCP Documentation
Own Code
It sounds like you're asking if pubsub functions deployed with the Firebase CLI (and not gcloud) can request ordering. Currently, this is not possible. When you deploy a pubsub function with the Firebase CLI, it will automatically create the pubsub topic for you. However, there is no option to set the ordering setting.
If you would like see this implemented, you should file a feature request on the Firebase CLI GitHub and/or contact Firebase support directly.
Alternatively, you can switch to use gcloud to establish the trigger and create its pubsub topic with your customizations.
You should note that enabling ordering does not necessarily ensure that function invocations will be perfectly serialized with respect to incoming messages. Cloud Functions is designed for incoming events to be handled in parallel as fast as possible by using multiple server instances as needed. There is no guarantee that pubsub messages will be processed fully in order. The ordering only applies to pubsub topic consumer code that runs in a single process. You could try to set the max instances of a function to 1, but that is not a hard guarantee.
If you need fully ordered serialized execution of some code in response to pubsub events, maybe Cloud Functions isn't the product you want to use. Maybe a single App Engine or Compute Engine instance would work better.
Going through this blog, I could see that my app would be better off using the Firebase cloud functions for Firebase Firestore rather than directly accessing Firestore using client-side SDK.
I can implement the Firestore READ operation using get(), WRITE operation using set() or update() &
DELETE using delete() methods. All these one-shot operations are fine.
Is possible to implement addSnapshotListener to fetch real-time updates? If yes, how?
While it technically is possible to use addSnapshotListener, there are not a lot of use-cases where this makes sense.
Cloud Functions are short-lived, a maximum of 9 minutes, so not well suited for scenarios where you need to listen to the database for longer times.
Typically, you'll want to instead define a separate Cloud Function that responds to the thing you'd otherwise want to listen to.
Realtime listeners are not compatible with the way Cloud Functions works. Functions can only run for a limited amount of time and are shut down after that timeout expires. Database listeners are indefinite, and will continue listening until they are removed. If you add a listener inside a function, the system will still shut it down after the function timeout expires.
It's unclear to me why you need a listener in a Cloud Functions, but it's almost certainly not the right thing to do, given the way functions work.
What is the scope of implementing a micro-service architecture using firebase cloud functions? Is it a correct way to do it or is it a step backward. As we have seen fire base is built to be server less application back-end, But with multiple triggers and support for HTTPS should we try to get back to micro services. Just to try I have implemented multiple services on firebase cloud functions which had multiple URLs, they had a really good response time averaging at 500ms
This is a very challenging question to answer. It is not a step backward, you can think of Cloud Functions as a tool that you can use along with other technologies to implement your microservice strategy. For instance, if you are going to be leveraging the Firebase Database, and other features within Firebase then it makes sense to use the Cloud Functions for Firebase.
Let's say you don't want to use Cloud Functions for Firebase and you choose another technology such as Kubernetes or App Engine. First, you'll have to add the firebase SDKs to that stack and make sure it can access your Firebase project. You get access for free in Cloud Functions for Firebase. Next, you will write the same code that you would implement in the Cloud Function. Finally, you will have additional steps for deploying those technologies. Leveraging Cloud Functions for Firebase will be quicker and more productive.
As time goes on it will become more apparent when to use an additional technology. I recently wrote a blog post about when I would choose Container Engine over Cloud Functions. This topic can become subjective since it's really based on your needs, features, and the technologies you are working with.
Cloud Functions vs Container Engine
I am using Firebase for my new project. This will be used on both Android and IOS. As you can guess I don't want to write the same code over and over again for both OS.
I am considering to code most of the work with Javascript on the cloud functions. In order to do that I need to use HTTP Requests to call my functions since firebase doesn't support any other way to call cloud functions.
There is two question in my head about this.
Is this possible and does it makes sense?
Since I've been using HTTP Requests all the time isn't that make my app open to listening with tools like Wireshark etc if there are multiple users on the same network? (I know Firebase now supports SSL but do I have to but a domain and license for that?)
What is the best way to do it in an engineer's perspective?
You can certainly move more of your app logic into Cloud Functions. But it's not really true that the only way to invoke a Cloud Function is via HTTP. You can also push data into your database to invoke a database trigger. I gave a talk on this at Google I/O yesterday about how I made a game with all the logic in Cloud Functions. You can watch it here.