I am currently learning Firebase Cloud Functions and wanted to understand if the express app is always running, my assumption is that it is always running on the Firebase platform otherwise how does it handle a request?
Your Express app on Cloud Functions runs in a container, and the default behavior is to spin down that container after there have not been any requests for a while (the exact period is undocumented). Since you're only paying for the time it's actually actively processing requests, otherwise Cloud Functions would end up with lots of containers that nobody is paying for.
When a new request comes in when there is no container instance, or while all the existing containers are busy, Cloud Functions spins up a new container to process that request, which it then will also shut down when it's been inactive for a while.
Since spinning up the container takes some time, you can configure Cloud Functions to keep a certain minimum number of containers active. But I'd usually recommend only looking at that when you're actively experiencing performance problems due to the startup time of new containers, as you'll pay (a lower amount) for each container that is kept active while not processing a request.
Related
After reading the docs on both Cloud Run and Firebase Cloud Functions, I have a few questions I want to clear up:
Does Cloud Run basically act as a container image storage/deploying mechanism? If I have 2 websites and have them as separate containerized images, does Cloud Run just deploy the specified one given the trigger?
Integrating Cloud Run with Firebase Cloud Functions as the trigger, will there be an additional layer of latency? While latency times are never known, FCFs inherently have warm-up times due to cold starts, will there be added latency due to Cloud Run cold-starting the images?
Does the Cloud Run images travel through the FCF to get to the user. Or does FCF merely redirect the user directly to the Cloud Run image?
Basically, is it like
Client -> FCF -> Image -> FCF -> Client
or
Client -> FCF -> Image -> Client
Typically on Stack Overflow one is supposed to limit to one question per post (to avoid being closed as "too broad"), but I'll try here. Please address followup questions as new posts.
Yes, it is just a containerized way of serving HTTP requests.
Cloud Run is not directly related to Cloud Function except where you write code to connect them. If you write a Cloud Function trigger that proxies to Cloud Run, it will incur all the latency costs of both products, as required (not all invocations require a cold start). All "serverless" compute options have a cold start time, since they all scale down to zero (based on current load), and you don't pay for virtual server instances to be allocated and immediately available all the time.
Again, they are not related. You can use either one without the other.
I need to initialize potentially thousands of Firebase app instances server-side using firebase-admin with a service account and then listen to specific Realtime DB and Firestore events on these app instances. These instances have nothing to do with one another and the only access I have to them is through their service account credentials files.
My main concern is scaling because new project app instances will be initialized on the fly. It is important that the events listened to should not be duplicated, so I can not have multiple server instances initialize listen to the events of all the apps.
Ideally, I would love to have an auto-scaling serverless solution and to keep it inside the Google Cloud Platform family. If there was a way to work with the app instances, while they are auto-scaled behind the scenes - like a load balancer for a collection of app instances. I've looked at the Google Cloud Pub/Sub solution to potentially do something like this, but could not come up with something concrete.
The non-scaling solution I have so far, using cloud functions:
EDIT: After feedback regarding Cloud Functions
on startup of my project's cloud functions, initialize all the current app instances
have a cloud function listen to newly added service accounts and initialize a new app instance for it. Currently, I have no idea how many project instances I can keep in memory before it becomes a problem.
Start up a single Node server to initialize all the current app instances. As new projects are added, simply add a reference for it. Monitor the server closely to ensure it doesn't run out of memory. Scale vertically as far as possible. When it gets to the point where necessary, start up a second server instance, but manually distribute project instances between servers, so there are no duplicates.
Bonus question: are only the individual cloud functions scaled as needed? ie. what happens to other standard JS functions/objects created outside a cloud function but inside the cloud functions index.js file?
Any advice would be greatly appreciated. Happy to elaborate on anything.
I have written some server-side code to use firebase-queue](https://github.com/firebase/firebase-queue) for scalability, however, with the release of Cloud Functions for Firebase (and its promise of automatic scalability), I am wondering if there is any need for Queue...Has anyone out there combined these two technologies to a greater purpose? Specifically to Firebase developers like #Frank van Puffelen, will Functions replace firebase-queue?
firebaser here
I'm not sure if firebase-queue is not obsolete. Time will have to tell.
But we definitely now use Cloud Functions for Firebase in a lot of scenarios where we'd previously have used firebase-queue and a node worker process. No longer needing to bring our own Node.js process gives an increase in development speed. The auto-scaling of Cloud Functions has proven to be invaluable already.
Combining Cloud Functions with firebase-queue seems illogical. If you simply append nodes to the database and consume them in your function, you have the same behavior without needing the extra library.
Update: one of our database engineers just gave a scenario where Functions can't replace the queue. When making backups, the workers need to mount and unmount remote disks. That task may be possible in a Cloud Function, it is much easier to do in a standalone, self-managed Node process.
I don't think Firebase cloud functions should render firebase-queue obsolete, and I don't think it's illogical at all to want to combine firebase-queue with cloud functions as ##Frank van Puffelen has said.
Firebase-queue provides much more than a way for tasks to listen to firebase and launch tasks. It provides a protocol for communication between parties assigning and and responding to task requests, coordinating retries of failed tasks, reporting progress, state and additional metadata. One of the things this allows is task chaining.
I think it would be useful for Firebase or a third party to develop a firebase-functions-queue package that extends the firebase-functions and allows you to write a firebase cloud function with the same signature as firebase-queue. Something like this:
const functions = require('firebase-functions');
const functions-queue = require('firebase-functions-queue'); //extends firebase-functions with onQueue function
admin.initializeApp(functions.config().firebase);
functions.database.ref('/queue').onQueue(options,function(data,progress,resolve,reject){
...
})
This new package would work just like firebase-queue and use the same metadata and options, except it wouldn't need to deal with managing multiple worker processes since cloud functions already do this automatically and seamlessly.
This would have the following advantages:
Would allow firebase developers to use a standard way of assigning jobs to queues, monitor progress, failure, etc.
An app could assign a job to a queue and not care if it was being handled by a cloud function or a different environment.
A developer could take an existing queue and move it from their node server to cloud functions without changing the client app.
It could even allow task chaining where one task could be handled by a cloud function and another by a different server within the same job.
I don't think it's as simple as saying it replaces firebase-queue. Ideally, you would use Functions for most of the common use cases for firebase-queue.
However, there are probably also some legitimate reasons to use firebase-queue anyway.
It's a pretty robust queuing system that allows you to use multi-state tasks and allows you to set the number of workers.
It runs in an environment you specify instead of App Engine (you could install binaries on your server that aren't available in GAE).
Could be cheaper depending on specifics of bandwidth vs invocations.
It can contact third party endpoints for free, while Functions requires a paid account for these.
Functions are still in Beta and don't offer an SLA, they also currently have some startup latencies that wouldn't be experienced in firebase-queue (this point should become invalid around GA release time).
We have an application that is running on Heroku and uses Firebase for data. There are some instances where we have to do some logic that require multiple nested Firebase calls (if method 1 succeeds, create object with method 2) and so on. Unfortunately, they seem to be a little more in-depth than what a transaction method on the client side could accomodate.
We are relying on the REST api to do these functions server-side, but have run into issues when stress-testing that they are frequently timing out (especially when issuing a PATCH request).
However, if we run the same test on a different server with the same code and Firebase database, they succeed no problem.
Is there any throttling going on for multiple requests from the same environment?
I am having a problem under the following circumstances:
1. Running a webapp with both firebase sdk and angularfire initiated connections
2. App has been running in background. Meaning user has launched other apps. During that time, internet connectivity could have come and gone.
3. User brings webapp back into foreground as active app. Firebase connectivity via angularfire has been lost and updates have been lost as well. Hence "2way" binds to existing views/controllers are not longer current and will not re-establish/sync. Only recovery is to actively kill the app and restart.
This is obviously undesirable. I have currently attempted a bruit force method where I am issuing a goOnline() call at the beginning of each of my controllers in an attempt to always try to re-establish a connection but for services that expect a 2way bind, I am not sure that everything will sync up?
Any thought or guidance on this would be very helpful as this is a serious issue.