Our goal is to set a minimum instance count for our firebase cloud function to be able to handle unexpected bursts of traffic. We've followed these instructions to set a minInstance count of 3. After deploying the update, I can see in the UpdateFunction log that minInstance value is set to 3.
However, the Active instances count in the cloud function dashboard regularly falls below 3 (see image below). Does "active" in this metric only count instances which are being executed and doesn't count idling instances? Or does this actually mean that the minInstance value is being ignored?
As Cloud Functions is stateless it may initialize the execution environment from scratch which is called Cold Start. Cold Starts may take a significant amount of time and could increase the application latency. To reduce the number of Cold Starts, Cloud Functions for Firebase allows setting a minimum number of instances by specifying minInstances by following this document. This will keep the specified number of instances ready or warm to serve the requests which will not go through cold starts.
Now in the Cloud Functions page of Google Cloud Console, the metrics Active Instances means the number of instances that are currently serving the request as mentioned here.
Setting up minInstances does not mean that there will always be that much number of Active Instances. Minimum instances are kept running idle (without CPU allocated), so are not counted in Active Instances.
Related
To minimize cold starts, I've set a minimum instance for my Google Cloud Function. I actually do it with the firebase admin SDK like this:
functions.runWith({ minInstances: 1 })
...but I can see it confirmed in Google Cloud Console:
I'm noticing that after every deployment, I still encounter one cold start. I would have assumed that one instance would be primed and ready for the first request, but that doesn't seem to be the case. For example, here are the logs:
You can see that ~16 hours after deployment, the first request comes in. It's a cold start that takes 8139ms. The next request comes in about another hour later, but there's no cold start and the request takes 556ms, significantly faster than the first request.
So is this the expected behaviour? Do we still encounter one cold start even if minimum instances is set? Should I then be priming the cloud function after every deployment with a dummy request to prevent my users from encountering this first cold start?
Tl;dr: The first execution of a function that has minimum instances set is not technically a cold start, but probably will be slower than later executions of that instance.
Minimum instances of a function will immediately be "warmed up" on deploy and in a warm but idle state, ready to respond to a request. However, the functions we write often need to do extra setup work when they're actually triggered for the first time.
For example, we may use dynamic imports to pull in a library or need to set up a connection to a remote DB. Even though the function instance is warm, the extra work that has to be done on the first execution means that it will probably be slower than later executions.
The benefit of the minimum instances setting is that later executions benefit from all the setup work done by the first execution, and can be much faster than if they were scaled back to zero and had to set themselves up all over again on the next request.
Update: Occasionally, an idle instance may be killed by the Cloud Functions backend. If this happens, another instance will be spun up immediately to meet the required minimum instances setting, but that new instance will need to go through its extra setup work again the first time it is triggered. However, this really shouldn't happen often.
The documentation does not make a hard guarantee about the behavior (emphasis mine):
To minimize the impact of cold starts, Cloud Functions attempts to
keep function instances idle for an unspecified amount of time after
handling a request.
So, there is an attempt (no guarantee), and it kicks in after a handling a request (not after deployment), but you don't know how long that will last. As stated, it sounds like you might want to make a request, along with the expectation that it might still not always work exactly the way you want.
Like many others, I'm struggling with cold starts for most of my cloud functions. I did all the optimization steps that the documentation recommended, but because almost all of my functions are using the admin SDK, and Firestore, a typical cold start takes around 5 seconds, meaning that the user has to wait at least 5 seconds for the data.
I'm thinking on replacing all my http trigger cloud functions with a single one, like /api/* and set it up to have some amount of minInstances.
What are the cons of this approach? What is the point of having multiple http functions in the first place?
TL;DR; Why shouldn't I use a single http cloud function for my REST like api to handle all the resources in order to eliminate cold starts?
Why shouldn't I use a single http cloud function for my REST like api to handle all the resources in order to eliminate cold starts?
Because this won't eliminate cold starts. In fact, you will be loading a bigger function every time you invoke your /api/* root function.
As in the example from this video, the point of having multiple functions is to instance them as needed, and also to be isolated between them.
To reduce cold starts, you could set
a minimum number of instances that Cloud Functions must keep ready to serve requests. Setting a minimum number of instances reduces cold starts of your application.
See also:
Cloud Functions Cold Boot Time (Cloud Performance Atlas)
Firebase has announced in September 2021 that it is possible now to configure its cloud function autoscaling in a way, so that a certain number of instances will always be running (https://firebase.google.com/docs/functions/manage-functions#min-max-instances).
I have tried to set this up, but I can not get it to work:
At first I have set the number of minimum instances in Google Cloud Console: Cloud Console Screenshot
After doing this I expected that one instance for that cloud function would run at any time. The metrics of that function indicate that it instances were still scaled down to 0: Cloud functions "Active Instances Metric"
So to me it looks a bit as if my setting is ignored here. Am I missing anything? Google Cloud Console shows me that the number of minimum instances has been set to 1 so it seems to know about it but to ignore it. Is this feature only available in certain regions?
I have also tried to set the number of minimum instances using the Firebase SDK for Cloud Functions (https://www.npmjs.com/package/firebase-functions). This gave me the same result, my setting is still ignored.
According to the Documentation, the Active Instances metrics shows the number of instances that are currently handling the request.
As stated in the Documentation :
Cloud Functions scales by creating new instances of your function. Each of these instances can handle only one request at a time, so large spikes in request
volume often causes longer wait times as new instances are created to handle the demand.
Because functions are stateless, your function sometimes initializes the execution environment from scratch, which is called a cold start. Cold starts can take
significant amounts of time to complete, so we recommend setting a minimum number of Cloud Functions instances if your application is latency-sensitive.
You can also refer to the Stackoverflow thread where it has been mentioned that
Setting up minInstances does not mean that there will always be that much number of Active Instances. Minimum instances are kept running idle (without CPU > allocated), so are not counted in Active Instances.
I am developing a new React web site using Firebase hosting and firebase functions.
I am using a MySQL database (SQL required for heavy data reporting) in GCP Cloud Sql and GCP Secret Manager to house the database username/password.
The Firebase functions are used to pull data from the database and send the results back to the React app.
In my local emulator everything works correctly and its responsive.
When its deployed to Firebase Im noticing the 1st and sometimes the 2nd request to a function takes about 6 seconds to respond. After that they respond less than 1 sec. For the slow responses I can see in the logs the database pool is initialized.
So the slow responses are the first hit to the instance. Im assuming in my case two instances are being created.
Note that the functions that do not require a database respond quickly regardless of it being the 1st or 2nd call.
After about 15 minutes of not using a service I have the same problem. Im assuming the instances are being reclaimed and a new instance is being created.
The problem is each function will have its own independent db pool so each function will initially provide a slow response (maybe twice for the second call).
The site will see low traffic meaning most users will experience this slow response.
By removing the reference to Secret Manager and hard coding username/password the response has dropped to less than 3 seconds. But this is still not acceptable.
Is there a way to:
Increase the time that a function is reclaimed if not used?
Tag an instance that it should not be reclaimed?
Is there a way to create a global db pool that does not get shutdown between recycles?
Is there an approach to work with db connections in Firebase Functions to avoid reinit of the db pool?
Is this the nature of functions and Im limited to this behavior?
Since I am in early development, would moving to AppEngine/Node.js (the Flexible Plan) resolve recycling issues?
First of all, the issues you have been experiencing with the 1st and the 2nd requests taking the longest time are called cold starts.
This totally makes sense because new instances are spun up. You may have a cold start when:
Your function has been deployed but not yet triggered.
Your function has been idle(not processing requests) enough that it has been recycled for resources.
Your function is auto-scaling to handle capacity and creating new instances.
I understand that your five questions are intended to work around the issue of Cloud Functions recycling the instances.
The straight answer from questions 1 to 4 is No because Cloud Functions implement the serverless paradigm.
This means that one function invocation should not rely on in-memory state(database pool) set by a previous invocation.
Now this does not mean that you cannot improve the cold start boot time.
Generally the number one contributor of cold start boot time is the number of dependencies.
This video from the Google Cloud Tech channel exactly goes over the issue you have been experiencing and describes in more detail the practices implemented to tune up Cloud Functions.
If after going through the best practices from the video, your coldstart shows unacceptable values, then, as you have already suggested, you would need to use a product that allows you to have a minimum set of instances spun up like App Engine Standard.
You can further improve the readiness of the App Engine Standard instances by later on implementing warm up requests.
Warmup requests load your app's code into a new instance before any live requests reach that instance. The last document talks about loading requests which is similar to cold starts in which it is the time when your app's code is being loaded to a newly created instance.
I hope you find this useful.
I have a Firebase website project, right now everything looks good except for the Firebase Cloud Functions generated pages flagged with reduce server response time on pagespeed insights. Insights would result 0.5 - 1.5 seconds.
Static pages get a score of 99 on mobile and 86 on desktop. With the cloud functions generated pages however the highest I get for mobile is 75 and lowest drops to as low as 30.
Here is some background on what is happening:
The page being requested is identified to ready the values needed for the Firestore queries.
4 Firestore queries (3 collections with <100 documents and 1 with <5000 documents, which are very small).
Data from the queries are organized and prepped for render.
Page renders with res.render()
Original Build:
Cloud Functions
Express
Cors
Handlebars (as the template engine)
This build would usually get me a score of 60-75. Server response time is still high but it is consistent within the range 0.5 - 1.5
Alternative Build:
This build I use the same code for the queries except for the template engine rendering step.
Cloud Functions
Express
Cors
PUG (as the template engine)
This build gives both the best and the worst scores. I don't get a medium score on this of 60-75, The highest I got is a perfect 100 and the lowest is a very low 30. The bad thing about this is that it is very inconsistent and I can't determine the reason.
Is it a Cold Start problem?
I have read about cold start so I tried experimenting if it was the problem.
First Attempt
I made a function that would ping the functions used by my Cloud Functions. I started it up and put it on a loop every 10seconds on my desktop. After a few minutes I checked Pagespeed if it had any effects, no improvements.
Second Attempt
I made a function that would fetch the page I am testing on Pagespeed. I started it up and put it on a loop also on my desktop. I checked Pagespeed and it also did not have any effects.
So I guess I should cross out the Cold Start issue? Or am I handling the cold start issue wrong? Or maybe its a templating thing since using two different template engines have different results?
Anyway, this is what I have done so far, any idea on how to reduce the server response time of Firebase Cloud Functions generated pages?
Cold start might be your problem.
If you keep one Cloud Function warm it will keep a container with that function warm - so that the next user can come and use the function in a warm state.
However - if you have multiple users attempting to use the cloud function it will allocate one warm container to a connection, then another couple of cold containers to the other users. It's how they scale - building out more containers.
I'm assuming whats happening is your tests are sometimes connecting to the warmed container - and when they are in use the tests are having to wait for a cold start on the other containers.
Read up on Cold Starts
A new function instance is started in two cases:
When a new function instance is automatically created to scale up to the load, or occasionally to replace an existing instance.