I read here endpoint spin-up is supposed to be transparent, which I assume means cold start times should not differ from regular execution times. Is this still the case? We are getting extremely slow and unusable cold start times - around 16 seconds - across all endpoints.
Cold start:
Function execution took 16172 ms, finished with status code: 200
After:Function execution took 1002 ms, finished with status code: 304
Is this expected behaviour and what could be causing it?
UPDATE: The cold start times seem to no longer be an issue with node 8, at least for me. I'll leave my answer below for any individuals curious about keeping their functions warm with a cron task via App Engine. However, there is also a new cron method available that may keep them warm more easily. See the firebase blog for more details about cron and Firebase.
My cold start times have been ridiculous, to the point where the browser will timeout waiting for a request. (like if it's waiting for a Firestore API to complete).
Example
A function that creates a new user account (auth.user().onCreate trigger), then sets up a user profile in firestore.
First Start After Deploy: consistently between 30 and 60 seconds, frequently gives me a "connection error" on the first try when cold (this is after waiting several seconds once Firebase CLI says "Deploy Complete!"
Cold Start: 10 - 20 seconds
When Warm: All of this completes in approximately 400ms.
As you can imagine, not many users will sit around waiting more than a few seconds for an account to be setup. I can't just let this happen in the background either, because it's part of an application process that needs a profile setup to store input data.
My solution was to add "ping" function to all of my API's, and create a cron-like scheduler task to ping each of my functions every minute, using app engine.
Ensure the ping function does something, like access a firestore document, or setup a new user account, and not just respond to the http request.
See this tutorial for app engine scheduling: https://cloud.google.com/appengine/docs/flexible/nodejs/scheduling-jobs-with-cron-yaml
Well it is about resource usage of Cloud Functions I guess, I was there too. While your functions are idle, Cloud Functions also releases its resources, at first call it reassignes those resources and at second call you are fine. I cannot say it is good or not, but that is the case.
if you try to return a value from an async function there won't be any variables in the main function definition (functions.https.onCall) and GCP will think that the function has finished and try to remove resources from it.
Previous breaking code (taking 16 + seconds):
functions.https.onCall((data, context) => {
return asyncFunction()
});
After returning a promise in the function definition the function times are a lot faster (milliseconds) as the function waits for the promise to resolve before removing resources.
functions.https.onCall((data, context) => {
return new Promise((resolve) => {
asyncFunction()
.then((message) => {
resolve(message);
}).catch((error) => {
resolve(error);
});
});
});
Related
Really bizarre that Firebase doesn't seem to work quite like typical Express app. Whatever I write in Express and copy-paste to Firebase Functions I typically get error. There is one that I can't figure out on my own though.
This endpoint is designed to start a function and live long enough to finish even longer task. That request is a webhook (send docs, we will transform them and ping you when it's done to specified another webhook). Very simplified example below:
router.post('/', (req, res) => {
try {
generateZipWithDocuments(data) // on purpose it's not async so request can return freely
res.sendStatus(201)
} catch (error) {
res.send({ error })
}
})
On my local machine it works (both pure Express app and locally emulated Firebase Functions), but in the cloud it has problems and even though I put a cavalcade of console.log() I don't get much information. No error from Firebase.
If generateZipWithDocuments() is not asynchronous res.sendStatus() will be immediately executed after it, and the Cloud Function will be terminated (and the job done by generateZipWithDocuments() will not be completed). See the doc here for more details.
You have two possibilities:
You make it asynchronous and you wait its job is completed before sending the response. You would typically use async/await for that. Note that the maximum execution time for a Cloud Function is 9 minutes.
You delegate the long time execution job to another Cloud Function and, then, you send the response. For delegating the job to another Cloud Function, you should use Pub/Sub. See Pub/Sub triggers, the sample quickstart, and this SO thread for more details on how to implement that. In the Pub/Sub triggered Function, when the job is done you can inform the user via an email, a notification, the update of a Firestore document on which you have set a listener, etc... If generateZipWithDocuments() takes a long time, it is clearly the most user friendly option.
My workflow is quite simple, I have two script, first script is ASYNC and the second is SYNC. In each script I have a loop from 0 to Integer.MAX_VALUE as follow
for(int i=0;i<Integer.MAX_VALUE;i++)
System.out.println("value is "+i);
When I run my process, it starts working and I can see in my log file that it is being filled. But when I want to stop it, I find nothing in my active process instances, neither in completed process or even in aborted. even if I check my data base, I have nothing related to this process in the ProcessInstanceInfo or even ProcessInstanceLog. So weird isn't it? what could be the reason?
The goal from creating this workflow is to see the difference between ASYNC and SYNC tasks, because as I know that ASYNC tasks when they start running, the workflow don't have to wait until this task finish, but what I have is that my task ASYNC is still running and it didn't go to next task. So my second question is can any one give me the difference between ASYNC and SYNC with a good example to learn. I would appreciate if I'll get at least one answer on one of my two questions. thanks
What do you stop? Do you abort the process instance ?
In the scripts you can populate the process variables with kcontext.setVariable("variable_name","variable_value"). This will reflect in DB if you have defined the process variable persistent in the process model.
The tasks, the sync one will return the flow control to the process when is completed. In contrast to the async one, process flow will continue immediately after it sends the async tasks to execute.
Suppose that I have the following firebase function that looks something like this:
functions = require('firebase-functions')
exports.myFunction = functions.https.onRequest((request, response) => {
// Do stuff...
})
After I deploy this function on the web and execute it for the first time it takes it around 10 seconds to finish, but every other execution after the first takes only 2 seconds to finish.
I assume this has something to do with cache.
I would like every execution of my function to run just like the first execution.
Why does this happen and how can I disable this feature?
It's not possible, and it's not really a "cache". Cloud Functions reuses server instances after creating one to service a request. That server instance will continue to run to handle more incoming requests for as long as it likes. There isn't some command or configuration that you can use to shut it down manually.
We're using Firebase for our app that needs to process a some data and then send out a series of e-mails after their data has been decided.
Right now I'm triggering a single handler via CRON (which uses pub/sub) that processes the data and then publishes a series of messages to a different pub/sub topic. That topic in turn has a similar trigger function that goes through a few processes and then sends an single email per execution.
// Triggered by CRON task
const cronPublisher = functions.pubsub.topic('queue-emails').onPublish(async () => {
//processing
...
// Publish to other topic
await Promise.all(
emails.map((email) =>
publisher.queueSendOffer(email)
)
);
});
// Triggered by above, at times twice
const sendEmail = functions.pubsub.topic('send-email').onPublish(async () => {
//processing and send email
});
The issue I'm running into is that the 2nd topic trigger at times is executed more than once, sending two identical emails. The main potential cause I've come across by way of Google just involves long execution times resulting in timeouts, and retries. This shouldn't be the case since our acknowledgment timeout is configured to 300 seconds and the execution times never exceed ~12 seconds.
Also, the Firebase interface doesn't seem to give you any control over how this acknowledgment is sent.
This CRON function runs everyday and the issue only occurs every 4-5 days, but then it duplicates every single email.
Any thoughts?
Appreciated.
If 'every single message' is duplicated, perhaps it is your 'cronPublisher' function that is being called twice? Cloud Pubsub offers at least once semantics, so your job should be tolerant to this https://cloud.google.com/pubsub/docs/subscriber#at-least-once-delivery.
If you were to persist some information in a firebase transaction that this cron event had been received, and check that before publishing, you could prevent duplicate publishing to the "send-email" topic.
I'm puzzled at what we see when running this setup:
FuncA: Google Cloud Function trigger-http
FuncB: Google Cloud Function trigger-topic
FuncA is called by a HTTP client. Upon being called, FuncA does some light work setting up a JSON object describing a task to perform, and stores this JSON into Google Cloud Storage. When the file has been written FuncA published a topic with a pointer to the gs file. At this point FuncA responds to the client and exits. Duration typically 1-2 seconds.
FuncB is informed that a topic has been published and is invoked. The task JSON is picked up and work beings. After processing FuncB stores the result info Firebase Realtime Database. FuncB exits at this point. Duration typically 10-20 seconds.
As FuncA and FuncB are in no way associated, live their individual process lifecycles on different function names (and triggers) and only share communication through pub/sub topic message passing (one-direction from A to B) we would expect that FuncA can run again and again, publishing topics at any rate. FuncB should be triggered and fan-out to scale with what ever pace FuncA is called with.
This is however not what happens.
In the logs we see results following this pattern:
10:00:00.000 FuncA: Function execution started
10:00:02.000 FuncA: Function execution took 2000 ms, finished with status: 'ok'
10:00:02.500 FuncB: Function execution started
10:00:17.500 FuncB: Function execution took 15000 ms, finished with status: 'ok'
10:00:18.000 FuncA: Function execution started
10:00:20.000 FuncA: Function execution took 2000 ms, finished with status: 'ok'
10:00:20.500 FuncB: Function execution started
10:00:35.500 FuncB: Function execution took 15000 ms, finished with status: 'ok'
...
The client calling FuncA clearly gets to wait for both FuncA and FuncB to finish, before being let through with the next request. It is expected that FuncA would finish, and allow a new call in immediately at what ever pace the calling client can "throw at it".
Beefing the client up with more threads only repeats this pattern, such that "paired" calls to FuncA->FuncB always waits for each other.
Dicsuss, clarify, ... stackoverflow, do your magic! :-)
Thanks in advance.