Currently, I'm developing app with Firebase(Firestore, Firebase functions, Storage, etc)
I have one pubsub job which runs monthly(This job iterates over thousands of items and each item takes 1-3 seconds to run.)
Therefore, firebase functions timeout in 540 seconds(max limit for firebase functions).
As I researched, i need to move to Cloud App Engine(or something like that) or split into multiple jobs(But not sure how to split it).
Could you share how you achived this problem?
Thank you.
The timeout cannot be increased. You can switch to Cloud Functions Gen 2 that has a 1 hour max timeout instead of 9 minutes. If that is insufficient then it would be best to use something like Cloud Compute.
Related
We're trying to fetch 86k of our firebase users, on local & firebase functions it takes 2 minutes for all, but in cloud run it is taking on average 20 seconds per call (you can only request 1k users per calls according to firebase docs).
Interestingly, get all firebase real time database uses takes 15s, but in cloud run it took 365s.
2022-06-17T00:03:04.986000061Zgrabbed users data from db, total: 86442 in 364.015s
2022-06-17T00:03:05.732000112ZProgress 1000 0.746s
2022-06-17T00:03:15.131999969ZProgress 2000 9.847s
2022-06-17T00:03:39.332999944ZProgress 3000 34.347s
2022-06-17T00:04:03.832999944ZProgress 4000 58.846s
2022-06-17T00:04:28.433000087ZProgress 5000 83.447s
2022-06-17T00:04:51.733000040ZProgress 6000 106.747s
2022-06-17T00:05:58.332000017ZProgress 7000 172.947s
Any thoughts on how to solve this? No special network settings in place on cloud run.
Background Info:
Cloud run instance is using NodeJS 14. 2GB Memory which stays at 8% usage. CPU usage stays around 10%. The user object is relatively small, but across all these users it's about 60-70 MB. In firebase functions, only 256 MB of memory are required to do the fetching.
PS: I've yet to test if region makes a difference, as cloud run is in us-east1 and functions are in us-central1. Will be testing soon.
Is there any pricing information regarding deploying cloud functions with minInstances greater than 0? Also, when I deployed the function with runWith({ minInstances: 1}), the edit function page at console.cloud.google.com does not reflect the change.
You can find pricing info on the Firebase Pricing Page. Note this info to figure out the cost of minimum instances
kept idle refers to idle billable time of instances kept warm using minimum instances.
You can also get a quote when you deploy:
Change your function to use minInstances
export const example = functions.https.onCall( ...
export const example = functions.runWith({minInstances: 1}).https.onCall( ...
Deploy via firebase deploy --only functions
The command line should prompt you with a quote and confirmation input such as:
functions: The following functions have reserved minimum instances. This will reduce the frequency of cold starts but increases the minimum cost. You will be charged for the memory allocation and a fraction of the CPU allocation of instances while they are idle.
With these options, your minimum bill will be $153.75 in a 30-day month
? Would you like to proceed with deployment? (y/N)
Also a tip: I use cloud scheduler to keep my functions warm as it's a small fraction of the cost and works just as well for my use case.
I am trying to call an api every minute for ski lift status and check for changes. I am going to store the value of if the lift is open or closed in firebase (Real Time Database) and read to see if value from api is different and only update/ write to that node when it's a different value. Then I can set up a cloud function that will listen for database changes and send push notifications to the list of FCM tokens from that channel. I am not sure if this is the most efficient way, but I was going to set up scheduled functions to call the third party api.
I have been using these docs:
https://firebase.google.com/docs/functions/schedule-functions
I was planning to do something like this:
exports.scheduledFunction = functions.pubsub.schedule('every 5 minutes').onRun((context) => {
CALL MY API IN HERE AND UPDATE DATABASE IF SNAPSHOT BACK IS DIFFERENT
});
I was wondering how would I run only between set times- say 8am-6pm EST. I am struggling to find anything about times to run. Should I just run the function every minute and then pause and resume by checking the time? In which case how does it know to keep checking the time when it is paused?
Firebase scheduled functions use Cloud Scheduler to implement the schedule. It accepts cron style time specifiers to indicate when a job should be run. The full spec for that can be found here. You will have to use ranges of numbers to indicate the valid times and frequency of the schedule. For example, you might use "8-18" in the hour field to limit the hours of execution.
I am making a spam counter ( on Firebase ). What I do is I use database trigger on firebase cloud functions to increment a path (/counter/${uid}). This path will hold an integer for each user that other path will have a security rule that reference to it and check whether it exceed the limit. However, I would like to clear the counter once a day.
When I search on google I found official way of firebase to do this by using another Google cloud service to deploy cron job. However, I wonder if I use setInterval on cloud function instead would work. This task would only be a one line execution ( admin.database().ref('/counter').set({}) . And it is not so serious that if it were to skip once or twice of the execution due to some problem, it should be ok.
Thanks
The use of setInterval won't work, and it's not really ever recommended to do so. You can use setInterval to keep a function alive for some amount of time, but you will be paying for that time even if the function is just waiting. Also you are still subject to the way Cloud Functions will time out your function (default 60 seconds, max 9 minutes by special configuration).
Don't use setInterval as you'll be paying for un-used compute time.
Instead see this video on YouTube; https://www.youtube.com/watch?v=CbE2PzvAMxA
They go into detail of how to setup a free-schedule service and setup a HTTP trigger that should achieve the result you're after.
Is it possible on Firebase or Parse to set up something kinda like a cron job?
Is there a way to set up some sort of timed operation that runs over the stored user data?
For example, I'm writing a program that allows people to RSVP for lunch everyday. If you have RSVPed by noon, then you get paired up with somebody else who has also RSVPed. Using JavaScript, the user can submit their RSVP in the browser.
The question is, can Firebase/Parse execute the code to match everyone at 12:00pm every day?
Yes, this can be done with Parse. You'll need to write your matching function as a background job in cloud code, and then you'll need to schedule the task in the dashboard. In terms of the flexibility in scheduling, it's not as flexible as cron but you can definitely run a task at the same time every day, or every x minutes/hours.
Tasks can take 15 mins max to execute before they're killed, so depending on the size of your database or the complexity of your task, you may need to break it up into different tasks or make it resumable.
Just to confirm about Firebase:
As #rickerbh said, it can be done with Parse, but currently there is no way for you to run your code on Firebase's server. There are 2 options for you 2 solve this:
You could use Firebase Queue and run your code in Node.js
You could use a different library such as Microsoft Azure (I still haven't tried this yet, I'm not sure if it provides Job Scheduling for Android)
However, Firebase is working on something called Firebase Trigger, which will solve our problem, however it is still not released with no confirmed release date.