I need to run a function in firebase every 10 seconds (I call an external api and process data).
With a normal cron I cant because it is limited to maximum once a minute. Using a setTimeOut is also not convenient since Functions charges per second of use.
My only idea so far is to use Cloud Tasks. But I don't know if it is the most convenient to use this tool for my purpose.
On Google Cloud Platform, Cloud Tasks is almost certainly the best/only way to get this job done without using other products in a way that they weren't intended. Cloud Functions is not good for the reasons you mentioned. With Cloud Tasks, you will need to have the prior task schedule the next task upon completion, as they will not repeat themselves automatically like cron.
Related
I need a bit of expert advice. I'm using the firebase cloud function to automate few things ( using this brilliant nodejs package "https://github.com/jdgamble555/adv-firestore-functions".
What happens is, it runs on - onWrite trigger, as I understand when onWrite triggers, it executes the function on each new document or childnode within a document being updated or created or deleted. That package has taken care many of things, but my concern is, executing the functions multiple times does any harm? I'm already making sure than if not required do not hit firestore by using condition checks. So (all) the function executes as I can see it in the log not writing/updating firestore db if not required.
I'm worried if all functions execute all the time will I finish my limits quickly. ( right now I'm testing on Firebase emulator), specially when userbase with increase.
Anything can be done to reduce these calls or its normal?
As per the firebase documentation, the first 2 million invocations are free per month, after that 0.40$ for every million invocations. Also, there is a resource limit for each function call. https://cloud.google.com/functions/pricing#free_tier
As of my knowledge and experience, this is normal. Just make sure that your code does not make any infinite function calls & database reads/writes.
I'm also using cloud functions for my social media platform which also uses triggers to execute & write to the database based on conditions. It never went beyond the free quota.
So, I'm making multiplayer mobile game using Xamarin and Firebase. In game there are many moment when I'm letting players decide what to do and send their decision to the server (by putting decision enum in player-specific Firebase database node). Decision is time limited (short time, no longer than 20s).
I set listener to that specific node in my Firebase functions to check if all player decided or player decision comes after time deadline, but I need to deal with case when: some players send their decision in time - sersnmart were will not execute next action, and that one player just will not send his decision (eave game or something) - server won't be poke again to check deadline and invoke functions.
That why I'm looking for something else, I found method for schedule functions using crontab, but the minimal time interval there seems to be minutes, which is way more to long for me.
Second idea includes wait that specific time interval in previous Firebase thread, but it seems too bad way to deal with this.
Which way is best for dynamic invoking short-interval scheduled Firebase functions?
The best way to schedule Cloud Functions to run at a specific time it through the Google Cloud Tasks schedules. See Doug's blog post for a full description of this: How to schedule a Cloud Function to run in the future with Cloud Tasks (to build a Firestore document TTL)
That said, I regularly use setTimeOut in my own Cloud Functions too when I need to delay an operation for a short period of time. Just keep in mind that you pay for the seconds that the function is sleeping, so cost-wise you'll want to trade that time off against what another invocation would cost.
So for now I decided to use setTimeout, free firebase plan seems to limit only functions invoke number, not working plan so this shouldn't be a problem. Depsite this, I'm still waiting for advice from you
I need to schedule certain daily tasks, example payment notifications, payments and other things. My question is if through Cloud Functions I can get this.
An example of a task I need, is to make a daily payment, for 8 months, from Monday to Friday.
Activation of this I can do with a Cloud Function and the payment schedule I want to implement node-schedule. The main reason is because I use Cloud Firestore and it comes in handy in the project to implement the functions of the cloud and the database.
That's why I open the post, to know if it is possible for the Cloud Function to load these tasks in memory and execute them when node-schedule requires it.
Thank you.
Cloud Functions have a maximum execution time of 9 minutes, and you are billed for CPU and memory usage the entire time an instance is running. Using an in-process scheduler like node-schedule isn't possible for long time periods and isn't generally recommended even for shorter ones due to the cost involved.
Instead, you can use scheduled functions to define an arbitrary cron-like repeating job that will execute a function on a set schedule. It should be very possible to establish a "Mon-Fri daily payment" in such a scheduled function.
Let's say I have a Cloud Firebase Function - called by a cron job - that produces 30+ tasks every time it's invoked.
These tasks are quite slow (5 - 6 second each in average) and I can't process them directly in the original because it would time out.
So, the solution would be invoking another "worker" function, once per task, to complete the tasks independently and write the results in a database. So far I can think of three strategies:
Pubsub messages. That would be amazing, but it seems that you can only listen on pubsub messages from within a Cloud Function, not create one. Resorting to external solutions, like having a GAE instance, is not an option for me.
Call the worker http-triggered Firebase Cloud Function from the first one. That won't work, I think, because I would need to wait for a response from the all the invoked worker functions, after they finish and send, and my original Function would time out.
Append tasks to a real time database list, then have a worker function triggered by each database change. The worker has to delete the task from the queue afterwards. That would probably work, but it feels there are a lot of moving parts for a simple problem. For example, what if the worker throws? Another cron to "clean" the db would be needed etc.
Another solution that comes to mind is firebase-queue, but its README explicitly states:
"There may continue to be specific use-cases for firebase-queue,
however if you're looking for a general purpose, scalable queueing
system for Firebase then it is likely that building on top of Google
Cloud Functions for Firebase is the ideal route"
It's not officially supported and they're practically saying that we should use Functions instead (which is what I'm trying to do). I'm a bit nervous on using in prod a library that might be abandoned tomorrow (if it's not already) and would like to avoid going down that route.
Sending Pub/Sub messages from Cloud Functions
Cloud Functions are run in a fairly standard Node.js environment. Given the breadth of the Node/NPM ecosystem, the amount of things you can do in Cloud Functions is quite broad.
it seems that you can only listen on pubsub messages from within a Cloud Function, not create one
You can publish new messages to Pub/Sub topics from within Cloud Functions using the regular Node.js module for Pub/Sub. See the Cloud Pub/Sub documentation for an example.
Triggering new actions from Cloud Functions through Database writes
This is also a fairly common pattern. I usually have my subprocesses/workers clean up after themselves at the same moment they write their result back to the database. This works fine in my simple scenarios, but your mileage may of course vary.
If you're having a concrete cleanup problem, post the code that reproduces the problem and we can have a look at ways to make it more robust.
I'm developing a cloud function that triggers on a database new object and needs to delete that object 8 hours later. Right now I'm using a setTimeout to schedule that operation, but I'm not comfortable with such method, as I know function execution should be fast (60 secs max I read somewhere).
Any idea on how to achieve this in a proper way?
The setTimeout() method is definitely not the way to go, in this case. There is no guarantee that the Cloud Function instance will still be running 8 hours later.
Google doesn't provide a scheduler for Cloud Functions yet, and your best bet would be to create a schedule queue of some sort. When the object is created, add a task to the queue to delete it 8 hours later. Periodically (every minute, say) run a cron job via a cron service that triggers an HTTPS Cloud Function that reads the queue to see if there are any objects to be acted on.
Alternately, if the object has a create time associated with it, you could run an HTTPS Cloud Function periodically (triggered by an external cron job, again) that does a query for expired objects based on their create time and removes them.