I have a large node in Realtime Database that I want to delete everyday using Cloud Functions schedule.
Is there a limit on how much data I can delete using Cloud Functions on Realtime Database? And where can I find the cost for delete?
I've read the billing doc (link) but I'm not sure where it is mentioned about delete cost.
I'll start by adding this link.
Combining the informations in your link and in this one I've added, the answer is: no, you'll not be billed if you just delete data. The important information here is: if you just delete it. You will still be billed if, before you delete it, you download it. In other words, if you get the reference to a node in your code, and then you just perform a remove(ref), you won't be billed.
There is a remote possibility that you can be billed for a huge CPU consumption. This could happen if the node you're deleting is really big, but you can estimate this by testing it out and checking the "Usage" tab in your Firebase console, under the voice "Load". If the load for a testing delete is low, you're good and you won't be billed.
Related
i've been searching for what is the concurrent users limit for the cloud firestore spark plan but couldn't find it.
https://firebase.google.com/docs/firestore/quotas
It did said 1.000.000 concurrent users limit, but did not mention whether it is for the spark plan or the blaze plan. I've also tried searching answer elswhere, but did not find it answered specifically (with a source).
Help would be appreciated, thank you.
Per the Cloud Firestore pricing information (which Firebase uses):
When you use Firestore, you are charged for the following:
The number of documents you read, write, and delete.
The amount of storage that your database uses, including overhead for metadata and indexes.
The amount of network bandwidth that you use.
There is also no mention of any connection limits on Firebase's pricing page or the quotas documentation that you linked.
Unlike the Realtime Database, Cloud Firestore does not charge on a per-connection basis.
This video series also covers the ins and outs of Firebase products and is well worth sitting through.
Think of Cloud Firestore like a folder on your computer, which can contain thousands of little text files, similar to how documents in Cloud Firestore are stored. Users can update them with little chance of collision and grabbing a single document file would only require feeding 1s and 0s back to the requestor. This is why you are charged for network bandwidth rather than by individual connection.
In comparison, the RTDB was similar to one large JSON text file, with many people all trying to update it at once. Because parsing this text file on the server side was required to read and write data from it, it required compute resources to be able to do so. For this reason (among others), the number of connections the RTDB manager processes handled on behalf of spark plans were rate-limited to prevent abuse.
The firebase documentation includes a warning that if you use a kill switch to stop using firebase when a budget cap is exceeded, as follows:
Warning: This example removes Cloud Billing from your project,
shutting down all resources. Resources might not shut down gracefully,
and might be irretrievably deleted. There is no graceful recovery if
you disable Cloud Billing. You can re-enable Cloud Billing, but there
is no guarantee of service recovery and manual configuration is
required.
I'm trying to investigate what gets irretrievably deleted. Does the datastore get deleted when the kill switch is activated? Is there any opportunity to save data previously stored in cloud firestore, before the deletion takes place? Is there a way to download the database so that I can keep a back up in this scenario?
Please review the following reply from Firebase Team member(samstern) to gain more clarity on this:
these things are handled on a per-product basis and each product has different thresholds for quota overages and different procedures for what happens to inactive resources in a bad state.
For instance I know in Realtime Database if your DB is too big for the
free plan after you downgrade from a paid plan we will not delete
your data automatically. Instead you'll just be stopped from using the
database until you restore your billing.
However that statement clearly says that some products may delete data
if you pull Cloud Billing. It could be for technical reasons, it could
be for policy reasons.
If you want to turn off a product without risking this pulling your
billing account is NOT the best way to do this. It's the nuclear
option if you get into a bad situation and want to get out at all
costs. Instead you should use per-product APIs to shut down the
individual product or service and prevent future usage. This could
include doing things like flipping off APIs in the APIs console,
changing security rules to prevent all further writes, deleting Cloud
Functions, etc
The best source of information I've been able to uncover in answer to this particular question is a discussion on reddit which indicates that you can't recover access to your data, until you pay the bill (including blow out charges) - so maybe that buys some time, but if you don't pay, the project gets deleted. There may also be lost data for things happening at the time the kill switch activates.
Hey so with my current feed database design, I am using Redis for the cache for super-fast reads, which are routed through my Google Cloud Functions. The Redis database handles all post data and timeline updates, which is great and all, but I forgot one of the most considerable caveats to this. Firebase Firestore only permits one document write per second, meaning that if I have a document that stores the post data (post_id, user_id, content, like_count), the like_count would be impossible to track with the possibility for many likes per second. Does anyone have any solutions to this?
You can shard your counter among multiple documents and query them in aggregate as needed.
You can also try Cloud Tasks queue to smooth out the write frequency. It will add considerable complexity to the system, but is really the only genericized way in GCP to manage the rate of some work. This might not work out the way you need, however.
If you use Cloud Tasks, your task will need to be configured with a rate limit, and it will have to deliver the document data to write to yet another function or other HTTP endpoint that will perform the write.
I am looking for clarification on how the actions triggered by cloud functions affect the cost of hosting an app with Firebase.
My situation:
I have a cloud function that is triggered when a post is made...that cloud function writes that post to all of the appropriate uid's on the /feed node mentioned below.
I am using Firestore to host a feed/follow system. It is setup with each user having their own feed at the following path
/feed/{uid}/posts
My question is, if a user has 1,000,000 followers...does that translate to 1,000,000 writes in the eyes of Firestore?
Does that mean that single post will cost $1.80 to distribute? Based on the cost of $0.18/100k writes.
I am just trying to see how this will scale with thousands of users posting dozens of posts...those distribution costs seem very expensive at scale for this situation.
EDIT
It looks like it does cost for every read/write in a cloud function base on the following video:
https://www.youtube.com/watch?time_continue=49&v=6NegFl9p_sE&feature=emb_logo
It depends on how you are writing to Firestore, but bottom line is that yes, you get billed for each read/write operation performed to Firestore as mentioned over at their documentation.
Additionally, you get billed per Cloud Function invocation as mentioned here, so you would need to keep this in mind if you want to keep your expenses low.
Hope you find this useful.
My goal is to have a firebase cloud function track the upload of three separate files to the same storage bucket. These uploads are preceded by a write to the real time database which would preferably be the trigger for the cloud function to track the uploads.
The context is a user is adding an item to her shopping cart. The data is written to the RTDB and then a custom 3d model and 2 images are copied into a storage bucket. If any of these files don't successfully upload, I need to know that and conduct a rollback of the 3 files in the storage bucket and also remove the entry in the database. I could handle this client side, but that isn't ideal since usually if the uploads fail, its because the connection with the client has failed.
I haven't been able to find any sort of batch add or transaction-type uploads to firebase storage. Sorry for not having any code to show, but I'm not even really sure how to get started on this. Any suggestions would be much appreciated. Thanks!
There are no transactions that cross products like this. Nor are there any transactions offered by Cloud Storage. You're going to have to check errors and manually undo things previously done. Or, have some job that checks for orphaned data and deletes it later.