I'm using the firebase realtime database for my Flutter app,
I don't see anything (in the documentation) about costs when you write data to the database.
So if I do the following.
dataRef = FirebaseDatabase.instance.reference().child('myChild');
await dataRef.update({
't': 'test',
});
I can do it for example 100 times without getting billed (also in the admin SDK?).
Except the SSL handshake (only once per session) and the data storage cost (5 dollars/month).
Is this correct?
Thanks in advance!
You don't get billed per write and read. In real time database you get billed based on the gigabytes stored in the database and based on the data downloaded.
Firebase bills for the data you store in your database and all outbound network traffic at the session layer (layer 5) of the OSI model. Storage is billed at $5 for each GB/month, evaluated daily. Outbound traffic includes connection and encryption overhead from all database operations and data downloaded through database reads. Both database reads and writes can lead to connection costs on your bill. All traffic to and from your database, including operations denied by security rules, leads to billable costs
https://firebase.google.com/docs/database/usage/billing
Related
i've been searching for what is the concurrent users limit for the cloud firestore spark plan but couldn't find it.
https://firebase.google.com/docs/firestore/quotas
It did said 1.000.000 concurrent users limit, but did not mention whether it is for the spark plan or the blaze plan. I've also tried searching answer elswhere, but did not find it answered specifically (with a source).
Help would be appreciated, thank you.
Per the Cloud Firestore pricing information (which Firebase uses):
When you use Firestore, you are charged for the following:
The number of documents you read, write, and delete.
The amount of storage that your database uses, including overhead for metadata and indexes.
The amount of network bandwidth that you use.
There is also no mention of any connection limits on Firebase's pricing page or the quotas documentation that you linked.
Unlike the Realtime Database, Cloud Firestore does not charge on a per-connection basis.
This video series also covers the ins and outs of Firebase products and is well worth sitting through.
Think of Cloud Firestore like a folder on your computer, which can contain thousands of little text files, similar to how documents in Cloud Firestore are stored. Users can update them with little chance of collision and grabbing a single document file would only require feeding 1s and 0s back to the requestor. This is why you are charged for network bandwidth rather than by individual connection.
In comparison, the RTDB was similar to one large JSON text file, with many people all trying to update it at once. Because parsing this text file on the server side was required to read and write data from it, it required compute resources to be able to do so. For this reason (among others), the number of connections the RTDB manager processes handled on behalf of spark plans were rate-limited to prevent abuse.
I want to use firestore in my app due to the scaling limit being 1 million concurrent connections. I have found the pricing to be quite high especially when compared with the real time database, but cannot use this as it only scales to around 200k.
I was wondering whether I could use firestore which will be directly accessed on the client side for some of my data that will need live document listeners and use the realtime data for storing larger chunks of data which will be queried indirectly using firebase functions.
My question is:
if the only way to read/write the realtime database is through a cloud function which is called by the client side, will this only count as 1 concurrent connection as the client side is not directly connected to it?
Thank you
but cannot use [Realtime Database] as it only scales to around 200k.
Keep in mind that this is per database instance. On a paid project, you can create additional database instances to scale much further (even beyond the 1m concurrents that Firestore supports), as long as you are able/willing to define how to distribute your users over the database instances (commonly referred to as a "sharding strategy").
On your actual question: each Cloud Functions instance counts as a single connection to the database. Keep in mind here that Cloud Functions auto-scale, so you will have as many connections from Cloud Functions as you have concurrently running Cloud Functions instances. So while it may well be more than a single connection, it is extremely unlikely you'll reach the limit of 200K connections through this means.
I am a Google blaze plan user and I have an express server containing a simple endpoint that just pulls from firestore. During high traffic hours, I can retrieve 5000+ simultaneous read requests which eventually throws this error below,
Error: 8 RESOURCE_EXHAUSTED: Quota exceeded.
After I wait a few minutes I am able to read the collection again.
Update:
Unsure why the downvotes without any explainations.. but I also have a Mutex system which I think may be leading to hitting these limits. If fails to lock a document using transactions, it goes down an array of snapshots until a lock is given. If the array becomes empty, it does another read to firebase for another set of N documents, which is only 50 in my case.
So my question is, is there a limit on the amount of transactions or reads per second that we're allowed to do from a single connection (my express server)? I don't think it's stated anywhere in the documents.
It looks like you're reaching one of the read/write/transaction limits stated in this page.
Might be this one Maximum writes per second per database: 10,000 (up to 10 MiB per second) but I'm only guessing...
To answer your question, according to this link, the maximum concurrent connections for mobile/web clients per Firebase database is 1,000,000. Thus, your connections seem to not exceed the limits.
For the Blaze Plan project, the limit for Cloud Firestore Document Read is 50K/day, since free usage from Spark Plan is included in Blaze Plan. The limit is such, unless you have set any budget limit in your Billing account. The usage will be reset at midnight of PST. If you upgrade your plan to Flame Plan, the limit is 250K/day.
Here you may read about the official Cloud Firestore Quotas and limits, such as maximum document reads, maximum size for a document, that can be useful. Furthermore you may monitor your database usage and check your plan's limits from the “Usage” tab in the Firebase console. You can check usage over the current billing period, the last 30 days, or the last 24 hours.
Stackdriver Monitoring is also a practical tool for monitoring document reads/writes/deletes, active connections and snapshot listeners.
A good practice, if you want to avoid unexpected charges on your billing account, would be to create an alerting policy based on the Cloud Firestore metrics, as stated here.
Additionally, you can estimate and verify your monthly costs on the “Blaze Plan” by using this Blaze Plan calculator.
For anyone who runs into this issue in the future, please check your App Engine budget settings under "Application Settings". I set the daily spending limit to avoid unnecessary charges during testing and it slipped my mind. I increased the budget and the error is currently gone.
AWS usually sends me an email when my budget has been exceeded.
When I say "transfer" I am referring only to billable transferred data, which I believe is downloaded data only.
Lets say you're listening to ref('/posts/').onValue() and then you call ref('/posts').once(), does Realtime Database know to transfer no data across the network because the client already has the most up-to-date version of the data?
Now lets say you're listening to ref('/posts/').onValue() and a new update is found. Does Realtime Database transfer the delta or the entire document?
Now lets say you are doing a filter like ref('/posts/').orderByChild('timestamp').limitToLast(10) and your /posts/ ref has 500 entries. Does Realtime Database transfer 10 children to the client or all 500?
Now lets say you register thousands of listeners with Realtime Database, are you billed for the setting and removal of listeners?
Note: I'm not sure if this should be multiple questions or if this shouldn't be posted on Stack Overflow. Such is the uncertainty of a stack newbie.
If you attach multiple listeners to a location at the same time, the data for that location will only be transferred once.
When there is an update to a part of a larger node that you listen to, Firebase tries to send only the delta over the wire. The size of the exact data depends on the total size of the node and the update under it.
If you have an index on the queried property, only the query results will be transferred. If you don't have an index on the property, the Firebase client will log an error, transfer all data at the location and filter client-side.
Note that there are tools to learn about these things:
Use the Firebase Database profiler to learn more about read/write speed, bandwidth, and unindexed queries.
Enable debug logging or check the network tab of your browser to see the exact wire traffic between the client and the database.
I just can not figure out where the megabytes of downloaded data from FireBase RealTime DataBase come from, whereas I'm requesting a specific value in a particular line with a size of 10-20 characters. Values come. Requests for such values were not more than one hundred.
Value Request string
Firebase ref = new Firebase("https://XXXXXXXX.firebaseio.com/");
ref.child("city").child("street").addValueEventListener(new ValueEventListener() {
...
String street = snapshot.getValue().toString();
Perhaps FireBase RealTime DataBase takes into account for downloading data viewing this data in the Developer Console?
From: https://firebase.google.com/docs/database/usage/billing
Outbound traffic includes connection and encryption overhead from all database operations and data downloaded through database reads. Both database reads and writes can lead to connection costs on your bill. All traffic to and from your database, including operations denied by security rules, leads to billable costs.
And:
Firebase console data: Although this isn't usually a significant portion of Realtime Database costs, Firebase charges for data that you read and write from the Firebase console
And:
Protocol overhead: Some additional traffic between the server and clients is necessary to establish and maintain a session. Depending on the underlying protocol, this traffic might include: Firebase Realtime Database's realtime protocol overhead, WebSocket overhead, and HTTP header overhead. Each time a connection is established, this overhead, combined with any SSL encryption overhead, contributes to the connection costs. Although this isn't a lot of bandwidth for a single request, it can be a substantial part of your bill if your payloads are tiny or you make frequent, short connections.
I think my downloaded data comes frome many small writes to the database and the overhead associated with it. I have an IoT application - now I am not sure if Firebase is the right choice for this.