Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I am aware for Cloud Firestore a read is a document (whether the documents has 5 or 50 nodes). How does this compare to the RTDB?
If I have a query that has a limit of 25, is this going to be 25 reads, or 25 times x amount of items in each node?
Cheers.
Your question is a bit of a non-sequitur, as realtime database doesn't bill by reads, it bills by data transferred (and storage, of course). So, the thing that affects your cost is the size of the items transferred, which is only indirectly based on the number of items due to a limit on the query. Currently, the costs are about US $1 per GB downloaded assuming you are on the Blaze plan.
To compare this with the costs for Firestore would require knowing a lot more about the shape of your traffic -- how many reads and writes, average size of a read, etc. Note that Cloud Firestore also indirectly charges for data transferred, but at a much lower rate, as it is only the Google Cloud Network pricing.
This means that you can generally get quite a large number of Firestore document reads for the cost that RTDB charges for transferring 1 GB.. (e.g. at current prices for egress to most of the internet excluding some asia/pacific destinations, you could get 1 GB + over 1.4M firestore document reads for your $1 of 1 GB RTDB transfer).
The documentation references several things you can do to help control costs, including (but not limited to):
Prefer the native SDKs to the REST API
Monitoring your data usage and use the profiler tool to measure read operations.
Use fewer, longer lived connections, as SSL and connection overhead can contribute to your costs (but generally are not the bulk of your cost).
Ensure your listeners are limited to only the data you care about, and are as low in the database tree as possible, and only download updates, (e.g. on() vs once()).
Related
It is well known that read/write cost of firebase rtdb is free. With little bit more digging, i could've find out that read/write can actually cost in non-direct ways. Ok, so i've been searching through docs and SO questions to figure out of "what is the exact difference between FIRESTORE READING COST($0.06 per 100,000 documents) and REALTIME DATABASE DOWNLOAD($1/GB) COST", but sadly i couldn't have managed to complete it.
Stored data cost for RTDB($5/GB) is really clear, and i understand that the price billed monthly(this one would be true, right?). But what is exactly a DOWNLOAD cost? Through a few SO questions and official docs, i could've figure out that rtdb download cost is really similar to firestore reading cost, and it is important to specify db.ref path clearly by diving into the final path. But, if the download cost is all about these operations, such as reading json data in a specific field or path, what is difference between concept of firestore reading and concept of these rtdb download operations?
If all these things are already happening in the Earth, the cost of RTDB when it comes to 'conceptual reading' is never free, even if we speak in direct manner. Then why some community members and articles always say "read/write cost for RTDB is free"? I was considering migration of some features from firestore to RTDB since it is well-known that rtdb is free for read and write. The feature is updating a single path(document for firestore) of 500B size hundreds time every month. But this issue makes me really confusing.
Let's say that 100,000 read for firestore is $0.04 and download for RTDB(which seems like reading) is $1/GB. In my calculation, 2,500,000 document reads from firestore is equal to a single GB download from RTDB. It means that if a single operation reads bunch of data larger than 400B(approx.), firestore read-cost is even cheaper than RTDB read-cost. Then there is no reason for me to use RTDB for reading data if single operation needs to retrieve data larger than 400B per operation. It feels like i've got caught by wrong concepts, but it is not easy to get out of this swamp.. ]:
So i hope to make clear of RTDB read/write cost(if it is really free of charge by itself), and the reason why it is better to use RTDB than firestore, when the app have to do lots of read operations(for me, ex. approx. 1,000 operations retrieving 400B-size data per month per a single user). I understand that a few firebase gurus are thankfully contributing SO's firebase tag. I've tried to write the question as clear as possible, but think there would be some unclear parts in the question. So, comments will be really appreciated! Hope this question would reach to you.. Thanks in advance [:
I have created a very handy spreadsheet calculator that calculates the rough size of the payload and scales per user while also factoring in the free tier usage as well. You can enter your values at the top and get a decent result.
But to summarize, Realtime DB is highly expensive to read per KB while Firestore is rated for up to 1mb (potential) per read while writing to Realtime is extremely cheap, I have confirmed that besides overhead, it is free to write to realtime db.
Realtime db is not as economical compared to Firestore and is designed to cover some caveats of Firestore. Realtime Billing for reads (download) is the (data + overhead) rounded up to nearest kb
TLDR:
Firestore is ideal for high reads, low writes, static information.
Realtime is better suited with low reads, high writes, volatile information.
When reading documents from Firestore you pay for:
Document reads - The cost to read the document on the server.
Network egress - The cost to download the data to the client.
In most scenarios we see the cost for developers using Firestore coming more from document reads, as the cost per GB is comparatively low.
When reading data from the Realtime Database, you only pay for:
GB downloaded - The cost to download the data to the client.
Here the cost mostly comes from the size of the data you download. It's quite similar to the Network egress from Firestore, but at a higher cost per byte read (and of course you then don't pay for the read operation on the server itself).
While a calculator (such as the one from DIGI Byte, or the one on the pricing page) is going to be best, the rough guidance is that if you perform many small reads and writes, RTDB is going to a better choice, while if you perform fewer writes and/or more larger reads, then Firestore is often the better choice.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I watched this video on data modelling in Cosmos DB.
In the video, it is explained that if you can model your data such that your most common queries are in partition queries, then you can minimize RUs, which in turn minimizes cost and maximizes performance.
The example used in the video is a blogging system. They showed that by moving things around such that Blog Posts and Comments are stored as separate entities in the same collection all partitioned by blogId they could achieve a low RU for a common query.
They then showed that searching for all blog posts by a specific user, being a cross partition query, is very expensive. So they then duplicate all blog post data and add each blog post as a separate entity to the users collection, which is already partitioned by userId. Searching for posts by a user is now cheap. The argument is storage is much cheaper than CPU time so this is a fine thing to do.
My question is: do I continue to follow this pattern when I want to make more things efficiently searchable? For example, I want to be able to search on blog topic (of which there could be many per blog post), a discrete blog rating, and so on.
I feel like extending this pattern for each search term is unsustainable. In these cases, do I just have to live with high RU searches or is there some clever way of making things efficient?
The essentially comes down to knowing whether the cost of using change feed to copy data from one container to another is less than the cost of doing cross-partition queries. This requires knowing the access patterns of your application and also requires measuring the average cost of these queries versus the cost of using change feed to make another copy. Change feed consumes 2 RU/s when it polls the container, then 1 RU for each 1Kb or less read from the source container and ~8 RU for each 1Kb or less insert on target container depending on your index policy. Multiply that by the rate at which data is inserted or updated. Then calculate this per day or per month to compare cost.
If what you're looking for is to do free text search on your data, you may want to look at using Azure Search. This is simpler than using the approach using change feed, but Azure Search can be quite expensive as well.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
EDIT: Basically do you get charge for the documents you query. The answers is no, if you query 100K documents but only get back 10 you only get charge for the 10 documents you get form your query.
Hi so I have been using firebase firestore and it's been great. However, I have some questions about it. Currently, I'm working on an app where users can place orders and by default, the 'isActive' property is true so that the admin can see the orders. When the order is completed the property turns false and doesn't show up. However, eventually, I will accumulate thousands of orders and my question is will I get charged for the documents that I read that is true or it will counts as reading all thousands of documents even though I don't use it.
You will most definitely be charged for stored data regardless of if you access it your not. If you do decide to access it you'll be charged for that as well. Fortunately you have 20K free reads a day and then you pay after that.
While I think that answers your question. If your app reads in 20,000 documents in one action (open a page and then it loads in everything) that's not going to scale well for you and that's a lot of data. Which means your client devices will have to process 20K documents which is not ideal.
I'd advise using limit and then do pagination or infinite scrolling.
It's fairly simple: if your document is read from/on the server, you will be charged for a document read.
If your clients are not requesting documents with isActive is false, they won't generate document read charges for those documents. I your admin is reading those documents, they will generate read charges for those documents.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I'm doing a fair bit of work on a set of Firestore collections and documents. It amounts to a good amount of writes and reads, as I'm setting two-way refs and whatnot. Multiple documents are being written to multiple times.
Since Firestore offers offline capability, is it possible to reduce the number of writes via preparing all the data locally, then sending it to the server?
I'm using the Admin SDK.
It depends on what you mean. One document write is always going to cost one document write, no matter when or how that document was written. Batch writes don't in any way reduce the number of documents written, they just make all the document writes take effect at the exact same moment in time.
If you're staging lots of changes to a single document to take effect later, then feel free to do that. Just write the document whenever you've figured out what final document looks like, and no sooner.
I'am moving away from google appengine standard Python 2.7 NDB to Svelte, Firestore and RxFire.
I was able to dramatically reduce the number of reads and writes by batching hundreds of appengine NDB entities (datastore / data objects) into a single document using a data object map.
Every data object has a batchId prop to optimize (batched) batch writes / document writes. (batchId = docId)
Most of the querying is now done in the client using filters. This resulted in very simple reactive Firestore queries using RxFire observables. This also dramatically reduced the number of composite indexes.
doc:
batchId: docId
map: data Objects
batchId: docId
other props ...
....
I also used maps of data objects for putting all kinds of configuration and definition data into single documents. This setup is easy to maintain and available with a single doc read. The doc reads are part of observables to react to doc changes.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I'm not sure if this is the place to ask this, but I have a best practices question.
I have a dashboarding service fed by Salesforce data that displays the number of Task X performed this week (X being Opportunities Closed - Won, Leads Created, etc).
Currently, the data is being pulled regularly and stored in a SQL database, which is mapped to a REST API that the Client App calls to get the aggregations between two date values, and will be fed additionally by Webhook calls via SF's Insert Triggers.
I want to know if having a Firestore Collection as a Cache for Aggregated SQL is a good idea, or if there is a better approach. The benefits I see are reduced traffic on my SQL server, instant updates (if the "cache" (Firestore) is updated, the client's value updates instantly as well).
When data is pulled from SF or a new record is received via the Insert Trigger/Webhook, I can update the Firestore record and the client will receive the change immediately.
My idea for a Firestore Document would be
{
user: "123",
sfOwnerId: "124",
sfTaskType: "Opportunities Closed Won This Week",
count: 23
}
Is this a good idea? Is there a better one out there?
Thank you in advanced!
Your strategy of storing the aggregated data is what the Firestore documentation suggests for aggregation, so I think it's pretty solid idea.
An alternative strategy would be to only store the Salesforce data in Firestore as it comes in, not aggregated, and let the client perform the aggregation. This can be achieved by subscribing to real-time updates to a query of Collection. In this setup, you would perform the calculation within the onSnapshot callback (assuming you're using the Web environment).
The advantage here is a possible increase to performance, since Cloud Functions often suffer from "cold start" latency.
Note: Several of the recommendations in this document center around
what is known as a cold start. Functions are stateless, and the
execution environment is often initialized from scratch, which is
called a cold start. Cold starts can take significant amounts of time
to complete. It is best practice to avoid unnecessary cold starts, and
to streamline the cold start process to whatever extent possible (for
example, by avoiding unnecessary dependencies).
Source