Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
EDIT: Basically do you get charge for the documents you query. The answers is no, if you query 100K documents but only get back 10 you only get charge for the 10 documents you get form your query.
Hi so I have been using firebase firestore and it's been great. However, I have some questions about it. Currently, I'm working on an app where users can place orders and by default, the 'isActive' property is true so that the admin can see the orders. When the order is completed the property turns false and doesn't show up. However, eventually, I will accumulate thousands of orders and my question is will I get charged for the documents that I read that is true or it will counts as reading all thousands of documents even though I don't use it.
You will most definitely be charged for stored data regardless of if you access it your not. If you do decide to access it you'll be charged for that as well. Fortunately you have 20K free reads a day and then you pay after that.
While I think that answers your question. If your app reads in 20,000 documents in one action (open a page and then it loads in everything) that's not going to scale well for you and that's a lot of data. Which means your client devices will have to process 20K documents which is not ideal.
I'd advise using limit and then do pagination or infinite scrolling.
It's fairly simple: if your document is read from/on the server, you will be charged for a document read.
If your clients are not requesting documents with isActive is false, they won't generate document read charges for those documents. I your admin is reading those documents, they will generate read charges for those documents.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 1 year ago.
Improve this question
I have been developing an app which has a feature to chat with other users. I have been been really confused for the past few days trying to find out which would be the better database to use in terms of pricing, I am talking about going at scale here.
Whenever a user sends a message he creates a document and while doing so I am also checking if the other user is online or not, for which I read another document. On average I have to make around 7-8 writes per message and 8-10 reads per message. Also whenever a user opens a conversation he sees the last 15 messages, and if he scrolls then he sees more. This increases the reads as well.
Also I want to know if I send a message and the other user gets it , who previously read 15 documents to see the last 15 messages , when I send a new message , the 15th message gets replaced with the new one , so do I get charged for the 15 document reads again?
Pricing is the main concern here, please help me find the best approach here.
Firebase has a database recommender and a pricing calculator in its documentation to help you answer this question. I'd expect it to point to the Realtime Database here, primarily based on the fact that you'll have many smaller write operations.
For your second question (please limit yourself to a single question per post going forward): if your listener remains active, or if your local cache already contains the 14 unmodified documents, you will be charged only one document read (for the new document that needs to be read on the server to return it to the client).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I am beginner on Firebase (and beginner on NoSQL).
I would like to learn the good practices about filtering data.
For example with this simple realtime database:
How you would go about filtering this posts?
Example 1: List all posts (without any filter):
firebase.database().ref('post/').once('value').then((snapshot) => {
});
Example 2: List all posts order by createdAt and limit at 3:
firebase.database().ref('post/').orderByValue('createdAt').limitToLast(3).once('value').then((snapshot) => {
});
Example 3: List all posts order by createdAt, endAt(1605972663986) and limit at 3:
firebase.database().ref('post/').orderByValue('createdAt').endAt(1605972663986).limitToLast(3).once('value').then((snapshot) => {
});
But:
How can I get all posts of user "et6e1AKrhk2GwqjCAKUHK5Bjlgu2" order by "createdAt" and limit at 3 ?
How can I get all posts in category [9, 12] order by "createdAt" ?
How can I get all posts exclude category [2, 4] order by "createdAt" ?
Should I retrieve all the posts and then filter them myself? (is it a good way? if I have 100 millions of posts, what should I do ?)
Sorry If my questions look like stupid but I am 100% beginner and I don't know not yet the goods practice (Currently I have over 1000 messages and I need to filter [with pagination] them based on the current user settings).
Thank you
Firebase charges per document read, so reading 100 million posts at once wouldn’t be a good idea as I’m sure you know.
The query questions you asked seem rather simple and should not be a problem. Firebase does have limitations on queries and you should review all the documentation.
When I started with Firebase I used real-time database initially but eventually switch to Cloud Firestone as I found the querying to be much more powerful using where conditions. (Fire store also has some major limitations: https://firebase.google.com/docs/firestore/query-data/queries#query_limitations )
It may be necessary to add fields to your documents for the sole purpose of filtering and querying. Sometimes it works out where you need to sort data client side, which shouldn’t be a big deal.
I suggest you run tests to ensure you can query all data properly and add fields and ensure the DB fits your needs/research limitations before you dive in too deep! Your DB structure looks good!
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I'm doing a fair bit of work on a set of Firestore collections and documents. It amounts to a good amount of writes and reads, as I'm setting two-way refs and whatnot. Multiple documents are being written to multiple times.
Since Firestore offers offline capability, is it possible to reduce the number of writes via preparing all the data locally, then sending it to the server?
I'm using the Admin SDK.
It depends on what you mean. One document write is always going to cost one document write, no matter when or how that document was written. Batch writes don't in any way reduce the number of documents written, they just make all the document writes take effect at the exact same moment in time.
If you're staging lots of changes to a single document to take effect later, then feel free to do that. Just write the document whenever you've figured out what final document looks like, and no sooner.
I'am moving away from google appengine standard Python 2.7 NDB to Svelte, Firestore and RxFire.
I was able to dramatically reduce the number of reads and writes by batching hundreds of appengine NDB entities (datastore / data objects) into a single document using a data object map.
Every data object has a batchId prop to optimize (batched) batch writes / document writes. (batchId = docId)
Most of the querying is now done in the client using filters. This resulted in very simple reactive Firestore queries using RxFire observables. This also dramatically reduced the number of composite indexes.
doc:
batchId: docId
map: data Objects
batchId: docId
other props ...
....
I also used maps of data objects for putting all kinds of configuration and definition data into single documents. This setup is easy to maintain and available with a single doc read. The doc reads are part of observables to react to doc changes.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I'm trying to find a good way to read a large amount of data in firestore. Currently , I have 10000 documents in a single collection "movie". These documents are records that contains fields about a movie ( author , title , id).
My client use a query to read 20 documents randomly with .limit(20) and a user can fire this query as the much as he wants to get new documents. The problem is that an user can ask for example for 1000 documents(movie) and this cost 1000 reads in firestore.
I was thinking if it's possible to reduce the numbers of reads by structuring my collection in a different way with for example 500 subcollections with 20 documents(movies) inside. Like this, I have my documents divided equally and my client can just read 1 subcollection that contains 20 documents. Does this reduce the number of reads to only 1 ?
You're saying:
My client can read 20 documents randomly using .limit(20) and when a user click on a button 20 new documents are read once again. The problem is that an user can ask for 1000 documents and this cost 1000 reads in firestore.
If you want to limit the user to read at most 20 documents at once, you can do so in the server-side security rules with:
// Limit documents per request to 50
allow list: if request.query.limit <= 20
Also see the reference documentation on the query object in security rules.
Of course this only limits the number of documents the user can request at one time. They can still perform 50 queries of 20 documents each, and get 1000 documents that way.
The only way to have fewer reads is to read fewer docs. This is true for sub-collection, too. One way to have fewer docs is to fill single docs with more data...
// in a single doc
movies: {
movie_id_a: { title: 'Gone With the Wind', rating: 'G' },
movie_id_b: { title: 'Casablanca', rating: 'PG' },
...
}
But this comes at a cost: You'll run into a doc size limit, and you won't be able to use database features associated with keeping logical chunks each in their own doc (like saving a reference to just one movie, or listening to just one movie, etc)
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 3 years ago.
Improve this question
I am aware for Cloud Firestore a read is a document (whether the documents has 5 or 50 nodes). How does this compare to the RTDB?
If I have a query that has a limit of 25, is this going to be 25 reads, or 25 times x amount of items in each node?
Cheers.
Your question is a bit of a non-sequitur, as realtime database doesn't bill by reads, it bills by data transferred (and storage, of course). So, the thing that affects your cost is the size of the items transferred, which is only indirectly based on the number of items due to a limit on the query. Currently, the costs are about US $1 per GB downloaded assuming you are on the Blaze plan.
To compare this with the costs for Firestore would require knowing a lot more about the shape of your traffic -- how many reads and writes, average size of a read, etc. Note that Cloud Firestore also indirectly charges for data transferred, but at a much lower rate, as it is only the Google Cloud Network pricing.
This means that you can generally get quite a large number of Firestore document reads for the cost that RTDB charges for transferring 1 GB.. (e.g. at current prices for egress to most of the internet excluding some asia/pacific destinations, you could get 1 GB + over 1.4M firestore document reads for your $1 of 1 GB RTDB transfer).
The documentation references several things you can do to help control costs, including (but not limited to):
Prefer the native SDKs to the REST API
Monitoring your data usage and use the profiler tool to measure read operations.
Use fewer, longer lived connections, as SSL and connection overhead can contribute to your costs (but generally are not the bulk of your cost).
Ensure your listeners are limited to only the data you care about, and are as low in the database tree as possible, and only download updates, (e.g. on() vs once()).