https://firebase.google.com/docs/firestore/manage-data/transactions
If we use the batch API to write something to firestore, is it going to be counted as a single write operation on pricing >
No. You are still charged for each individual document write.
Related
Currently I am using Firestore for my database and I have a users collection. Whenever a user document is created or updated to the users collection, a cloud function takes the user document and saves it in Elastisearch.
I am starting to be concerned about the scalability to this architecture. For example, suppose that several thousand cloud functions started writing documents to Elasticsearch at once, is Elasticsearch going to handle this load. Is there a better solution to this in Google cloud?
For example, can those cloud functions write the user documents in a queue and have cloud functions at the other end of the queue take a 100 documents and bulk write them to Elasticsearch.
I am new to Google cloud and would appreciate if you give me ideas, videos, and things to read.
Thanks
ElasticSearch has no limits on number of documents it can have per index but there are some limits such as maximum doc size and bulk writes mentioned in their documentation.
Maximum Document Size: 100KB [configurable in 7.7+]
Maximum Indexing Payload Size: 10MB
Bulk Indexing Maximum: 100 documents per batch
As far as I know, Google Cloud has no full text search API.
Talking of bulk writes, if realtime availability (data to be available immediately after adding) if not a concern, then you can store the new documents in Firestore along with a timestamp they were added and a boolean value if a document has been indexes in Elasticsearch.
Then instead of running a cloud function with onCreate trigger, you can run a scheduled cloud function every N minutes which will:
Query documents which have not been added in Elasticsearch
Make batches of 100 (for the 1000/batch limit)
Upload them to Elasticsearch
This way you are are more documents per cloud function run so that'll be a bit efficient but if you need your new data to be available immediately then this won't work.
how to check firebase cloud firestore size in a project?
according to firebase the free plan gives you 1GB for firebase cloud firestore, i already create some collections with documents inside, but where can i check the total size i am using?
i already check some statistics in firebase console but i can only see the numbers of reads and writes.
You can check your Cloud Firestore size or Cloud Firestore stored data in Google Cloud App Engine Quotas page. You can go direct to Google Cloud using the link because when you create a Firebase project, you're also creating a project in the Google Cloud.
Inside the App Engine Quotas page you can see also the other Cloud Firestore usage information including reads, writes, index writes, deletes, and network egress.
Example images:
UPDATE:
You can check your usage up to last 30 days in Firestore Database > Usage
Go to the Firestore Usage tab
Click "View in Usage and Billing" (bottom right)
This will show a summary of usage including total bytes stored, bandwidth, reads, writes, and deletes.
The answer is you can’t check the usage of bytes stored per month in Firestore. Only per day.
Usage per day can be checked in the app engine
It’s confusing because you don’t know how much data you have stored in total in Firestore.
Being on the Blaze plan, what are the cost implications when it comes to firestore triggers?
Do I assume correctly that a trigger by itself doesn't generate any extra reads from database?
I still have a feeling that triggers are not free from perspective of CPM/memory consumption. And from that perspective do I understand correctly that firestore triggers can be treated just like any other firebase functions?
The Firestore document that triggers your Cloud Function is included in the context. There is no document read or bandwidth charge for accessing this document.
You will be charged for the invocation and CPU/memory usage of the Cloud Function itself, as well as for any additional Firestore access you perform inside your Function's code.
According to the Firebase docs:
Firestore Triggers
With Cloud Functions, you can handle events in
Cloud Firestore with no need to update client code.
So Firestore triggers can only be done via Cloud Functions. For cloud functions there are costs for running a function (https://firebase.google.com/pricing) and the costs of listening to changes (snapshots) or reading/writing to Firestore will be added to that. Depending on what you are planning to do you also have to account for the internet traffic generated between the Firestore and the Cloud Functions, but this depends on some variables.
So, yes you are correct.
I'm monitoring document read count in Firebase Console > Firestore > Usage section.
However, I want to find a way to see document read count by collection so that I can know which query to optimize.
Is there anyway for that?
I looked into GCP monitoring but I couldn't achieve what I want to do here.
I read documents from js SDK and nodejs Admin SDK
Firestore provides no accounting other than a total running read count for the entire database for the last 30 days. If you need something more specific, you'll have to record it yourself somehow.
I am understand that we have been charges for every read, write and delete operation, but my question are these operation also being counted in the firebase console?
Sometimes, we just wanna see the data through firebase console, and when click at the collection, it will automatically display the documents data and also sub-collection of the documents. Are we gonna get charged through activity in console too?
Are these operation also being counted in the firebase console?
Yes, all read, write or delete operations performed even from the cosole are counted.
Are we gonna get charged through activity in console too?
Yes.