On an Angular app using Firebase/Firestore with Offline persistence mode, one of my users is getting the following error with Chrome:
main.6cba4fc1c2ed99809a4b.js:1 [2022-06-07T10:45:20.998Z] #firebase/firestore: Firestore (8.10.0): AsyncQueue Failed to persist write: IndexedDbTransactionError: [code=unavailable]: IndexedDB transaction 'Locally write mutations' failed: QuotaExceededError```
I noticed that the application storage is full, but what is strange is that the quota is only 300 MB.
I checked the hard disk which has 190 Gb of free space.
What could explain such a low quota ?
Related
We're trying to fetch 86k of our firebase users, on local & firebase functions it takes 2 minutes for all, but in cloud run it is taking on average 20 seconds per call (you can only request 1k users per calls according to firebase docs).
Interestingly, get all firebase real time database uses takes 15s, but in cloud run it took 365s.
2022-06-17T00:03:04.986000061Zgrabbed users data from db, total: 86442 in 364.015s
2022-06-17T00:03:05.732000112ZProgress 1000 0.746s
2022-06-17T00:03:15.131999969ZProgress 2000 9.847s
2022-06-17T00:03:39.332999944ZProgress 3000 34.347s
2022-06-17T00:04:03.832999944ZProgress 4000 58.846s
2022-06-17T00:04:28.433000087ZProgress 5000 83.447s
2022-06-17T00:04:51.733000040ZProgress 6000 106.747s
2022-06-17T00:05:58.332000017ZProgress 7000 172.947s
Any thoughts on how to solve this? No special network settings in place on cloud run.
Background Info:
Cloud run instance is using NodeJS 14. 2GB Memory which stays at 8% usage. CPU usage stays around 10%. The user object is relatively small, but across all these users it's about 60-70 MB. In firebase functions, only 256 MB of memory are required to do the fetching.
PS: I've yet to test if region makes a difference, as cloud run is in us-east1 and functions are in us-central1. Will be testing soon.
I've a project using firebase realtime database for over 2 years now... it is a big project with many rules and over 1GB data and around 10k users dayly
I was checking the firebase rules monitor and i notice something unwanted
In my happy world there would be ZERO denies and zero errors. The denies I'm monitoring on my own, but I dont know any way to monitore the errors or debug them
I understand that errors come from firebase trying to evaluate some specific rule and it throwing an exception... BUT HOW CAN I DETERMINE WHICH RULE WAS IT? so i can fix the error...
when dealing with the client side firebase only return PERMISSION DENIED without any further explanation so it is impossible to me guess from the client side if this was a normal permission denied or anything related to a rule error [making it impossible to log or debug]
does anyone know ANY WAY to determine which rules are having errors?
================UPDATE===================
I've pasted the rules in this bin
https://ghostbin.co/paste/3bj823
This normally isn't possible with the limitations firebase gives us.
However, a sound solution is to have a catch on your realtime requests that pushes a log object to realtime db in a separate bucket or a dedicated node. Since realtime db writes are fundamentally free besides storage, this creates an environment where you can debug the users' request, their app state, and any data they may be trying to send.
example:
realtimeReference.push({
user: uid,
path: "path/that/failed",
payload: Json.stringify(payload),
meta: {additional:data,etc:etc},
timestamp: firebase.database.ServerValue.TIMESTAMP
})
To ensure that your app isn't always pushing data and to avoid constantly updating your app, you can use global variables with Remote Config:
https://firebase.google.com/docs/remote-config
If you are using cloud functions, you could automatically save all errors into a separate log file with the full trace of the requests chains
If a transaction reads 3 docs and then updates 2 documents successfully but something after this causes the transaction to fail... will I be charged for the 3 reads and 2 writes that were made even though they are rolled back?
Edit---
Also will the get() below only cost 1 read? Where col2 is a subcollection of doc1.
db.collection('col1').doc('doc1').collection('col2').doc('doc2').get();
Edit 2
The firebase website states the following
For example, if a transaction reads documents and another client modifies any of those documents, Cloud Firestore retries the transaction. This feature ensures that the transaction runs on up-to-date and consistent data.
So say my transaction performs 10 reads on 10 different documents. If this gets called and during exectution some of the same documents are updated by other users, which will make the transaction retry, am I going to be hit with 10 * Number of retries for my reads?
Edit 3
I have read more about the transactions here https://firebase.google.com/docs/firestore/transaction-data-contention and it states that the server side transactions will lock the documents and wait for the transaction to finish.
q1) As the transaction is locking and not retrying over and over... will multiple concurrent calls to the firebase function that has a transaction not cost any extra reads/writes and will the functions just take longer to execute because of the lock?
q2) The webpage also has a banner at the bottom stating
Note: Only the server client libraries support transactions with read operations after write operations. For transactions in the mobile/web SDKs, document reads must come before document writes.
I just tried this on my firebase function and recieve the following error...
Error: Firestore transactions require all reads to be executed before all writes.
at Transaction.get (/srv/node_modules/#google-cloud/firestore/build/src/transaction.js:76:19)
I am using firebase admin version "^8.8.0", is performing reads after writes a feature that has been added in newer versions?
If transactions fail, will I still be charged?
Yes. A read was completed so you are charged for it. (I am unsure if there are any "rollback charges" - as the change now needs to be reversed.)
What is the cost of a sub-collection document read?
The doc1 was not read - so it would not be charged. You are charged for only one read.
I couldn't find a clear text in the documentation and these answers are from my personal usage for Firebase for over a couple years. A Firebasers confirmation would be helpful.
Whenever I try to retrieve large data from Firestore in my angular application, I get below error.
#firebase/firestore: Firestore (5.5.9): FirebaseError: [code=resource-exhausted]: Resource has been exhausted (e.g. check quota)
What can be done in this case? Once this error comes, Chrome browser tab also hangs. I have no option than to kill the browser. Moreover, if the same code is written in Python and run from PyCharm Community Edition, no such error is received.
However I need the data to be retrieved in my angular application. What can be done in this case?
I'm encountering a problem where, the Azure Cosmos DB is giving Status Code: 403 Storage quota for 'Document' exceeded message for documents which are less than, 2 MB in size. I know, there is a limit on the document size but, in this case I tried with 5 KB document and it gave the same message. I'm using documentdb for storing JSON data.
According to
https://learn.microsoft.com/en-us/rest/api/documentdb/http-status-codes-for-documentdb
403 is also returned during a POST to create a resource when the resource quota has been reached. An example of this is when trying to add documents to a collection that has reached its provisioned storage.
I have encountered this when I have tried to push more data into a partition when its size has already reached 10 GB.