I am building a hospital management app. In this app there are bunch of features of which major features are messaging, appointment booking, video appointment and much more.
In the messaging feature what I am trying to do is loading all the previous messages from cache using bellow code
QuerySnapshot<Map<String, dynamic>> localMessageDocs = await _db
.collection("users")
.doc(AuthService.uid())
.collection("messages")
.orderBy('time', descending: true)
.get(GetOptions(source: Source.cache));
and after that I am querying latest messages using below code
_db
.collection('users')
.doc(AuthService.uid())
.collection('messages')
.where('time', isGreaterThan: _lastLocalMessageTime)
.orderBy('time', descending: true)
.snapshots()
but still I am afraid of bunch of reads that may occur if this system fails, like the cache size allowed in android and the firebase cache limitations(Default is 40MB) as messages for single user can be increase exponentially, which may cause thousand dollar bill.
Any help will be appreciated!
Related
I'm building a small chat app with expo, connected to Firestore. Here is the code to fetch the chat data:
useEffect(() => {
console.log("Loading snapShots on firebase");
const unsubscribe = db.collection('chats').onSnapshot(snapshot => (
setChats(snapshot.docs.map(doc => ({
id: doc.id,
data: doc.data()
})))
))
setTimeout(()=>{
unsubscribe();
}, 1000);
}, [])
This code is normally if I followed correctly documentation, supposed to close the snapShot listener after one second. If it does, I still get a [FirebaseError: Quota exceeded.] message and my app is very small, the data too.
Firebase quotas are reset daily at midnight (Pacific time). According to your timezone, it may differ. If you're located in Europe, it actually may be in the middle of the day. So if you reach the daily limitation, there is nothing you can do, but wait until the "next" day. Or you can update to the Spark Plan.
But remember, once you got the quota exceeded message your project will not be accessible until the quotas are reset.
As also #Dharmaraj mentioned in his comment, you might also consider using a get() call, and not listen for real-time changes. In this way, you attach a listener that is discounted automatically once you got the data.
Please also remember to not keeping your Firebase console open, as it is considered another Firestore client that reads data. So you'll be also billed for the reads that are coming from the console.
What happens if I call this query once which is used in a StreamBuilder which wrap a ListView :
Firestore.instance
.collection('users')
.orderBy('createdAt', descending: true)
.limit(3)
.snapshots();
and then I run this same query a second time but with a limit of 6 :
Firestore.instance
.collection('users')
.orderBy('createdAt', descending: true)
.limit(6)
.snapshots();
Does the 3 first snapshots are called a second time or kept in cache ?
Does the StreamBuilder rebuild all the ListView ?
The second query will fetch all 6 documents again. None of them will not come from cache, so they will all be billed as reads on the server, and take time to transfer. The only way query results will come from cache is if the client app is offline, or you specify a query source of cache using getDocuments and specify a Source of cache.
I have a firebase HTTP function which in turns calls some firestore operations. If I call the HTTP function several times, letting each call finish before calling the next, I get the following error in the firebase functions log:
(node:2) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added. Use emitter.setMaxListeners() to increase limit
The firebase function is an import task which takes the data to import, check for duplicates by calling a firestore query, and if there is none, it adds the data to the firestore DB by another DB operation.
Here is the firebase function, with parts removed for brevity:
module.exports = functions.https.onCall(async (obj, context) => {
// To isolate where the problem is
const wait = (ms: number) => new Promise(resolve => setTimeout(resolve, ms))
try {
const photo = getPhoto(obj)
// Query to look for duplicates
const query = db
.collection(`/Users/${context.auth.uid}/Photos`)
.where('date', '==', photo.date)
.where('order', '==', photo.order)
.limit(1)
await wait(300)
log.info('Before query')
const querySnap = await query.get()
log.info('After Query')
await wait(300)
// And then the rest of the code, removed for brevity
} catch (error) {
throw new functions.https.HttpsError('internal', error.message)
}
})
I inserted a pause before and after the const querySnap = await query.get() to show that it really is this invocation that causes the error message.
I also set the firestore logger to output its internal logging to help debug the issue, by doing this:
import * as admin from 'firebase-admin'
admin.initializeApp()
admin.firestore.setLogFunction(log => {
console.log(log)
})
So the more complete log output I get is this: (read it bottom to top)
12:50:10.087 pm: After Query
12:50:10.087 pm: Firestore (2.3.0) 2019-09-13T19:50:10.087Z RTQ7I [Firestore._initializeStream]: Received stream end
12:50:10.084 pm: Firestore (2.3.0) 2019-09-13T19:50:10.084Z RTQ7I [Firestore._initializeStream]: Releasing stream
12:50:10.084 pm: Firestore (2.3.0) 2019-09-13T19:50:10.084Z RTQ7I [Firestore.readStream]: Received response: {"document":null,"transaction":{"type":"Buffer","data":[]},"readTime":{"seconds":"1568404210","nanos":76771000},"skippedResults":0}
12:50:10.026 pm: (node:2) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 close listeners added. Use emitter.setMaxListeners() to increase limit
12:50:10.020 pm: Firestore (2.3.0) 2019-09-13T19:50:10.020Z RTQ7I [Firestore.readStream]: Sending request: {"parent":"[redacted]/documents/Users/SpQ3wTsFzofj6wcsF7efRrSMrtV2","structuredQuery":{"from":[{"collectionId":"Photos"}],"where":{"compositeFilter":{"op":"AND","filters":[{"fieldFilter":{"field":{"fieldPath":"date"},"op":"EQUAL","value":{"stringValue":"2019-06-26"}}},{"fieldFilter":{"field":{"fieldPath":"order"},"op":"EQUAL","value":{"integerValue":0}}}]}},"limit":{"value":1}}}
12:50:10.019 pm: Firestore (2.3.0) 2019-09-13T19:50:10.019Z RTQ7I [ClientPool.acquire]: Re-using existing client with 100 remaining operations
12:50:10.012 pm: Before query
The interesting thing is that I usually run these imports in batches of 10. I seem to only get the error during the first batch of 10. If I then quickly run more batches, I don't seem to get the error again. But if I wait some time, it returns. Also, it is not consistent in which invocation within a batch the error occurs. It may be the 9th or 2nd or invocation, or any other.
Finally, the error doesn't stop execution. In fact, the imports seem to never fail. But, I don't like have unaccounted for errors in my logs! I won't be able to sleep at night with them there. :-)
I'm grateful for any help you can offer.
I got a useful response from the Firebase support team. They told me to try to install the latest version of firebase-admin (which upgraded it from 8.5.0 to 8.6.0) and that resolved the issue, even without the workaround of installing grpc. So, I think this should be the correct answer now.
Looks like this bug MaxListenersExceededWarning: Possible EventEmitter memory leak detected #694 might the problem here.
Workaround is to use npm install #grpc/grpc-js#0.5.2 --save-exact until bug is fixed and the Firestore library starts using it.
Firestore documentation says:
"In the case of a concurrent edit, Cloud Firestore runs the entire transaction again. For example, if a transaction reads documents and another client modifies any of those documents, Cloud Firestore retries the transaction. This feature ensures that the transaction runs on up-to-date and consistent data."
I am using the cloud_firestore package and I noticed that doing
final TransactionHandler transaction = (Transaction tx) async {
DocumentSnapshot ds = await tx.get(userAccountsCollection.document(id));
return ds.data;
};
return await runTransaction(transaction).then((data){
return data;
});
the transaction may run multiple times but always return after the first transaction. Now in case of concurrent edits, the first transaction data may be incorrect so this is a problem for me.
How can I wait for the transaction to actually finish even if it will run multiple times and not return after the first one finished?
Your transaction code doesn't make any sense. It's not getting the contents of any documents. You only need to use a transaction if you intend to read, modify, and write at least one document.
The transaction function might only be run once anyway. There is only need for it to run multiple times if the server sees that there are a lot of other transactions occurring on documents, and it's having trouble keeping up with them all.
I'm developing a app that uses Firebase's Firestore to send data to the web. One of the functions of the app is being able to save data in the device while being offline and send it to Firestore when internet connection is restored.
I activated offline persistence but it dosen't work.
DEBUG CONSOLE:
W/OkHttpClientTransport(28536): Failed closing connection
W/OkHttpClientTransport(28536): javax.net.ssl.SSLException: Write error: ssl=0x7f7acfc408: I/O error during system call, Broken pipe
W/OkHttpClientTransport(28536): at com.google.android.gms.org.conscrypt.NativeCrypto.SSL_write(Native Method)
W/OkHttpClientTransport(28536): at com.google.android.gms.org.conscrypt.NativeSsl.write(:com.google.android.gms#14798020#14.7.98 (040406-222931072):4)
How can I activate offline persistence and sync with Firestore when internet is restored?
MY CODE:
Future<Null> sendFirebaseData(var selectedModel) async {
Firestore.instance.enablePersistence(true);
var certID = await getIDCertificado();
var dateTime = new DateTime.now();
var nowHour = new DateFormat('kk:mm:ss').format(dateTime);
Map<String, dynamic> dataHeader = {
'ID': certID,
};
Map<String, dynamic> finalDataMap = {}..addAll(dataGeneral)
..addAll(dataInstrumento)..addAll(dataPadrao)
..addAll(dataAdicional)..addAll(dataHeader);
await Firestore.instance.collection('certificados').document((certID.toString()))
.setData(finalDataMap);}
when you use offline persistence in Firestore, don't use Transactions or await for response.
so, change this :
await Firestore.instance.collection('certificados').document((certID.toString()))
.setData(finalDataMap);
to this:
Firestore.instance.collection('certificados').document((certID.toString()))
.setData(finalDataMap);
When you restore your internet connection your data will be sync automatically, even if you are in background.
Doesn't work when your app is closed.
Context Of Promises & Callbacks when Offline
Why the above code change to remove "await" works.
Reference: Firebase Video - How Do I Enable Offline Support 11:13
Your callback won't be called and your promise won't complete until the document write has been successful on the server. This is why if your UI waits until the write completes to do something, it appears to freeze in "offline mode" even if the write was actually made to the local cache.
It is OK to not use async / await, .then() or callbacks. Firestore will always "act" as if the data change was applied immediately, so you don't need to wait to be working with fresh data.
You only need to use callbacks and promises when you need to be sure that a server write has happened, and you want to block other things from happening until you get that confirmation.
I think the currrent answer is outdated. According to the firebase documentation, offline persistentence is enabled by default for Android and iOS. For the web, it is not.
In flutter, the firestore implementation is based on the underlying OS. So you're safe on mobile apps, but not with flutter for web.
It is enabled by default but still only when you are not using await or transactions, further you can use timeout to stop listening to network connection by firestore after a specific time.
ref.setData(newNote.toMap()).timeout(Duration(seconds: 2),onTimeout:() {
//cancel this call here
print("do something now");
});