I am trying to figure out the best way to execute my cloud function for my firestore database, when data is being read.
I have a field on all of my documents with the timestamp of when the document was last used, this is used to delete documents that haven't been used in two days. The deletion is done by another cloud function.
I want to update this field, when the documents is being used AKA read from my db. What would be the best way to do this?
onWrite(), onCreate(), onUpdate() and onDelete() is not an option.
My database is used by a Android App written in Kotlin.
There are no triggers for reading data. That would not be scalable to provide. If you require a last read time, you will have to control access to your database via some middleware component that you write, and have all readers query that instead. It will be responsible for writing the last read time back to the database.
Bear in mind that Firestore documents can only be written about once every second, so if you have a lot of access to a document, you may lose data.
Related
I am using a collection in Firebase Firestore to log some activities but I don't want this log collection to grow forever. Is there a way to set a limit to the number of documents in a collection or a size limit for the whole collection or get a notification if it passes a limit?
OR is there a way to automatically delete old documents in a collection just by settings and not writing some cron job or scheduled function?
Alternatively, what options are there to create a rotational logging system for client activities in Firebase?
I don't want this log collection to grow forever.
Why not? There are no downsides. In Firestore the performance depends on the number of documents you request and not on the number of documents you search. So it doesn't really matter if you search 10 documents in a collection of 100 documents or in a collection of 100 MIL documents, the response time will always be the same. As you can see, the number of documents within a collection is irrelevant.
Is there a way to set a limit to the number of documents in a collection or a size limit for the whole collection or get a notification if it passes a limit?
There is no built-in mechanism for that. However, you can create one mechanism yourself in a very simple way. Meaning, that you can create a document in which you can increment/decrement a numeric value, each time a document is added or deleted from the collection. Once you hit the limit, you can restrict the addition of documents in that particular collection.
OR is there a way to automatically delete old documents in a collection just by settings and not writing some cron job or scheduled function?
There is also no automatic operation that can help you achieve that. You can either use the solution above and once you hit the limit + 1, you can delete the oldest document. Or you can use a Cloud Function for Firebase to achieve the same thing. I cannot see any reason why you should use a cron job. You can use a Cloud Scheduler to perform some operation at a specific time, but as I understand you want it to happen automatically when you hit the limit.
Alternatively, what options are there to create a rotational logging system for client activities in Firebase?
If you still don't want to have larger collections, maybe you can export the data into a file and add that file to Cloud Storage for Firebase.
My app has a for loop that writes data to my Firestore database.
However, right now, when I click my update button, Firestore updates the documents one by one, using transactions.
Thus, this results in me having to read every single document before being able to update it, which is extremely inefficient.
Is it possible for batched writes to perform an update feature similar to how transactions do?
For my case, the field I intend to update is a number, thus,
I am wondering if its possible to update the field by adding to it.
await transaction.update(stockListDocRef,
{'Num': outerStockListSnapshot.data['Num'] + Add});
You can use FieldValue.increment(x) to increment a field value in any sort of update operation, including batches.
See also: FieldValue.increment for Cloud Firestore in Flutter
I know what you are probably thinking, "why does it matter? Don't try to over-complicate it just to optimize pricing". In my case, I need to.
I have a collection with millions of records in Firestore, and each document gets updated quite often. Every-time one gets updated, I need to do some data-cleaning (and more). So I have a function trigger by onUpdate that does that. In the function there's two parameters: document before update and document after update.
My question is:
Because the document is been passed as an argument, does that count as a database read?
The event generated by Cloud Firestore to send to Cloud Functions should not count as an extra read beyond what what was done by the client to initially trigger that event.
I am use Firestore and try to remove race condition in Flutter app by use transaction.
I have subcollection where add 2 document maximum.
Race condition mean more than 2 document may be add because client code is use setData. For example:
Firestore.instance.collection(‘collection').document('document').collection('subCollection’).document(subCollectionDocument2).setData({
‘document2’: documentName,
});
I am try use transaction to make sure maximum 2 document are add. So if collection has been change (For example new document add to collection) while transaction run, the transaction will fail.
But I am read docs and it seem transaction use more for race condition where set field in document, not add document in subcollection.
For example if try implement:
Firestore.instance.collection(‘collection').document('document').collection('subCollection').runTransaction((transaction) async {
}),
Give error:
error: The method 'runTransaction' isn't defined for the class 'CollectionReference'.
Can transaction be use for monitor change to subcollection?
Anyone know other solution?
Can transaction be use for monitor change to subcollection?
Transactions in Firestore work by a so-called compare-and-swap operation. In a transaction, you read a document from the database, determine its current state, and then set its new state based on that. When you've done that for the entire transaction, you send the whole package of current-state-and-new-state documents to the server. The server then checks whether the current state in the storage layer still matches what your client started with, and if so it commits the new state that you specified.
Knowing this, the only way it is possible to monitor an entire collection in a transaction is to read all documents in that collection into the transaction. While that is technically possible for small collections, it's likely to be very inefficient, and I've never seen it done in practice. Then again, for just the two documents in your collection it may be totally feasible to simply read them in the transaction.
Keep in mind though that a transaction only ensures consistent data, it doesn't necessarily limit what a malicious user can do. If you want to ensure there are never more than two documents in the collection, you should look at a server-side mechanism.
The simplest mechanism (infrastructure wise) is to use Firestore's server-side security rules, but I don't think those will work to limit the number of documents in a collection, as Doug explained in his answer to Limit a number of documents in a subcollection in firestore rules.
The most likely solution in that case is (as Doug also suggests) to use Cloud Functions to write the documents in the subcollection. That way you can simply reject direct writes from the client, and enforce any business logic you want in your Cloud Functions code, which runs in a trusted environment.
I am trying to make feed/timeline where a user can follow - Category,Album or another User. Every time a picture is added to Category,Album,User it should appear on the timeline. I am trying to model my database so it requires 1-2 get requests only.
One idea for the solution is fan-out structure, But how do i make the multi-path update in Firestore? How can i update all the followers timelines when someone uploads a photo ?
How do i structure the database when i cant query on sub-collections? Should i just make one collection which contains all user timeline posts as separate documents, which will be ridiculous amount of duplicated data.
Is there any other way instead of fan-out to structure a user timeline ?
But how do I make the multi-path update in Firestore?
The equivalent of Firebase Realtime Database's multi-path updates, are called batched writes in Cloud Firestore. You can read more in the documentation on batches writes.
Flat.
root
pictures
uid-abc123
url:"http://test.com/img1.jpg"
owner:useriduid,
created: 1529333679449
uid-abc1billion
url:"http://test.com/img1billion.jpg"
owner:useriduid,
created: 1529333679300
Querying and security rules are then easy as pie. You can add indexing and it's very scalable.
Re multipath writes, use batched writes.