I really like the offering that's Cloud Firestore. But does it have an audit trail?
Why am I asking? While some use of databases is essentially "state" (e.g. multiplayer games), for other uses also the history matters. Also, at times being able to see the history would be useful for debugging purposes (time travel).
So what's the means I would have to ask e.g. "who deleted a value in this document" or "who has touched this doc since its inception"?
Do people use additional commentary documents to do this at the application level, or can I just enable a setting somewhere that would create such a document behind the scene, automatically? (I could then describe access rights just as for any other Firestore document.)
Addons, plugins etc. can also be suggested. I don't have a pressing use case for this right now - just want to understand where Cloud Firestore would fit and where not.
Btw, the Getting to know Cloud Firestore series in Youtube is plain awesome! :)
Edit: 2022-10-25
As Frank mentioned in his answer, yes, by the time I have written this answer, there wasn't a solution for audit logging. However, when it comes to today, this option is available and it can be also configured based on what you need.
For a better understanding, I recommend you check the official documentation regarding Firestore audit logging where it is said that:
Google Cloud services write audit logs to help you answer the questions, "Who did what, where, and when?" within your Google Cloud resources.
Old answer:
> **Note:** This answer was correct at the time of writing, but is now outdated. See [this answer](https://stackoverflow.com/a/74154622/209103) for details.
So what's the means I would have to ask e.g. "who deleted a value in this document" or "who has touched this doc since its inception"?
Cloud Firestore doesn't store such metadata. If you find yourself in a position in which you need such details, you'll need to create a mechanism to store them. So in case you want to know who deleted a specific value, you should create an array of objects where you should store the UID of the user who deleted that value and a timestamp.
Do people use additional commentary documents to do this at the application level
Yes, we usually do.
or can I just enable a setting somewhere that would create such a document behind the scene, automatically?
There is no such setting that can be enabled.
If you need a history version of the document, a possible schema that might help you is in my answer from the following post:
Firestore: Version history of documents
While Alex's answer was correct at the time of writing, Firestore nowadays provides audit logging capabilities that you can enable and configure based on your needs.
See the documentation on Firestore audit logging for full details on the Admin Activity audit logs and Data Access audit logs that can be set up.
Related
I really like the offering that's Cloud Firestore. But does it have an audit trail?
Why am I asking? While some use of databases is essentially "state" (e.g. multiplayer games), for other uses also the history matters. Also, at times being able to see the history would be useful for debugging purposes (time travel).
So what's the means I would have to ask e.g. "who deleted a value in this document" or "who has touched this doc since its inception"?
Do people use additional commentary documents to do this at the application level, or can I just enable a setting somewhere that would create such a document behind the scene, automatically? (I could then describe access rights just as for any other Firestore document.)
Addons, plugins etc. can also be suggested. I don't have a pressing use case for this right now - just want to understand where Cloud Firestore would fit and where not.
Btw, the Getting to know Cloud Firestore series in Youtube is plain awesome! :)
Edit: 2022-10-25
As Frank mentioned in his answer, yes, by the time I have written this answer, there wasn't a solution for audit logging. However, when it comes to today, this option is available and it can be also configured based on what you need.
For a better understanding, I recommend you check the official documentation regarding Firestore audit logging where it is said that:
Google Cloud services write audit logs to help you answer the questions, "Who did what, where, and when?" within your Google Cloud resources.
Old answer:
> **Note:** This answer was correct at the time of writing, but is now outdated. See [this answer](https://stackoverflow.com/a/74154622/209103) for details.
So what's the means I would have to ask e.g. "who deleted a value in this document" or "who has touched this doc since its inception"?
Cloud Firestore doesn't store such metadata. If you find yourself in a position in which you need such details, you'll need to create a mechanism to store them. So in case you want to know who deleted a specific value, you should create an array of objects where you should store the UID of the user who deleted that value and a timestamp.
Do people use additional commentary documents to do this at the application level
Yes, we usually do.
or can I just enable a setting somewhere that would create such a document behind the scene, automatically?
There is no such setting that can be enabled.
If you need a history version of the document, a possible schema that might help you is in my answer from the following post:
Firestore: Version history of documents
While Alex's answer was correct at the time of writing, Firestore nowadays provides audit logging capabilities that you can enable and configure based on your needs.
See the documentation on Firestore audit logging for full details on the Admin Activity audit logs and Data Access audit logs that can be set up.
My assumption was that Firestore would not consider the client cache when evaluating Read usage. However, when I have persistence enabled in my application, it seems I can run the same queries endlessly and not see any change in my usage metrics. I've tested this over and over - when persistence is disabled, I see the usage metrics go up.
Given the amount of testing I've done, I feel that I should be able to safely declare that Firestore must be recognizing my local cache and only charging me for Reads on updated documents. However, I cannot find documentation around this anywhere, and I have combed through the docs. I'd certainly think if Google wasn't charging me for these cached Reads that they would want to highlight that benefit, whereas they really only highlight enablePersistence as an offline benefit - so I am perplexed.
Does someone have some insight into what is going on here?
Example enable persistence: firebase.firestore().enablePersistence()
Example query: myCollection.where(condition).get()
According to this documentation
Offline data persistence feature caches a copy of the Cloud Firestore data that your app is actively using, so your app can access the data when the device is offline
So, it is clear that you will only be charged when calling get() on the server to-read documents, mentioned in this answer
Also, be aware that
Offline Persistance is enabled by default for Android and iOS.
The pricing model for Cloud Firestore
This is a really great question. To clarify what the question is asking: "If my app is online and makes a query, but the query is fully returned by the offline cache, will I still be charged?"
I suspect yes.
Context: I am total Google Cloud begginer and I have just convinced my company headers to use Firestore Realtime Database for pushing transaction status to our mobile application. We have around 4 millions users that will use significantly our application for small money transfers. Now-a-days we use the concept of polling from Android/IOS to our Microservice endpoints and it will replaced by Firebase SDK imported to our Mobile app which will listen/observe to our Firestore Collection following few Firestore Rules. Since all money transfer will be confirmed/denied in short time (from few seconds to 1 or 2 minutes) the idea of replacing polling by a real reactive approach straigh from Firestore sounded and is already ongoing coding.
The issue: Firstly I don't what to compare solutions. It is just my reality: the prodution support operators must look after our internal Dashboard. Isn't allowed to them look at Google Dashboard Console (please accept this for this question). I need get on demand metrics of our FIrestore. It is nothing to do with Google pricing. It is just our demand: they want to see metrics like:
how many users listening at the same time now
how many users took some exception during connection
is there any user holding connection for more than X minute
when was the connection pick this morning
any exception of any type surrounding our Firestore database
I read Code Samples carefully follow the sample step-by-step trying to figure out some idea if there is some API providing the answers I am looking for.
So, my straight question is: is there such type of Google API providing metrics about my Firestore Database? Maybe following the same idea we found in Performance Monitor which works on Mobile side also some similar aproach on Firestore side.
*** Edited
Future readers may find worth read also about a way to get Firestore metrics info striagh from curl/postman
A couple of things: You mentioned both Firestore and Realtime Database; just wanted to make sure that you are aware that those are two different databases offered under the Firebase umbrella.
how many users listening at the same time now
is there any user holding connection for more than X minute
Yes, there's a dashboard: https://support.google.com/firebase/answer/6317517?hl=en. Including lots of options, like users active in the last 30 mins.
how many users took some exception during connection
any exception of any type surrounding our Firestore database
Yes, you can track errors and other logging via Stack Driver logging. These can give you reports on your cloud functions.
https://cloud.google.com/functions/docs/monitoring
Where can I find Stackdriver in Firebase console?
when was the connection pick this morning
For this one, I'm not sure if you mean A. when did somebody log on in the morning, or B. what was the time that there was the peak \ most usage. If B see 1. If A,
Real-time database has the concept of presence, which lets you know if a user is currently logged in or not. See examples here from the official documentation:
https://firebase.google.com/docs/firestore/solutions/presence
and this post
How to make user presence mechanism using Firebase?
Also applies to your
is there any user holding connection for more than X minute
..............
Edit in response to comments: I believe you are experiencing the XY problem https://meta.stackexchange.com/questions/66377/what-is-the-xy-problem where you are focused on a particular solution, even though your problem has other solutions. User metrics, database events, and errors are all accessible through both dashboards and cloud functions. You can cURL cloud functions if you wish, or set up cron functions to auto report, or set up database trigger functions to log errors. So, while the exact way you want this to work may not exist, you just need to connect existing tools to get the result you want.
I have been introduced to Firebase a few days ago. Since then I have been trying to discover more features and usage of Firebase, in particular, the usage of the Firestore/Realtime Database.
However, as I am reading into more details, I start to think of some questions. I hope someone that is familiar with Firebase can help me answer them
As for User authentication, I understand that I can use Firebase Authentication where I won't see their real password. However, technically, everything that user stores in my Firestore would be visible to me since I am the owner of the Firebase.
For example, if I were to develop a note/chat app, in which the user can access their notes/chat on the iOS app and also Android. That means all their notes data would be saved in my database. If it happens to contain some private data, then I would be able to read it?
Even if I set security rules, that would be only facing client-side, whereas, for me, the owner of the entire firebase data, I could see the whole thing. Surely, as a customer, you won't want to use an app knowing that I can see everything you write
I am not sure if what I said is true or not. If true, is there a possible solution?
Both of your statements are correct. This happens in most of the apps-websites, the admin or some core developers have full access to the data. That's why privacy policy and GDPR exist. You must specify what data you collect and for what reason. If you intend to use your user data for any other reason you must inform them. Be aware that if you disclose any user information without his permission you can be held liable.
I have seen a firebase api in which you can create firebase projects and it occurred to me for example, when a user wants to counter a SaaS, make it easier for me to create a project and connect it to their SaaS but I have the following questions:
If there are supposed to be limits to creating firebase projects in a google account, will there be a time when databases can no longer be created for new users?
If the above is true, how can this be solved?
I have seen that you can ask for more projects, but how many can I have?
I know that with firestore, I can model data and only in a database have all the information, but for example, each user may have special requirements in their system and also provide security and information saving operations that would be impossible if all information is in a single database.
Thanks for the help.
EDIT
"How many projects can you have as a developer?" Yes, that's what I mean, having all the user bases in a single firebase google account. For example, on a web page, the user wants to pay for the application, with firebase admin and google cloud functions, I can automatically create a project and have all the databases in a google account. And what I want to know is if you can have multiple databases. I have seen that you can ask google to give you permission to host several projects but, for example, can you have more than 100 projects or even 1000? (I may never have reached that number, but in that case, I would like to know the limits that can be reached).
Edit 2
This first structure I have all the documents in a "universal" collection and I identify them with an id to know the user who used it.
But the second structure seems to me more organized and clean, in addition to the fact that users at no time need to share information among others. I have read that having nested collections is not good but over the years and the progress that firestore has had, this is no longer or was a problem only that the limitation is that you can only have up to 100 collections anidades but I never think to reach that quantity.
Inside of list, have all products for that user, because inside of document, only can have 1 mb of data and download 1 mb and is not the best option.
in the firestore documentation I see that it does not reveal other problems, it only says that it is difficult to delete the subcollection, but I do not allow users to do that and if I need to delete the subcollections, with the Google Cloud functions i can do it.
So, the second structure is more intuitive for me, but is the best option for that? or actually firestore is not good for this strcucutre?