I'm looking at writing the date a user was last active to my firestore users table. This information is available in the metadata of the user - lastRefreshTime.
https://firebase.google.com/docs/reference/admin/node/admin.auth.UserMetadata
Has anyone already done this before?
I am looking for an efficient way to do this with minimal writes.
I could run a daily process that checks all users and the dates and updates if changed but wondering if there is a better more efficient way.
How about having each client write it themselves when they go online?
It won't be guaranteed (as malicious users may call the API themselves without writing the value), but it will prevent you from having to have an administrative process over a data set that will be hard to predict the growth of.
Related
I have been scouring the internet for days on a solution to this problem.
That is, how to handle aggregation when there is no network connection? I have a task management app that looks to aggregate meta data about user tasks. For example, the task can contain tags that can be aggregated to be shown in a dashboard to the user on a daily basis. This would be easy if the user is always online, so I could use transaction or cloud function to aggregate, but when the user is offline, the aggregation will appear to be incorrect, until the user restores their network connection.
Aggregation queries are explained here:
https://firebase.google.com/docs/firestore/solutions/aggregation
Which states a limitation:
Offline support - Client-side transactions will fail when the user's
device is offline, which means you need to handle this case in your
app and retry at the appropriate time.
However, there has yet to be any example or documentation on how to 'handle this case'. How would I go about addressing this problem?
Some thoughts:
I could cache the item if a transaction fails. This item will be aggregated on top of the stored aggregation. However, going down this line would mean that I can't take advantage of the Firestore's "offline mode", because I'm using my own cache on every write while offline anyway.
I could aggregate on demand. That is, never store the aggregation. This is going to be very heavy on read depending on how many tasks a user has. Furthermore, if the aggregation will need to be shared as insights to other users, this option will not work because other users do not have access to the tasks.
I'm at a loss and any help would be appreciated, thanks!
After a lot of research and trial and error I found a solution that can address this problem gracefully.
FieldValue.increment to the rescue.
What FieldValue.increment does is bypass the use of transaction while respecting the default Firestore's offline cache behaviour. It requires the use of set or update on the field directly. The drawback is the inability to use the 'withConverter' on the collection for type safety. I'm willing to live with the drawback considering how useful FieldValue.increment is.
I've done multiple tests and can confirm that the values can be incremented/decremented multiple times locally while offline. This offline value is reflected in a get or snapshot call to the cache. When the network connection is restored, the values are updated on the server.
The value itself is not stored on the cache, it simply stores the "difference" in the FieldValue sentinel for when it is time to update it on the server.
This method only works with incrementing and decrementing values. Storing averages will not be possible using this method. That is because the true total number of items is not known at the time of its calculation when offline.
Instead, the total number of items are stored along side the total value. The average is then calculated when and as needed. In this way the average will always be accurate from a local perspective when offline, and it will also be accurate when online when the total value and count has been synced.
I am looking for some advice on an approach. I am using firebase firestore. I am using batch and transaction updates (depending on situation) to keep things as atomic as possible .
My application has many transactions (10's and possibly 100's of thousands). On the user's dashboard I loop through and total some of the fields in those transactions by looping ALL transactions and totaling by user, team and challenge. Clearly, this is too costly to do each time a user hits the dashboard. I have tried a few approaches and currently doing the following.
I store all totals in a collection. i.e. overall totals, team totals, user totals. These are the only items I need to fetch to show dashboard so its very minimal compared to all transactions.
I am doing all this work using firebase functions.
If I find a collection of totals does NOT exists, I create it from scratch by looping through all transactions and when done, saving results/totals to the totals collection.
I then have a firebase function firestore listener that triggers whenever a transaction changes. Then I just update the totals based on the updated transaction so I don't need to read them all again.
The thing I worry about is that if I receive an update while the main loop/calculation is running I will lose the data for that new/updated transaction and it will cause the totals to get out of sync.
Does anyone have an idea of how to get around this issue? Should I try and use google tasks to queue things? Any ideas how to even keep track of if that listener is running? Sort of like a semaphore or something (which I don't think firebase functions support).
I apologize if this is too vague. I don't need code or anything like that, was just looking for some ideas on what would be a reasonable approach. I
Suppose I am creating a transaction app.
How will I store transactions?
I know I need to denormalize.
Would I save the transaction within a transaction node at the first
db level? Or would i save the transaction node under each user's node? Or would i save it in both the transaction node on the first level and the
transaction node under each user's node?
What if the user changed their name, how would I reflect these
changes in both the transaction history of the user and the business?
I feel like the best way is to put it in just the first level of the database and have the user's query the entire list to see their transaction history.
But, If i have a lot of users wouldn't this be extremely slow?
Or is firebase smart enough and fast enough to handle such queries.
Does the user's internet speed affect this querying, especially on a
mobile device?
Can you display the transaction on the screen as it is being loaded?
Would firebase indexing allow me to do these very large dataset queries easily? Perhaps indexing a user's username that is contained inside each transaction?
First, rather than filtering history of transaction data using username I would suggest using userId which will never changed and always unique.
Second, I think saving the transaction globally (without using '/userId') is better. Because :
We need to able to summarize all transactions for accounting reason
If you think the query will be slow even after using index, you can consider loading part of query result using limitToFirst() just like pagination in web (infinite scroll in android). There is great tutorial here
The problem
I have a firebase application in combination with Ionic. I want the user to create a group and define a time, when the group is about to be deleted automatically. My first idea was to create a setTimeout(), save it and override it whenever the user changes the time. But as I have read, setTimeout() is a bad solution when used for long durations (because of the firebase billing service). Later I have heard about Cron, but as far as I have seen, Cron only allows to call functions at a specific time, not relative to a given time (e.g. 1 hour from now). Ideally, the user can define any given time with a datetime picker.
My idea
So my idea is as following:
User defines the date via native datepicker and the hour via some spinner
The client writes the time into a seperate firebase-database with a reference of following form: /scheduledJobs/{date}/{hour}/{groupId}
Every hour, the Cron task will check all the groups at the given location and delete them
If a user plans to change the time, he will just delete the old value in scheduledJobs and create a new one
My question
What is the best way to schedule the automatic deletion of the group? I am not sure if my approach suits well, since querying for the date may create a very flat and long list in my database. Also, my approach is limited in a way, that only full hours can be taken as the time of deletion and not any given time. Additionally I will need two inputs (date + hour) from the user instead of just using a datetime (which also provides me the minutes).
I believe what you're looking for is node schedule. Basically, it allows you to run serverside cron jobs, it has the ability to take date-time objects and schedule the job at that time. Since I'm assuming you're running a server for this, this would allow you to schedule the deletion at whatever time you wish based on the user input.
An alternative to TheCog's answer (which relies on running a node server) is to use Cloud Functions for Firebase in combination with a third party server (e.g. cron-jobs.org) to schedule their execution. See this video for more or this blog post for an alternative trigger.
In either of these approaches I recommend keeping only upcoming triggers in your database. So delete the jobs after you've processed them. That way you know it won't grow forever, but rather will have some sort of fixed size. In fact, you can query it quite efficiently because you know that you only need to read jobs that are scheduled before the next trigger time.
If you're having problems implementing your approach, I recommend sharing the minimum code that reproduces where you're stuck as it will be easier to give concrete help that way.
I'm looking to make my Firebase Database update at a particular time.
The way it should work is that, for a group, the leader sets a deadline time. The group votes on some stuff. At the deadline time, I would like the database to automatically tabulate the votes and store the response within.
I'm not sure how to set these types of rules for the database without doing a check whenever a member of the group is online and refreshes their feed. Also, this would allow any member to write to the vote-result field, which seems bad when I want it to just be automatic. It seems like there should be an easier way than this, but I just can't find anything.
It seems like the other option would be to set up a separate server that counts through all the time-frames and sends an update request when the time has allotted. But it seems like Firebase should have this built in. I'm sure I'm missing something. Thank you in advance.
EDIT: Here is a more comprehensive look at my usecase. I am looking into cron stuff now, as I think it will solve my problem, but I don't know.
1) Leader creates a group and invites friends to it. Event is created is firebase database. Group is created with a specific deadline.
2) Before deadline, leader and friends can vote on certain options. Basically they submit a dictionary to database with their votes.
3) On deadline, either just need to change the state of the group (from voting to closed) or calculate the vote response. Same problem, which is that I don't know to do do it at a certain time w/o using user clients.