I have authenticated users and would like to be able to protect myself from users spamming reads on a particular ref (thus driving up costs), how do you do this? I see the question here:
Firebase rate limiting in security rules?
That involves rate limiting writes by:
"The trick is to keep an audit of the last time a user posted a
message" - Kato
Is there a way to determine the last time a user read, and then limit their next read to some time interval from their last read? Probably better is limiting the amount of reads in a certain timeframe (say n reads per hour)?
Thanks
I just read that firebase uses a burstable billing plan, as seen here:
https://en.wikipedia.org/wiki/Burstable_billing
Such that you are not charged for spikes from a malicious user doing what I describe here or from a DDOS.
How about this:
A structure:
posts
post_id_0
msg: "a post about posting"
post_id_1
msg: "a post about pizza"
and a users node
users
uid_0
name: "biff"
post_activity
post_id_0
last_activity: "20170128100200"
pseudo-code since we don't know the platform
display a lists of posts in a table
a post about posting
a post about pizza
user taps or clicks 'a post about posting'
in code, get the last activity, which was today at 10:02 am
lastActivity = uid_0/post_activity/post_id_0/last_activity
and then compare the last activity to the current time, and if it's been accessed less then a minute ago, don't allow them to read it again
let currentTimestamp = current time (say it's 10:04 am)
if currentTimestamp - lastActivity > 1 minute then
show post details
update the lastActivity node to current timestamp
else
print("Posts can only be reviewed every minute")
In this case the last time the post was read 2 minutes ago, so allow it to be read again. If it was less than a minute it would be denied.
Also, if the user would tap/click post_id_1 the post activity would not be found which means the user has never viewed it before; in that case add it to the uid_0/post_activity node.
The same technique could be used with a counter instead of a time to limit the number of times the user reads a post.
Related
this is more of a logic question. So I’m building up an app that requires 30 days trial period mechanism. What would be the right way to do this?
I’m relying on Flutter/Firebase to do this!
This is how I’m planning to do it but not sure if that is correct way to do it
Adding the field named “free_trial” (boolean) in the firebase and setting it to true initially
Now I need to make that field false once the user crossed the 30 days trial period! How to do this logic?
Let's assume, this is a registration page and once the user signed up, there will be a field inserted in the firebase database which would be "free_trial" and it will be initially set to true
CollectionReference users = FirebaseFirestore.instance.collection('users');
users.add({
"free_trial": true
});
How to make it false once the user has crossed 30 days period. What logic/algorithm would be required to achieve this? like fetching today date and then checking against registration date (something like this)
Any help would be appreciated
A better approach would be to store the free_trial_expiration timestamp and derive whether the user is still in their free trial from that. That way you won't have to change any value when their free trial expires, you just have to check whether they are still in their free trial in your queries.
If you do want to set a field after a certain time period, you can do that with Cloud Functions. The two most common approaches are to either:
Periodically check all potentially expired documents (e.g. all documents that have "free_trial": true, and update the ones that have expired, OR
Create a Cloud Task for each trial user (tasks are quite cheap), and set that task to trigger another Cloud Function when the trial period expires, so that you can update that specific document.
For more on these approaches, also see:
Is there any TTL (Time To Live ) for Documents in Firebase Firestore
How to create an auto delete mechanism for firestore? (deleting data after time period)
Can a Firestore query listener "listen" to a cloud function?
Maybe this repo link would help you.
This is the current sample structure
Posts(Collection)
- post1Id : {
viewCount : 100,
likes : 45,
points : 190,
title : "Title",
postType : image/video
url : FileUrl,
createdOn : Timestamp,
createdBy : user20Id,
userName : name,
profilePic: url
}
Users(Collection)
- user1Id(Document):{
postsCount : 10,
userName : name,
profilePic : url
}
viewed(Collection)
- post1Id(Document):{
viewedTime : ""
}
- user2Id(Document)
The End goal is
I need to getPosts that the current user did not view and in points field descending order with paging.
What are the possible optimal solutions(like changing structure, cloud functions, multiple queries from client-side)?
I'm working on a solution to show trending posts and eliminate posts that are already seen by users or poor content. It's really painful to deal with two queries especially when the user base is increasing.
It's difficult to maintain the "viewed" collection and filter the new posts. Imagine having 1 million viewed posts and then filter for the un-seen posts.
So I figured a solution, which is not that great, but still cool.
So here is our data structure
posts(Collection)
--postid(document)
Title.
Description.
Image.
timestamp.
priority
This is a simple post structure with basic details. You can see I have added a Priority field. This field will do the magic.
How to use Priority.
We should query the posts that start with the higher priority and ends with lower priority.
When a user posts a new Post. Assign the current timestamp as the default priority.
When the user upvotes (Likes) a post increase the priority by 1 minute(60000 milliseconds)
When the user downvotes (Dislike) a post decrease the priority by 1 minute (60000 ms)
You can reset the priority every 24 hours. If you start browsing the feed today morning you will see posts with the last 24 hours in past. Once the 24-hour duration reached you can reset the priority to the present time. The 24-hour limit can be changed according to your needs. You may want to reset the limit every 15 min. because in every 15 min 100s of new posts might have added. This limit will ensure the repetition of content in the feed.
So when you start scrolling the feed you will get all the trending posts first then lower priority posts later. If you post a post today and people start upvoting it. It will get an increased lifetime, thus overpowers the poor content and when you downvote it, it will push down the post as long as users will not reach it.
Using timestamp as a priority because the old posts should lose priority with time. Even the trending posts today should lose the priority tomorrow.
Things to consider:
The lifetime can vary according to your needs.
The bigger the user base. You should lower the lifetime value. because if a post posted today is upvoted by 10,000 users it trends 6.9 days in the future. And if there are more than 100 posts that have been upvoted by more than 10,000 users then you will never get to see a new post in those 6.9 days.
So a trending post should hardly last a day or two.
So in this case you can give 10 seconds lifetime, it will give 1.1 day lifetime for 10,000 upvotes.
This is not a perfect solution but it may help you get started.
Edit: 11th June 2021
Nowadays, there are two more options that can help you solve such a problem. The first one would be the whereNotEqualTo method and the second one would be whereNotIn. You might choose one, or the other according to your needs.
Seeing your database structure, I can say you're almost there. According to your comment, you are hosting under the following reference:
Users(Collection) -> userId(Document) -> viewed(Collection)
As documents, all the posts a user has seen and you want to get all the post that the user hasn't seen. Because there is no != (not equal to) operator in Firestore nor a arrayNotContains() function, the only option that you have is to create an extra database call for each post that you want to display and check if that particular post is already seen or not.
To achieve this, first you need to add another property under your post object named postId, which will hold as String the actual post id. Now everytime you want to display the new posts, you should check if the post id already exist in viewed collection or not. If it dons't exist, display that post in your desired view, otherwise don't. That's it.
Edit: According to your comments:
So, for the first post to appear, it needs two Server calls.
Yes, for the first post to appear, two database calls are need, one to get post and second to see if it was or not seen.
large number of server calls to get the first post.
No, only two calls, as explained above.
Am I seeing it the wrong way
No, this is how NoSQL database work.
or there is no other efficient way?
Not I'm aware of. There is another option that will work but only for apps that have limited number of users and limited number of post views. This option would be to store the user id within an array in each post object and everytime you want to display a post, you only need to check if that user id exist or not in that array.
But if a post can be viewd by millions of users, storing millions of ids within an array is not a good option because the problem in this case is that the documents have limits. So there are some limits when it comes to how much data you can put into a document. According to the official documentation regarding usage and limits:
Maximum size for a document: 1 MiB (1,048,576 bytes)
As you can see, you are limited to 1 MiB total of data in a single document. So you cannot store pretty much everything in a document.
This is the current sample structure
Posts(Collection)
- post1Id : {
viewCount : 100,
likes : 45,
points : 190,
title : "Title",
postType : image/video
url : FileUrl,
createdOn : Timestamp,
createdBy : user20Id,
userName : name,
profilePic: url
}
Users(Collection)
- user1Id(Document):{
postsCount : 10,
userName : name,
profilePic : url
}
viewed(Collection)
- post1Id(Document):{
viewedTime : ""
}
- user2Id(Document)
The End goal is
I need to getPosts that the current user did not view and in points field descending order with paging.
What are the possible optimal solutions(like changing structure, cloud functions, multiple queries from client-side)?
I'm working on a solution to show trending posts and eliminate posts that are already seen by users or poor content. It's really painful to deal with two queries especially when the user base is increasing.
It's difficult to maintain the "viewed" collection and filter the new posts. Imagine having 1 million viewed posts and then filter for the un-seen posts.
So I figured a solution, which is not that great, but still cool.
So here is our data structure
posts(Collection)
--postid(document)
Title.
Description.
Image.
timestamp.
priority
This is a simple post structure with basic details. You can see I have added a Priority field. This field will do the magic.
How to use Priority.
We should query the posts that start with the higher priority and ends with lower priority.
When a user posts a new Post. Assign the current timestamp as the default priority.
When the user upvotes (Likes) a post increase the priority by 1 minute(60000 milliseconds)
When the user downvotes (Dislike) a post decrease the priority by 1 minute (60000 ms)
You can reset the priority every 24 hours. If you start browsing the feed today morning you will see posts with the last 24 hours in past. Once the 24-hour duration reached you can reset the priority to the present time. The 24-hour limit can be changed according to your needs. You may want to reset the limit every 15 min. because in every 15 min 100s of new posts might have added. This limit will ensure the repetition of content in the feed.
So when you start scrolling the feed you will get all the trending posts first then lower priority posts later. If you post a post today and people start upvoting it. It will get an increased lifetime, thus overpowers the poor content and when you downvote it, it will push down the post as long as users will not reach it.
Using timestamp as a priority because the old posts should lose priority with time. Even the trending posts today should lose the priority tomorrow.
Things to consider:
The lifetime can vary according to your needs.
The bigger the user base. You should lower the lifetime value. because if a post posted today is upvoted by 10,000 users it trends 6.9 days in the future. And if there are more than 100 posts that have been upvoted by more than 10,000 users then you will never get to see a new post in those 6.9 days.
So a trending post should hardly last a day or two.
So in this case you can give 10 seconds lifetime, it will give 1.1 day lifetime for 10,000 upvotes.
This is not a perfect solution but it may help you get started.
Edit: 11th June 2021
Nowadays, there are two more options that can help you solve such a problem. The first one would be the whereNotEqualTo method and the second one would be whereNotIn. You might choose one, or the other according to your needs.
Seeing your database structure, I can say you're almost there. According to your comment, you are hosting under the following reference:
Users(Collection) -> userId(Document) -> viewed(Collection)
As documents, all the posts a user has seen and you want to get all the post that the user hasn't seen. Because there is no != (not equal to) operator in Firestore nor a arrayNotContains() function, the only option that you have is to create an extra database call for each post that you want to display and check if that particular post is already seen or not.
To achieve this, first you need to add another property under your post object named postId, which will hold as String the actual post id. Now everytime you want to display the new posts, you should check if the post id already exist in viewed collection or not. If it dons't exist, display that post in your desired view, otherwise don't. That's it.
Edit: According to your comments:
So, for the first post to appear, it needs two Server calls.
Yes, for the first post to appear, two database calls are need, one to get post and second to see if it was or not seen.
large number of server calls to get the first post.
No, only two calls, as explained above.
Am I seeing it the wrong way
No, this is how NoSQL database work.
or there is no other efficient way?
Not I'm aware of. There is another option that will work but only for apps that have limited number of users and limited number of post views. This option would be to store the user id within an array in each post object and everytime you want to display a post, you only need to check if that user id exist or not in that array.
But if a post can be viewd by millions of users, storing millions of ids within an array is not a good option because the problem in this case is that the documents have limits. So there are some limits when it comes to how much data you can put into a document. According to the official documentation regarding usage and limits:
Maximum size for a document: 1 MiB (1,048,576 bytes)
As you can see, you are limited to 1 MiB total of data in a single document. So you cannot store pretty much everything in a document.
Below question is related to NetSuite Support Module.
We want to send an email notification to the support rep assigned to the case if the time since last modification of the case has exceeded 48 hours. This notification needs to be sent for each case as soon as it ages over 48 hours since last modification,
I tried a saved search notifiaction, but that does not work as the case which exceeds 48 hours is not a new record.
I am not able to figure out what the trigger would be for a workflow or a script to make this notification work.
Any ideas?
Thanks
Use a workflow
make initiation "Scheduled"
use a condition that the Case is not closed or whatever works for you
under Saved Search set up a search that ids cases older than 48 hours
the saved search will run every half an hour and pick up your aging cases.
To finish this you need to decide if the same case will get another every half an hour until it's been dealt with. If so the workflow can end when the email goes out. If not then the workflow needs to go into some state waiting for the next escalation or waiting for some delay until you can ping the assignee again.
As the title states, how can i stop two users from booking the same appointment time.
Example, two users logging, on there screen they can both see that a 1pm appointment is available. They both try to book themselves into that appointment time.
How can i stop this from happening and ensure only one user can book it, then refresh the screen to show the next available booking time to the other user.
Thanks.
That's a fairly classic use case. You can simply display the appointment plan at a specific time. That can remain static or you can set up a periodic process (for example, every five seconds) to update the plan with new information.
Then, when the user/operator decides to book a free timeslot, it tries to do an (atomic) update that will fail if someone else has slipped in (using primary key or some other unique constraint). The atomicity of the update operation guarantees that only one person can book the timeslot. If the update works, voila, you have your time booked.
If it fails, notify the user of that fact and then load up the new appointment plan.
Rinse and repeat until the user has their booking or they wander off, disgruntled.
I do not think that this is optimal. Why don't you opt for a first clicked/first served pattern ?
What may happen if you have 10 users or more viewing the same page ? A user viewing a page does not mean that he will click on a time slot. So IMHO wait for a user to REALLY click and then notify eventually other users that a time slot they are viewing has been booked (As it happens with StackOverFlow when somebody answers a question while you are answering)