I have this situation:
I have a collection named Lists -> lists of sent e-mails;
I have a collection named Emails -> all the sent emails;
I need to show the sent e-mails for an specific list, lets say "list_id: 1";
And, on the same page, I need to show the total sent e-mails on that day, without filtering by list_id, but by submitted_date;
How's that possible on Meteor?
Thank you!
Actually I've found a good workaround for my issue. Subscribing to a count:
Meteor subscribe to a count
There is an example on Meteor's documentation:
http://docs.meteor.com/#meteor_publish
Maybe this will help someone in the future.
Those requests will depend on the structure of your Lists and Emails collections.
If your Lists collection is base on _id and an array of Email _id.
Then to show e-mail for a specific list:
Meteor.publish('emails_for_list', function(list){
return Emails.find({
_id: {$in: list.emails}
});
});
As for counting the total number of emails sent on a day.
return Emails.find({submitted_date: today}).count();
Again it all depends on how you build you collections.
Instead of having a Lists collection you can as easy put an list_number on each email object.
Related
I'm toying with several ideas for using Firestore for a social media feed. So far, the ideas I've had haven't panned out, so for this one I'm hoping to get the community's feedback.
The idea is to allow users to post information, or to record their activity, and to any user following/subscribed to that information, display it. The posts information would be in a root collection called posts.
The approaches, as far as I can tell, require roughly the same number of reads and writes.
One idea is to have within the users/{userId} have a field called posts which is an array of documentIds that I'm interested in pulling for the user. This would allow me to pull directly from posts and get the most up-to-date version of the data.
Another approach seems more Firebasey which is to store documents within users/{userId}/feeds that are copies of the posts themselves. I can use the same postID as the data in posts. Presumably, if I need to update the data for any review, I can use a group collection query to get all collections called feeds, where the docID is equal (or just create a field to do a proper "where", "==", docId).
Third approach is all about updating the list of people who should view the posts. This seems better as long as the list of posts is shorter than the lists of followers. Instead of maintaining all posts on every follower, you're maintaining all followers on each post. For every new follower, you need to update all posts.
This list would not be a user's own posts. Instead it would be a list of all the posts to show that user.
Three challengers:
users/{userId} with field called feed - an array of doc Ids that point to the global posts. Get that feed, get all docs by ID. Every array would need to be updated for every single follower each time a user has activity.
users (coll)
-> uid (doc)
-> uid.feed: postId1, postId2, postId3, ...] (field)
posts (coll)
-> postId (doc)
Query (pseudo):
doc(users/{uid}).get(doc)
feed = doc.feed
for postId in feed:
doc(posts/{postId}).get(doc)
users/{userId}/feed which has a copy of all posts that you would want this user to see. Every activity/post would need to be added to every relevant feed list.
users (coll)
-> uid (doc)
-> feed: (coll)
-> postId1 (doc)
-> postId2
-> postId3
posts (coll)
-> postId (doc)
Query (pseudo):
collection(users/{uid}/feed).get(docs)
for post in docs:
doc(posts/{post}).get(doc)
users/{userId}/feed which has a copy of all posts that you would want this user to see. Every activity/post would need to be added to every relevant feed list.
users (coll)
-> uid (doc)
posts (coll)
-> postId (doc)
-> postId.followers_array[followerId, followerId2, ...] (field)
Query (pseudo):
collection(posts).where(followers, 'array_contains', uid).get(docs)
Reads/Writes
1. Updating the Data
For the author user of every activity, find all users following that
user. Currently, the users are stored as documents in a collection, so this is followerNumber document reads. For each of the users, update their array by prepending the postId this would be followerNumber document writes.
1. Displaying the Data/Feed
For each fetch of the feed: get array from user document (1 doc read). For each postId, call, posts/{postId}
This would be numberOfPostsCalled document reads.
2. Updating the Data
For the author user of every activity, find all users following that
user. Currently, the users are stored as documents in a collection, so this is followerNumber document reads. For each of the users, add a new document with ID postId to users/{userId}/feed this would be followerNumber document writes.
2. Displaying the Data/Feed
For each fetch of the feed: get a certain number of posts from users/{userId}/feed
This would be numberOfPostsCalled document reads.
This second approach requires me to keep all of the documents up to date in the event of an edit. So despite this approach seeming more firebase-esque, the approach of holding a postId and fetching that directly seems slightly more logical.
3. Updating the Data
For every new follower, each post authored by the person being followed needs to be updated. The new follower is appended to an array called followers.
3. Displaying the Data
For each fetch of the feed: get a certain number of posts from posts where uid == viewerUid
Nice, when I talk about what is more optimal I really need a point or a quality attribute to compare, I' will assume you care about speed (not necessary performance) and costs.
This is how I would solve the problem, it involves several collections but my goal is 1 query only.
user (col)
{
"abc": {},
"qwe": {}
}
posts (col)
{
"123": {},
"456": {}
}
users_posts (col)
{
"abc": {
"posts_ids": ["123"]
}
}
So far so good, the problem is, I need to do several queries to get all the posts information... This is where cloud functions get into the game. You can create a 4th collection where you can pre-calculate your feed
users_dashboard
{
"abc": {
posts: [
{
id: "123", /.../
}, {
id: "456", /.../
}
]
}
}
The cloud function would look like this:
/* on your front end you can manage the add or delete ids from user posts */
export const calculateDashboard = functions.firestore.document(`users_posts/{doc}).onWrite(async(change, _context) {
const firestore = admin.firestore()
const dashboardRef = firestore.collection(`users_dashboard`)
const postRef = firestore.collection(`posts`)
const user = change.after.data()
const payload = []
for (const postId of user.posts_ids) {
const data = await postRef.doc(postId).get().then((doc) => doc.exists ? doc.data() : null)
payload.push(data)
}
// Maybe you want to exponse only certain props... you can do that here
return dashboardRef.doc(user.id).set(payload)
})
The doc max size is 1 MiB (1,048,576 bytes) that is plenty of data you can store in, so you can have like a lot of posts here. Let's talk about costs; I used to think firestore was more like to have several small docs but I've found in practice it works equally well with big size into a big amount of docs.
Now on your dashboard you only need query:
const dashboard = firestore.collection(`users_dashboard`).doc(userID).get()
This a very opinionated way to solve this problem. You could avoid using the users_posts, but maybe you dont want to trigger this process for other than posts related changes.
It looks like your second approach is best in this situation.. I don't really understand what #andresmijares was trying to do and he mentioned something like storing posts in a document which is not a good approach, imagine if you have more than 20K posts (which what I think a document can hold) then the document won't be able to store any more data.. a better approach is to store posts as a document inside a Collection (just like in your 2nd option).. So let's recall here what's the best approach.
1)_ You share a post in the (posts "Collection") and in users you're following's (Feed "Collection").. maybe this can be done with cloud function and let's not forget to aggregate (with cloud functions also) the number of posts that needs to appear in the user's profile.
2)_ You follow a user and get all of their posts from the (posts "Collection") into your (Feed "Collection") this way you get to see all of their posts on your feed.
with this approach, there will be a lot of writes once but the read will be fast.. and if your app is about reading more and writing less then there's nothing to worry about unless i'm wrong.
I have a list of contacts (List A) that I would like to send a particular email to. However, the issue here is that the contacts in this List A are present in the Bronto Contacts database already, and I have no way of applying segmentation so that I get the same contacts as the ones in List A.
Is there a way I can send these contacts an email from Bronto?
If you're looking to do this programmatically, Bronto has an API that you can use to look up the contacts, as well as send a message. Otherwise you'd likely have to create a new list that mirrors List A (which would be very time-consuming - but can also be done programmatically via their API).
I am looking create a social-media feed using Firebase. My data is structured like this:
users: {
uid: {
... // details
}
}
friends: {
uid: {
friends: { // sub collection
fuid: {
... // details
}
}
}
}`
posts: {
postId: {
postedBy: uid
... // details
}
}
Now I am trying to get the posts from all friends of the user, limit it to the most recent 10 posts, and then create a scrolling directive that queries the next set of 10 posts so that the user doesn't have to query and load posts^N for friends^N on the page load. But I'm not really sure how to query firebase in an effective manner like this, for the user's friends and then their posts.
I have the scrolling directive working, taken from Jeff Delaney's Infinite Scrolling Lesson on AngularFirebase.com. But it only handles the posts (boats in the tutorial) collection as a whole, without selectively querying within that collection (to check if the user is a friend).
The only solution that I could think of was to query all of the user's friends posts, store that in an array, and then chunk load the results in the DOM based on the last batch of posts that were loaded. This just seems like it could be really inefficient in the long-haul if the user has 100's of friends, with 100's of posts each.
If I get it right, you are duplicating the post for each user in the user's friend list right? I don't think it is a good idea if your app escalates... At this time, the cost for 100k doc writes is $0,18, so:
Imagine that a user of your app have 1000 friends. When he posts anything, you are making 1000 writes in the database. imagine that you have 1000 active users like him. You have just made 1.000.000 writes now and paid $1.80.
Now even worse: you probably have on each post, a duplicated field for user displayName and a profileImageUrl. Imagine that this user has 500 posts in his history and have just changed his profile picture. You will have to update one of the fields for each post on each of his 1000 friend's feed right? You will be doing 1000 * 500 = 500.000 writes just for updating the profileImageUrl! and if the user didn't like the photo? he tries 3 new photos and now in 10 minutes you had made 2.000.000 writes in the database. This means you will be charged $3.60. It may not seems too much, but pay attention that we're talking about 1 single user in a single moment. 1000 users changing profile picture 4 times in the same day and you are paying $3,600.00.
Take a look at this article: https://proandroiddev.com/working-with-firestore-building-a-simple-database-model-79a5ce2692cb#7709
I ended up solving this issue by leveraging Firebase Functions. I have two collections, one is called Posts and the other is called Feeds. When a user adds a post, it gets added to the Posts collection. When this happens, it triggers a Firebase Function, which then grabs the posting user's UID.
Once it has the UID, it queries another collection called Friends/UID/Friends and grabs all of their friend's UID's.
Once it has the UID's, it creates a batch add (in case the user has more than 500 friends), and then adds the post to their friend's Feeds/UID/Posts collection.
The reason I chose this route, was a number of reasons.
Firebase does not allow you to query with array lists (the user's friends).
I did not want to filter out posts from non-friends.
I did not want to download excessive data to the user's device.
I had to paginate the results in order from newest to oldest.
By using the above solution, I am now able to query the Feeds/UID/Posts/ collection, in a way that returns the next 10 results every time, without performance or data issues. The only limitation I have not been able to get around completely is it takes a few seconds to add the post to the user's personally feed, as the Function needs time to spin up. But this can be mitigated by increasing the memory allocation for that particular function.
I also do the above listed for posts that are edited and or deleted.
I think i have a solution for Firestore Social Feed queries. Not sure if it works but here it is;
A Friends collection keeps the friends UUID'S list as an array in a document. Every document in this collection is for a user. So when the user logs in we first have the friends list with a cloud function with "one read" right? All friends id's are in one document. And we also put a lastchecked time stamp to this document. Everytime we get friends array we record the date.
Now a cloud function can check all users posts one by one. As i understand latest IN queries allow an array up to 10 UUID's. So if user has 100 friend query will end in ten rounds. Now we have sth to serve.
Instead of directly serving the posts we create a collection for every user. We will put all this collected data to document but we slice it to days. Let's pretend we already have older posts in this usersfeed collection (every day as a document). So we had a last time check on our friends document. We query now -> last checked date. This way we only fetched unseen posts and sliced them daily (if they belong to more days ofcourse)
So while this happens on cloud function we already served the previous feed document. And when collection has new document firestore already listens and adds right? If the user scrolls down we get the previous days document. So every document will have more then one posts data as map / array.
This saves many read counts i guess.
I have below firestore collections.
-Converstions(collection)
(document) {participants: {userid1: true, userid2: true}, messages: [subcollection]}
-Users(collection)
(document)(userid1){userName: 'Test1', ...}
(document)(userid2){userName: 'Test2', ...}
Now I need to query for conversations a users is in, I can do this with
firebase.firestore().collection('conversations')
.where(`participants.${uid}`, '==', true);
What this does is gets all conversation a users is participating in, I need to now get the user details from id for each document in those conversation. If we make another call to UserRef to get the user details it will make extra request for each conversation data. I wanted to know if there is easy way to get user details in single call to the firebase.
When a user is added to a document, you could also add some display information about that user (either from the app or Cloud Functions).
There is no way to return data referenced elsewhere. You either need to duplicate or make multiple fetches.
I'm designing a chat app much like Facebook Messenger. My two current root nodes are chats and users. A user has an associated list of chats users/user/chats, and the chats are added by autoID in the chats node chats/a151jl1j6. That node stores information such as a list of the messages, time of the last message, if someone is typing, etc.
What I'm struggling with is where to make the definition of which two users are in the chat. Originally, I put a reference to the other user as the value of the chatId key in the users/user/chats node, but I thought that was a bad idea incase I ever wanted group chats.
What seems more logical is to have a chats/chat/members node in which I define userId: true, user2id: true. My issue with this is how to efficiently query it. For example, if the user is going to create a new chat with a user, we want to check if a chat already exists between them. I'm not sure how to do the query of "Find chat where members contains currentUserId and friendUserId" or if this is an efficient denormalized way of doing things.
Any hints?
Although the idea of having ids in the format id1---||---id2 definitely gets the job done, it may not scale if you expect to have large groups and you have to account for id2---||---id1 comparisons which also gets more complicated when you have more people in a conversation. You should go with that if you don't need to worry about large groups.
I'd actually go with using the autoId chats/a151jl1j6 since you get it for free. The recommended way to structure the data is to make the autoId the key in the other nodes with related child objects. So chats/a151jl1j6 would contain the conversation metadata, members/a151jl1j6 would contain the members in that conversation, messages/a151jl1j6 would contain the messages and so on.
"chats":{
"a151jl1j6":{}}
"members":{
"a151jl1j6":{
"user1": true,
"user2": true
}
}
"messages":{
"a151jl1j6":{}}
The part where this gets is little "inefficient" is the querying for conversations that include both user1 and user2. The recommended way is to create an index of conversations for each user and then query the members data.
"user1":{
"chats":{
"a151jl1j6":true
}
}
This is a trade-off when it comes to querying relationships with a flattened data structure. The queries are fast since you are only dealing with a subset of the data, but you end up with a lot of duplicate data that need to be accounted for when you are modifying/deleting i.e. when the user leaves the chat conversation, you have to update multiple structures.
Reference: https://firebase.google.com/docs/database/ios/structure-data#flatten_data_structures
I remember I had similar issue some time ago. The way how I solved it:
user 1 has an unique ID id1
user 2 has an unique ID id2
Instead of adding a new chat by autoId chats/a151jl1j6 the ID of the chat was id1---||---id2 (superoriginal human-readable delimeter)
(which is exactly what you've originally suggested)
Originally, I put a reference to the other user as the value of the chatId key in the users/user/chats node, but I thought that was a bad idea in case I ever wanted group chats.
There is a saying: https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it
There might a limitation of how many userIDs can live in the path - you can always hash the value...