How exactly to merge multiple streams in firebase firestore - firebase

Before you say this is a duplicate question or that I should use nested stream builders, please hear me out.
I am designing a social media type application. And I want users to receive updates whenever someone they are following posts something in their "my Followers Posts" collection.
In the app, the app will check firebase rtdb for the current user's following list(the people he is following) and make a list of their uids.
I plan on using said list to create a list of streams (ordered by time, of course) and merge them into one stream that shall then be fed into a stream builder on the private feed page.
On this page, the user will be able to easily follow what their people of interest have been posting.
I figured such a system is a lot more cost efficient than every user having a document in the "private Feed" collection and whenever someone posts something, the app reads their list of followers and then promptly posts an update in each and every one of their private feeds.
Because... Picture someone with 2 million followers. That's 2 million writes instantly. And later on, 2 million reads.
I figured it's a lot more cost efficient for the poster to just put the post in their "publicFeed" and the different followers simply listen in onto that feed and keep up to tabs with them.
But.. This requires implementing a merging of multiple streams (more than 2). How do I do this?
I have tried reading into RxDart but it is total Greek to me. I am relatively a beginner in dart. I've only been coding for about 5 months now.

You can use StreamGroup from the async package: https://pub.dev/documentation/async/latest/async/StreamGroup-class.html to group events from multiple streams - it's well documented and maintained by the dart team. If you have no RxDart experience this is a good choice. It does not have all the features of rx but for a beginner it should be easier to wrap your head around this

I recently had a similar case, and what I suggest You to do is this
(I'm using cloud firestore, but I'm sure You have your streams already written so the important part is the usage of multiple streams):
You have to add this plugin to pub spec.yaml:
https://pub.dev/packages/rxdart
Here is the repository for (in your case posts, let's say newPosts, oldPosts):
class PostRepository {
static CollectionReference get collection => yourCollectionRef;
static Stream<List<Post>> newPosts() {
Query query = collection
.where('Your condition like was viewed', isEqualTo: false)
.orderBy('updateDate', descending: true)
.limit(50);
return query.snapshots().map<List<Post>>((querySnapshot) {
final _newPosts = querySnapshot.documents.map((doc) {
final post = Post.fromDoc(doc);
return post;
}).where((p) => p != null);
return _newPosts
});
}
static Stream<List<Post>> oldPosts() {
Query query = collection
.where('Your condition like was viewed', isEqualTo: true)
.orderBy('updateDate', descending: true)
.limit(50);
return query.snapshots().map<List<Post>>((querySnapshot) {
final _oldPosts = querySnapshot.documents.map((doc) {
final post = Post.fromDoc(doc);
return post;
}).where((p) => p != null);
return _oldPosts
});
}
}
Then to get multiple streams (those two from above combined), do like this in your widget class:
IMPORTANT! you have to import this - import 'package:rxdart/streams.dart';
List<Post> newPosts;
List<Post> oldPosts;
Widget _pageContent() {
return SingleChildScrollView(
child: Column(
children: [
ListView.builder(
shrinkWrap: true,
physics: NeverScrollableScrollPhysics(),
itemCount: newPosts.length,
itemBuilder: (context, index) {
return ListTile(
title: Text(newPosts[index].title)
);
}
),
ListView.builder(
shrinkWrap: true,
physics: NeverScrollableScrollPhysics(),
itemCount: oldPosts.length,
itemBuilder: (context, index) {
return ListTile(
title: Text(oldPosts[index].title)
);
}
)
]
)
);
}
Widget _posts() {
return StreamBuilder(
stream: CombineLatestStream.list([
PostRepository.getNewPosts(),
PostRepository.getOldPosts()
]),
builder: (context, snap) {
if (snap.hasError) {
debugPrint('${snap.error}');
return ErrorContent(snap.error);
} else if (!snap.hasData) {
return Center(
child: CircularProgressIndicator(),
);
}
newPosts = snap.data[0];
oldPosts = snap.data[1];
return _pageContent();
}
);
}
I wrote the code like from head, so there may be some small errors, but I hope You get the point, enjoy :)

Related

How can I mix a statenotifier, with streamproviders that fetch 2 collections from Firebase to get constant update on a chat list

This is my first question here and I hope I’m not making it too complex.
So, I’m a junior programmer and I’ve start learning flutter, firebase and riverpod a couple of months ago and I’m stuck in a specific project.
For part of the app, what I need to do is something very similar to WhatsApp:
A screen with all the user chats,
This screen (or part of it) should update every time a chat gets a new message, showing the last message snippet and turning into bold if not read,
The shown chats should change between unarchived and archived, depending on a button in the UI.
Couldn’t have picked up an easier starting project, right? ;) (now every time I look at WhatsApp I say wow!)
Regarding firebase/firestore I’m fetching 2 different collections for this:
the sub-collection ‘chats’ within the ‘user_chats’ collection: where I get all the chat Ids plus it’s status (if this chat is archived and if the last message was read), for the current user,
the main ‘chats’ collection, where I have the main info of each chat.
At this moment I’m doing this:
In the chats_screen (UI) I’m fetching my chat provider: userChatsProvider
(note: this is just an example of the UI implementation. I have another implementation for it, but as long I get the chatName and lastMsgContent updated for each chat, perfect.)
class ChatsScreen extends ConsumerWidget {
const ChatsScreen({Key? key}) : super(key: key);
#override
Widget build(BuildContext context, WidgetRef ref) {
(…)
return Scaffold(
appBar: (…) // not relevant for this question
body: Center(
child: Consumer(
builder: (BuildContext context, WidgetRef ref, Widget? child) {
return ref.watch(userChatsProvider).when(
loading: () => const CircularProgressIndicator(),
error: (err, st) => Center(child: Text(err.toString())),
data: (chatData) {
return Column(
children: [
// button to get archived chats
child: TextButton(
child: (…)
onPressed: () {}),
),
Expanded(
child: ListView.builder(
itemCount: chatData.length,
itemBuilder: (ctx, index) => Row(
children: [
Text (chatData[index].chatName!),
Text (chatData[index].lastMsgContent!),
]
),
),
),
]
);
}
);
}
)
)
);
}
}
In the chats_provider (provider) I’m fetching the 2 repository providers and joining them into a specific model I’ve created for this screen:
final userChatsProvider = FutureProvider.autoDispose<List<ChatScreenModel>>((ref) async {
const String userId = ‘XXX’; // this will be substituted by a dynamic variable with the current user Id
try {
final List<UserChat> userChats =
await ref.watch(userChatsRepositoryProvider).get(userId: userId);
final List<String> chatIdList = userChats.map<String>((e) => e.id).toList();
final List<Chat> chats =
await ref.watch(chatsRepositoryProvider).get(chatIds: chatIdList);
// ref.maintainState = true;
return toChatScreenModel(chats, userChats);
} on Exception catch (e) {
throw const CustomException();
}
});
In the repositories I’m fetching the firestorm collections I mentioned above. Here’s an example:
final userChatsRepositoryProvider =
Provider<UserChatsRepository>((ref) => UserChatsRepository(ref.read));
class UserChatsRepository {
final Reader _read;
const UserChatsRepository(this._read);
Future<List<UserChat>> get({required String userId}) async {
try {
final snap = await _read(firebaseFirestoreProvider)
.collection('users_chats/$userId/chats')
.get();
// Maybe this step is not necessary, but I’ve decided to transform the data into temporary models, before sending to provider
List<UserChat> userChats =
snap.docs.map((doc) => UserChat.fromJson(doc.data(), doc.id)).toList();
return userChats;
} on FirebaseException catch (e) {
throw CustomException(message: e.message);
}
}
}
And by the way, this is the model I’m sending to the UI:
class ChatScreenModel {
final String? id;
final String? chatName;
final String? chatImage;
final String? lastMsgContent;
final String? lastMsgDate;
final ChatType? chatType;
final bool? archived;
final bool? msgRead;
ChatScreenModel({
this.id,
this.chatName,
this.chatImage,
this.lastMsgContent,
this.lastMsgDate,
this.chatType,
this.archived,
this.msgRead,
});
Problems with this implementation:
I’m getting the user chats in the screen, but they don’t update since I’m not using a stream. So I get a snapshot, but it will only update if I leave and enter that chats_screen again. And it would be important to have it updating with a stream.
I’m showing all the chats, and not a filtered list with only the unarchived chats.
Also, related with the previous point, I still don’t have the archived button working, to only show the archived chats.
I’ve lost many, many hours trying to understand how I could implement a stream provider and a state notifier provider in this workflow.
Tried many combinations, but without success.
Can anyone help me understand how to do this?
Priority: transform these providers into stream providers (so it updates the UI constantly).
Nice to have: also include the archived/unarchived dynamic to filter the chats that appear and be able to switch between them.
Thanks a lot. :)

Does Flutter Streambuilder use cache on setState or retrieve all documents again from the firestore database?

I am building a chat functionality for my Flutter application.
To ensure that the latest message of a user is always showed, I need the real-time functionality of a Streambuilder.
However, I need to introduce pagination as well to avoid that all chat messages from the database are loaded from the database each time the Widget is rebuilt.
I've implemented this behaviour successfully with the code below. Each time the maximum scrollExtent is reached, the Widget is rebuilt with a higher documentLimit.
My question is now the following:
When setState is called, will the data from the streambuilder that was already there be read again from the database? Or will the Streambuilder use its cache where possible and only use reads for the documents that it does not have?
In other words, with a documentLimit of 20, will each set state only cost me maximum 20 extra reads? Or will it cost me the entire new documentLimit?
void initState() {
maxMessageToDisplay = 20;
_scrollController = ScrollController();
_scrollController.addListener(() {
if (_scrollController.position.pixels == _scrollController.position.maxScrollExtent) {
setState(() {
maxMessageToDisplay += 20;
});
}
});
super.initState();
}
Widget build(BuildContext context) {
return StreamBuilder<QuerySnapshot>(
stream: _firestore.collection('chat').limit(maxMessageToDisplay).orderBy('timestamp', descending: true).snapshots(),
builder: (context, snapshot) {
final messages = snapshot.data.documents;
messages.sort((a, b) => b.data['timestamp'].compareTo(a.data['timestamp']));
var format = new DateFormat("Hm");
List<MessageBubble> messageBubbles = [];
for (var message in messages) {
final messageText = message.data['text'];
final messageSender = message.data['sender'];
final messagePhoto = message.data['photo'];
final messageUserId = message.data['uid'];
final messageTime = format.format(DateTime.fromMillisecondsSinceEpoch(message.data['timestamp'], isUtc: false));
final messageBubble = MessageBubble(
sender: messageSender,
text: messageText,
photo: messagePhoto,
time: messageTime,
userId: messageUserId,
);
messageBubbles.add(messageBubble);
}
return Expanded(
child: ListView(
controller: _scrollController,
reverse: true,
padding: EdgeInsets.symmetric(vertical: 10.0, horizontal: 10.0),
children: messageBubbles,
),
);
},
);
}
}
By changing the limit you create a whole new query and all data for it is loaded as it is for each new query.
To avoid loading all data you could implement a pagination as shown here. Where you would use the last loaded doc and startAfter to load the next 20 elements from the point where the previous query stoped. It would make your code more complicated to handle multiple realtime queries and keep track of the last docs.
You could also consider to have only the last 20 messages in realtime and if you need the older ones to get them with the get call only once because they would probably not change. Make sure to store all messages from the listener and get calls into a single List and make sure not to store duplicates accidentaly.
I would also consider if it is so importand to avoid loading all data when watching older messages. Does that happen so often in your app? I have multiple chat apps with Firebase that use the same logic with just increasing the limit. Users very rarely watch older messages so implementing such workflows with pagination would not have much effect on the billing and total read amount.

Does Firestore perform a read every time a page is built?

I'm designing an app with Flutter and using Firestore as my database.
I have this page which is basically taking everything from a specific collection in Firestore and listing it for the user.
I'm not yet so familiar with how things work in the Firestore side and would like to make this is optimized as possible, to avoid unnecessary database reads. My question in this situation is: will Firestore perform a read in this collection every time the user opens this screen? Or only when something in this collection changes?
Let's say the user opened this list, but then went to his profile, and after that went back to this list. If the collection didn't have any changes in the meanwhile, will there be 1 or 2 reads in the database?
Would I need to use a Stream for that to happen?
Implementation of this List in Flutter
FutureBuilder<List<Store>>(
future: DatabaseService().storeList,
builder: (context, snapshot) {
if (!snapshot.hasData) {
return Loading();
} else {
return ListView.builder(
scrollDirection: Axis.vertical,
shrinkWrap: true,
itemBuilder: (context, index) => StoreTile(store: snapshot.data[index]),
itemCount: snapshot.data.length,
);
}
},
);
Implementation of Database stuff
class DatabaseService {
// Collection reference
final CollectionReference storeCollection = FirebaseFirestore.instance.collection('stores');
// Make a store list from snapshot object
List<Store> storeListfromSnapshot(QuerySnapshot snapshot) {
return snapshot.docs.map((doc){
return Store(
name: doc.data()['name'] ?? '',
description: doc.data()['description'] ?? '',
image: doc.data()['image'] ?? ''
);
}).toList();
}
// Get instantaneous stores list
Future<List<Store>> get storeList async {
return storeListfromSnapshot(await storeCollection.get());
}
}
Every get() call will fetch the data from the server. So here since you are using get() on a collection and let's say you have 5 documents, then it will be 5 document read. If you enter the page again, then it will also count 5 document reads.
If you want to get data from the cache then use snapshots() which returns data from the cache and if there are any modification then it will return data from the server.
You can also add the parameter GetOptions to the get() method for example:
get(GetOptions(source : Source.cache))
This will always get the data from cache, ignoring the server completely.
You can also monitor document read here:
https://firebase.google.com/docs/firestore/monitor-usage
Also note if you keep the firebase console open then you will get unexpected reads.

How to get sub-collections of every collection in Firestore using Flutter

I'm building a task management app where every user has multiple projects (collections), inside every project there are tasks (collections), and inside every task there are sub-tasks.
I was able to retrieve all the projects using this code:
User user = Provider.of<User>(context);
CollectionReference projects = Firestore.instance.collection('users').document(user.uid).collection("projects");
But my question is how can I get all the tasks and sub-tasks (if they exist) from every project and list them in a StreamBuilder?
Here's my current code:
Flexible(
child: StreamBuilder<QuerySnapshot>(
stream: projects.snapshots(),
builder: (BuildContext context,
AsyncSnapshot<QuerySnapshot> snapshot) {
if (snapshot.hasError) {
return Text('Something went wrong');
}
if (snapshot.connectionState == ConnectionState.waiting) {
return Text("Loading");
}
return new ListView(
children:
snapshot.data.documents.map((DocumentSnapshot document) {
return new ListTile(
title: new Text(document.data['project_name']),
subtitle: new Text(document.data['project_description']),
);
}).toList(),
);
},
),
),
If the project has tasks I want to display them below the project description, and if the task has sub-tasks I want to show them below its parent task.
It sounds like you're essentially asking how to list nested subcollections, which is not possible for web and mobile clients. See also:
How to list subcollections in a Cloud Firestore document
How to list subcollection on firestore?
Also, all Firestore queries are shallow, meaning they only consider documents in one collection at a time. There are no deep or recursive queries.
Consider instead maintaining the list of projects in a document that you can get() directly. You can use that list to then query each subcollection directly.
There are different kind of implementations for this scenario, basically subcollections are located under one single document, the query identifies a group of documents, when you know that the query only identifies one document, the Firestore client doesn't know that, and even if it did, it doesn,t know the full path for that document.
You need to execute a query for getting individual documents, and then you can get the subcollection of each one.
You can find a good example using flutter here.
I had a similar issue, and used bloc provider alongside streams,
I first created a model for Json parsing (In my case I had an app whereby users can create campaigns, these campaigns were saved as a sub-collection in the UserData collection, the document IDs were the userID from FirebaseAuth(now User in flutter migration). The objective was to loop through UserData collection for each user's subcollection (UserCampaigns) and retrieve all of them and display in the UI with bloc. userCampaignRepository:
List<UserCampaignModel> UserCampaignList = [];
CreateListofAllUserCampaigns(QuerySnapshot snapshot) async {
var docs = snapshot.docs;
for (var Doc in docs) {
UserCampaignList.add(
UserCampaignModel.fromEntity(UserCampaignEntity.fromSnapshot(Doc)));
}
return UserCampaignList;
}
Stream<List<UserCampaignModel>> getListUserCampaigns() async* {
final collection = FirebaseFirestore.instance.collection('UserData');
final List list_of_users = await collection.get().then((value) => value.docs);
for (int i = 0; i < list_of_users.length; i++) {
FirebaseFirestore.instance.collection("UserData")
.doc(list_of_users[i].id.toString())
.collection("UserCampaigns")
.snapshots()
.listen(CreateListofAllUserCampaigns);
}
yield UserCampaignList;
}
In the bloc: I have 2 events, one to initiate the Load of Events by listening to the stream and calling the loader event.
if (event is InitiateLoadOfAllUserCampaignsEvent) {
yield AllUserCampaignsState.initial();
try {
_userCampaignSubscription?.cancel();
_userCampaignSubscription = _userCampaignRepository
.getListUserCampaigns()
.listen((allCampaigns) {
print("Load event is called");
add(
LoadAllUserCampaignsEvent(allUserCampaigns: allCampaigns),
);
});
} catch (e) {
print("error is ${e}");
yield AllUserCampaignsState.allUserCampaignLoadFailure();
}
} else if (event is LoadAllUserCampaignsEvent) {
yield* _mapLoadAllUserCampaignsEventToState(event);
}}}

Filter list by firebase conditions in flutter

I have a StreamBuilder feeding data to a list which is later displayed in a stack.
I want to filter (remove) certain cards from being added to the List[] by these rules:
I'm the owner of the card ('ownerId' == currentUserId)
I've liked the card ('liked.' contains = currentUserId)
I choose to do a StreamBuilder because I'd like to update the stack automatically after the user interacts with it (by liking it, it disappears and shows the next). For my understanding, a stream serves that purpose. But please correct me if I'm wrong and if there's a better approach.
streamCard() {
return StreamBuilder(
stream: postRef
.orderBy("timestamp", descending: true)
.limit(10)
.snapshots(),
builder: (context, snapshot) {
if (!snapshot.hasData) {
return circularProgress();
}
List<PostCard> cards = [];
snapshot.data.documents.forEach((doc) {
cards.add(PostCard.fromDocument(doc));
});
If "cards" isn't empty:
return Stack(
alignment: Alignment.center,
children: cards,
In Firebase, I have:
"Cards" > "cardID" > with "ownerId" and "liked" fields ("liked" contains every uid who has liked it).
The idea is to compare the current uid with the contents of each cardID. If they're present, dismiss that card and try the next.
I've been struggling with this problem for a while, I appreciate any help!
Edit Firestore structure:
Basically what you are trying to do is a logical OR query on your firestore, which is very limited as described in the documentation. For comparison, with you where trying to make a logical AND all you had to do it stack 2 where() calls in your query.
So you can't simply limit this in the query itself (in this case, your stream) but you can operate it in the stream's builder, which could be done doing something this:
builder: (context, snapshot) {
if (!snapshot.hasData) {
return circularProgress();
}
List<PostCard> cards = [];
snapshot.data.documents.forEach((doc) {
List<String> liked = List.castFrom(doc.get("liked"));
// currentUserId is previously populated
if(doc.get("ownerId") == currentUserId || liked.contains(currentUserId)){
cards.add(PostCard.fromDocument(doc));
}
});
return cards;
}

Resources