Current State
I have a Flutter app which shows me a list of data from Firebase in a list view.
return new ListView(
children: snapshot.data.docs.map((DocumentSnapshot documentSnapshot) {
return _createRows(
documentSnapshot.data()['id'],
documentSnapshot.reference.id,
);
}).toList());
Problem/Question
But the list will get bigger and therefore the loading times will increase but much more important the usage of the read processes will increase exponentially. I also plan to add a search function.
Firebase docs:
[...] downloading an entire collection to search for fields
client-side isn't practical.
Is there a possibility to only query the used data from the ListView.builder and to do the search via Firebase?
(One possibility is shown here. However, this is not too advantageous for data storage use)
Also, there are a few third party sites, but I couldn't find any free ones in addition, I'm not sure whether the effort to implement in Flutter is worth it. e.g.elastic
I am curious to hear your suggestions
I decided to download all the data at the start of the app and then pass it around inside the app, as I have comparatively little data to download and a long stay in the app, it is most worthwhile.
I realised the search with the plugin TypeAhead.
Related
There are several articles (firestore and firebase realtime database) explaining how to build a user presence system but I cannot find a resource for a friend presence system.
A simple user presence system is not perfect for some applications such as chat apps where there are millions of users and each user wants to listen to only his/her friends. I've found similar questions:
exact same question on stackoverflow
exact same issue on github
Two ok solutions with a realtime database are: (solutions are from the above stackoverflow post)
Use many listeners (one for each friend) with a collection of users. Possibly have a cap on the number of friends to keep track of.
Each user has friends collections and whenever a user's status changes, his/her status changes wherever he/she shows up in some user's friends collection as well.
Is there a better way to do? What kind of databases do chat apps like discord, whatsapp and etc. use to build their friends presence system?
I came to two approaches that might be worth looking into. Note, that I have not tested how it will scale longer term as I just pushed to prod. First step, write a users presence on their user document (will need firebase, cloud functions, and cloud firestore per https://firebase.google.com/docs/firestore/solutions/presence).
Then take either approach:
Create an array field on your user documents (users> {userID}) called friends. Every time you add a friend add your id to this array, and vice versa. Then, on the client run a function like:
db.collection(users).where("friends", "array-contains", clientUserId).onSnapshot(...)
In doing so, all documents with friends field that contains the clientUserId will be listened to for real-time updates. For some reason, my team didn't approve of this design but it works. If anyone can share their opinion as to why I'd appreciate it
Create a friend sub-collection like so: users>{userID}>friends
. When you add a friend, add a document to your friend sub-collection with the id equal to your friends userID. When a user logs on, run a get query for all documents in this collection. Get the doc IDs and store into an array (call it friendIDs). Now for the tricky part. It'd be ideal if you can read use the in operator for unlimited comparison values because you can just run an onSnapshot as so:
this.unSubscribeFriends = db.collection(users).where(firebase.firestore.FieldPath.documentId(), "in", friendIDs).onSnapshot((querySnapshot) => {get presence data}). Since this onSnapshot is attached to this.unSubscribeFriends you just need to call this once to detach the listener:
componentWillUnmount() {
this.unSubscribeFriends && this.unSubscribeFriends()
}
Because a given users friends can definetely increase into the hundreds I had to create a new array called chunkedFriendsArray consisting of a chunked version of friendIDs (chunked as in every 10 string IDs I splice into a new array to bypass the in operator 10 comparison values limit). Thus, I had to map chunkedFriendsArray and set an onSnapshot like the one above for every array of a max length of 10 inside chunkedFriendsArray. The problem with this is that the all the listeners are attached to the same const (or this.unSubscribeFriends in my case). I have to call this.unSubscribeFriends as many times as chunkedArrays exist in chunkedFriendsArray:
componentWillUnmount() {
this.state.chunkedFriendsArray.forEach((doc) => {
this.unSubscribeFriends && this.unSubscribeFriends()
})
}
It feels weird having many listeners attached to the same const (method this.unSubscribeFriends) and calling the same exact one to stop listening to them. I'm sure this will lead to bugs in my production code.
There are other decentralize approaches but the two I listed are my best attempts at avoiding having a bunch of decentralized presence data.
I am trying to build a mobile app which has a NewsBulletin feature using a NoSQL Cloud Firestore. I am trying to get the unique post view by keeping the user's uid into an array called "views" and count it by getting the length of the array. Is this recommendable or are there other better solution for this? Thank you
Currently this is the structure of my database:
News(Collection)
-DummyNews1(Document)
-newsTitle
-posterName
-bodyMessage
-timeCreated
-views(array)
-dummyuid1
-dummyuid2
I like your solution as it is easy to implement. You don't actually have to manually check for duplicate uids, as firestore has a built in feature that does that for you.
Here is an example:
FirebaseFirestore.instance.collection('news').doc('documentId').update({
'views': FieldValue.arrayUnion([viewerUid]),
});
FieldValue.arrayUnion will check if the contents exists in the database, and only when it does not will add the content.
Now, although I am a fan of you solution, and I do use this method for like type of feature in my own published apps, there are some limitations that you should be aware in case your app becomes super popular.
Maximum document size in firestore is 1MiB. Since firebase auth's uid is 28 characters long, that would be about 37,400 views maximum to be stored in a document ignoring other fields.
But if this is a new application, I would not worry too much about this limit. Besides, once you get close to this limit, you should have more than enough resources to pivot to another method that scales.
I'm beginner in flutter-fire app [: And here i've got a simple issue but not easy to figure out.
When we use Streambuilder in the app, we can usually see awesome synchronization between UI and DB on time. I'm really feeling joyful when using Streambuilder. It seemed like only beautiful things would happen with Streambuilder.
Meanwhile, a suspicious question popped out. When we enter print('hello world!'); in the Streambuilder, we can see that RUN console is printing out the phrase every milliseconds, otherwise would be just printed once. It means that RAM usage is extremely increased. When it comes to DB synchronization, we can easily guess that the use of Streambuilder leads to huge usage of client-server communication fee, which in my case is Firebase fee.
So, here is my question.
Can i feel free to use Streambuilder when the stream is connected to DB(in my case, firebase)?
I'm worrying about communication fee between UI and firebase, because streambuilder seems like literally using huge amount of energy every milliseconds(IMO additionally, server fee) unlike normal builders. Especially when the length of collection is so long that reading the collection once may cost a lot of energy, the fee would increase a lot more on and on, because the streambuilder have to check thousands of firebase documents only to figure out a single line of condition.
I guess many backend-connected flutter methods use the Streambuilder, so someone could clearly figure it out how much we're gonna pay for Google when we use Streambuilder. I know it's quite ambiguous question, but hope you understand. [:
Content coming from the Firebase database, whether FutureBuilder or StreamBuilder, pays only for the query value that has been processed once, and after that, in case the same response value is the same for the same query, it does not pay for that cost and displays the stored list stored in the cache on the client screen again. .
And check that it's not being called on something like setState. if so, of course StreamBuilder is called again.
Hi I am working with a small start-up which is building a web application. The initial tech stack we had chosen was, a React JS front end, Python on the server-side to handle some external data requests, and Googles Fire-base (real-time DB) as the back-end.
We specifically looked at Firebase due to the documented plugins to the Google suite of tools including google analytics and big query, and the already provided users authentication that comes along with the Google Firebase dashboard.
However since engaging a group of developers, concerns have been raised by them in two areas when using Firebase with our application. Firebase has documented limitations to its depth and complexity of search that it supports.
In that for queries that require complex joins across tables or search criteria that is partial or requires returns of similar or LIKE, Firebase is said to either have no capability or very limited capability.
with regards to users the product is said to have limited capability when building an environment that requires user groups and roles.
It has been therefore suggest we look at moving away from Firebase. Or we consider reducing the use of Firebase to simpler elements of our application environment, moving critical data and data that is found and displayed via complex searching / data queries, onto alternative database technologies that have greater support for data and search complexity.
To that end I am looking to understand if anyone else has their entire web application back-end in either of the two Firebase database offerings (real time db or FireServe) and if you have faced any issues around performance, lack of functionality, lack of capability when trying to do complex things within your back-end.
Then if you did how did you resolve the issue. Did you add on to Firebase with third-party plugins, did you move part or all of your data off Firebase to alternative database technologies, or completely moved away from Firebase altogether?
And lastly I would like to know if using Firebase in a more limited way, for example to manage user access to your application while the critical data resides in another database (for example MongoDB or SQL) is possible or are we over complicating the infrastructure build by leveraging two different database technologies?
Thanks for to anyone who offers their advise. Duncan
This isn't exactly an answer but reading the comments shows that one requirement of the post was a partial string query. According the comment, this is the concern
One user types in “Al” in an attempt to search for his friend “Alex”.
You want to fetch all users in the database whose name contains the
text “Al” to help the user narrow down his search. This task is not
possible through Cloud Firestore alone.
So examining that one requirement, let's suppose we want to perform that task; return all users that have a name that starts with 'Al'
func findPartialString(startingWith: String) {
let usersColl = self.db.collection("users")
let endingWith = startingWith + "\u{f8ff}"
let query = usersColl.whereField("name", isGreaterThanOrEqualTo: startingWith).whereField("name", isLessThan: endingWith)
query.getDocuments { querySnapshot, err in
if let err = err {
print("there was an error: \(err.localizedDescription)")
return
}
if let snap = querySnapshot {
if snap.count > 0 {
for document in snap.documents {
let key = document.documentID
let name = document.get("name") as! String
print(key, name)
}
} else {
print("no matches")
}
} else {
print("no data returned in snapshot")
}
}
}
and would be called like this
findPartialString(startingWith: "Al")
and the result is
uid_5 Alex
uid_4 Albert
There are several other ways to perform this same task, so clearly it's not only very possible to do, it's also compact code that easy to maintain.
I'm trying to create a list of cards, those cards take information from Firestore a query (need .where) it all worked out until I passed my quota, a bit later I found out about this: https://www.youtube.com/watch?v=Lb-Pnytoi-8. I need to read specific documents with .where and post them in a streambuilder, I've thought of multiple options (mostly around Future, JSON) which non of them worked. I've theoretically thought of somehow pausing a streambuilder and reenabling it every pull-to-refresh or making a quota through a futurebuilder but I had no luck.
Here's my code:
Code
I couldn't find an efficient way to do this, any luck anyone?
Apologies for this question. For community members who seek the same answer: using methods such as initState, async or anything alike will cause the widget to rebuild - ergo send once more an API-request.