I've found this thread but I would like to extend my question based on the answers.
My goal is to implement something close to the full text search solution provided by the firebase documentation. While Algolia seems to be a great choice, it does not come cheap and introduces another third-party.
Question: I have the idea of building a search index based on Firestore data and save it as JSON in Cloud Storage. This JSON would be accessed by a Cloud Function that would use a library like fuse.js to execute a search (see the page for an example) and return it to a user.
Is this a reasonable thing to do for searching a collection with around 20000 documents / JSON size of around 3MB maximum?
Related
What is that query builder which is shown on the Firebase Firestore page, what's its use case and how does it benefit my project?
From the documentation, I get that it's used to query collections and sub-collections in that actual Firestore database.
But does this used just for visualizing the queries from it? or does it prepare those types of queries to be ready when requesting them so it will cause some performance and speed of requesting improvements?
and when should I use it, and when I shouldn't?
The new query building in the Firestore console is just a visual way to build a query that then determines what data the console shows. I find it most helpful to limit the amount of data the console shows, and to see if a query is going to be possible before I translate it in to code.
Aside from that, the query build shows the resulting documents in a tabular view (rather than the panel view that already existed), which makes it possible to compare documents at a glance and fits more data in less space.
I am trying to build a mobile app which has a NewsBulletin feature using a NoSQL Cloud Firestore. I am trying to get the unique post view by keeping the user's uid into an array called "views" and count it by getting the length of the array. Is this recommendable or are there other better solution for this? Thank you
Currently this is the structure of my database:
News(Collection)
-DummyNews1(Document)
-newsTitle
-posterName
-bodyMessage
-timeCreated
-views(array)
-dummyuid1
-dummyuid2
I like your solution as it is easy to implement. You don't actually have to manually check for duplicate uids, as firestore has a built in feature that does that for you.
Here is an example:
FirebaseFirestore.instance.collection('news').doc('documentId').update({
'views': FieldValue.arrayUnion([viewerUid]),
});
FieldValue.arrayUnion will check if the contents exists in the database, and only when it does not will add the content.
Now, although I am a fan of you solution, and I do use this method for like type of feature in my own published apps, there are some limitations that you should be aware in case your app becomes super popular.
Maximum document size in firestore is 1MiB. Since firebase auth's uid is 28 characters long, that would be about 37,400 views maximum to be stored in a document ignoring other fields.
But if this is a new application, I would not worry too much about this limit. Besides, once you get close to this limit, you should have more than enough resources to pivot to another method that scales.
I wanted to get some community consensus on how to achieve the following with the Firebase JS SDK (e.g., in React):
Suppose I have a collection users and I wanted to paginate users that do not have document IDs matching a subset of IDs (O(100-1000)). This subset of excluded IDs is dynamic based on the authenticated user.
It seems the not in query only supports up to 10 entries, so this is out of the question.
It also seems it's not possible to fetch all document IDs and filter on the client side, at least not in the 'firebase' JS SDK.
The only workaround I can think of is to have a document that keeps an array of all users document IDs, pull that document locally and perform the filtering/pagination logic locally. The limitation here is that a document can be at most 1MB, so realistically the single document can store at most O(10K) IDs.
Firestore has a special bunch of methods for pagination which may be useful for you. Those are called "query cursors".
You can use them to define the start point startAt() or startAfter() and to define an end point endAt() or endBefore(). Additionally, if needed, those can be combined with limit method.
I strongly encourage you to check this tutorial. Here you can find a quick video explaining the matter and lot of examples in all popular languages.
I have a small database in Firebase realtime database which contains posts added by users. Everytime I open the console, each and every post is in the correct position.
I'm trying to migrate to Cloud Firestore. I've created a script to copy every post objects from Firebase realtime database to Cloud Firestore and works fine.
The problem is when I switch the tab to Firestore, the posts are not ordered anymore and I find hard time to find a post.
How can I order the posts in the exact same way that were in Firebase and also in the ListView?
Strange Firestore console situation
It is not strange, is the normal sorting scheme provided (by default) in Firebase console.
Unlike the Fireabse realtime database ids, Cloud Firestore ids are actually purely random. There's no time component included. That is why (by default) there is no order. However, as Frank van Puffelen mentioned in his comment:
The Cloud Firestore console support sorting/filtering of the documents on any field (since June 2018), by clicking the "filter" icon above the document list.
Beside that, if know the id of a document, just simply CTRL + F in your browser and you'll be able to find the desired document very easy. In my opinion, this sorting scheme makes it relatively intuitive to find the document you might be looking for.
If you want to order your elements in your ListView, you should add those requirements into queries. The order in your query is not related with order you see in the dashboard.
Those ids are required so an app like yours can work perfectly on a big scale. At a big scale, it doesn't matter what order the documents appear in the console because the console is useless for viewing large amounts of documents.
A solution for ordering your posts in the ListView, would be to order your posts according to a timestamp property.
For Android, here you can find how to add the date using a model class or FieldValue.serverTimestamp().
I have a small web application that uses Firestore for the data.
I am currently storing around 8 million objects.
The data size in the Firestore database is 10 times bigger than it is in "raw" format (json files).
Reading this wiki I came to the conclusion that there is a high chance that the automatic indexes are guilty of the humongous size of the database.
I have read the indexing documentation and it says how to create and remove custom indexes from the console, but I haven't find a way of removing automatic indexes.
Since I am only doing direct access to the objects, I don't need fields indexing at all.
During the Beta there is no way to remove automatic generated indexes in Firestore.
So far you are only able to manage composite indexes by yourself.
Source: A Google Firestore employee mentioned this as a part of an answer on another question on Stackoverflow. Though, I can't find the link to the post anymore.