I'm trying to add a feature in my app to search users by their name in Firebase, so I've tried to make a query that returns all the users where their name contains the search.
For example, if we have 5 users and their names are: "Bernard, Francois, Connard, Antoine, Penard" and the search is "nar", the query would return "Bernard, Connard, Penard". Does someone have an idea how to do the query? Or else a better idea to find the users?
Unfortunately, Firebase does not offer substring-match functionality. There are some hacks available with good thought on data structures and a bit of code work if you only want PREFIX based matches, but there is no operator for INFIX matches.
Your best option is to acknowledge that Firebase is an excellent database, but not an excellent search engine. There is nothing wrong with this - it does its actual job very well. When you need search, you will have the best results "pairing" it with an actual search engine such as ElasticSearch, Algolia, CloudSearch, etc.
Related
I'm building a fairly simple Next.js app that allows users to browse potentially millions of user profiles by 4-5 key facets (last login date, team, tags, a few other basic fields).
I soon discovered that Firestore won't allow me to run the queries I need to, if I wish to support multiple filters. I could get, for example, all the profiles with a matching team, but to filter all the other facets I'd have to do it on the front end and end up invoking far more document reads than I'd like to.
Algolia to the rescue! Works great, but boy is that cost going to rack up. I can't feasibly launch this app with Algolia or I'll have to remortgage my house. I also absolutely do not need text search, just basic faceting, so I really feel like Algolia is overkill here regardless of the cost implications.
I really do not want to start querying Firestore for an insane number of documents and then run the filtering on the client, unless this really is the BEST option for my needs. But I need advice.
Thanks guys!
I soon discovered that Firestore won't allow me to run the queries I need to if I wish to support multiple filters.
That's not true, Firestore does for sure allow multiple filters.
I could get, for example, all the profiles with a matching team, but to filter all the other facets I'd have to do it on the front end
You can create a query and chain multiple whereEqualTo() functions. This means that you can filter by multiple fields at once. Here is an example:
rootRef.whereEqualTo("lastLoginData", lastLoginData).whereEqualTo("team", nameOfTeam);
You can chain as many functions as you want. That's an example for Android, but you can find the corresponding queries for each platform and programming language. If you also need to order the results, don't forget to create an index.
Algolia to the rescue!
Algolia is useful only if you need full-text searches in your app, otherwise, for simple queries, you can use the filters that are currently available.
Getting documents and filtering them o the client is not an option. Always filter the documents on the server and get only the results that you need.
I wanted to get some community consensus on how to achieve the following with the Firebase JS SDK (e.g., in React):
Suppose I have a collection users and I wanted to paginate users that do not have document IDs matching a subset of IDs (O(100-1000)). This subset of excluded IDs is dynamic based on the authenticated user.
It seems the not in query only supports up to 10 entries, so this is out of the question.
It also seems it's not possible to fetch all document IDs and filter on the client side, at least not in the 'firebase' JS SDK.
The only workaround I can think of is to have a document that keeps an array of all users document IDs, pull that document locally and perform the filtering/pagination logic locally. The limitation here is that a document can be at most 1MB, so realistically the single document can store at most O(10K) IDs.
Firestore has a special bunch of methods for pagination which may be useful for you. Those are called "query cursors".
You can use them to define the start point startAt() or startAfter() and to define an end point endAt() or endBefore(). Additionally, if needed, those can be combined with limit method.
I strongly encourage you to check this tutorial. Here you can find a quick video explaining the matter and lot of examples in all popular languages.
I have a book collection for example, each book has 'genre' field. How to query all books which genre is "fantasy" or "historical"? Like SQL SELECT * FROM book WHERE genre in ("fantasy", "historical"). Pretty usual SQL query as my opinion.
I found this feature request in GitHub, which isn't resolved yet. The Firestore documentation says: "Logical OR queries aren't supported". So, I cannot do such simple query in Firestore? What is a workaround? Should I query each genre separately and join the result?
Note, that I found similar questions like "How to get multiple documents by set of ids?", but not about custom properties. For ids, there is 'getAll()' method in the js admin sdk for example.
It is not supported in firestore as you correctly identified. You can send two separate queries and merge the results locally. This is how it needs to be done unfortunately. If yours is a simple usecase, sending two separate requests shouldn't be a problem.
However, have a look at rxfire package which is officially supported by firebase. Note that it doesn't support or queries per se but with the help of rxjs, it makes such tasks easier to manage. Here's an excerpt from the link below on what it can do:
Firebase provides realtime streams and async callbacks to do all sorts
of awesome things in your app. Getting data from one realtime stream
is easy enough, but what if you want to join it from another?
Combining multiple async streams of data can get complex. At Firebase,
we wanted to help simplify things, so we created a new JavaScript
library: RxFire.
Link to introductory blogpost: RxFire
I'm trying to perform a filter by pattern over a Firestore collection. For exemple, in my Firestore database I have a brand called adidas. The user would have an search input, where typing "adi", "adid", "adida" or "adidas" returns the adidas document. I pointed out several solutions to do this :
1. Get all documents and perform a front-end filter
var brands = db.collection("brands");
filteredBrands = brands.filter((br) => br.name.includes("pattern"));
This solution is obviously not an option due to the Firestore pricing. Moreover it could be quite long to perform the request if the number of documents is high.
2. Use of Elasticsearch or Algolia
This could be interesting. However I think this is a bit overkill to add these solutions' support for only a pattern search, and also this can quickly become expensive.
3. Custom searchName field at object creation
So I had this solution : at document creation, create a field with an array of possible search patterns:
{
...
"name":"adidas",
"searchNames":[
"adi",
"adida",
"adidas"
],
...
}
so that the document could be accessed with :
filteredBrands = db.collection("brands").where("searchNames", "array-contains", "pattern");
So I had several questions:
What do you think about the pertinence and the efficiency of this 3rd solution? How far do you think this could be better than using a third party solution as Elasticsearch or Algolia?
Do you have any other idea for performing pattern filter over a firestore collection?
IMHO, the first solution is definitely not an option. Downloading an entire collection to search for fields client-side isn't practical at all and is also very costly.
The second option is the best option considering the fact that will help you enable full-text search in your entire Cloud Firestore database. It's up to you to decide if it is worth using it or not.
What do you think about the pertinence and the efficiency of this 3rd solution?
Regarding the third solution, it might work but it implies that you create an array of possible search patterns even if the brand name is very long. As I see in your schema, you are adding the possible search patterns starting from the 3rd letter, which means that if someone is searching for ad, no result will be found. The downside of this solution is the fact that if you have a brand named Asics Tiger and the user is searching for Tig or Tige, you'll end up having again no results.
Do you have any other ideas for performing pattern filters over a Firestore collection?
If you are interested to get results only from a single word and using as a pattern the staring letters of the brand, I recommend you a better solution which is using a query that looks like this:
var brands = db.collection("brands");
brands.orderBy("name").startAt(searchName).endAt(searchName + "\uf8ff")
In this case, a search like a or ad will work perfectly fine. Besides that, there will be no need to create any other arrays. So there will be less document writing.
I have also written an article called:
How to filter Firestore data cheaper?
That might also help.
I am facing a quite strange issue searching with Lucene. I have a query with 3 clauses. If I launch the query with just 2 clauses in share, it returns several documents, included the one I am seeking for. Nevertheless, if I add the third clause to the query and perform the query in share it returns no results, but it returns the document I am looking for when I launch it in the alfresco console!.
I guess it is not a grant issue since I get the document I am looking for when the query is less restrictive. The query with the third clause just fails for a specific value, for the others it works fine.
It maybe an indexing problem but in that case I think it should fail when launching the query in alfresco console as well.
Any help?
Querying in Alfresco Share differs from querying in the Nodebrowser or directly through JavaScript.
If you take a look at: alfresco/templates/webscripts/org/alfresco/slingshot/search/search.lib.js which is a repository webscript triggered by Share. You'll see in the code it fail saves results.
So you'll need to play around to get the right results.