Following is one of the examples in Cloud Firestore list query
citiesRef.where("state", ">=", "CA").where("state", "<=", "IN")
What does it mean?
Is this the way I can do a query for searching substrings. For example,
I have a collection of users and all my user documents have first_name and last_name. Now as a user I am trying to search others by typing a portion of the name and searching. I should be able to write a query to get all user who's name contains that string. How do I do that in Firestore?
[Googler here] To answer your first question, state <= "IN" is a query for all documents where the value of the state property sorts less than or equal to the value "IN". So if the state was "IM" that would be true but not if it was "IO" or "IP" as those come later lexicographically.
The use of "IN" may have been confusing here, it's meant to be the abbreviation for the state of Indiana but it's also a reserved word in many other database systems.
To answer the second question, Cloud Firestore does not support any native "contains", operations or other common string queries like "beginsWith", "endsWith", "like", etc. The reason for this is that all Cloud Firestore queries must hit an index and right now we don't index textual fields in a way that would make those queries fast.
Currently we recommend using a third-party search provider like Algolia, and we provide some guidance on doing so here:
https://firebase.google.com/docs/firestore/solutions/search
Related
I have a request to Firestore to check if the collection of tickets contains some duplicated IDs:
firestore.collection("tickets").where("extId", "==", "Test 2").get();
The problem is - where method looks up only for case-sensitive IDs.
Is there a chance I can return a document with test 2 or tesT 2 extId?
Unfortunately Firestore queries are case sensitive and there is no way to avoid this. Here are some approaches you could take to solve this problem depending on your use case:
1. Store several versions of your data (eg store both an upper and lower case version of your data)
2. Don't allow users to type in their query, and instead provide them with predefined filter items (for example chips, or a dropdown menu)
3. Do some client side processing before executing the query (eg. convert query strings into upper or lowercase to ensure it matches the format in the document)
4. Use a search engine platform on top of Firestore such as Algolia, Typesense, Meilisearch, among others as they contain typo-tolerance and other additional features
As also #Hydra mentioned in her answer, the queries in Firestore are case-sensitive, meaning that will always return documents with the exact match. If you want to get documents with test 2 OR tesT 2, then you should consider using in operator:
Use the in operator to combine up to 10 equality (==) clauses on the same field with a logical OR. An in query returns documents where the given field matches any of the comparison values.
In your particular case, the following query will do the trick:
firestore.collection("tickets").where("extId", "in", ["test 2", "tesT 2"]).get();
I have a collection where the documents are uniquely identified by a date, and I want to get the n most recent documents. My first thought was to use the date as a document ID, and then my query would sort by ID in descending order. Something like .orderBy(FieldPath.documentId, descending: true).limit(n). This does not work, because it requires an index, which can't be created because __name__ only indexes are not supported.
My next attempt was to use .limitToLast(n) with the default sort, which is documented here.
By default, Cloud Firestore retrieves all documents that satisfy the query in ascending order by document ID
According to that snippet from the docs, .limitToLast(n) should work. However, because I didn't specify a sort, it says I can't limit the results. To fix this, I tried .orderBy(FieldPath.documentId).limitToLast(n), which should be equivalent. This, for some reason, gives me an error saying I need an index. I can't create it for the same reason I couldn't create the previous one, but I don't think I should need to because they must already have an index like that in order to implement the default ordering.
Should I just give up and copy the document ID into the document as a field, so I can sort that way? I know it should be easy from an algorithms perspective to do what I'm trying to do, but I haven't been able to figure out how to do it using the API. Am I missing something?
Edit: I didn't realize this was important, but I'm using the flutterfire firestore library.
A few points. It is ALWAYS a good practice to use random, well distributed documentId's in firestore for scale and efficiency. Related to that, there is effectively NO WAY to query by documentId - and in the few circumstances you can use it (especially for a range, which is possible but VERY tricky, as it requires inequalities, and you can only do inequalities on one field). IF there's a reason to search on an ID, yes it is PERFECTLY appropriate to store in the document as well - in fact, my wrapper library always does this.
the correct notation, btw, would be FieldPath.documentId() (method, not constant) - alternatively, __name__ - but I believe this only works in Queries. The reason it requested a new index is without the () it assumed you had a field named FieldPath with a subfield named documentid.
Further: FieldPath.documentId() does NOT generate the documentId at the server - it generates the FULL PATH to the document - see Firestore collection group query on documentId for a more complete explanation.
So net:
=> documentId's should be as random as possible within a collection; it's generally best to let Firestore generate them for you.
=> a valid exception is when you have ONE AND ONLY ONE sub-document under another - for example, every "user" document might have one and only one "forms of Id" document as a subcollection. It is valid to use the SAME ID as the parent document in this exceptional case.
=> anything you want to query should be a FIELD in a document,and generally simple fields.
=> WORD TO THE WISE: Firestore "arrays" are ABSOLUTELY NOT ARRAYS. They are ORDERED LISTS, generally in the order they were added to the array. The SDK presents them to the CLIENT as arrays, but Firestore it self does not STORE them as ACTUAL ARRAYS - THE NUMBER YOU SEE IN THE CONSOLE is the order, not an index. matching elements in an array (arrayContains, e.g.) requires matching the WHOLE element - if you store an ordered list of objects, you CANNOT query the "array" on sub-elements.
From what I've found:
FieldPath.documentId does not match on the documentId, but on the refPath (which it gets automatically if passed a document reference).
As such, since the documents are to be sorted by timestamp, it would be more ideal to create a timestamp fieldvalue for createdAt rather than a human-readable string which is prone to string length sorting over the value of the string.
From there, you can simply sort by date and limit to last. You can keep the document ID's as you intend.
Assuming I have a list of data I would like to store with Firebase realtime database, and search it later.
What would be the best way to store the data and query it to get the best performance?
My data is a list of names (containing a lot of names).
["Bob", "Rob", ...]
Note that I have multiple clients searching in a given time.
If the names are supposed to be unique and order doesn't matter, you'll want to store them as a mathematical set. In the Firebase Realtime Database you'll model this as:
"usernames": {
"Bob": true,
"Rob": true
}
A few things of note about this model:
We use the names as keys, which means that each name is by definition unique (since each key can exist only once in its containing node).
The true values have no specific meaning. They are just needed, since Firebase can't store a key without a value.
Certain characters (such as . and /) cannot be used in keys. If a name contains such characters, you will have to filter them out (or encode them) in the key. For example someone named Jes.sie will have to be stored as Jes.sie (lossy) or e.g. Jes%2Esie (with URL encoding).
In such cases you could store the original unfiltered/unencoded name as the value. So: "Jes%2Esie": "Jes.sie".
A few more general notes about (text) searching in the Firebase Realtime Database:
Firebase can only do prefix matches, it has no support for searching strings that contain or end with a certain substrings. This means that in the original data it can search for everything starting with an B (with orderByKey().startAt("R").endAt("R\uF7FF")), but it can't search for everything ending with ob.
Searches are case-sensitive. If you want to be able to search case-insensitive, consider storing the keys as all-lowercase:
"usernames": {
"bob": "Bob",
"rob": "Rob",
"jes%2esie": "Jes.sie"
}
If you need better support for text-search, consider integrating a third-party search engine. Common recommendations are Elastic-search (self-hosted) or Algolia (cloud-based).
For more information on many of these topics, see:
this article on NoSQL data modeling
the video series Firebase for SQL developers
Cloud Firestore Case Insensitive Sorting Using Query (while written for Firestore, the same applies here)
I am trying to query all Documents where DocumentID begins with my string. My research tells me it's possible for field values by using '<=' in where(), but I have not found anything online that tells how to do this on Document ID fields.
I am doing this because my table contains Geohashes as DocumentIDs (high precision) with data, and I would like to get those documents on bases of low precision Geohash.
I also found this as a potential solution:
docs = mycolRef.where(firebase.firestore.FieldPath.documentId(), '>', somestring).stream();
However, this gives the error: "NameError: name 'firebase' is not defined"
I am importing firestore through google.cloud as mentioned in this guide: (https://cloud.google.com/firestore/docs/quickstart-servers)
from firebase import firestore, or import firebase resulted in errors.
How to query Documents where DocumentID begins with a substring?
There is currently no way you can achieve that.
I have not found anything online that tells how to do this on Document ID fields.
You haven't found anything because such a query does not exist.
My research tells me it's possible for field values by using '<=' in where()
Yes, that's the only option you have now. So add all those Geohashes which are now documents ids as values of a new (geohash) property within your document.
I also found this as a potential solution:
docs = mycolRef.where(firebase.firestore.FieldPath.documentId(), '>', somestring).stream();
Unfortunately, you cannot do that, hence that error.
One more thing to note is that in Firestore, it is not possible to change the document ids once it exists. All that data is immutable. What you can do instead is to read all the documents, change the document id and then put it all back.
Assuming I have a list of data I would like to store with Firebase realtime database, and search it later.
What would be the best way to store the data and query it to get the best performance?
My data is a list of names (containing a lot of names).
["Bob", "Rob", ...]
Note that I have multiple clients searching in a given time.
If the names are supposed to be unique and order doesn't matter, you'll want to store them as a mathematical set. In the Firebase Realtime Database you'll model this as:
"usernames": {
"Bob": true,
"Rob": true
}
A few things of note about this model:
We use the names as keys, which means that each name is by definition unique (since each key can exist only once in its containing node).
The true values have no specific meaning. They are just needed, since Firebase can't store a key without a value.
Certain characters (such as . and /) cannot be used in keys. If a name contains such characters, you will have to filter them out (or encode them) in the key. For example someone named Jes.sie will have to be stored as Jes.sie (lossy) or e.g. Jes%2Esie (with URL encoding).
In such cases you could store the original unfiltered/unencoded name as the value. So: "Jes%2Esie": "Jes.sie".
A few more general notes about (text) searching in the Firebase Realtime Database:
Firebase can only do prefix matches, it has no support for searching strings that contain or end with a certain substrings. This means that in the original data it can search for everything starting with an B (with orderByKey().startAt("R").endAt("R\uF7FF")), but it can't search for everything ending with ob.
Searches are case-sensitive. If you want to be able to search case-insensitive, consider storing the keys as all-lowercase:
"usernames": {
"bob": "Bob",
"rob": "Rob",
"jes%2esie": "Jes.sie"
}
If you need better support for text-search, consider integrating a third-party search engine. Common recommendations are Elastic-search (self-hosted) or Algolia (cloud-based).
For more information on many of these topics, see:
this article on NoSQL data modeling
the video series Firebase for SQL developers
Cloud Firestore Case Insensitive Sorting Using Query (while written for Firestore, the same applies here)