The latest Firecast, Doug Stevenson mentioned request.query, however, he only discussed request.query.limit and request.query.orderBy (here is the timestamp).
The documentation names a third property, i.e. request.query.offset:
offset - query offset clause.
In all the time I have been using Cloud Firestore, I have never seen the "offset clause".
Can someone explain what this offset clause is and how the request.query.offset property is implemented?
That currently doesn't do anything. Offset is currently only available to server SDKs (for example: here); it's not an option in web and mobile client SDKs. Since server SDKs always bypass security rules, there's nothing you can do with request.query.offset that would affect way rules would evaluate.
The reference to this should actually be removed from the documentation altogether.
Related
I was watching this Firebase video, and one stuff that wasn't clear to me is that the "||" or OR operator isn't supported especially with the way Firebase store indexes.
It was stated that you have to make separate queries and join it on the client side instead of on the firebase side.
Isn't the in operator essentially just a convenience method and acts like multiple OR statements?
https://firebase.blog/posts/2019/11/cloud-firestore-now-supports-in-queries
It is a well known feature of Firestore indexes which are good for range queries. With the indexes applied , for an inequality query, the backend would still have to scan every document in the collection in order to come up with results, and thus will affect the performance when the number of documents increases with time.
So, as per your question regarding the backend logical working of the “in” operator when used in a query and which is also mentioned in this thread on addition of IN queries not only address this performance issue but also supports up to 10 equality clauses on the same field with a logical OR".The arguments which are passed in the “In” operator query, are compared when searching a document.This will allow you to fetch documents with your filter criteria and thus result in function operation to take less time rather than goind one by one through each item.
For the example you could do:
// Get all documents in 'foo' where status is open or upcoming
db.collection('foo').where('status','in',['open','upcoming']).get()
I would also recommend you to check these following similar examples:
How to perform compound queries with logical OR
How to make queries on firestore
Firebase database operator working
Firestore IN operator working
Firestore Query limitation
I am aware it would be very difficult to query by a value that does not exist in an array but is there a way to do this without doing exactly that?
Here is my scenario - I have a subscription based service where people can opt in and "follow" a specific artist. In my backend, this creates a subscription doc with their followerId, the id of the artist they want to follow (called artistId), and an array called pushed. The artist can add new releases, and then send each follower a notification of a specific song in the future. I would like to keep track of which follower has been pushed which release, and this done in the aforementioned pushed array. I need a way to find which followers have already been pushed a specific release and so...
I was thinking of combining two queries but I am not sure if it is possible. Something like:
db.collection('subscriptions').where('artistId', '==', artistId)
db.collection('subscriptions').where('artistId', '==', artistId).where('pushed', 'array-contains', releaseId)
And then take the intersection of both query results and subtract from the 1st query to get the followers that have not been pushed a specific release.
Is this possible? Or is there a better way?
There is no way to query Firestore for documents that don't have a certain field or value. It's not "very difficult", but simply not possible. To learn more on why that is, see:
Firestore get documents where value not in array?
Firestore: how to perform a query with inequality / not equals
The Get to know Cloud Firestore episode on how Firestore queries work.
Your workaround is possible, and technically not even very complex. The only thing to keep in mind is that you'll need to load all artists. So the performance will be linear to the number of artists you have. This may be fine for your app at the moment, but it's something to definitely do some measurements on.
Typically my workaround is to track not what releases were pushed to a user, but when the last push was sent to a user. Say that a release has a "notifications sent timestamp" and each user has a "last received notifications timestamp". By comparing the two you can query for users who haven't received a notification about a specific release yet.
The exact data model for this will depend on your exact use-case, and you might need to track multiple timestamps for each user (e.g. for each artist they follow). But I find that in general I can come up with a reasonable solution based on this model.
For Elasticsearch case you need to sync with your database and elasticsearch server. And also need to make firewall rules at your Google Cloud Platform, you need keep away the arbitrarily request to your server, since it may cause bandwith cost.
The not-in operator is now available in Firestore!
citiesRef.where('country', 'not-in', ['USA', 'Japan']);
See the docs for a full list of examples:
https://firebase.google.com/docs/firestore/query-data/queries#in_not-in_and_array-contains-any
citiesRef.where('country', 'not-in', [['USA']]);
Notice the double array around [['USA']]. You need this to filter out any docs that have 'USA' in the 'country' array.
Single array ['USA'] assumes that 'country' is a string.
In the Firestore security rules you can access resource properties. I would like to use these properties in my queries, but I can't find any documentation on it.
Currently I am manually writing updatedAt timestamps into documents where I need them, but that is cumbersome and fragile, because it is easy to forget to update the timestamp. It also feels redundant, since the resource already has this data.
Is it, for example, possible to query all documents in a collection that have been updated since yesterday?
It is not possible to query on these, they are specific to the Security Rules layer.
While we can inspect the server update time for a specific document once retrieved, we cannot query for them since it is not indexed (and handled at a layer lower than our indexing engine).
I have a legacy Firebase project i contribute to. In it I have the following rules for the resource songs:
"songs": {
".indexOn": ["artist_timestamp"]
},
Which allows me to do things like curl htttp://my-fire-base-ref/songs.json?orderBy="artist_timestamp"
However I can also do orderBy="$priority" which is a property we add to all song objects. This works even though it is not explicitly in the rules json definition. Is this a secretly allowed property??
The .priority of each node is implicitly indexed, so you don't need to define an index for it.
Why are you using priorities though? While they still work, using named properties allows you to accomplish the same with more readable code. See What does priority mean in Firebase?
According to the documentation for indexing data:
Firebase provides powerful tools for ordering and querying your data.
Specifically, Firebase allows you to do ad-hoc queries on a collection
of nodes using any common child key. As your app grows, the
performance of this query degrades. However, if you tell Firebase
about the keys you will be querying, Firebase will index those keys at
the servers, improving the performance of your queries.
This means you can order by any key at any time without specifying it as an index, but without a specific index specified for a key, performance may be very bad for large sets of data.
I have read the Firebase docs about priorities but I don't think I understand it yet.
I think I understand that it is related to querying and sorting data. To give my question (and the answers) some weight, in what instances might you use priorities?
From the docs I read that you can set priorities when you set a value at some reference, and then when you query that reference priority determines the ordering based on its type and value. And that makes some sense but I'm not quite understanding it.
Disclosure: I work for Firebase.
Priorities are an optional (numeric or alphanumeric) value of each node, which is used to sort the children under a specific parent or in a query if no other sort condition is specified. The priority of a node is hidden from most views of the data. In cases where a priority is specified for a node, it can be found as a .priority property in the exportVal() of a snapshot.
Since Firebase added the ability to order children on a specified property, priorities have lost most of their value. They are a left-over artifact from the time before Firebase had orderByChild queries. If you are starting on a Firebase project today, you should use orderByChild instead of relying on priorities.