I have a very large database of users that I need to paginate through. The structure is as follows
users > -userId > logs > -logId > {type, timestamp, ...}
Not all users have logs so according to the data order docs if I orderBy "logs" these users should appear first (https://firebase.google.com/docs/database/web/lists-of-data#data-order)
When using orderByChild(), data that contains the specified child key
is ordered as follows:
Children with a null value for the specified child key come first.
As a result I am trying to use the following code to retrieve the last 5 users with logs but am running out of memory on the cloud platform (2GB). Should I be adding an index to assist with this query or is my syntax here incorrect?
admin.database().ref("users").orderByChild("logs").limitToLast(5).once("value", (snapshot) => {
Is there a better way to fetch a limited group of users who have logs? Any feedback would be appreciated
Should I be adding an index to assist with this query or is my syntax here incorrect?
The answer to that is always yes. Without an index in your security rules, the server will send all data at the location to the client, which then orders and filters it.
So for your query, your rules will need to contain an index on logs under the users node.
Related
I have a situation where a user can create a doc and then share it with a group of other users. They could share it to multiple different groups. I don't know how to set a rule for this.
Here is the database structure:
So in the group you have a list of docs that have been shared to it. My app loads the group that a user is in, then wants to load all the docs in the documents array. I need a way server side to say that this is OK. Up until now only the owner of the doc can read it.
I put a field in each doc that contains ids for each group its shared to. I think I want to say "check if the user is a member of any groups in the sharedToGroups" list but I can't work out how to do that unless I maintain another list somewhere say in the userProfile doc that has a list of circles the user is a member of. Even then I'd be trying to compare 2 lists and I'm not sure I can do that client side.
It would be nice to be able to get the group Id somehow from where the request is being issued from and just see if that is in the sharedToGroups array.
Any help or comments on how this can be achieved would be greatly appreciated, maybe it needs a different db structure.
You can try an approach of this sort:
I am not sure if this will help you but off the top of my head maybe you could enable permissions on firestore for the group document. As in, in the rules, for the group set up a function that validates the user with the user ID stored in the document with the ID attached in the auth via the firebase auth
Therefore, rather than trying to restrict access per document, restrict access per group.
I'm going to answer my own question. Not sure if its the correct protocol here (not a professional programmer or experienced Stack Overflower) but it might help someone.
I ended up adding a field in the user_profiles document that has a list of each group they are in. This list needs to be maintained as I create and add / remove people from groups along with the members list in the group itself.
The benefit of this is that I can use the users id from the request object to get that document from the data base in the security rule. I then have a 'sharedToGroup' array in the doc I'm trying to access and a "inGroups" array in the user_profile that I can access also. Then I use the hasAny operator to compare the two arrays and allow access if the sharedToGroup array has any values from the inGroups array.
My rule becomes:
match /_group/{groupId}{
allow create: if isSignedIn();
allow read: if isOwner()
|| resource.data.sharedToGroup.hasAny(get(/databases/$(database)/documents/user_profiles/$(request.auth.uid)).data['inGroups']);
allow write: if isOwner();
}
Only thing left to do is to secure the user_profiles doc to make sure not even the user can write to it since I don't want someone manually adding groups into their array.
I hope this might help someone someday - like I said I'm a not a pro here so take it with a grain of salt.
I am new in Flutter - Firestore
I am learning flutter with firebase and creating a sample dating app
I have a list of users that I get in a stream and display it using List view
Firestore.instance.collection('users').snapshots()
I have learnt to filter this like so
.where((user) => user.age < settings.agemax && user.age > settings.agemin))
and all this works.
I also have a subcollection called shortlist (list of users that current user has shortlisted) that I get using,
Firestore.instance.collection('users').document(uid).collection('shortlist').snapshots()
Now I am trying to redefine my first query GetUsers with filters based on following
How do I exclude shortlisted users that I am fetching in a stream from all users stream
Similarly would also need to filter out "matched users" and "Blocked / declined" users as well !
I believe my question is how do I query Users Collection and exclude records with uid's that contained in a Shortlist subcollection. I am planning to use the same logic for matches and blocked !? am I on the right track ?
also ... do I need to refetch all records when a users shortlists/matches/blocks someone, as the stream would change or is there a way to remove that one record from the listview without rebuilding, may be I should separate this question in two.
If I understand correctly you are looking for the (just introduced) not-in operator, so I recommend also checking out this blog post.
I expect that this operator hasn't landed in the Flutter libraries yet, as that may take some time. I recommend checking the upcoming releases to see when it lands, or checking/filing an issue on the repo.
Until then, there's no way to exclude results from a query, so you will have to exclude the items from the stream of results in your application code.
I have a real-time database on firebase which consists of ListFields. Among these fields, one field, participants is a list of strings and two usernames. I want to make a query to firebase database such that it will return the documents in which a particular username is present in the participants list.
The structure of my document is as follows :
I want to make a query such that Firebase returns all the documents in which the participants list consists aniruddh. I am using Flutter with the flutterfire plugins.
Your current data structure makes it easy to find the participants for a conversation. It does however not make it easy to find the conversations for a user.
One alternative data structure that makes this easier is to store the participants in this format:
imgUrls: {},
participants: {
"aniruddh": true,
"trubluvin": true
}
Now you can technically query for the the conversations of a user with something like:
db.child("conversations").orderByChild("participants/aniruddh").equalTo(true)
But this won't scale very well, as you'll need to define an index for each user.
The proper solution is to add a second data structure, known as an inverted index, that allows the look up of conversations for a user. In your case that could look like this:
userConversations: {
"aniruddh": {
"-LxzV5LzP9TH7L6BvV7": true
},
"trubluvin": {
"-LxzV5LzP9TH7L6BvV7": true
}
}
Now you can look up the conversations that a user is part of with a simple read operation. You could expand this data structure to contain more information on each conversation, such as the information you want to display in your list view.
Also see my answer heres:
Firebase query if child of child contains a value (for more explanation on why the queries won't work in your current structure, and why they won't scale in the first structure in my answer).
Best way to manage Chat channels in Firebase (for an alternative way of naming the chat rooms).
I have a question regarding authentication using Google Firebase.
For an app, I want to build an authentication similar to the one Slack uses: first, the user provides the input as to which group they want to log in to. If there exists a group with the same name as provided in the input, the user is then taken to a login/signup screen.
I've thought about storing users in the realtime database as follows, but I think there must be a better way to do this (since I don't think I can use the firebase authentication in this case):
groups: {
"some_group_name": {
"users": [
"user1": {
.. user 1 information
},
"user2": {
.. user 2 information
}
],
"group_details": {
"name": ..,
"someGroupDetail": ..
}
},
"some_other_group_name": {
...
}
}
I haven't realized if there is an obvious answer yet, so I'm open to suggestions. How would you suggest I tackle this?
Thanks
PS: I'm building the application using Nativescript and Angular, and (so far) there is no server or database involved other than Firebase.
Another suggestion that might work, is by using Firebase Auth Custom Claims. That way, you only need to store the group ID and group name in your realtime database, without worrying to keep changing the database each time user is added or removed.
This is one way you can do it:
Store database exactly like you have it, with it's group ID and name.
In your backend script (I recommend Cloud Function), each time a User is registering themselves, add custom claims in your user: Specifying what group is the User belong to.
Every time user authenticate, retrieve the group ID from custom claims. And there you get it!
Note: be careful not to put too much information in your custom claims as it cannot exceed 1000 bytes.
Read more about it here: https://firebase.google.com/docs/auth/admin/custom-claims
I would suggest you to implement Root-level collections.
Which is to create collections at the root level of your database to organize disparate data sets(as shown in the image below).
Advantages: As your lists grow, the size of the parent document doesn't change. You also get full query capabilities on
subcollections.
Possible use case: In the same chat app, for example, you
might create collections of users or messages within chat room
documents
Based on the reference from the firebase cloud firestore
Choose a data structure tutorial (I know you are using Realtime database but structuring the database is the same since both are using the NoSQL Schema)
For your case:
Make 2 Collections: Users, Groups
Users: User info is stored in the form of document
Groups: In the Groups Collection, here comes the tricky part, you can either store all groups subcollection under 1 document or split into multiple documents (based on your preference)
In the group-subcollection, you can now store your group info as well as the user assigned where you can store user assigned in the form of array, therefore whenever a user access the group, query the user assigned first, if yes, then allow (assuming users can view all group)
You do the thinking now
I'm having trouble setting up a search query within my Firebase database. I put the rules all the way of my search, but I always get the index error.
Below is my structure:
My Firebase database structure:
My query:
https://zoome-production-users.firebaseio.com/country.json?orderBy="fullname"&equalTo="Vitor Darela"
Erro:
{
"error": "Index not defined, add \".indexOn\": \"fullname\", for path \"/country\", to the rules"
}
In your REST call, you run a query on /country in your database. This query inspects each direct child node of /country to see if it matches your filter. In your JSON data /country/AD does not have a property fullname, so the query will not return anything. Even if you were to add the index that the error message tells you about, it will not return anything.
The query you are trying to do is not possible with your current data structure. You will need to modify your data structure to allow it. It seems that you're trying to to find the countries that have a user with a certain name. To allow that query, add an additional data structure that holds precisely that:
"countries_by_fullname": {
"Vitor Darela": {
"AD": true
}
}
With this additional structure you can find the list of countries by simply reading /countries_by_fullname/Vitor Darela.
Also see:
Firebase Query Double Nested
Firebase query if child of child contains a value