This question already has answers here:
Firestore: Query by item in array of document
(5 answers)
Closed 4 years ago.
I have read this https://firebase.google.com/docs/firestore/solutions/arrays but I still couldn't figure out how it solves the issue of querying not-hardcoded user-generated items in array.
Let's say I have a Party collection:
Party {
name: "Birthday party",
invitees: [56448, 869987, 230232, 202030] # user id's of invited people
}
How can I query only parties where I'm in the invitees array without making an index for each possible ID?
https://firebase.google.com/docs/firestore/solutions/arrays
Limitations
The solution shown above is a great way to simulate array-like structures in Cloud Firestore, but you should be aware of the following limitations:
Indexing limits - A single document can have only 20,000 properties in order to use Cloud Firestore built-in indexes. If your array-like data structure grows to tens of thousands of members, you may run into this limit.
So the short answer is no.
If I were using Firebase Database, I would create a cloud function to watch the party/invitees branch on creation and modification, then propagate the invitation to a branch on the userID profile. Because you can be invited and uninvited, the changeset contains old and new and you can remove invitations from people who are in the previous but not in the next.
I haven't explored Cloud Functions for Firestore (yet) but I suspect that you can do something similar.
If you follow the method described in the documentation you point to (i.e. https://firebase.google.com/docs/firestore/solutions/arrays) and instead of storing the users in an array you save an object like
Party {
name: "Birthday party",
invitees: {
"56448": true,
"869987": true,
"230232": true,
.....
}
}
you will be able to query like
var yourUserID = .....; //e.g. '869987'
db.collection('invitees')
.where('invitees.' + yourUserID, '==', true)
.get()
.then(() => {
// ...
});
And as the doc says "This technique relies on the fact that Cloud Firestore creates built-in indexes for all document fields, even fields in a nested map.", so no worries for maintaining the indexes.
Related
I am using Vue.js and Firebase Firestore. Now I have two collectionsusers and orders. In the orders collection, I have already stored the id of each document of user collection. I now have to fetch the details of the corresponding users from this. How am I supposed to go about it?
This is what I've done so far
let orderRef = db.collection("orders")
orderRef.onSnapshot(snapshot => {
snapshot.docChanges().forEach(change => {
if (change.type == "added") {
let doc = change.doc;
this.orders.push({
id: doc.id,
orderData: doc.data().orderData,
user_id: doc.data().user_id,
userInfo: db.collection("users").doc(user_id),
});
}
});
});
I need to store user data in userInfo.Thanks in advance
Firestore doesn't support foreign key like SQL database, so you can't retrieve nested data like SQL.
In Firestore you need to fetch referenced data separately, either you can fetch all users data separately in parallel with orders data and store it in map, or if you don't need users data initially then fetch each users data when needed like when you check details of each order.
I think that you're structuring your data as if you were working in a relational database. Firestore is a no-SQL database that doesn't have any notion of reference, the only thing Firestore understands are key-values, documents and collections, everything else has to me modeled on top of that. See Firestore data model.
In Firestore relationships are usually modeled by storing the id of the document you'd like to "reference". In that sense you might not need to store the 'users' document inside the 'order' but the field 'user_id' would suffice. The caveat is that this data layout comes at the price of having to fetch the 'user_id' from orders before you can fetch the actual user data. You could read more about data denormalization and modeling relationships in articles link1, link2 and this other thread.
Also, it's worth noting that Firestore documents are limited in size to 1MB so with your actual configuration if the amount of info of 'user' documents increases it may get to a point where it would be necessary to reshape your documents structure.
All in all, if you don't want to change your data layout you would need to follow Ked suggestions and first retrieve the 'users' data to inline it into the 'userInfo' field.
I'm trying to figure out if there's a reasonable way of doing this:
My problem:
Exceeding my daily quota for reads in firestore pretty fast.
My database and what I do:
My database looks like this (simplified):
sessions: { // collection
sessionId: { // document
users: { // collection
userId: { // document
id: string
items: { // collection
itemId: trackObject
}
}
}
}
}
Now I want to retrieve from one session, all users and their items. Most sessions have 2-3 users but some users have around 3000 items. I basically want to retrieve an array like this:
[
{
userId,
items: [
...items
],
},
...users
]
How I go about it currently:
So I get all users:
const usersRef = db.collection(`sessions/${sessionId}/users`);
const userSnapshots = await usersRef.get();
const userDocs = userSnapshots.docs;
Then for each user I retrieve their items:
(I use a for-loop which can be discussed but anyhow)
const user = userDocs[i].data();
const itemsRef = usersRef.collection(`${user.id}/items`);
const itemSnapshots = await itemRef.get();
const items = itemSnapshots.docs
Finally I retrieve the actual items through a map:
user.items = items.map(doc => doc.data());
return user;
My theory:
So it looks like if I do this on a session where a user has 3000 items, the code will perform 3000 read operations on firestore. After just 17 runs I eat up my 50000 operations a day.
This reasoning is somewhat based on this answer.
My question:
Is there any other way of doing this? Like getting all tracks in one read-call? Should I see if I can fit all the items into an array-key in the user-object instead of storing as a collection? Is the free version of firestore simply not designed for this many documents being retrieved in one go?
If you're trying to reduce the number of document reads, you'll need to reduce the number of documents that you need to read to implement your use-case.
For example, it is fairly unlike that a user of your app will want to read the details of all 3000 items. So you might want to limit how many items you initially read, and load the additional items only on demand.
Also consider if each item needs to be its own document, or whether you could combine all items of a user into a single document. For example, if you never query the individual items, there is no need to store them as separate documents.
Another thing to consider if whether you can combine common items into a single document. An example of this is, even if you keep the items in a separate subcollection, to keep the names and ids of the most recent 30 items for a user in the user's document. This allows you to easily show a user and their 30 most recent items. Doing this you're essentially pre-rendering those 30 items of each user, significantly reducing the number of documents you need to read.
To learn more on data modeling considerations, see:
Cloud Firestore Payments
Going over read-quota in firebase firestore
the video series Getting to know Cloud Firestore, specifically What is a NoSQL Database? How is Cloud Firestore structured? and How to Structure Your Data
this article on NoSQL data modeling
I've been working with Google Cloud Firestore. I'm about to import 13000+ records from a CSV to the firestore back-end. I'll be using this collection for look up and auto-completion purposes.
I'm curious and concerned to know if this is a good idea. Also, I'm looking for some suggestions on what techniques should I be using to make retrieval of this this data as efficient as possible. I'm working with Angular 5 and using AngularFire2 to connect with Firestore.
The document itself is really small such as:
{
address: {
state: "NSW"
street: "19 XYZ Road"
suburb: "Darling Point"
},
user: {
name: "ABC",
company: "Property Management Company"
}
file_no: "AB996"
}
Most of the searching would be based on file_no property of the document.
Update
I just imported all 13k+ records to Firestore. It is really efficient. However, I have one issue. After importing the records, I'm getting the message on my Firestore console that my daily limit for Read Operations is reached (0.05 of 0.05 Million Ops). I just wrote data and displayed those records in a Data Table. I used the following query:
this.propertyService
.getSnapshotChanges()
.subscribe(properties => {
this.properties = properties;
this.loadingIndicator = false;
});
getSnapshotChanges(): Observable < any > {
return this.afs.collection(this.propertiesCollection).snapshotChanges()
.map((actions) => {
return actions.map((snapshot) => {
const data = snapshot.payload.doc.data();
data.id = snapshot.payload.doc.id;
return data;
});
});
}
How dos this makes my reading limit exceed?
The number of documents in a collection is of no consequence when you use Cloud Firestore. That's actually one of its bigger perks: no matter how many documents are in a collection, the queries will take the same amount of time.
Say you add 130 document and (for sake of example) it takes 1 second to get 10 documents out of it. That's the performance you'll get no matter how many documents are in the collection. So with 1300 documents it will also take 1 second, with 13K it will take 1 second, and with 13M, it will also take 1 second.
The problem more developers run into is to make their use-cases fit within the API of Firestore. For example: the only way to search for strings is with a so-called prefix match, there is no support for full-text search. This means that you can search for Prop* and find Property Management Company, but not for *Man* to find it.
I had to rephrase this question since it was a bit misleading (my fault).
Here is my dilemma, let's say I have a party collection:
parties {
status: "open",
invitees: [56486,68978,897650], # user ids of invited users
scheduled_at: 1948089050 # timestamp
}
I'd like to query only "open" parties, that I'm invited to (my user id in the invitees array), and sorted by scheduled_at
I could solve the first part of querying the array by turning it into a hash (thanks to #renaud and #james poag):
parties {
status: "open",
invitees: {
56486: true,
68978: true,
897650: true
}
scheduled_at: 1948089050
}
Now performing this:
db.collection('parties').where('status', '==', 'open').where('invitees.56486', '==', true').orderBy('scheduled_at')
Results in a firebase error asking me to make a composite index for status + invitees.56486 + scheduled_at. as you can see it's impractical for me to add an index for each user id.
Any ideas?
It looks like you're trying to make a query against a schema that doesn't really support that query. You're going to have to adjust your schema (possibly duplicating data between collections) to support your intended query. This sort of practice is normal for NoSQL type databases.
You're going to need a new collection that relates a single party with a single invitee, one for each combination, that effectively serves as a "join" between them:
party-invitees
- party_id
- party_status ("open")
- party_scheduled_at
- attendee_id
Now you can find out which open parties an attendee is invited to:
db.collection('party-invitees')
.where('party_status', '==', 'open')
.where('attendee_id', '==', 'whatever')
.orderBy('party_scheduled_at')
Bear in mind that you'll have to change this collection along with any other collections with the same data as they change. Fortunately, batch writes and transactions make this easier to do atomically.
I want to get the total number of the documents that exist in firestore.
I don't want to get the data only the total number of inside Products collection I have 200.000 items is that possible with Angular 4-5, not angular.js
Can someone expert tell me how I can achieve that ??
My code so far and is not work
get_total_messages() {
this.messages_collection = this.afs.collection<MessageEntity>('messages');
return this.messages_collection.snapshotChanges();
}
End this is how I try to get the data but is not what I want;
this.firebase_Service.get_total_messages().subscribe( data => {
console.log(data);
});
There is no API to get the count of the number of documents in a Firestore collection. This means that the only ways to get the count are:
Get all documents and count them client-side.
Store the count as a separate property and update that as you add/remove documents.
Both approaches are quite common in NoSQL databases, with the second of course being a lot more efficient as the number of documents grows.
Firebase provides a sample of using Cloud Functions to keep a counter. While this sample is written for the Firebase Realtime Database, it can easily be modified to work on Cloud Firestore too.
Firestore also provides documentation on running aggregation queries and running distributed counters. Both seem slightly more involved than the first sample I linked though.
this.firebase_Service.get_total_messages().subscribe( data=>this.totalnumber=data.length);
//now, you can get total number of messages
luckily , i've solved somehow using the code,
try this, and it works well .
this.db.collection('User').valueChanges()
.subscribe( result => {
console.log(result.length);
})