Firebase/Firestore Experts.
So I have got a question about structure, as I have now run into a problem, and it got me thinking if I actually have the right structure.
I'll try to explain...
I have an application where it is possible for companies to create a profile. Each company has a companyId. Each company can create courses, which means I have a collection named "Courses". This collection looks like this:
CourseCollection
Doc (CompanyID)
Subcollection (Categories)
Subcollection (Courses)
Doc (CompanyID)
Subcollection (Categories)
Subcollection (Courses)
The reasoning for structuring like this was so it was easy to have a collection that had the specific companies and the company's own categories and courses. I thought that might be a smarter solution than having a flat collection of all courses ever created and having a companyId as a property on the course. I don't know if this solution is better or if it is better to have a flat structure.
Just to mention, a course document holds an "events" array, where each "event" has a start date.
Anyway, now I want to set up a function to run daily. The function I am setting up needs to run through all courses for all companies and see if it has any active events for "today" and then send out an email to a user that is signed up for the course. However, then I found myself not being able to query the "Courses" collection - The following returns me nothing
export const dailyCourses = async () => {
const coursesByCompanies = query(collection(db, 'courses'));
const checkCourseForNewClientProgramSnapshot = await getDocs(coursesByCompanies);
checkCourseForNewClientProgramSnapshot.forEach((courseSnapshot) => {
console.log('courseSnapshot', courseSnapshot.data());
});
};
However, if I specify a companyId and the subcollection, I can get all the courses for the given companyId
export const dailyCourses = async () => {
const coursesByCompanies = query(collection(db, 'courses/**someId**/courses'));
const checkCourseForNewClientProgramSnapshot = await getDocs(coursesByCompanies);
checkCourseForNewClientProgramSnapshot.forEach((courseSnapshot) => {
console.log('courseSnapshot', courseSnapshot.data());
});
};
Now, that got me thinking that it might not be possible to loop over all root documents in the courses collection and get all the "courses" (subcollection) for each company, without me knowing each companyId beforehand to look up each course in the subcollection?? Which means I could have to query all companies in my "companies" collection before and get all ids of the companies which might not be the best, as all companies may haven't created any courses, which would cause me to do a lot of unnecessary reads?
So now I'm thinking that I might be better off actually having a flat structure, after all, containing all courses and then querying them by companyId. Then I can loop all courses and the events and see which events match the current day (today).
As you might hear, I'm a bit confused about which is the best solution, which is why I'm looking for any expert advice. I'm thinking I'm not the first to have run into an issue like this or similar before.=
I know a lot of you would say "Read the docs" or "It depends on the use case", etc - I have read the docs, however, this doesn't say anything about which solution I would/should use in THIS specific case, so I'm REALLY hoping that someone could give me their thoughts and perspectives. It should also be said I'm still quite new to Firebase/Firebase, so if you can see that there is an even better solution for it than I have proposed, I'm all ears :)
THANK YOU for your time in advance :) I'm really hoping a helpful soul is coming my way ;)
it got me thinking if I actually have the right structure.
We are usually structuring a Firestore database according to the queries that we want to perform. So the following answer will most likely help.
What is the correct way to structure this kind of data in Firestore?
The reasoning for structuring like this was so it was easy to have a collection that had the specific companies and the company's own categories and courses.
The shared structure looks good to me.
I thought that might be a smarter solution than having a flat collection of all courses ever created and having a companyId as a property on the course. I don't know if this solution is better or if it is better to have a flat structure.
None of the solutions is better than the other. If any structure allows you to query and get the correct results, then that's the one you should go ahead with.
The function I am setting up needs to run through all courses for all companies
That can be achieved in a very simple way, by using a collection group query, which will allow you to get the desired data from all sub-collections called for example, "courses".
Besides that, your first code snippet doesn't work because there is no top-level collection called "courses". The second snippet works because you are pointing exactly to the courses of a particular company.
Now, that got me thinking that it might not be possible to loop over all root documents in the courses collection and get all the "courses" (subcollection) for each company
No need for that.
As you might hear, I'm a bit confused about which is the best solution, which is why I'm looking for any expert advice.
There is no best solution. However, your schema can solve the queries that you were talking about.
For simplicity I'd prefer having a separate collection for courses with a reference to the company it belongs to.
Not only would it reduce reads, but it makes your code a little easier to write and maintain. Plus if there are some option in the future to have public courses etc.
Not really sure if there is more I can add to it, but it's as simple as the above if I was to do it.
Related
I am writing an app where there is not a lot of interaction with other users. Set and retrieve your own data only.
In Firebase Firestore how could I model this so that everything fits under a users UID?
Something that would look like this?
users/{uid}/user/
users/{uid}/settings/
users/{uid}/weather/
If I want to achieve something like this, then I need to create another UID:
users/{uid}/user/{uid}/{userInfo}
This feels a bit off to me.
Is this wrong? Would it be better if I moved every subcollection into its own collection?
Is this faster / more efficient?
Any help is appreciated!
The most common approaches for me:
Store the profile information, settings and weather in the user document (your {uid}) itself. This most common for the profile information, but it's always worth considering for other types too: do they really need to be in their own documents?
Have a default name for a single subcollection for each user, and then have each information type as a document with a known name in there. So /users/$uid/documents/profile, /users/$uid/documents/settings, and /users/$uid/documents/weather. So now each information type is in a separate document, meaning you can for example secure access to them individually.
If the information for a certain type is repeated, I'd put that in documents in a known/named subcollection. So if there are many weathers, you'd get /users/$uid/weather/$weatherdocs. So with this you can now have an endless set of the specific type of information.
Neither of these is pertinently better/worse, as it all depends on the use-cases of your app.
There will be performance differences between these approaches, as they require a different number of network requests. If this is a concern for your app, I'd recommend testing all approaches above to measure their relative performance against your requirements.
I have a simple question. I am building a Instagram clone app and I want to show each user to their friends. Also they can see the friends list. I am using cloud firestore approach. However I'm a little bit confused about how to store user's friends data? . Should I create a new collection as friendsList
or should I hold the data in users collection as a friends array ?
In the first approach I will create the user data again when some user adds a new friend. Am a new for both firestore and NoSql I would be thankful If anyone can explain.
I'm not going to "answer" as such, but explain the philosophy of NoSQL a bit. The best approach is to design your queries first (i.e. what do you want to get from the database), then design your database schema to make getting the results of those queries efficient and affordable. There are many ways to organize data; you want to take advantage of NoSQL "schema-less" to make your schema match your needs, not the other way around.
Other things to keep in mind: DRY is less critical to NoSQL. Static data (i.e. never or rarely changes) can be stored in multiple places (i.e. a friend's name might be in their profile and in a friends-list) if that saves reads & writes (which are the biggest factor in costs).
So how to organize your database? I don't know; what do you want your database to do?
I should read to this tutorial.This tutorial about is MySql but not important for me if you understand this tutorial you can apply firebase.
I leave a tip below.
I'm building a one to one messaging feature the intent behind is the following:
There is a unique project and people (two or more) can chat about the project so we can think a project is a room, I've been looking to different modeling structures the most common is something like the following:
Chats
- projectId (room)
- messages
message
userId
name
profilePicture
posted (timestamp)
But I've been thinking in a flat structure something like
Messages
ProjectId
Message
userId
name
profilePicture
posted
The chat feature is going to have a huge impact on the web app I'm building, being said that is quite important to make the right desition (I'm sure there is no always a right or wrong but consider the purpose of the chat)
Just some questions that come to my mind:
are there any implications in performance by using a flat structure?
what are the advantages of using a nested structure like the mentioned in example #1
which solution is cheaper? (reads/writes)
There are befenits from both the solutions you proposed. Let's dive into them:
performance: they are pretty similar from this point of view. In fact, if you want to get a chat from Firestore, in the second case simply make a query for the messages of a particular chat and parse the required information from the first document you receive (since in each message you have the userID, name, profilePicture, etc ...). With the first approach this operation is straightforward since you already asking for a Chat document.
structure: the first solution is the one that I prefer because it's clear what it does and since Firestore is schemaless it enforces a clear design. With the second approach you are basically flattening your DB but you are also exposing your messages to privacy issues. In fact, setting up rules in the first case is pretty straightforward, simply let the users access only the chats they are involved in. But in this case, all the users can, "possibly", read each other messages which should not be something which you want.
cost: this basically depends on what you will do with these documents. In fact, the cost of Firestore either depended on the number of documents read/written but also on the amount of data you store. Here, the first solution is clearly better since you are not adding redundancy for fields like profilePicture, name, userID, etc ... This fields logically belong to the Chat entity, and not to its messages.
I hope this helps since properly setting up a database is vital for any good project.
I'm working on a library app, and am using Firestore with the following (simplified) two collections books and wishes:
Book
- locationIds[] # Libraries where the book is in stock
Wish
- userId # User who has wishlisted a book
- bookId # Book that was wishlisted
The challenge: I would like to be able to make a query which gets a list of all Book IDs which have been wishlisted by a user AND are currently available in a library.
I can imagine two ways to solve this:
APPROACH 1
Copy the locationIds[] array to each Wish, containing the IDs of every location having a copy of that book.
My query would then be (pseudocode):
collection('wishes')
.where('userId' equals myUserId)
.where('locationIds' contains myLocationId)
But I expect my Wishes collection to be pretty large, and I don't like the idea of having to update the locationIds[] of all (maybe thousands) of wishes whenever a book's location changes.
APPROACH 2
Add a wishers[] array to each Book, containing the IDs of every user who has wishlisted it.
Then the query would look something like:
collection('books')
.where('locationIds' contains myLocationId)
.where('wishers' contains myUserId)
The problem with this is that the wishers array for a particular book may grow pretty huge (I'd like to support thousands of wishes on each book), and then this becomes a mess.
Help needed
In my opinion, neither of these approaches are ideal. If I had to pick one, I will probably go with Approach 1 simply because I don't want my Book object to contain such a huge array.
I'm sure I'm not the first person to come across this sort of problem, is there a better way?
You could try dividing the query in two different requests. For instance, in pseudocode:
wishes = db.collection('wishes').where('userId', '==', myUserId)
book_ids = [wish.bookId for wish in wishes]
books = db.collection('books').where('bookId', 'in', book_ids)
result = [book.bookId for book in books if book.locationIds]
Notice that this is just an example, this code probably doesn't work, since I haven't tested it and the keywork in just supports 10 values. But you get the idea. A good idea would be adding the length of the locationIds or whether it's empty or not in a separate attribute so you could omit the last iteration querying the books with:
books = db.collection('books').where('bookId', 'in', book_ids).where('hasLocations', '==', True)
Although you would still have to iterate to only get the bookId.
Also, you should avoid using arrays in Firestore since it doesn't have native support for them, as explained in their blog.
Is it mandatory to use NoSQL? Maybe you could do this M:M relation better in SQL. Bear in mind that I'm no database expert though.
I'm trying to perform a filter by pattern over a Firestore collection. For exemple, in my Firestore database I have a brand called adidas. The user would have an search input, where typing "adi", "adid", "adida" or "adidas" returns the adidas document. I pointed out several solutions to do this :
1. Get all documents and perform a front-end filter
var brands = db.collection("brands");
filteredBrands = brands.filter((br) => br.name.includes("pattern"));
This solution is obviously not an option due to the Firestore pricing. Moreover it could be quite long to perform the request if the number of documents is high.
2. Use of Elasticsearch or Algolia
This could be interesting. However I think this is a bit overkill to add these solutions' support for only a pattern search, and also this can quickly become expensive.
3. Custom searchName field at object creation
So I had this solution : at document creation, create a field with an array of possible search patterns:
{
...
"name":"adidas",
"searchNames":[
"adi",
"adida",
"adidas"
],
...
}
so that the document could be accessed with :
filteredBrands = db.collection("brands").where("searchNames", "array-contains", "pattern");
So I had several questions:
What do you think about the pertinence and the efficiency of this 3rd solution? How far do you think this could be better than using a third party solution as Elasticsearch or Algolia?
Do you have any other idea for performing pattern filter over a firestore collection?
IMHO, the first solution is definitely not an option. Downloading an entire collection to search for fields client-side isn't practical at all and is also very costly.
The second option is the best option considering the fact that will help you enable full-text search in your entire Cloud Firestore database. It's up to you to decide if it is worth using it or not.
What do you think about the pertinence and the efficiency of this 3rd solution?
Regarding the third solution, it might work but it implies that you create an array of possible search patterns even if the brand name is very long. As I see in your schema, you are adding the possible search patterns starting from the 3rd letter, which means that if someone is searching for ad, no result will be found. The downside of this solution is the fact that if you have a brand named Asics Tiger and the user is searching for Tig or Tige, you'll end up having again no results.
Do you have any other ideas for performing pattern filters over a Firestore collection?
If you are interested to get results only from a single word and using as a pattern the staring letters of the brand, I recommend you a better solution which is using a query that looks like this:
var brands = db.collection("brands");
brands.orderBy("name").startAt(searchName).endAt(searchName + "\uf8ff")
In this case, a search like a or ad will work perfectly fine. Besides that, there will be no need to create any other arrays. So there will be less document writing.
I have also written an article called:
How to filter Firestore data cheaper?
That might also help.