I have a DB with users and items.
Every user has languages, which is an array of languages, for example ['en', 'ar']
Every item has language, which is a string, for example 'en'.
How can I index my items, such that I can get a list of the last X items in an array of languages? (i.e - latest 5 items who are either 'en' or 'ar')
For a single language the solution is simple - have an index that has the language key, and array of item keys ordered by whatever.
Please note that the Firebase official documentation recommends against using arrays. IMHO, the main problem in your project is that you trying to use an array, which is an anti-pattern when it comes to Firebase. Beside that, one of the many reasons Firebase recommends against using arrays is that it makes the security rules impossible to write. Because a Firebase database is structured as pairs of key and values, the best option is to use a Map.
To achieve what you want, i recomand you using a database structure which looks like this:
Firebase-root
|
--- users
| |
| --- UID1
| | |
| | --- languages
| | |
| | --- languageId1: true
| | |
| | --- languageId2: true
| |
| --- UID2
| |
| --- languages
| |
| --- languageId3: true
| |
| --- languageId4: true
|
--- languages
| |
| --- languageId1
| | |
| | --- languageName: "en"
| | |
| | --- items
| | |
| | --- itemId1: true
| |
| --- languageId2: "ar"
| | |
| | --- languageName: "ar"
| | |
| | --- items
| | |
| | --- itemId2: true
|
--- item
|
--- itemId1
| |
| --- title: "ItemTitleEn"
| |
| --- en: true
|
--- itemId2
|
--- title: "ItemTitleAr"
|
--- ar: true
Now, with this database structure you can achieve everything you want. For example, you can query your database to display all languages from you database. You can also display all the languages of a single user.
If you want to query your database for the last x item which have the language set to en, you just need to put a listener on the items node and create a query using functions like: orderByChild(), equalsTo() and limitToLast(). Such a query should look like this:
query = rootRef.child("items")
.orderByChild("en")
.equalsTo(true)
.limitToLast(5)
.addListener(/* ... */)
EDIT: Unfortunately Firebase does not allow multiple conditions in a query. So in Firebase there is no where clause that sounds like this: WHERE language = "en" AND language = "ar". So to solve this, you need to put a listener on the other node, on languages node.
The flow is as follows:
yourRef = rootRef.child("languages"); //only one listener
yourRef.addListener(
1. Create a list
2. get items from dataSnapshot.child("languageId1").child("items").getChildren()
3. add **en** items to the list
4. get items from dataSnapshot.child("languageId2").child("items").getChildren()
5. add **ar** items to the list
6. display the list that contains data from both languages
)
Hope it helps.
Related
I am writing firebase security rules, and I am attempting to get a document that may have a space in its documentID.
I have the following snippet which works well when the document does not have a space
function isAdminOfCompany(companyName) {
let company = get(/databases/$(database)/documents/Companies/$(companyName));
return company.data.authorizedUsers[request.auth.uid].access == "ADMIN;
}
Under the collection, "Companies", I have a document called "Test" and another called "Test Company" - Trying to get the document corresponding to "Test" works just fine, but "Test Company" does not seem to work, as the company variable (first line into the function) is equal to null as per the firebase security rules "playground".
My thought is that there is something to do with URL encoding, but replacing the space in a documentID to "%20" or a "+" does not change the result. Perhaps spaces are illegal characters for documentIDs (https://cloud.google.com/firestore/docs/best-practices lists a few best practices)
Any help would be appreciated!
EDIT: As per a few comments, I Will add some additional images/explanations below.
Here is the structure of my database
And here is what fields are present in the user documents
In short, the following snippet reproduces the problem (I am not actually using this, but it demonstrates the issue the same way)
rules_version = '2';
service cloud.firestore {
match /databases/{database}/documents {
match /Users/{user} {
allow update: if findPermission(resource.data.company) == "MASTER"
}
function findPermission(companyName) {
let c = get(path("/databases/" + database + "/documents/Companies/" + companyName));
return c.data.authorizedUsers[request.auth.uid].access;
}
}
}
When I try to update a user called test#email.com (which belongs to company "Test"), the operation is permitted, and everything works exactly as expected.
The issue arises when a user, called test2#email.com, who belongs to company "Test Company" comes along and makes the same request (with authorization email/uid updated in playground, to match what is actually found in the company structure), the request fails. The request fails, since the get() call (line 1 of the function) cannot find the Company document corresponding to "Test Company" - indicated by the variable "c" being null in the screenshot (see below) - IT IS NOT NULL WHEN LOOKING FOR "Test"
Below is a screenshot of the error message, as well as some of the relevant variables when the error occurs
Check to see what type of space just in case it is another non-printable character. You could convert it to Unicode, and check what it might be. However, it is considered bad practice to use spaces in naming variables and data structures. There are so many different types to consider.
| Unicode | HTML | Description | Example |
|---------|--------|--------------------|---------|
| U+0020 |   | Space | [ ] |
| U+00A0 |   | No-Break Space | [ ] |
| U+2000 |   | En Quad | [ ] |
| U+2001 |   | Em Quad | [ ] |
| U+2002 |   | En Space | [ ] |
| U+2003 |   | Em Space | [ ] |
| U+2004 |   | Three-Per-Em Space | [ ] |
| U+2005 |   | Four-Per-Em Space | [ ] |
| U+2006 |   | Six-Per-Em Space | [ ] |
| U+2007 |   | Figure Space | [ ] |
| U+2008 |   | Punctuation Space | [ ] |
| U+2009 |   | Thin Space | [ ] |
| U+200A |   | Hair Space | [ ] |
I'm looking for the efficient way to model users friend's posts
so #Frank van Puffelen, #Doug Stevenson, #Alex Mamo ... all other champions, a simular question is posted 3 years ago Firebase Firestore, query a users friend's posts
so I need to make sure that we are up to date for this topic on 2021 :)
data structured :
Firestore-root
|
--- users (collection)
| |
| --- uid (documents)
| |
| --- name: "name"
| |
| --- email: "email"
|
--- friends (collection)
| |
| --- uid (document)
| |
| --- friends (collection)
| |
| --- uid (documents)
| |
| --- uid (documents)
|
--- posts (collection)
| |
| --- postId (documents)
| |
| --- uid: "user id"
| |
| --- postId: "postid"
| |
| --- userName: "userName"
| |
| --- message: "message"
| |
| --- lastModified: "yyyymmdd hhmm"
| |
| --- likeCount: "xxx"
| |
| --- commentCount: "yyy"
| |
| --- likes (collection)
| | |
| | --- uid (documents)
| | |
| | --- uid (documents)
| |
| |
| --- comments (collection)
| |
| --- commentId (documents)
| |
| --- commentId (documents)
|
--- feeds (collection)
| |
| --- uid (documents)
| |
| --- posts (collection)
| |
| --- postId (documents)
| |
| --- uid: "user id"
| |
| --- postId: "postid"
| |
| --- userName: "userName"
| |
| --- message: "message"
| |
| --- lastModified: "yyyymmdd hhmm"
| |
| --- likeCount: "xxx"
| |
| --- commentCount: "yyy"
NOTE : I'm using this feeds structure because Firestore does not allow query with array lists more then 10 element (the user's friends).
workflow :
user adds a post -> post added to posts collection
When this happens it triggers a Firebase Function :
get uid from post ( field uid in post doc )
find friends ids from friends/uid/friends ( probably have friends list as array in user doc is better to avoid reading multiple docs in friends collection ) --> this will return list of friends ids
loop on friends ids and copy the post to each friends feeds/friendUserId/feeds/postId
user like a post -> uid added to the posts/postId/likes/uid
When this happens it triggers a Firebase Function :
increment likeCount field in posts/postId doc
increment likeCount field in feeds/{anyId}/posts/postId ( use collectionGroup )
user comment a post -> uid added to the posts/postId/comments/uid
When this happens it triggers a Firebase Function :
increment commentCount field in posts/postId doc
increment commentCount field in feeds/{anyId}/posts/postId ( use collectionGroup )
query :
get current user and friends posts order by most recent
FirebaseFirestore.instance.collection('feeds').doc(currentUserId).collection('posts').orderBy('lastModified', descending: true)
get top N liked posts
FirebaseFirestore.instance.collection('posts').orderBy('likeCount', descending: true)
NOTE : purge Firebase Function can be trigger to delete all posts > X months
Question : Should I go with this structure ? if No any suggestion will be appreciated :)
The NoSQL Firestore has no table, what will be the best way for tagging, just store multiple tags in an array?
The NoSQL Firestore has no table.
That's right, the database is in a JSON format.
What will be the best way for tagging, just store multiple tags in array?
According to the use case of your app, you can choose between two approaches. If your tags are of type String, then you can store this literal strings in an array. This would be the first approach and the database schema might look like this:
Firestore-root
|
--- questions (collections)
|
--- questionId (document)
|
--- questionId: "02cubnmO1wqyz6yKg571"
|
--- title: "Question Title"
|
--- tags ["History", "Geography"]
As you can see, I took as an example a collection of questions in which each document has an array of tags.
If you need more details about a tag, the second approach would be to create an object for each tag and store this tag objects in a collection. In the question document, you'll only need to store the ids of the tags also in an array, as in the following schema:
Firestore-root
|
--- questions (collections)
| |
| --- questionId (document)
| |
| --- questionId: "02cubnmO1wqyz6yKg571"
| |
| --- title: "Question Title"
| |
| --- tags ["tagId", "tagId"]
|
--- tags (collections)
|
--- tagId (document)
| |
| --- tagId: "yR8iLzdBdylFkSzg1k4K"
| |
| --- tagName: "History"
| |
| --- //Other tag properties
|
--- tagId (document)
|
--- tagId: "tUjKPoq2dylFkSzg9cFg"
|
--- tagName: "Geography"
|
--- //Other tag properties
What is really this denormalization all about when talking about Firebase Cloud Firestore? I read a few articles on the internet and some answers here on stackoverflow and most of the answers recommend this approach. How does this denormalization really help? Is it always necessary?
Is database flatten and denormalization the same thing?
It's my fist question and hope I'll find an answer that can help me understand the concept. I know is different, but I have two years of experience in MySQL.
What is denormalization in Firebase Cloud Firestore?
The denormalization is not related only to Cloud Firestore, is a technique generally used in NoSQL databases.
What is really this denormalization?
Denormalization is the process of optimizing the performance of NoSQL databases, by adding redundant data in other different places in the database. What I mean by adding redundant data, as #FrankvanPuffelen already mentioned in his comment, it means that we copy the exact same data that already exists in one place, in another place, to suit queries that may not even be possible otherwise. So denormalization helps cover up the inefficiencies inherent in relational databases.
How does this denormalization really help?
Yes, it does. It's also a quite common practice when it comes to Firebase because data duplication is the key to faster reads. I see you're new to the NoSQL database, so for a better understanding, I recommend you see this video, Denormalization is normal with the Firebase Database. It's for Firebase realtime database but the same principles apply to Cloud Firestore.
Is it always necessary?
We don't use denormalization just for the sake of using it. We use it, only when it is definitely needed.
Is database flatten and denormalization the same thing?
Let's take an example of that. Let's assume we have a database schema for a quiz app that looks like this:
Firestore-root
|
--- questions (collections)
|
--- questionId (document)
|
--- questionId: "LongQuestionIdOne"
|
--- title: "Question Title"
|
--- tags (collections)
|
--- tagIdOne (document)
| |
| --- tagId: "yR8iLzdBdylFkSzg1k4K"
| |
| --- tagName: "History"
| |
| --- //Other tag properties
|
--- tagIdTwo (document)
|
--- tagId: "tUjKPoq2dylFkSzg9cFg"
|
--- tagName: "Geography"
|
--- //Other tag properties
We can flatten the database by simply moving the tags collection in a separate top-level collection like this:
Firestore-root
|
--- questions (collections)
| |
| --- questionId (document)
| |
| --- questionId: "LongQuestionIdOne"
| |
| --- title: "Question Title"
|
--- tags (collections)
|
--- tagIdOne (document)
| |
| --- tagId: "yR8iLzdBdylFkSzg1k4K"
| |
| --- tagName: "History"
| |
| --- questionId: "LongQuestionIdOne"
| |
| --- //Other tag properties
|
--- tagIdTwo (document)
|
--- tagId: "tUjKPoq2dylFkSzg9cFg"
|
--- tagName: "Geography"
|
--- questionId: "LongQuestionIdTwo"
|
--- //Other tag properties
Now, to get all the tags that correspond to a specific question, you need to simply query the tags collection where the questionId property holds the desired question id.
Or you can flatten and denormalize the database at the same time, as you can see in the following schema:
Firestore-root
|
--- questions (collections)
| |
| --- questionId (document)
| |
| --- questionId: "LongQuestionIdOne"
| |
| --- title: "Question Title"
| |
| --- tags (collections)
| |
| --- tagIdOne (document) //<----------- Same tag id
| | |
| | --- tagId: "yR8iLzdBdylFkSzg1k4K"
| | |
| | --- tagName: "History"
| | |
| | --- //Other tag properties
| |
| --- tagIdTwo (document) //<----------- Same tag id
| |
| --- tagId: "tUjKPoq2dylFkSzg9cFg"
| |
| --- tagName: "Geography"
| |
| --- //Other tag properties
|
--- tags (collections)
|
--- tagIdOne (document) //<----------- Same tag id
| |
| --- tagId: "yR8iLzdBdylFkSzg1k4K"
| |
| --- tagName: "History"
| |
| --- questionId: "LongQuestionIdOne"
| |
| --- //Other tag properties
|
--- tagIdTwo (document) //<----------- Same tag id
|
--- tagId: "tUjKPoq2dylFkSzg9cFg"
|
--- tagName: "Geography"
|
--- questionId: "LongQuestionIdTwo"
|
--- //Other tag properties
See, the tag objects are the same as well in users -> uid -> tags -> tagId as in tags -> tagId. So we flatten data to group somehow existing data.
For more information, you can also take a look at:
What is the correct way to structure this kind of data in Firestore?
Because you say you have a SQL background, try to think at a normalized design which will often store different but related pieces of data in separate
logical tables, which are called relations. If these relations are stored physically as separate disk files, completing a query that draws information from several relations (join operations) can be slow. If many relations are joined, it may be prohibitively slow. Because in NoSQL databases, we do not have "JOIN" clauses, we have to create different workarounds to get the same behavior.
For some applications my team creates authenticated users with a password/email combination. This will get the user an firebase user uid. The problem with this is that the keys in firebase itself are external id's, and they do not match the auth.uid. How would I go about creating security rules then?
Sample auth.uid:
9dkad6c7-s649-9623-99e2-5a0dbgf5dfdz
Then a sample of the structure:
database
|
—— conversations
|
——{external id 1}
| |
| ——{external id 2}
| |
| {data here}
|
messages
|
——{externalid1|externalid2}
| |
| —{-KFasdahsduids}
| |
| {data here}
|
|
users
|
——{externalId}
| |
| {first name}
| {last name}
| {firebaseUID}
| {more data here}
|
——{externalId2}
|
{first name}
{lastname}
{firebaseUID}
{more data here}
The problem really is that the auth.uid is not the same as the external ones, and we really need those external id's. Can I do something with the UID that is stored in the /users/? Any suggestions?