I have this simple struct:
{
"items": {
"item1": {
"likes": 4
},
"item2": {
"likes": 5
}
}
}
I enable offline mode:
Firebase.getDefaultConfig().setPersistenceEnabled(true);
Multiple users can increase the "likes" fields at the same time. How can I increase the likes filed by 1 in offline mode? If I get the likes value than set it with the increased value, what happens it the device reaches the server but another users before that have increased the same likes field? Will it overwite the value?
I can imagine my data structure is not so good because I'm new to NoSQL. I can add Firebase userids to the likes fields instead of a counter integer and summarize them to get the result. But this means lots of unneccessary data beacuse I don't need to know who liked I just need the number of likes.
Please help me to find out the best solution! Thank you!
Related
I have around 2 million users at a branch in a database. Each user may contain a property called activeSessionId which tells me if this user has ever started a web session to my app. There is also an array called sessions that contains data for all the web sessions this user has ever started. I have found some bugs in the code which may cause the users who have started a web session to the app to have some invalid data causing the mobile app to become unresponsive. I want to traverse all the users who have started a web session and fix that data.
If I traverse over all the users, the code fails with firebase complaining about too much data to read. So, I am trying to think of a query that could just give me the users that have an activeSessionId defined and I just traverse over those. But I can't think of any such query. The equalTo method doesn't allow for an undefined value for a property.
Below is what a user looks like in firebase that has an activeSessionId.
Top level
userInfos: {
"-NL1sdfee29E7bQ53_rJTW": {
"activeSessionId": "-NL1aKF29E7bQ53_rJTW",
"alarmsCount": {
"ownAlarms": {
"Personal": {
"edit": 1244,
"new": 436
}
}
},
"contactsLastUpdatedAt": 1675435956007,
"deviceLocale": "en-US",
"deviceOs": "android",
"joinDate": 1649084020051,
"lastActiveAt": 1675435953311,
"maxAllowedAlarms": 5,
"rateTheApp": {
"done": true,
"lastAsked": 1675435981073
},
"release": "7.4.3",
"sessions": {
"-NL1aKF29E7bQ53_rJTW": {
"browser": "Chrome",
"id": "-NL1aKF29E7bQ53_rJTW",
"lastActive": "10 Jan 2023, 11:22 AM",
"platform": "Windows"
}
},
"timezone": "America/Denver",
"timezoneOffset": 420,
"totalNumberOfContacts": 282
}
}
Is it possible to create such a query for Firebase Realtime Database?
In the Realtime Database, there is no way you can query a node based on a field that doesn't exist. So if you want to perform a query that returns the users that have the activeSessionId field set to a particular value and the users that don't have the activeSessionId field at all, that isn't possible.
A possible workaround would be to denormalize the data and create a node that contains only the users that have already set the activeSessionId field or don't have the activeSessionId field at all.
However, if you consider at some point in time using Cloud Firestore, then please note that you can achieve something similar using whereIn() in a query that looks like this:
db.collection("products").whereIn("activeSessionId", Arrays.asList("someId", "defaultId"))
This means that you have to set a default value to all users who don't have set a specific ID.
Also, in the case of Firestore, a query like this:
db.collection("products").whereIn("activeSessionId", Arrays.asList("someId", null));
Won't work. Besides that, you cannot also query for non-existing fields. There is no "undefined" value for Firestore fields. Here are the supported data types. You can see that null is a supported data type but not undefined.
So we cannot query for fields that don't exist in Firestore. We can only search for fields that exist in the index, hence the presence of the defaultId in that array.
I have this data model.
/tests/testId
{
"photos":{
79075240-f6c3-11ea-9d76-c328c656dbfc:{
"url":"",
"votes":0
},
7a394290-f6c3-11ea-bd51-5d216a9dfad9:{
"url":"urlperPhoto"
"votes":0
}
},
"moderated":false,
"owner":o8SIEjIByyaNciEgCFH5Kfh4ngh2,
"active":false,
"votes":0
}
/tests/testId/votes
{
photoId: 'xxx',
birthday: null,
sex: false,
votedDate: null
}
I would like to get a list of posts without which I voted. Because I have voted in other collections so I can add additional field for the post model.
Example:
votedUsers: [user1, user2, user3] or votedUsers: {user1: true, user2: true}
But... I don't have in firebase filter like "not exists". How can I display posts for the user, without this which he voted?
This sort of query is not possible with Firestore, as there are no indexes for data that doesn't exist. You can only query for data that does exist, and is indexed. This means that you will need to execute one query to get some possible items that the user has not voted on, then compare that to the results of another query that checks to see if that user has voted on, and remove those from the result set. Yes, this is difficult, and potentially expensive. But this is just not the sort of problem that Firestore is good at.
You might want to consider using another data along with Firestore in order to maintain this sort of relationship between users and things they have not yet seen or done. (It just won't scale like Firestore.)
See also:
Firebase Firestore Structure for getting un-seen trending posts - Social
How to query Cloud Firestore for non-existing keys of documents
Summary
How could I model my database in Firebase to keep, for example, reviews in a specific page updated with the users info, this is, if a user changes it's avatar or name, the reviews should also display the updated data of the user.
I've used MongoDB most of the time, with Mongoose, and I am now working on a mobile app with Firebase. In Mongo I would just store a ref to the user in the review, and populate the field to retrieve the data I wanted from the document. Is there something like this in Firebase, and is it even a good or acceptable practice?
Quick Questions
Is there something like ".populate()" in Firebase?
Should I model the documents as much as possible to have the data that will be used in the view, and avoid "joins"?
Example
We have a users collection, and a store collection with reviews in it.
As far as I've read, you should minimize the doc reads, and so we should model our data with the specific values we need for the view were they will be used, so that we only need to do one query.
For the sake of simplification, let's say:
User has a name, email, avatar
users: {
user_id_1: {
email: "user1#gmail.com",
name: "John Doe",
avatar: "some_firestore_url"
}
}
Should the store collection:
Have nested collection of reviews like this
stores: {
store_id_1: {
name: "Dat Cool Store!",
reviews: {
user_id_1: {
name: "John Doe",
avatar: "some_firestore_url",
text: "Great store love it!",
timestamp: "May 07, 2020 at 03:30"
}
}
}
}
The problem I see with this, is that unless we use a function that updates every field in every document with the new values there is no other way to update the data in name and avatar.
Have the user_id in a field and query for the user information after:
stores: {
store_id_1: {
name: "Dat Cool Store!",
reviews: {
review_id_1: {
user: "user_id_1",
text: "Great store love it!",
timestamp: "May 07, 2020 at 03:30"
}
}
}
}
This is the mimicking the way I would do in MongoDB.
Sorry if some of it sounds confusing or I didn't explain myself the best way, but it's 4 o'clock in the morning here and I'm just trying to get it right :)
How could I model my database in Firebase to keep, for example, reviews in a specific page updated with the user's info, this is, if a user changes its avatar or name, the reviews should also display the updated data of the user.
Without knowing the queries you intend to perform, it's hard to provide a viable schema. We are usually structuring a Firestore database according to the queries that we want to perform.
In Mongo I would just store a ref to the user in the review, and populate the field to retrieve the data I wanted from the document. Is there something like this in Firebase, and is it even a good or acceptable practice?
Yes, there is. According to the official documentation regarding Firestore supported data-types, a DocumentReference is one of them, meaning that you can store only a path to a document and not the entire document. In the NoSQL world, it's quite common to duplicate data, so to have the same data in more than one place. Again, without knowing the use-case of your app it's hard to say whether using normalization it's better than holding only a reference. For a better understanding, I recommend you read my answer from the following post:
What is denormalization in Firebase Cloud Firestore?
And to answer your questions:
Is there something like ".populate()" in Firebase?
If you only store a DocumentReference, it doesn't mean that the data of the document that the reference is pointing to will be auto-populated. No, you first need to get the reference from the document, and right after that, based on that reference, you have to perform another database call, to actually get the data from the referenced document.
Should I model the documents as much as possible to have the data that will be used in the view, and avoid "joins"?
Yes, you should only store the data that you actually need to be displayed in your views. Regarding a JOIN clause, there isn't something like this supported in Firestore. A query can only get documents in a single collection at a time. If you want to get, for example, data from two collections, you'll have at least two queries to perform.
Another solution would be to add a third collection with data already merged from both collections so you can perform a single query. This is already explained in the link above.
Some other information that might be useful is explained in my answer from the following post:
Efficiency of searching using whereArrayContains
Where you can find the best practice to save data into a document, collection, or subcollection.
For me, the way I would go ahead with structuring my json collection also depends on the size of data, I am trying to store in the collection.
Let's say the number of users if small and I only want to support a thousand users. So in that case, I can go with this structure.
{
"store_id_1": {
"name": "Dat Cool Store!",
"reviews": [
{
"user_id_1": {
"name": "John Doe",
"avatar": "some_firestore_url"
},
"text": "Great store love it!",
"timestamp": "May 07, 2020 at 03:30"
},
{
"user_id_2": {
"name": "John Doe 2",
"avatar": "some_firestore_url 2"
},
"text": "Great store love it! TWO",
"timestamp": "May 27, 2020 at 03:30"
}
]
}
}
So now, you can have all the user info embedded in the stores collection. This will reduce your reads too.
But in case you want to scale it, then, I would suggest only store the users metadata and then make another read from users collection.
Hope this helps!
I've been working with Google Cloud Firestore. I'm about to import 13000+ records from a CSV to the firestore back-end. I'll be using this collection for look up and auto-completion purposes.
I'm curious and concerned to know if this is a good idea. Also, I'm looking for some suggestions on what techniques should I be using to make retrieval of this this data as efficient as possible. I'm working with Angular 5 and using AngularFire2 to connect with Firestore.
The document itself is really small such as:
{
address: {
state: "NSW"
street: "19 XYZ Road"
suburb: "Darling Point"
},
user: {
name: "ABC",
company: "Property Management Company"
}
file_no: "AB996"
}
Most of the searching would be based on file_no property of the document.
Update
I just imported all 13k+ records to Firestore. It is really efficient. However, I have one issue. After importing the records, I'm getting the message on my Firestore console that my daily limit for Read Operations is reached (0.05 of 0.05 Million Ops). I just wrote data and displayed those records in a Data Table. I used the following query:
this.propertyService
.getSnapshotChanges()
.subscribe(properties => {
this.properties = properties;
this.loadingIndicator = false;
});
getSnapshotChanges(): Observable < any > {
return this.afs.collection(this.propertiesCollection).snapshotChanges()
.map((actions) => {
return actions.map((snapshot) => {
const data = snapshot.payload.doc.data();
data.id = snapshot.payload.doc.id;
return data;
});
});
}
How dos this makes my reading limit exceed?
The number of documents in a collection is of no consequence when you use Cloud Firestore. That's actually one of its bigger perks: no matter how many documents are in a collection, the queries will take the same amount of time.
Say you add 130 document and (for sake of example) it takes 1 second to get 10 documents out of it. That's the performance you'll get no matter how many documents are in the collection. So with 1300 documents it will also take 1 second, with 13K it will take 1 second, and with 13M, it will also take 1 second.
The problem more developers run into is to make their use-cases fit within the API of Firestore. For example: the only way to search for strings is with a so-called prefix match, there is no support for full-text search. This means that you can search for Prop* and find Property Management Company, but not for *Man* to find it.
Considering the following structure of the Firebase database:
root
game1
$playerUidA
score: 50
$playerUidB
score: 10
.....
game2
$playerUidC
score: 20
$playerUidD
score: 30
.....
game3
.....
I want to run a query that will return all game nodes, where the children of each game node (the players) will be sorted based on the score. The game node contains the UID of each player, and each UID node contains the score of the player. I am storing other data as well, however, for the sake of this example I will be only using the score.
Can I do that with a single query? Something like
rootRef.child("root").orderByChild("score")? Unfortunately that doesn't seem to work.
Or the only way to achieve that is by manually sorting the items on the client?
#Puf - Hope you'll answer that :)
Although this question is rather old there might be people (like me) stumbling over it. Especially because it is pretty intuitive to structure the database in a similar way the author of the question did, to create (for example) a leaderboard system for a game. Since the answer is a bit outdated I wanted to add some things.
Some time ago the devs added the possibility to order by deeply nested children! (See Ordering by a specified child key) To do so you basically have to do the exact same thing the author did and additionally use the first part of the answer given by #adolfosrs. Basically you have to do two things:
Work with .indexOn (as described by #adolfosrs)
Use the OrderByChild() command.
To make this work on the example given by #steliosf you would have to do the following:
First set the .indexOn in your database rules:
{
"rules": {
"$gameId": {
".indexOn": "score",
"$playerUid": {
...
}
}
}
}
Second use the Command the author of the question already used:
rootRef.Child("root").OrderByChild("score")
I would recommend that you always add a LimitToFirst() or LimitToLast() command to avoid that you pull the whole database which might be a lot of data (depending on the size of your database of course). To get for example the top 10 scores you could use:
rootRef.Child("root").OrderByChild("score").LimitToLast(10)
Since the data is ordered in ascending order you need to use LimitToLast().
If you want all the games sorted by the player score all you need to do is to work with your .indexOn rule.
{
"rules": {
"$gameId": {
".indexOn": "score",
"$playerUid": {
...
}
}
}
}
This will keep your data sorted in the database so when you retrieve it you will have all the data ready. But keep in mind that if you want to retrieve all the games and all the players it means you will be fetching the whole database. So you should look at the needs of your application and maybe rethink the structure.
Something that could help when scaling is to iterate over the games and retrieve a limited amount of users with firebase.database().ref(gameId).limitToLast(10).
Update
For your example you will have all the games with the following request:
firebase.database().ref().once('value', snap => {
//prints all players for each game sorted ascending.
snap.forEach(game => {
game.forEach(player => {
console.log(player.val());
});
});
});