I'm working with Firestore and Typescript.
For the data models I have types definitions. For example User could be this:
interface User {
name: string;
age: number;
}
The users are stored in the database in the users collection under a unique name/id.
In Firebase when you query a collection, the ids of the documents are available on the document reference, and do not come with the data. In a common use-case for front-end, you want to retrieve an array of records with their ids, because you probably want to interact with them and need to identify each.
So I made a query similar to the code below, where the id is merged into the resulting array:
async function getUsers(): Promise<any[]> {
const query = db.collection("users")
const snapshot = await query.get();
const results = snapshot.docs.map(doc => {
return { ...doc.data(), id: doc.id };
});
}
Now the problem is, that I have a User type, but it can't be used here because it does not contain an id field.
A naïve solution could be to create a new type:
interface UserWithId extends User {
id: string
}
And write the function like:
async function getUsers(): Promise<UserWithId[]> {}
But this doesn't feel right to me, because you would have to potentially do this for many types.
A better solution I think would be to create a generic type:
type DatabaseRecord<T> = {
id: string,
data: T
}
Thus keeping data and ids separate in the returning results:
const results = snapshot.docs.map(doc => {
return { data: doc.data(), id: doc.id };
});
... and use the function signature:
async function getUsers(): Promise<DatabaseRecord<User>[]> {}
I would favour the second over the first solution, because creating new types for each case feels silly. But I am still not sure if that is the best approach.
This seems like such a common scenario but I didn't manage to find any documentation on this. I have seen developers simply write the id in the model data, essentially duplicating the document name in its data, but that to me seems like a big mistake.
I can imagine that if you don't use Typescript (of Flow) that you just don't care about the resulting structure and simply merge the id with the data, but this is one of the reasons I really love using type annotation in JS. It forces you think more about your data and you end up writing cleaner code.
Related
Given the data structure below in firebase, i want to run a query to retrieve the blog 'efg'. I don't know the user id at this point.
{Users :
"1234567": {
name: 'Bob',
blogs: {
'abc':{..},
'zyx':{..}
}
},
"7654321": {
name: 'Frank',
blogs: {
'efg':{..},
'hij':{..}
}
}
}
The Firebase API only allows you to filter children one level deep (or with a known path) with its orderByChild and equalTo methods.
So without modifying/expanding your current data structure that just leaves the option to retrieve all data and filter it client-side:
var ref = firebase.database().ref('Users');
ref.once('value', function(snapshot) {
snapshot.forEach(function(userSnapshot) {
var blogs = userSnapshot.val().blogs;
var daBlog = blogs['efg'];
});
});
This is of course highly inefficient and won't scale when you have a non-trivial number of users/blogs.
So the common solution to that is to a so-called index to your tree that maps the key that you are looking for to the path where it resides:
{Blogs:
"abc": "1234567",
"zyx": "1234567",
"efg": "7654321",
"hij": "7654321"
}
Then you can quickly access the blog using:
var ref = firebase.database().ref();
ref.child('Blogs/efg').once('value', function(snapshot) {
var user = snapshot.val();
ref.child('Blogs/'+user+'/blogs').once('value', function(blogSnapshot) {
var daBlog = blogSnapshot.val();
});
});
You might also want to reconsider if you can restructure your data to better fit your use-case and Firebase's limitations. They have some good documentation on structuring your data, but the most important one for people new to NoSQL/hierarchical databases seems to be "avoid building nests".
Also see my answer on Firebase query if child of child contains a value for a good example. I'd also recommend reading about many-to-many relationships in Firebase, and this article on general NoSQL data modeling.
Given your current data structure you can retrieve the User that contains the blog post you are looking for.
const db = firebase.database()
const usersRef = db.ref('users')
const query = usersRef.orderByChild('blogs/efg').limitToLast(1)
query.once('value').then((ss) => {
console.log(ss.val()) //=> { '7654321': { blogs: {...}}}
})
You need to use limitToLast since Objects are sorted last when using orderByChild docs.
It's actually super easy - just use foreslash:
db.ref('Users').child("userid/name")
db.ref('Users').child("userid/blogs")
db.ref('Users').child("userid/blogs/abc")
No need of loops or anything more.
In the example's src(UserPage.js):
const mapStateToProps = (state, ownProps) => {
// We need to lower case the login due to the way GitHub's API behaves.
// Have a look at ../middleware/api.js for more details.
const login = ownProps.params.login.toLowerCase()
const {
pagination: { starredByUser },
entities: { users, repos }
} = state
const starredPagination = starredByUser[login] || { ids: [] }
const starredRepos = starredPagination.ids.map(id => repos[id])
const starredRepoOwners = starredRepos.map(repo => users[repo.owner])
return {
login,
starredRepos,
starredRepoOwners,
starredPagination,
user: users[login]
}
}
I notice that there is many templates like xxx.ids.map(id => someEntities[id]),I am not sure why use this pattern to work.IMO,I would use something like import { map } from 'lodash'; someList && map(someList, item => {...}) in the container component and just pass the entities in the mapStateToProps.
So,could someone explains it's purpose?Thanks.
The standard suggestion for normalizing data in Redux is to store data items in an object, with IDs as the keys and the items as the values. However, an object doesn't have an inherent order to it. (Technically, the order of iteration for object keys should be consistent, but it's a bad practice to rely on that as the sole means of ordering.)
Because of that, it's also standard to store an array of just the IDs as well. A typical example might look like:
{
byId : {
qwerty : { },
abcd : { },
aj42913 : { }
},
items : ["qwerty", "aj42913", "abcd"],
sorted : ["abcd", "aj42913", "qwerty"],
selected : ["qwerty", "abcd"]
}
In this example, items contains all item IDs, probably in the order they were insert. sorted contains the IDs in some sort of sorted order, while selected contains a subset of the IDs.
This allows the items themselves to only be stored once, while multiple representations of those items can be saved using various arrays of IDs.
From there, you can pull together a list of the actual items by mapping over whatever array of IDs you care about, and retrieving the items by their IDs.
So, ultimately the answer is that relying just on the keys of the byId object doesn't give you any kind of ordering, and doesn't allow defining subsets of the data.
I have a game built on Meteor framework. One game document is something like this:
{
...
participants : [
{
"name":"a",
"character":"fighter",
"weapon" : "sword"
},
{
"name":"b",
"character":"wizard",
"weapon" : "book"
},
...
],
...
}
I want Fighter character not to see the character of the "b" user. (and b character not to see the a's) There are about 10 fields like character and weapon and their value can change during the game so as the restrictions.
Right now I am using Session variables not to display that information. However, it is not a very safe idea. How can I subscribe/publish documents according to the values based on characters?
There are 2 possible solutions that come to mind:
1. Publishing all combinations for different field values and subscribing according to the current state of the user. However, I am using Iron Router's waitOn feature to load subscriptions before rendering the page. So I am not very confident that I can change subscriptions during the game. Also because it is a time-sensitive game, I guess changing subscriptions would take time during the game and corrupt the game pleasure.
My problem right now is the user typing
Collection.find({})
to the console and see fields of other users. If I change my collection name into something difficult to find, can somebody discover the collection name? I could not find a command to find collections on the client side.
The way this is usually solved in Meteor is by using two publications. If your game state is represented by a single document you may have problem implementing this easily, so for the sake of an example I will temporarily assume that you have a Participants collection in which you're storing the corresponding data.
So anyway, you should have one subscription with data available to all the players, e.g.
Meteor.publish('players', function (gameId) {
return Participants.find({ gameId: gameId }, { fields: {
// exclude the "character" field from the result
character: 0
}});
});
and another subscription for private player data:
Meteor.publish('myPrivateData', function (gameId) {
// NOTE: not excluding anything, because we are only
// publishing a single document here, whose owner
// is the current user ...
return Participants.find({
userId: this.userId,
gameId: gameId,
});
});
Now, on the client side, the only thing you need to do is subscribe to both datasets, so:
Meteor.subscribe('players', myGameId);
Meteor.subscribe('myPrivateData', myGameId);
Meteor will be clever enough to merge the incoming data into a single Participants collection, in which other players' documents will not contain the character field.
EDIT
If your fields visibility is going to change dynamically I suggest the following approach:
put all the restricted properties in a separated collection that tracks exactly who can view which field
on client side use observe to integrate that collection into your local player representation for easier access to the data
Data model
For example, the collection may look like this:
PlayerProperties = new Mongo.Collection('playerProperties');
/* schema:
userId : String
gameId : String
key : String
value : *
whoCanSee : [String]
*/
Publishing data
First you will need to expose own properties to each player
Meteor.publish('myProperties', function (gameId) {
return PlayerProperties.find({
userId: this.userId,
gameId: gameId
});
});
then the other players properties:
Meteor.publish('otherPlayersProperties', function (gameId) {
if (!this.userId) return [];
return PlayerProperties.find({
gameId: gameId,
whoCanSee: this.userId,
});
});
Now the only thing you need to do during the game is to make sure you add corresponding userId to the whoCanSee array as soon as the user gets ability to see that property.
Improvements
In order to keep your data in order I suggest having a client-side-only collection, e.g. IntegratedPlayerData, which you can use to arrange the player properties into some manageable structure:
var IntegratedPlayerData = new Mongo.Collection(null);
var cache = {};
PlayerProperties.find().observe({
added: function (doc) {
IntegratedPlayerData.upsert({ _id : doc.userId }, {
$set: _.object([ doc.key ], [ doc.value ])
});
},
changed: function (doc) {
IntegratedPlayerData.update({ _id : doc.userId }, {
$set: _.object([ doc.key ], [ doc.value ])
});
},
removed: function (doc) {
IntegratedPlayerData.update({ _id : doc.userId }, {
$unset: _.object([ doc.key ], [ true ])
});
}
});
This data "integration" is only a draft and can be refined in many different ways. It could potentially be done on server-side with a custom publish method.
I'm implementing an orbit.js adapter for firebase, orbit-firebase.
I'm looking for an efficient way to query for multiple records so that I can resolve relationships between objects e.g. course.participants
{
course: {
'c1': {
participants: ['p1', 'p2']
}
},
participant: {
'p1': {
name: "Jim"
},
'p2': {
name: "Mark"
}
}
}
Given I have the ids 'p1' and 'p2' what's an efficient way to query for both of them?
I can't use a query because I'm using security rules with the participants i.e. the user that's trying to resolve course.participants doesn't have access to all of the participants (bear in mind this is a contrived example).
I'd recommend that you move away from arrays in your JSON structures. These are nothing but pain in real-time, distributed data and don't work particularly well with security rules and situations like this.
Given this structure:
course: {
'c1': {
participants: {
'p1': true, 'p2': true
}
}
}
I could join these fairly easily. You can get a normalized ref that behaves just like a Firebase ref by using Firebase.util's NormalizedCollection:
var ref = new Firebase(...);
var coll = new Firebase.util.NormalizedCollection(
ref.child('course/c1/participants'),
ref.child('participant')
).select('participant.name').ref();
coll.on('child_added', function(snap) {
console.log('participant ' + snap.key(), snap.val());
});
Note that this data structure (sans the array) will also make it simpler to enforce read rules on participant data and the like by allowing you to directly reference the user ids under $courseid/participants/, since they are now keys that can match a $ variable.
I'm trying to work with associating documents in different collections (not embedded documents) and while there is an issue for that in Mongooose, I'm trying to work around it now by lazy loading the associated document with a virtual property as documented on the Mongoose website.
The problem is that the getter for a virtual takes a function as an argument and uses the return value for the virtual property. This is great when the virtual doesn't require any async calls to calculate it's value, but doesn't work when I need to make an async call to load the other document. Here's the sample code I'm working with:
TransactionSchema.virtual('notebook')
.get( function() { // <-- the return value of this function is used as the property value
Notebook.findById(this.notebookId, function(err, notebook) {
return notebook; // I can't use this value, since the outer function returns before we get to this code
})
// undefined is returned here as the properties value
});
This doesn't work since the function returns before the async call is finished. Is there a way I could use a flow control library to make this work, or could I modify the first function so that I pass the findById call to the getter instead of an anonymous function?
You can define a virtual method, for which you can define a callback.
Using your example:
TransactionSchema.method('getNotebook', function(cb) {
Notebook.findById(this.notebookId, function(err, notebook) {
cb(notebook);
})
});
And while the sole commenter appears to be one of those pedantic types, you also should not be afraid of embedding documents. Its one of mongos strong points from what I understand.
One uses the above code like so:
instance.getNotebook(function(nootebook){
// hey man, I have my notebook and stuff
});
While this addresses the broader problem rather than the specific question, I still thought it was worth submitting:
You can easily load an associated document from another collection (having a nearly identical result as defining a virtual) by using Mongoose's query populate function. Using the above example, this requires specifying the ref of the ObjectID in the Transaction schema (to point to the Notebook collection), then calling populate(NotebookId) while constructing the query. The linked Mongoose documentation addresses this pretty thoroughly.
I'm not familiar with Mongoose's history, but I'm guessing populate did not exist when these earlier answers were submitted.
Josh's approach works great for single document look-ups, but my situation was a little more complex. I needed to do a look-up on a nested property for an entire array of objects. For example, my model looked more like this:
var TransactionSchema = new Schema({
...
, notebooks: {type: [Notebook]}
});
var NotebookSchema = new Schema({
...
, authorName: String // this should not necessarily persist to db because it may get stale
, authorId: String
});
var AuthorSchema = new Schema({
firstName: String
, lastName: String
});
Then, in my application code (I'm using Express), when I get a Transaction, I want all of the notebooks with author last name's:
...
TransactionSchema.findById(someTransactionId, function(err, trans) {
...
if (trans) {
var authorIds = trans.notebooks.map(function(tx) {
return notebook.authorId;
});
Author.find({_id: {$in: authorIds}, [], function(err2, authors) {
for (var a in authors) {
for (var n in trans.notebooks {
if (authors[a].id == trans.notebooks[n].authorId) {
trans.notebooks[n].authorLastName = authors[a].lastName;
break;
}
}
}
...
});
This seems wildly inefficient and hacky, but I could not figure out another way to accomplish this. Lastly, I am new to node.js, mongoose, and stackoverflow so forgive me if this is not the most appropriate place to extend this discussion. It's just that Josh's solution was the most helpful in my eventual "solution."
As this is an old question, I figured it might use an update.
To achieve asynchronous virtual fields, you can use mongoose-fill, as stated in mongoose's github issue: https://github.com/Automattic/mongoose/issues/1894