I am working on a card game platform with a lot (10,000+) of cards dynamically updated with real-world data. Cards are populated/updated once a day.
I have two basic collections at the foundation (asides from users):
1) data - all individual items with different data values for same data fields/parameters (for example, various car models with their specifications). I update this collection once a day from a json API I have on another server for another purpose.
2) cards - "printed" cards with unique IDs but duplicates are off course possible (so we can have 10 Ford Focus 2010. cards).
Cards collection has a couple of most important fields from data collection (model, brand, top performance parameter(s) of the card) to provide efficient user card browsing, and a "dataId" field which links it to data collection for detailed info.
Cards in collection "cards" should be inserted ("issued" or "printed") with functions/methods on server side but in response to client side events (such as new-game etc). When a new card is inserted/dispatched, it first gets a unique "admin-owner" with a user _id from users table for one-to-one relationship, which is later updated to create ownership.
So, on client side, cards collection is like a user "deck" (all cards where owner is user). If I am correct, it should be written on the server side as:
Meteor.publish('cards', function() {
return Cards.find({"userID":this.userId});
});
This is all quite clear and up to that point Meteor is fantastic as it saves me months of work!
But, I am not sure about:
1) I would like to have a client-side data collection publication to cover client detailed card view (by linking cards with data). It should off course have only all data items from data collection with details for each card in client card collection ("deck"). I see it as something like:
Meteor.publish('data', function (dataIds *array with all unique data item ids in client card collection *) {
return Data.find("dataID":{$in:dataIds);
});
2) I need a server/client method to add/insert new cards from data-items ("create 10 news Ford Focus 2010 cards") with an empty/admin user by executing Meteor.call methods from client console of "admin" user, and a server/client method to change ownership of a random card so that it becomes a part of a client cards collection ("cast random card to user").
Where would I place those methods? How can I access server methods from client console (if a certain admin user is logged)?
4) I need a clever way of handling a server publication/client subscription of data collection that will have only the data used in cards from client cards collection.
Should I use client side minimongo query to create an array with all dataIds needed to cover local cards collection? I am new to mongo, so I am not sure how would I write something like SELECT DISTINCT or GROUP BY to get that. Also, not sure if that is the best way to handle that, or should I do something server side as a publication?
Having a clear idea on 1-4 would get me going and then I guess I would dig my way around (and under :)
1) The publish function you wrote makes perfect sense. Of course, there's a bit confusion in the term "client-side data collection publication": publications are on the server side, while on the client side you've got subscriptions. Also, while you didn't specify your schema, I suppose you've got dataID field in cards collection, that joins with _id in data collection, so your find should say {_id: {$in: dataIds}}.
2) Read this carefully, there's all you need for that. Remember to check user privileges within the server side method. A rule of thumb for security is that you should never trust the client.
3) There's no point 3?
4) I'm not sure how the question here is different from 1. However, you should probably familiarize with this method, which you can use in your subscription to ensure the _ids in the array are unique.
Related
I am building an iOS app that is using Cloud Firestore (not Firebase realtime database) as a backend/database.
Google is trying to push new projects towards Cloud Firestore, and to be honest, developers with new projects should opt-in for Firestore (better querying, easier to scale, etc..).
My issue is the same that any relational database developer has when switching to a no-SQL database: data modeling
I have a very simple scenario, that I will first explain how I would configure it using MySQL:
I want to show a list of posts in a table view, and when the user clicks on one post to expand and show more details for that post (let say the user who wrote it). Sounds easy.
In a relational database world, I would create 2 tables: one named "posts" and one named "users". Inside the "posts" table I would have a foreign key indicating the user. Problem solved.
Poor Barry, never had the time to write a post :(
Using this approach, I can easily achieve what I described, and also, if a user updates his/her details, you will only have to change it in one place and you are done.
Lets now switch to Firestore. I like to think of RDBMS's table names as Firestore's collections and the content/structure of the table as the documents.
In my mind i have 2 possible solutions:
Solution 1:
Follow the same logic as the RDBMS: inside the posts collection, each document should have a key named "userId" and the value should be the documentId of that user. Then by fetching the posts you will know the user. Querying the database a second time will fetch all user related details.
Solution 2:
Data duplication: Each post should have a map (nested object) with a key named "user" and containing any user values you want. By doing this the user data will be attached to every post it writes.
Coming from the normalization realm of RDBMS this sounds scary, but a lot of no-SQL documents encourage duplication(?).
Is this a valid approach?
What happens when a user needs to update his/her email address? How easily you make sure that the email is updated in all places?
The only benefit I see in the second solution is that you can fetch both post and user data in one call.
Is there any other solution for this simple yet very common scenario?
ps: go easy on me, first time no-sql dev.
Thanks in advance.
Use solution 1. Guidance on nesting vs not nesting will depend on the N-to-M relationship of those entities (for example, is it 1 to many, many to many?).
If you believe you will never access an entity without accessing its 'parent', nesting may be appropriate. In firestore (or document-based noSQL databases), you should make the decision whether to nest that entity directly in the document vs in a subcollection based on the expect size of that nested entity. For example, messages in a chat should be a subcollection, as they may in total exceed the maximum document size.
Mongo, a leading noSQL db, provides some guides here
Firestore also provided docs
Hope this helps
#christostsang I would suggest a combination of option 1 and option 2. I like to duplicate data for the view layer and reference the user_id as you suggested.
For example, you will usually show a post and the created_by or author_name with the post. Rather than having to pay additional money and cycles for the user query, you could store both the user_id and the user_name in the document.
A model you could use would be an object/map in firestore here is an example model for you to consider
posts = {
id: xxx,
title: xxx,
body: xxx,
likes: 4,
user: {refId: xxx123, name: "John Doe"}
}
users = {
id: xxx,
name: xxx,
email: xxx,
}
Now when you retrieve the posts document(s) you also have the user/author name included. This would make it easy on a postList page where you might show posts from many different users/authors without needed to query each user to retrieve their name. Now when a user clicks on a post, and you want to show additional user/author information like their email you can perform the query for that one user on the postView page. FYI - you will need to consider changes that user(s) make to their name and if you will update all posts to reflect the name change.
I've been using redux-crud, and really like it. I'm still very new to Redux itself, so the following question may seem a bit noob.
I'm at the point where I want to fetch just a single record freshly from the server when I enter the edit form for it. I can't rely on the record that might have been previously fetched into the state, to be the most accurate representation of it for editing purposes.
Based on my current understanding, it doesn't seem that redux-form is suited to fetching singles, rather it seems to suggest that I pull the record for edit out of the collection of records already in the state (previously fetched with the out of the box crud actions and reducers).
I have a type of record called Providers.
Am I right to say that I'm going to have to create a separate set of fetch actions and reducers that are suited to singular fetching?
So where Redux Crud would give me PROVIDERS_FETCH_SUCCESS, I would then need to go on and implement PROVIDER_FETCH_SUCCESS in a singular fashion? Or is there a simpler way out of the box with Redux Crud that I'm not seeing clearly?
Thanks!
redux-crud will manage updating items for you when you need to fetch individual entities. actionCreatorsFor and reducersFor both take optional secondary arguments to specify what
reduxCrud.actionCreatorsFor('providers', {key: '_id'});
reduxCrud.Map.reducersFor('providers', {key: '_id'});
You can then mark the action as "replace" to replace existing items that have matching key fields;
fetchSuccess(providers, {replace: true});
Or
fetchSuccess([provider], {replace: true});
I have a problem with subscriptions which should have fill same collection on the client with the different set of records.
For example:
I have a Books collection and two different publications:
Meteor.publish(‘books’, () => Books.find({ status: { $ne: 3 } });
publish “booksForReservation” which is returning an array of books (books are filtered based on the reservation and some other data)
Problem occurs on the client, when I coming from one route to another. All Books collection are in the main component, and when I need those booksForReservation collection on the client is not updating with only that specific set of books.
I have subscribed on the client like:
Meteor.subscribe(‘booksForReservation’, reservationsIds);
let books = Books.find({}).fetch();
but I’m still getting all books collection displayed. when I have filtered Books collection on the client side with the same query used on the server, collection is updated. But then filtering it on the server doesn’t have any point.
How can I update same collection with different subscription?
You have to filter collection on the client side with a query from the server. You subscribe to two publications, so on the client you will have data from both of them. The point of filtering collection on the server side is security. By doing it you won't publish any unwanted data to the client. You can read more about it here https://guide.meteor.com/data-loading.html#specific-queries
I'm designing a chat app much like Facebook Messenger. My two current root nodes are chats and users. A user has an associated list of chats users/user/chats, and the chats are added by autoID in the chats node chats/a151jl1j6. That node stores information such as a list of the messages, time of the last message, if someone is typing, etc.
What I'm struggling with is where to make the definition of which two users are in the chat. Originally, I put a reference to the other user as the value of the chatId key in the users/user/chats node, but I thought that was a bad idea incase I ever wanted group chats.
What seems more logical is to have a chats/chat/members node in which I define userId: true, user2id: true. My issue with this is how to efficiently query it. For example, if the user is going to create a new chat with a user, we want to check if a chat already exists between them. I'm not sure how to do the query of "Find chat where members contains currentUserId and friendUserId" or if this is an efficient denormalized way of doing things.
Any hints?
Although the idea of having ids in the format id1---||---id2 definitely gets the job done, it may not scale if you expect to have large groups and you have to account for id2---||---id1 comparisons which also gets more complicated when you have more people in a conversation. You should go with that if you don't need to worry about large groups.
I'd actually go with using the autoId chats/a151jl1j6 since you get it for free. The recommended way to structure the data is to make the autoId the key in the other nodes with related child objects. So chats/a151jl1j6 would contain the conversation metadata, members/a151jl1j6 would contain the members in that conversation, messages/a151jl1j6 would contain the messages and so on.
"chats":{
"a151jl1j6":{}}
"members":{
"a151jl1j6":{
"user1": true,
"user2": true
}
}
"messages":{
"a151jl1j6":{}}
The part where this gets is little "inefficient" is the querying for conversations that include both user1 and user2. The recommended way is to create an index of conversations for each user and then query the members data.
"user1":{
"chats":{
"a151jl1j6":true
}
}
This is a trade-off when it comes to querying relationships with a flattened data structure. The queries are fast since you are only dealing with a subset of the data, but you end up with a lot of duplicate data that need to be accounted for when you are modifying/deleting i.e. when the user leaves the chat conversation, you have to update multiple structures.
Reference: https://firebase.google.com/docs/database/ios/structure-data#flatten_data_structures
I remember I had similar issue some time ago. The way how I solved it:
user 1 has an unique ID id1
user 2 has an unique ID id2
Instead of adding a new chat by autoId chats/a151jl1j6 the ID of the chat was id1---||---id2 (superoriginal human-readable delimeter)
(which is exactly what you've originally suggested)
Originally, I put a reference to the other user as the value of the chatId key in the users/user/chats node, but I thought that was a bad idea in case I ever wanted group chats.
There is a saying: https://en.wikipedia.org/wiki/You_aren%27t_gonna_need_it
There might a limitation of how many userIDs can live in the path - you can always hash the value...
I don't understand the concept of Meteor.subscribe.
It is supposed to receive records from the server, and attach it to collections with the same name, right?
[subscribe] will queue incoming attributes until you declare the Meteor.Collection on the client with the matching collection name.
So, why the example in docs uses different names? What is the relation between allplayers and players?
Meteor.subscribe("allplayers");
...
// client queues incoming players records until ...
...
Players = new Meteor.Collection("players");
There are two names:
The name of the collection ('players' in this case).
The name of the subscription ('allplayers' in this case).
A subscription is a way to get records into a client side collection. The name this collection that the records go into is decided (on the server side) by the use of this.set() in the relevant Meteor.publish function, but usually it is just name of the collection that is queried on the the server side[1].
Many subscriptions can deposit data into the same collection, so certainly the name of the subscription doesn't need to correspond to the name of the collection. In fact, it's probably only a good idea to have them be the same if you are doing a fairly straightforward single subscription to that collection.
[1] If you return a cursor (e.g. return players.find();) in Meteor.publish, it automatically wires up calls to this.set(name) for you, where name is inferred from the server side players collection.