I have created app whose structure looks like this. Current structure for one company only.
let current = {
products: {
product1: {}//...
},
customers: {
customer1: {},// ...
},
orders: {
order1: {},// ...
},
}
Now I have design data structure to make it multi company app. Suppose companies are ABC, PQR, XYZ but the customers are same. So, a customer can see products from different companies.
Option 1: Add Company property in every lists doc.
let option1 = {
company: {
products: {
product1: {
company: 'ABC'
},
},
customers: { //Also we can put it at root with field company as array. Customers are not primary concern
customer1: {
company: 'PQR'
}
},
orders: {
order1: {
company: 'ABC'
}
},
}
}
My Remarks: I have to put company property in every list which may be more than these. It doesn't look like right solution. Querying products in different companies looks easy.
Option 2: Copy the current root structure for different companies.
let option2 = {
company1: {
products: {
product1: {}
},
customers: {//Also we can put it at root with field company as array. Customers are not primary concern
customer1: {},
},
orders: {
order1: {},
},
},
company2: {
products: {
product1: {}
},
customers: {
customer1: {},
},
orders: {
order1: {},
},
},
// ...
}
My Remarks: I don't know firestore limitations and upcoming features. Querying products in different companies may not be easy.
let option3= {} //your suggestions.
In same firestore project, assume customer handling will not be a problem.
What can be done here? What are things I am missing?
Products
It is unusual for several companies to sell the same product, without wanting to create their own stock item, description, price, etc. Also, if you have an array / map of companies who sell the product, within the product document, customers will see where they can buy the product and there may be no loyalty to any one company. If this is what you're hoping to achieve, then your options could work for you.
Customers
Companies will also want to keep certain data about their customers separate from other companies.
My proposal
I would suggest that you create a Cloud Firestore collection companies and have a document for each company. Within that document, you can create sub-collections for customers, orders and products.
A separate users collection at the root level will allow users to maintain their own data and allow companies to collect whatever they need to keep their records up to date. Adding public and private sub-collections of the user data can manage this easily for you.
Todd Kerpelman from the Firebase team, has made an excellent video which will really help with data modelling in Cloud Firestore.
I hope that this helps
Related
I'm trying to set up a structure for my app moving from SQL structure to Firebase structure. At the moment I'm using the following:
-SQL-
Product: {id, name, workPrice}
Material: {id, name, unitCost}
Product_Material{ pId, mId, amount}
I'm using a table to set the number of materials used so I can get an overall cost of the product.
I read about firebase structuring but I don't know how to apply to this case. What is recommended when associating the two would be the following:
-FB-
Product: {
boxId: {
name: "Wooden box"
workPrice: "5"
materials: {
"woodId": true
}
}
},
Material: {
woodId: {
name: "wood",
unitCost: "10"
}
}
But since I need an amount, it doesn't fit. How would this apply to my case? Do I need to make a third object the same as the third table in SQL?
Since Firebase is a No-SQL database, you can relation two objects by id's, for example, if a product has materials, you can reference the material id of that product, then you query for the product and get that material id, and with that material id, you go to the material node and lookup for that material.
Example
Product: {
name: "Wooden box"
workPrice: "5"
materials: {
"materialId": true,
"materialId2": true
}
},
Material: {
materialId: {
name: "wood",
unitCost: "10"
},
materialId2: {
name: "Plastic",
unitCost:"15"
}
}
So, in this example, lets say you query the wooden box product, when you iterate over the materials sub node of product, you can get each material ID, and then you can relation those material id's with each product.
you can generate random materials ids with the .push() method in Realtime Database or .add() with Firestore
How to insert data in to datastore?
The data could be like the one below:
{
'food': [{
"item_name": item,
'price': price
}, {
"item_name": item,
'price': price
}],
'beverages': [{
''
'beverage_name': beverage,
'beverage_price': b_price
}, {
''
'beverage_name': beverage,
'beverage_price': b_price
}]
}
The data that you are trying to add to the Google Cloud Datastore is a JSON string. The way you have it in your question is wrong structured. The proper JSON example would be:
{
"food": [
{ "food_name":"NAME1", "food_price":"PRICE1" },
{ "food_name":"NAME2", "food_price":"PRICE2" },
{ "food_name":"NAME3", "food_price":"PRICE3" }
],
"beverages":[
{ "beverage_name":"NAME1", "beverage_price":"PRICE1" },
{ "beverage_name":"NAME2", "beverage_price":"PRICE2" }
]
}
To add the data from the JSON string to the Datastore you have to:
Load the JSON string as JSON object to be able to go through its fields
Create a client to access Google Datastore
Set the key food for the Kind value in Datastore
Use the entity to add the data to the Datastore
Set the key beverages for the Kind value in Datastore
Use again entity to add the data to the Datastore
For further information, you can refer to Google Cloud Data Store Entities, Properties, and Keys documentation.
I have done a little bit coding myself and here is my code example in GitHub for Python. You can take the idea of how it works and test it. It will create two different Kind values in Datastore and add the food data in foods and beverage data to beverages.
Requirements:
Users can register to company & unsubscribe freely.
Users should get only the posts of companies they are registered to.
Users shouldn't see posts they already liked/disliked
Currently, what we are doing is, when a user joins a company, we copy the relevant posts to the user User_Feed, and the client fetches the posts.
Now assume we have 40K posts & 500K users. Each user that joins a company, we need to copy 40K of data to the user feed. Or when a company posts a new post, we need to build a huge fan-out object to pass to 500K USER_FEEDs.
This is not scalable.
David East in his post states that the fan-out technique supports millions of records, but even in his example, how can he handle 1M followers?
We feel like we're going about it the wrong way.
Is there a better solution to our problem?
We currently have the following structure:
Companies(~20 companies)
{
companyId: {
...comanyInfo
}
}
Users (~500K users)
uid: {
...userInfo
}
User_Companies (500k * max(20 companies))
uid: {
company1: true,
company2: true,
....
}
Company_Users (20 * max(500K users))
companyId: {
uid1: true,
uid2: true,
....
}
Company_POSTS (2K posts/company)
{
companyId: {
postId: {
...postInfo
}
}
}
User_Feed (For each user (500K))
{
uid:
posId : {
like: true
}
}
This post is also opened in Firebase Google Group
I have a game built on Meteor framework. One game document is something like this:
{
...
participants : [
{
"name":"a",
"character":"fighter",
"weapon" : "sword"
},
{
"name":"b",
"character":"wizard",
"weapon" : "book"
},
...
],
...
}
I want Fighter character not to see the character of the "b" user. (and b character not to see the a's) There are about 10 fields like character and weapon and their value can change during the game so as the restrictions.
Right now I am using Session variables not to display that information. However, it is not a very safe idea. How can I subscribe/publish documents according to the values based on characters?
There are 2 possible solutions that come to mind:
1. Publishing all combinations for different field values and subscribing according to the current state of the user. However, I am using Iron Router's waitOn feature to load subscriptions before rendering the page. So I am not very confident that I can change subscriptions during the game. Also because it is a time-sensitive game, I guess changing subscriptions would take time during the game and corrupt the game pleasure.
My problem right now is the user typing
Collection.find({})
to the console and see fields of other users. If I change my collection name into something difficult to find, can somebody discover the collection name? I could not find a command to find collections on the client side.
The way this is usually solved in Meteor is by using two publications. If your game state is represented by a single document you may have problem implementing this easily, so for the sake of an example I will temporarily assume that you have a Participants collection in which you're storing the corresponding data.
So anyway, you should have one subscription with data available to all the players, e.g.
Meteor.publish('players', function (gameId) {
return Participants.find({ gameId: gameId }, { fields: {
// exclude the "character" field from the result
character: 0
}});
});
and another subscription for private player data:
Meteor.publish('myPrivateData', function (gameId) {
// NOTE: not excluding anything, because we are only
// publishing a single document here, whose owner
// is the current user ...
return Participants.find({
userId: this.userId,
gameId: gameId,
});
});
Now, on the client side, the only thing you need to do is subscribe to both datasets, so:
Meteor.subscribe('players', myGameId);
Meteor.subscribe('myPrivateData', myGameId);
Meteor will be clever enough to merge the incoming data into a single Participants collection, in which other players' documents will not contain the character field.
EDIT
If your fields visibility is going to change dynamically I suggest the following approach:
put all the restricted properties in a separated collection that tracks exactly who can view which field
on client side use observe to integrate that collection into your local player representation for easier access to the data
Data model
For example, the collection may look like this:
PlayerProperties = new Mongo.Collection('playerProperties');
/* schema:
userId : String
gameId : String
key : String
value : *
whoCanSee : [String]
*/
Publishing data
First you will need to expose own properties to each player
Meteor.publish('myProperties', function (gameId) {
return PlayerProperties.find({
userId: this.userId,
gameId: gameId
});
});
then the other players properties:
Meteor.publish('otherPlayersProperties', function (gameId) {
if (!this.userId) return [];
return PlayerProperties.find({
gameId: gameId,
whoCanSee: this.userId,
});
});
Now the only thing you need to do during the game is to make sure you add corresponding userId to the whoCanSee array as soon as the user gets ability to see that property.
Improvements
In order to keep your data in order I suggest having a client-side-only collection, e.g. IntegratedPlayerData, which you can use to arrange the player properties into some manageable structure:
var IntegratedPlayerData = new Mongo.Collection(null);
var cache = {};
PlayerProperties.find().observe({
added: function (doc) {
IntegratedPlayerData.upsert({ _id : doc.userId }, {
$set: _.object([ doc.key ], [ doc.value ])
});
},
changed: function (doc) {
IntegratedPlayerData.update({ _id : doc.userId }, {
$set: _.object([ doc.key ], [ doc.value ])
});
},
removed: function (doc) {
IntegratedPlayerData.update({ _id : doc.userId }, {
$unset: _.object([ doc.key ], [ true ])
});
}
});
This data "integration" is only a draft and can be refined in many different ways. It could potentially be done on server-side with a custom publish method.
I know Firebase does not support JOINs between nodes (like SQL Server does between tables), but that is exactly what I need to accomplish. Here's my situation:
I have a transactions node in Firebase like this (where I am including the category name for each transaction):
"transactions":
{
"-Jruazf35b9a_gAVmZBe":
{
payee: "McDonalds", amount: "2.35", category: "Eating Out"
}
"-JruadR11b4a_aTVmZFi":
{
payee: "Walmart", amount: "78.12", category: "Household"
}
"-Jruazf35b9a_AgvNWCq":
{
payee: "CapitalOne", amount: "150.00", category: "Debt"
}
"-JryJF2c33ijbjbBc24p":
{
payee: "FootLocker", amount: "107.54", category: "Personal Blow"
}
"-Jrz0T-aL61Vuw4SOqRb":
{
payee: "Starbucks", amount: "2.88", category: "Eating Out"
}
}
And I have a Categories node like this (where I am including the transactions under each category):
"categories":
{
"-Jruazf35b2a_gAVmZRy":
{
categoryname: "Eating Out",
categorytype: "Expense"
}
"transactions": {
"-Jruazf35b9a_AgvNWCq": {
payee: "McDonalds", amount: "2.35"
}
.
.
.
}
}
}
So far so good. My data is flat. I'm able to show the list of transactions with the category name (screenshot below) and I can show the list of transactions under each category in the expenses per category section (screenshot not shown here).
The problem I have is that if I rename a category the change is only reflected for future transactions. Past transactions show the old category name.
This is obvious due to the way I'm saving the data. So my first logical reaction was to save the category unique ID in the transactions node instead of the category name. However, that presents the challenge where, in my SQL Server little brain, I would need a JOIN so I can get the list of transactions and also include the name of the category for each transaction.
How can I structure my data so that I can:
show a list of transactions including the name of the category (as it does today)
allow a user to rename a category and show the change reflected for ALL transactions (past and future)
show a list of transactions under each category (I think the current approach would still be valid)
Joining data from two lists is inherently a slow operation, especially on NoSQL databases.
I would recommend keeping the categoryName and adding a categoryId. That way you can show your current screen with a single read, but also link to the category. For how to deal with updating the categoryName in each transaction, see How to write denormalized data in Firebase and https://medium.com/#collardeau/es6-promises-with-firebase-76606f36c80c.
Alternatively: the list of categories is likely to be relatively small. So you could also pre-load it and perform a client-side lookup while you're iterating the transactions.