I have some question regarding Firebase database structure. I am not familiar with it as I just started to learn it couple days ago. Basically 1 account can have many receipts, 1 receipt can have many items of different types, 1 receipt for 1 store and 1 receipt for 1 currency.
I have came up with the database design as below:
receipts {
accountID1 : {
receiptID1 : {
date:
store: {
storeName: store1
storeAddr: addr1
}
currency: {
currencyName: currency1
currentcySymbol: $
}
totalAmount: 50.00
}
receiptID2 : { ... }
}
accountID2 : { ... }
},
itemlists {
receiptID1: {
items: {
itemID1 : {
type: food
name: snack
price: 10.00
quantity: 2
}
itemID2 : { ... }
}
}
receiptID2: { ... }
},
receiptIDsByStore {
storeID1: {
receiptID1: true
receiptID2: true
}
},
receiptIDsByCurrency {
currencyID1: {
receiptID1: true
receiptID2: true
}
},
stores {
storeID1: {
storeName: store1
storeAddress: addr1
}
},
currencies {
currencyID1: {
currencyName: currency1
currencySymbol: $
}
}
itemIDsByType {
food: {
itemID1: true,
itemID2: true,
}
entertainment: {
itemID3: true,
itemID4: true,
}
}
So my question is:
Is there any redundancy mistake I made for the design above?
I can get the total amount of spend by each user from receipts, am I right? I can just query like receipts/accountID1 to get all the receipts then sum up the total amount.
How can I actually sum up the total spend by each user for each type of items? For instance, I wanted to find for food. So I query itemIDsByType/food, then get list of itemIDs, then from there query itemlists and check if receiptID is belonged to that particular account, it it is then get the unit price?
EDIT
receipts {
accountID1 : {
receiptID1 : {
date : "07/07/2017"
store : {
storeName : "store1"
storeAddr : "addr1"
}
currency : {
currencyName : "currency1"
currentcySymbol : "$"
}
totalAmount : "50.00"
items : {
itemID1 : true,
itemID2 : true,
}
}
receiptID2 : {
date : "08/07/2017"
store : {
storeName : "store1"
storeAddr : "addr1"
}
currency : {
currencyName : "currency1"
currentcySymbol : "$"
}
totalAmount : "20.00"
items : {
itemID3 : true,
itemID4 : true,
}
}
}
accountID2 : {
receiptID3 : {
date : "08/07/2017"
store : {
storeName : "store2"
storeAddr : "addr2"
}
currency : {
currencyName : "currency1"
currentcySymbol : "$"
}
totalAmount : "100.00"
items : {
itemID5 : true,
itemID6 : true,
}
}
}
},
items {
itemID1 : {
type : "food"
name : "snack"
unitprice : "10.00"
quantity : "2"
}
itemID2 : {
type : "entertainment"
name : "gaming equipment"
unitprice : "150.00"
quantity : "1"
}
itemID3 : {
type : "food"
name : "fruit juice"
unitprice : "4.00"
quantity : "1"
}
itemID4 : {
type : "entertainment"
name : "new year clothes"
unitprice : "100.00"
quantity : "1"
}
itemID5 : {
type : "healthcare"
name : "fever meds"
unitprice : "100.00"
quantity : "1"
}
itemID6 : {
type : "healthcare"
name : "flu meds"
unitprice : "100.00"
quantity : "1"
}
},
receiptIDsByStore {
storeID1 : {
receiptID1 : true,
receiptID2 : true,
}
storeID2 : {
receiptID3 : true,
}
},
receiptIDsByCurrency {
currencyID1 : {
receiptID1 : true,
receiptID2 : true,
receiptID3 : true,
}
},
stores {
storeID1 : {
storeName : "store1"
storeAddress : "addr1"
}
storeID2 : {
storeName : "store2"
storeAddress : "addr2"
}
},
currencies {
currencyID1 : {
currencyName : "currency1"
currencySymbol : "$"
}
},
itemIDsByType {
food : {
itemID1 : true,
itemID3 : true,
}
entertainment: {
itemID2 : true,
itemID4 : true,
}
healthcare : {
itemID5 : true,
itemID6 : true,
}
}
Your database is very well structured. One thing you need to remeber when it comes to Firebase database structuring a is to have the data as flatten as possible and to make everything for the view. If you want to learn more about Firebase database structure i recomand you reading this posts: NOSQL DATA MODELING TECHNIQUES and Structuring your Firebase Data correctly for a Complex App.
Regarding your questions, as you you'll probably see in those posts, having the same data in different location is not a mistake, actually it's the opposite, it's ca ommon practice in Firebase. Yes, you can just query like receipts/accountID1 and get all the receipts. If you need a count, you can just simply use getChildrenCount() method directly on the DataSnapshot. Nested queries are not prohibited in Firebase. You can query once, to get those ids and second to get the desired data according to those ids.
Not at least, if you have a SQL background i recomand you watching this youtube tutorial sesries, The Firebase Database For SQL Developers.
Hope it helps.
Related
I am trying to create capturecontext using https://apitest.cybersource.com/flex/v2/sessions with below request payload
{ "fields" : {
"paymentInformation" : {
"card" : {
"number":"4111111111111111",
"expirationMonth":"12",
"expirationYear":"2031",
"type":"001",
"securityCode":"737"
}
} } }
and getting below response
"{"correlationId":"303ff124-e250-42fc-97ef-388369e3e2af","details":[{"location":"fields.paymentInformation.card","message":"Unknown
field definition property"}],"message":"One or more validation
errors occurred","reason":"VALIDATION_ERROR"}"
Blockquote
I am following exactly same pattern mentioned on Cybersource developer guide.Can somebody please help me to identify the issue here.
The error says what's wrong:
"location":"fields.paymentInformation.card"
"message":"Unknown field definition property"
If you check in https://developer.cybersource.com/docs/cybs/en-us/digital-accept-flex/developer/all/rest/digital-accept-flex/flex-api-2/flex-api-2-generate-capture-context.html, the request should be something like:
{
"fields" : {
"paymentInformation" : {
"card" : {
"number" : { },
"securityCode" : {
"required" : false
},
"expirationMonth" : {
"required" : false
},
"expirationYear" : {
"required" : false
},
"type" : {
"required" : false
}
}
}
}
}
I have a Firebase database structure like this
{
"5ZhJG1NACDdG9WoNZWrBoYGkIpD3" : {
"Company" : {
"5ZhJG1NACDdG9WoNZWrBoYGkIpD3" : {
"authTime" : 1532061957,
"companyName" : "Scopic Software",
"contactName" : "Hoang Scopic",
"email" : "hoang.trinh#scopicsoftware.com",
"firebaseID" : "5ZhJG1NACDdG9WoNZWrBoYGkIpD3",
"isFirstLogin" : false,
"phoneNumber" : "1234567890",
"schoolName" : "MIT",
"teachers" : {
"aOpjnzHpDiZ7uwQQqJoinGvM9ZD3" : "0"
}
}
}
},
"AhZc9B02goOtZ6qBNhz9W0K6Esg2" : {
"Subscription" : {
"-LHlQ4OhijzzFY5HZOT4" : {
"firebaseID" : "-LHlQ4OhijzzFY5HZOT4",
"period" : {
"endAt" : "1533194625",
"startAt" : "1531985025"
},
"status" : "trial"
}
},
"Teacher" : {
"AhZc9B02goOtZ6qBNhz9W0K6Esg2" : {
"authTime" : 1532061932,
"email" : "hoang.trinhj#gmail.com",
"firebaseID" : "AhZc9B02goOtZ6qBNhz9W0K6Esg2",
"isFirstLogin" : false,
"name" : "Hoang Trinh",
"schoolName" : "HUST",
"subscriptions" : {
"-LHlQ4OhijzzFY5HZOT4" : "0"
}
}
}
},
"aOpjnzHpDiZ7uwQQqJoinGvM9ZD3" : {
"Subscription" : {
"-LHlWnpNZazBC5lpXLi0" : {
"firebaseID" : "-LHlWnpNZazBC5lpXLi0",
"period" : {
"endAt" : "1533196388",
"startAt" : "1531986788"
},
"status" : "trial"
}
},
"Teacher" : {
"aOpjnzHpDiZ7uwQQqJoinGvM9ZD3" : {
"Company" : {
"5ZhJG1NACDdG9WoNZWrBoYGkIpD3" : "0"
},
"authTime" : 1532060884,
"email" : "piavghoang#gmail.com",
"firebaseID" : "aOpjnzHpDiZ7uwQQqJoinGvM9ZD3",
"isFirstLogin" : false,
"subscriptions" : {
"-LHlWnpNZazBC5lpXLi0" : "0"
}
}
}
},
"ooR32SjABdOYEkWX6dzy4fE5Kym1" : {
"Admin" : {
"email" : "admin#test.com",
"firstName" : "Hoang",
"lastName" : "Trinh",
"password" : "123456xx"
},
"isAdmin" : true
}
}
This is data in an existing system, so I can't change its structure.
Now I need to write an API to list these records filtered by "email" attribute there (it's nested inside a key).
How can I do a search for this email?
There is one solution that I thought about. It's just getting the whole json back and process data in code (not using Firebase database query functions).
But I can't do that, because AFTER filtering, I need to do paging also.
In order to make the pagination, I need to use query functions like "sortByChild" or "limitToLast", so I have to use these query functions with filtering.
Thank you.
EDIT:
My current implementation for pagination:
getUsers: async (page, perPage, searchTerm) => {
const db = fbAdmin.db();
let dbRef = await db.ref();
if (searchTerm) {
// TODO: Add filter logic here
}
let totalItems = await dbRef.once('value');
let totalItemsNumber = await Object.keys(totalItems.toJSON()).length;
// Calculate for pagination
let lastItemId;
if ((page - 1) * perPage + 1 > totalItemsNumber) {
lastItemId = null;
} else {
let lastItems = await dbRef.orderByKey().limitToLast((page - 1) * perPage + 1).once('value');
lastItemId = Object.keys(lastItems.toJSON())[0];
}
// Do the pagination
let snap;
let data;
if (lastItemId) {
snap = await dbRef.orderByKey().endAt(lastItemId).limitToLast(perPage).once('value');
data = snap.toJSON();
} else {
data = {};
}
let users = Object.keys(data).map(key => {
if (data[key][KEY]) { // Check if it's an admin
return {
role: TYPE_ADMIN,
email: data[key][KEY].email,
name: data[key][KEY].firstName + " " + data[key][KEY].lastName
}
} else if (data[key][Company.KEY]) { // Check if it's a company member
return {
role: TYPE_COMPANY_MEMBER,
email: data[key][Company.KEY][key].email,
name: data[key][Company.KEY][key].name,
schoolName: data[key][Company.KEY][key].schoolName
}
} else if (data[key][Teacher.KEY]) { // Check if it's a teacher
if (data[key][Teacher.KEY][key][Company.KEY]) { // Check if it's a company teacher
return {
role: TYPE_COMPANY_TEACHER,
email: data[key][Teacher.KEY][key].email,
name: data[key][Teacher.KEY][key].name,
schoolName: data[key][Teacher.KEY][key].schoolName,
subscription: data[key][Subscription.KEY]
}
} else { // Check if it's an individual teacher
return {
role: TYPE_INDIVIDUAL_TEACHER,
email: data[key][Teacher.KEY][key].email,
name: data[key][Teacher.KEY][key].name,
schoolName: data[key][Teacher.KEY][key].schoolName,
subscription: data[key][Subscription.KEY]
}
}
} else {
return null;
}
});
return users || null;
},
My firebase database structure is as below. I would like to search with a particular "Info/Email", let's say "abc#abc.com".
{
"-KhWrBcYyMluJbK7QpnK" : {
"Info" : {
"Email" : "xyz#gmail.com"
},
"Settings" : {
"Status" : "Accepted"
}
},
"-KhX0tgQvDtDYqt4XRoL" : {
"Info" : {
"Email" : "abc#abc.com"
},
"Settings" : {
"Status" : "Accepted"
}
},
"-KhX1eo7uFnOxqncDXQ5" : {
"Info" : {
"Email" : "abc#abc.com"
},
"Settings" : {
"Status" : "Pending"
}
}
}
I added a rule too
"Invitation" : {
".indexOn": ["Info/Email","Settings/Status"]
}
My AngularFire code is as follows:
var rootRef = firebase.database().ref().child('Invitation');
var userInvitations = rootRef.child("Info").orderByChild("Email").equalTo("abc#abc.com");
var allInvitations = $firebaseArray(userInvitations);
But I am getting a FIREBASE WARNING: Using an unspecified index. Consider adding ".indexOn": "Email" at /Invitation/Info to your security rules for better performance. and of course I am not receiving any data.
What are the mistakes I made here? Also can I add multiple orderByChild, for example: if I want to find details of the record, which has "Info/Email" equal to "abc#abc.com" and "Settings/Status" equal to "Pending" ?
The Firebase API only allows you to filter children one level deep
So with reference to your data structure:
if it were to be this way,
{
"-KhWrBcYyMluJbK7QpnK" : {
"Email" : "xyz#gmail.com",
"Settings" : {
"Status" : "Accepted"
}
},
"-KhX0tgQvDtDYqt4XRoL" : {
"Email" : "abc#abc.com",
"Settings" : {
"Status" : "Accepted"
}
},
"-KhX1eo7uFnOxqncDXQ5" : {
"Email" : "abc#abc.com",
"Settings" : {
"Status" : "Pending"
}
}
}
You should write your rules this way instead.
"Invitation" : {
"Info" : {
".indexOn": "Email",
},
"Settings" : {
".indexOn": "Status",
}
}
Then you will be able to query the data
var allInvitations = [];
var rootRef = firebase.database().ref().child('Invitation');
var userInvitations = rootRef.orderByChild("Email").equalTo("abc#abc.com");
userInvitations.on('value', snap => {
var invitations = snap.val()
for (prop in invitations ) {
allInvitations.push(invitations[prop ]);
}
console.log(allInvitations);
})
I currently have a data structure that looks like this
Rooms
-k-abcw2222
Messages
-k-112w12123 : {
text : 'hello'
msgdate : 1487558835433
}
-k-112w12125 : {
text : 'hello2'
msgdate : 1487558835633
}
I want to listen to every new message published to a room and am doing a
firebase
.database()
.ref('/Rooms/' + roomkey + '/Messages/')
.orderByChild('msgdate')
.on(...)
but I get the exception
No index defined for orderBy
I created the index as follows
{
"rules" : {
"Rooms" : {
"$Roomsid" : {
"Messages" : {
".indexOn": ["msgdate"]
}
...
}
and also tried creating the index as follows
{
"rules" : {
"Rooms" : {
"$Roomsid" : {
"Messages" : {
"$MessageId" : {
".indexOn": ["msgdate"]
}
...
}
but still did not work. Any suggestions?
I have some data structured like the following in firebase. My client is built with angularjs + angularfire.
'all_records' is nested like this to split the data up and also as the majority of the querying is time orientated.
The 'all_records' nest is equal to: all_records > year > month > date > records.
I want to associate the records with the user that posted the record and therefore on record creation have saved the associated ID to the user in 'my_records'.
This example is obviously a very small dataset however to allow for scalability and potentially having having hundreds of thousands of records what would be the best way to query 'all_records' to get just the users 'my_records'.
Thanks
all_records : {
2015 : {
2 : {
3: {
-JgJJaLaL7LuuRL_IO7c : {
data: big data;
},
-JgJfaOsyfZe_mZPTApA : {
data: lots of data;
},
...
},
...
},
1 : {
28 : {
-JgJJaLaL7kddEl_ddfE : {
data: some more data;
}
},
27 : {
-JgJfadkdjEk_mZPTApA : {
data: even more;
}
},
...
},
...
},
2014 : {
12 : {
24: {
-JgJmnvkFdqc_y63lqcv : {
data: decemeber data;
},
-JgK6xbXY_E58lshaiVa : {
data: and a bit more data;
}
},
...
},
11 : {
...
},
...
}
}
user: {
name : James,
my_records : {
-JgK6xbXY_E58lshaiVa: true,
-JgJJaLaL7LuuRL_IO7c: true
}
}
Your user's my_records currently doesn't contain enough information to to directly map to the record. You'd need to add the date (year, month and day) to the information to allow a direct lookup.
I would instead structure things differently:
a list of just the records with their push-generated -J key
a list of users, with references to their records
a list/tree of dates, with references to the records for each date
Your application should never try to load the list of all records (#1). Instead it should load either the relevant user (from #2) or the relevant date(s) from (#3) and then load only the records that are referenced from there.
all_records : {
-JgJJaLaL7LuuRL_IO7c : { data: big data; },
-JgJfaOsyfZe_mZPTApA : { data: lots of data; },
-JgJJaLaL7kddEl_ddfE : { data: some more data; }
-JgJfadkdjEk_mZPTApA : { data: even more; }
}
dates: {
2015 : {
2 : {
3: {
-JgJJaLaL7LuuRL_IO7c : true,
-JgJfaOsyfZe_mZPTApA : true,
...
},
...
},
1 : {
28 : {
-JgJJaLaL7kddEl_ddfE : true
},
27 : {
-JgJfadkdjEk_mZPTApA : true
},
...
},
...
}
}
user: {
name : James,
my_records : {
-JgK6xbXY_E58lshaiVa: true,
-JgJJaLaL7LuuRL_IO7c: true
}
}
You might also want to consider structuring the dates as a simple list, instead of a tree:
dates: {
20150203: {
-JgJJaLaL7LuuRL_IO7c : true,
-JgJfaOsyfZe_mZPTApA : true,
...
},
20150128 : {
-JgJJaLaL7kddEl_ddfE : true
},
20150127 : {
-JgJfadkdjEk_mZPTApA : true
},
}
While it's tempting to build a tree for the dates, I doubt you'll get much benefit from it. The above structure is simpler and allow for a wider variety of queries, such as "give me all dates in Q1 of 2015": ref.child('dates').startAt('20150101').endAt('20150331').