NoSQL data integrity (Firebase + Redux) - firebase

I saw the following code from one of the firebase example apps (Link):
// Write the new post's data simultaneously in the posts list and the user's post list.
var updates = {};
updates['/posts/' + newPostKey] = postData;
updates['/user-posts/' + uid + '/' + newPostKey] = postData;
When you add a new post, it writes the same data object into two different places. This seemed odd to me because it means now there are two exactly same data that you have to take care of if you want to update or delete them. I'd rather do something like this:
post: {
id: 1,
username: 'awesomeMan', // store necessary author's metadata
title: 'hello',
body: 'body
}
author: {
id: 1,
username: 'awesomeMan',
email: 'hello#gmail.com',
posts: {
1: true // index posts by their ids
}
}
If I do it this way, I still have a bit of data duplicate because I store user's metadata username inside a post. But isn't it better than having two exactly same data in two places?
Edit: Follow Up Question
I want to add Comments resource to Posts. Given these two options, which one is better, and why?
Option 1.
Store all comments related to postId under this path:
/post-comments/${postId}
This is the only path for comments and we don't store anything into comments/
Option 2.
Store comments in comments/ and index them by ids in posts.
/post/${postId}/comments #only consists of commentIds that belong to this post
/comments/${commentId}

Related

How to efficiently update a field in Firestore from Google Sheets

I am working with Google Sheets, and I am trying to send data to my Firestore database. I have been able to write to Firestore from Google Sheets, but I can't seem to update a field without completely messing things up.
This is my current testing code:
function getFireStore() {
const email = "your#email.gserviceaccount.com"
const key = "-----BEGIN PRIVATE KEY-----\n your key here \n-----END PRIVATE KEY-----\n";
const id = "project_id";
var firestore = FirestoreApp.getFirestore(email, key, id);
var spreadsheet = SpreadsheetApp.getActive()
var sheet = spreadsheet.getActiveSheet()
var data = {
numIndividuals: sheet.getRange(23, individuals).getValue(),
numTeams: sheet.getRange(23, teams).getValue(),
schoolID: sheet.getRange(23, schoolID).getValue(),
uid: sheet.getRange(23, uid).getValue(),
};
firestore.createDocument("competitions/" + sheet.getRange(23, compId).getValue() + "/registration/abcdefg", data)
}
I understand after playing around with this that it will create a new subcollection titled "registration" with the document "abcdefg." The same thing happens when I use the updateDocument function, as well.
For the website that is reading and writing to this particular Firestore database, I use a similar function .update() to update the document with the correct information. However, in Google Sheets, while it would work the same way it is much more convoluted and tedious to do so.
The way that I came up with for trying to update the document was basically copying everything and adding in the new data.
However, this is seriously tedious and messy. Just copying the data that isn't changed looks like this:
var data = {
compDate: competitions.fields.compDate.stringValue,
contact: competitions.fields.contact.stringValue,
email: competitions.fields.email.stringValue,
grade: competitions.fields.grade.stringValue,
id: competitions.fields.id.integerValue,
maxTeams: competitions.fields.maxTeams.integerValue,
regDate: competitions.fields.regDate.stringValue,
schTeams: competitions.fields.schTeams.integerValue,
schedule: competitions.fields.schedule.stringValue,
site: competitions.fields.site.stringValue,
status: competitions.fields.status.stringValue,
timestamp: competitions.fields.timestamp.integerValue,
user: competitions.fields.user.stringValue,
year: competitions.fields.year.stringValue,
}
The data I want to change is a .mapValue with multiple fields where one of the fields can have multiple fields, which also have multiple fields.
Here's the hierarchy for the field I need to update:
first registration and first team
I know I could do multiple for-loops and whatnot on this, but my question is: is there a simpler way to do this, or do I have to go through and loop over everything to extract only what I want?
As a sidenote, what gets sent to Firestore if I put in the data I got from Firestore using the spread operator, without any editing, it includes every child from the above image. As in, I would have registration -> mapValue -> fields -> 0 -> mapValue -> fields -> etc. And, I don't want those mapValue and fields included, just that actual data (i.e. registration -> 0 -> {schoolID, uid, names, etc.}).

How to access an object into another object in a query database from firebase functions?

I am trying to access to a object into another object in my firebase database, i have a structure like this:
I want to get all the objects that have the email that i send by parameters, i am using .child to access to the childs into my object but i am not success with the query, this is my code
$ ref_db.child("/groups").child("members").orderByChild("email").equalTo(email).once("value", (snapshot)=>{
console.log(snapshot.val());
});
The snapshot.val() always is undefined.
could you help me with the query?
One efficient way to get "all the groups that have that email inside the members object" would be to denormalize you data and have another "main node" in your database where you store all "members" (i.e. their email) and the "groups" they belong to.
This means that each time you add a "member" node under a "group" (including its email) you will also add the group as a child of the member email, in this other "main node".
More concretely, here is how would be the database structure:
Your current structure:
- groups
- -LB9o....
...
- members
- -LB9qbd....
-email: xxxx#zzz.com
- -LBA7R....
-email: yyyyy#aaaa.com
And the extra structure:
- groupsByMembers
- xxxxxx#zzzcom
- Grupo1: true
- yyyyy#aaaacom
- Grupo1: true
- Grupo2: true
- bbbcccc#dddcom
- Grupo6: true
- Grupo8: true
Note that in the "extra structure" the dots within an email address are removed, since you cannot include a point in a node id. You will have to remove them accordingly when writing and querying.
This way you can easily query for the list of groups a member is belonging to, as shown below. Without the need to loop several times over several items. This dernomalization technique is quite classic in NoSQL databases.
const mailToSearchFor = xxxx.xx#zzz.com;
const ref = database.ref('/groupsByMembers/' + mailToSearchFor.replace(/\./g, ''));
ref.once('value', snapshot => {
const val = snapshot.val();
for (let key in val) {
if (val.hasOwnProperty(key)) {
console.log(key);
}
}
});
In order to write to the two database nodes simultaneously, use the update method as explained here https://firebase.google.com/docs/database/web/read-and-write#update_specific_fields
This is because you have a random key before members, you need to go through the path and not skip a node, to be able to access the values:
ref_db.child("groups").child("-LB9oWcnE0wXx8PbH4D").child("members").orderByChild("email").equalTo(email).once("value", (snapshot)=>{
console.log(snapshot.val());
});

Efficiently storing and getting likes in FireStore / Document DB

I have a page with posts and likes for each post.
In FireStore a collection of posts and a collection of likes, and I update the total_likes and recent likes array when a user likes or unlikes a post with cloud functions.
However, I can't figure out how to show for each post if the currently logged in user liked it or not. What's an efficient way to do that for.
Any pointers?
I believe you might need to look at data aggregration. Even though this example is with Angular, I also use the same principle in a different application: https://angularfirebase.com/lessons/firestore-cloud-functions-data-aggregation/
Alternatively, you could store the post_id's that your user likes in their own 'like_array'. Knowing which posts the user currently sees, you can cross reference the shown post_id's with the (single object) 'like_array' from the user to determine if he/she has liked a particular post. In the long run, you could disambiguate like_arrays based on days or weeks, and only query the like_arrays of this and last day/week - based on what post you are showing. If you are working with categories of posts, similar applies: different like_arrays for different categories.
Hope this helps!
One solution would be to have another collection in your Firestore database where you create a document by user, in which you save (and update) an object containing all the posts this user has liked.
Like
- likers (Collection)
- UserUID (doc)
- postIds {
post1_UID: true,
post2_UID: true
}
The idea is to use the technique described in the doc, here: https://firebase.google.com/docs/firestore/solutions/arrays#solution_a_map_of_values
I don't know which language you use in the front end but in JavaScript you would do:
var postToTestId = ....; <- You set this value as you need (e.g. as a function parameter)
firebase.auth().signInWithEmailAndPassword("...", ".....")
.then(function (info) {
var postId = 'azer';
return db.collection('likers')
.where('postIds.'+ postToTestId, '==', true)
.get();
})
.then(function(querySnapshot) {
if (querySnapshot.size > 0) {
console.log("USER LIKES THIS POST!!!");
}
})
.catch(function (error) {
console.log(error);
});
I don't think there is any solution without storing somewhere all the posts each user liked...

Filter and sort on multiple values with Firebase

I'm building a social app using Firebase. I store posts in Firebase like this:
posts: {
"postid": {
author: "userid"
text: "",
date: "timestamp"
category: "categoryid"
likes: 23
}
}
Each post belong to a category and it's possible to like posts.
Now, I'm trying to show posts that belong to a specific category, sorted by the number of likes. It's possible I also want to limit the filter by date, to show only the most recent, most liked posts in a category. How can I do this?
Firebase query functionality doesn't seem to support multiple queries like this, which seems strange...
You can use only one ordering function with Firebase database queries, but proper data structure will allow you to query by multiple fields.
In your case you want to order by category. Rather than have category as a property, it can act as an index under posts:
posts: {
"categoryid": {
"postid": {
author: "userid"
text: "",
date: "timestamp",
category: "categoryid",
likes: 23
}
}
}
Now you can write a query to get all the posts underneath a specific category.
let postsRef = Firebase(url: "<my-firebase-app>/posts")
let categoryId = "my-category"
let categoryRef = postsRef.childByAppendingPath(categoryId)
let query = categoryRef.queryOrderedByChild("date")
query.observeEventType(.ChildAdded) { (snap: FDataSnapshot!) {
print(snap.value)
}
The code above creates a reference for posts by a specific category, and orders by the date. The multiple querying is possible by the data structure. The callback closure fires off for each individual item underneath the specified category.
If you want to query further, you'll have to do a client-side filtering of the data.

Optimizing Firebase data structure for two large paths

I think I've wrapped my head around denormalization as a primary method of optimization when storing data in Firebase as mentioned in question like this one and in this blog post but I'm getting stuck on one small detail.
Assuming I have two things in my domain, users and posts as in the blog article I mentioned, I might have 20,000 users and 20,000 posts. Because I denormalized everything like a good boy, root/users/posts exists as does root/posts. root/users/posts has a set of post keys with a value of true so that I can get all post keys for a user.
users: {
userid: {
name: 'johnny'
posts: {
-Kofijdjdlehh: true,
-Kd9isjwkjfdj: true
}
}
}
posts: {
-Kofijdjdlehh: {
title: 'My hot post',
content: 'this was my content',
postedOn: '3987298737'
},
-Kd9isjwkjfdj: {
title: 'My hot post',
content: 'this was my content',
postedOn: '3987298737'
}
}
Now, I want to list the title of all posts a user has posted. I don't want to load all 20,000 posts in order to get the title. I can only think of the following options:
Query the root/posts path in some way using the subset of keys that are set to true in the root/users/posts path (if this is possible, I haven't figured out how)
Store the title in the root/users/posts so that each entry in that path has the title duplicated looking like this:
posts: {
-Kofijdjdlehh: true
}
becomes
posts: {
-Kofijdjdlehh: {
title: 'This was my content'
}
}
This seems reasonable, but I haven't seen a single example of doing this, so I'm concerned that it's some anti-pattern.
Another way I haven't been able to find
I appreciate any pointers you might have or documentation I might have missed on this use case.
Either are valid solutions. #1 would be more work for whoever is reading the data, while #2 would be more work when data is saved. Also for #2, you'd have to handle updates to post's titles, though this would be pretty easy with the new multi-path updates.
To implement #1, you'd have you essentially do two queries. Here's a really basic solution which only handles adding posts. It listens for posts being added to the user, and then hooks up a listener to each post's title.
var usersPosts = {};
ref.child('users').child(userId).child('posts').on('child_added', function(idSnap) {
var id = idSnap.key();
ref.child('posts').child(id).child('title').on('value', function(titleSnap) {
usersPosts[id] = titleSnap.val();
});
});
For a third solution, you could use firebase-util, which automagically handles the above scenario and more. This code would essentially do the same as the code above, except it comes with the bonus of giving you one ref to handle.
new Firebase.util.NormalizedCollection(
[ref.child('users').child(userId).child("posts"), "posts"],
[ref.child("posts"), "post"]
).select(
{
key: "posts.$value",
alias: "x"
},
{
key: "post.title",
alias: "title"
}
).ref();
Note that the x value will always be true. It's necessary to select that because firebase-util requires you to select at least one field from each path.

Resources