Db structure:
--followers
-followedUser1
-user1
-followedUser2
-user1
-user2
--users
-user1
-followed
-followedUser1
-followedUser2
-user2(followedUser1)
-followed
-followedUser2
-user3(followedUser2)
Everytime user follows(onCreate) & unfollows(onDelete) under followers/{followedUser}/{followerUser} path, it triggers function which increment or derement count and assigning or detaching posts from follower. It's working via fanout method, and there're no problems. Now, worse part comes when some user deletes account completely together with detaching his followers from himself(because his account will be a ghost), i've set trigger onDelete to indicate whenever it'll happen, then iterating through this user's (i.e.user3) followers removes himself from corresponding followers plus his account, it looks like this then:
--followers
-followedUser1
-user1
-followedUser2
-user1
-user2
--users
-user1
-followed
-followedUser1
-user2(followedUser1)
Now, problematic part - when promise returns i'd like to also remove whole follower/followedUser2(because it's a ghost now) path but... there is a trigger which unfortunately executes for every follower under(onDelete). So, is there any chance to remove path above(levelup) deletetion trigger without triggering children itself? Or any other approach would be great, thanks
edit: Dont get me wrong, it'll work but if number of followers under followedUser would be "a lot" server will die after 100... trigger
At this time, there is no way to filter what deletion events will trigger the function, so you are correct, the function will be triggered once for each user the deleted user was following. I recognize this is one of many use cases where such functionality would be useful, so if you get a chance, please fill out a feature request here.
It looks like your multiple-update problem can be fixed with a multi-location updates
In very quickly hacked together and not tested typescript:
export const cleanupFollowers = functions.auth.user().onDelete(event => {
const user = event.data.userId;
const followersNode = admin.database().ref(`followers/${user}`);
const followers = _.keys(await followersNode.once('value'));
// Every follower also has a reverse node for this user. Get the list of keys:
const reverseNodesToDelete = followers.map(follower => `followers/${follower}/${user}`);
// Model this update as a map of deep key -> null to delete all at once
let cleanup = _.keyBy(reverseNodesToDelete, null);
// add one more update: deleting full node for the deleted user.
cleanup[`followers/${user}`] = null;
// do all deletions as one database request:
return admin.database().ref().update(cleanup);
}
Note that this will still fire your counting function, but that should be fine to run in parallel. It probably makes your app simpler to have each invariant captured separately.
Related
really appreciate the helps
I have been following this video with this code.
My code looks like this
function angerEmotionCapture(agent) {
const angryTo = agent.parameters.angryDirectedTo;
return admin.database().ref('directedTo').transaction((directedTo)=>{
let target = directedTo;
agent.add(`previous entry was ${target}`);
target = angryTo;
agent.add(`new entry is ${target}`);
return directedTo;
});
}
The purpose of this is to capture a conversation topic and store it in the database.
I'm planning to use it for multiple purposes that's why I don't use context.
This code is only the first step to see if I can capture it properly.
When doing this, the agent response always look like this
previous entry was null
new entry is boss
previous entry was friends
new entry is boss
Here "friends" and "boss" are expected. However, the first repetition is not expected and it always gives null. Despite of that, this correctly update the database
I want to understand why is there a repetition here
Thanks, really appreciate the time
I built action on google using dialogflow for experience sampling purpose.
It's idea is: it asks specific users about their mood 3 times per day.
It sends these users then every week a weekly overview about their mood after it has been analysed by researchers.
So I need to save each user info with his mood entries on a database so they can be accessed later by researchers, analysed and sent back to users.
I'm using dialogflow fulfilment with index.js to connect to Firebase database to save the entries.
This agent should be integrated as action on google
On the database I get users names and moods but they are not related to each other so I cannot know which user entered which mood and, also I cannot do the userID check.
I would really appreciate if anybody could help me with the functions since I am totally unfamiliar with node.js or databases but I have to do it that way.
here is my code.
// See https://github.com/dialogflow/dialogflow-fulfillment-nodejs
// for Dialogflow fulfillment library docs, samples, and to report issues
'use strict';
const functions = require('firebase-functions');
const {WebhookClient} = require('dialogflow-fulfillment');
const {Card, Suggestion} = require('dialogflow-fulfillment');
//initialise DB connection
const admin = require('firebase-admin');
admin.initializeApp();
process.env.DEBUG = 'dialogflow:debug'; // enables lib debugging statements
exports.dialogflowFirebaseFulfillment = functions.https.onRequest((request, response) => {
const agent = new WebhookClient({ request, response });
console.log('Dialogflow Request headers: ' + JSON.stringify(request.headers));
console.log('Dialogflow Request body: ' + JSON.stringify(request.body));
function saveName(agent) {
const nameParam = agent.parameters.name;
const context = agent.getContext('awaiting_name_confirm');
const name = nameParam || context.parameters.name;
agent.add('Hi ' + name + ' Are you ready to answer my question?' || 'Hi' + name + 'Have you got a moment for me? ' );
//agent.add('Hi' + name + 'Have you got a minute for me? ');
return admin.database().ref('/names').push({name: name}).then((snapshot)=>
{
console.log('database write sucessful: ' + snapshot.ref.toString());
});
}
function saveMood(agent) {
const moodParam = agent.parameters.mood;
const mood = moodParam;
agent.add('That is good! keep it up. Thanks for sharing with me! Bye ');
//agent.add('Hi' + name + 'Have you got a minute for me? ');
return admin.database().ref('/moods').push({mood: mood}).then((snapshot)=>
{
console.log('database write sucessful: ' + snapshot.ref.toString());
});
}
// Run the proper function handler based on the matched Dialogflow intent name
let intentMap = new Map();
intentMap.set('Get Name', saveName);
intentMap.set('Confirm Name Yes', saveName);
// intentMap.set('Confirm Name Yes', getName);
intentMap.set('attentiveness', saveMood);
agent.handleRequest(intentMap);
});
You have a few issues in your code and approach that you'll need to address:
You need to design your database so that you can associate the user's mood with their account. (And possibly other user information, such as their name or email address, while you're at it.)
You need a unique identity for the person.
A person's name isn't a unique identity. Two people may have the same name, or the system may hear the name differently each time, so you need to have a way to know who you're talking to.
You also need to know their email address and possibly other information, so you can send them their report at the end of the week.
You need to make sure you have their identity between calls to your Action during the same conversation.
Fortunately, you do one thing that is typically missed - you make your calls to the database using Promises. So that part works.
Unique Identity
Your example code asks the user for their name, which it sounds like you intend to use as their identity. Unfortunately, this is a bad idea for a few reasons:
The name isn't an identity. What happens if two people with the same name access your Action?
Names are easily discoverable, so other people could use it and report misleading information. This may not be too serious in your case, but it can still have trustworthiness implications.
Names can be Personally Identifiable Information (PII), so may be covered by additional privacy laws.
Users might want to terminate their account, and can't do this without "changing" their name.
Additionally, you may need other identity information later, such as their email address, and asking for that every time may become troublesome.
You have a few ways to deal with this:
If you're developing for the Google Assistant, you can also use Google Sign In for Assistant which will tell you the user's Google identifier, which you can use as a unique ID. You also get their email address and name as part of their profile.
You can ask for this information (name, email, etc) and save it against a user ID that you generate or a user name the user provides. This ID becomes the identifier. If you're developing for the Google Assistant, you can save this ID in the user's private storage - only you and the user have access to it or can delete it. If not, you may need to use the database to look up the ID. More on this later.
You may wish to use variants on this later point, depending what information you're getting and how you want the user to identify themselves every time. But the important part is that they need to identify themselves with something unique and that you can easily capture.
Use identity in the same session
If you're using Google Sign In, you don't have to worry about this. You'll get the same ID each session and for each call during a session.
If you're using the user's private storage with the Google Assistant, you'll have this as part of the userStore object.
But if you're not, you need to make sure that you get the user's ID in an early intent, and saving this as part of a Context so it is preserved in between calls to your webhook. In subsequent handlers, you can get the ID out of the context and then use it to access other information.
You don't need to store it in the database at this point. All you have is an identifier - this becomes the key that you will use for other information. You just need to remember it for later parts of the conversation.
So in your saveName() function, it might look something like
function saveName(agent) {
const nameParam = agent.parameters.name;
agent.add('Hi ' + nameParam + ' Are you ready to answer my question?');
agent.setContext({
name: 'user',
lifespan: 99,
parameters: {
id: nameParam
}
};
}
As an aside - your handler seems to try to determine if this is the user saying their name, or confirming their name. This is probably better handled as separate intents and separate handlers. Trying to combine them will confuse things.
Structuring and Accessing your Database
We have an ID. We have the user reporting the data. How do we associate the two?
There are a lot of ways to structure the data, and Firebase goes into some detail depending on how you intend to use it, access it, and make it available to the users or others.
In this case, it seems pretty straightforward that you want to store records about the user. Each record can use their ID as a key, and then contain some information about the user, including their mood.
One nice thing about the Firebase database is that you can (mostly) treat it like a Javascript object. If we think about it this way, it might look something like
{
"user": {
"id1":{...},
"id2":{...},
"id3":{
"moods": [
{"mood":"good"},
{"mood":"tired"}
]
},
"id4":{...}
}
}
And so forth. With Firebase, we would reference the moods of user "id3" with a path such as user/id3/moods. If we have the user id in a variable name, we might use the following code to get that reference
var ref = admin.database().ref('user').ref(name).ref('moods');
and then use code such as this to push an object with the mood onto the array (and return the Promise that we need to do):
var obj = {
mood: mood
};
return ref.push( obj ).then( snapshot => {
// Do stuff, including acknowledge to the user you saved it.
});
Keep in mind that you may want to also use this to store more information about each user (such as their name or email) on the user level, or more about the moods (such as a timestamp) in the mood object.
This is more of a theoretical how database should be setup, and less about programming.
Lets say I have a news feed full of cards, which each contain a message and a like count. Each user is able to like a mesesage. I want it to be displayed to a user if they have already liked that particular card. (The same way you can see the post you like on facebook, even if you come back days later)
How would you implement that with this Firestore type database? Speed is definitely a concern..
storying it locally isn't an option, my guess would be on each card object, you would have to reference a collection that just kept a list of people who liked it. The only thing is that is a lot more querying.. which feels like it would be slow..
is there a better way to do this?
TL;DR
This approach requires more to setup, ie a cron service, knowledge of Firestore Security Rules and Cloud Functions for Firebase. With that said, the following is the best approach I've come up with. Please note, only pseudo-rules that are required are shown.
Firestore structure with some rules
/*
allow read
allow update if auth.uid == admin_uid and the
admin is updating total_likes ...
*/
messages/{message_key} : {
total_likes: <int>,
other_field:
[,...]
}
/*allow read
allow write if newData == {updated: true} and
docId exists under /messages
*/
messages_updated/{message_key} : {
updated: true
}
/*
allow read
allow create if auth.uid == liker_uid && !counted && !delete and
liker_uid/message_key match those in the docId...
allow update if auth.uid == admin_uid && the admin is
toggling counted from false -> true ...
allow update if auth.uid == liker_uid && the liker is
toggling delete ...
allow delete if auth.uid == admin_uid && delete == true and
counted == true
*/
likes/{liker_uid + '#' + message_key} : {
liker_uid:,
message_key:,
counted: <bool>,
delete: <bool>,
other_field:
[,...]
}
count_likes/{request_id}: {
message_key:,
request_time: <timestamp>
}
Functions
Function A
Triggered every X minutes to count message likes for potentially all messages.
query /messages_updated for BATCH_SIZE docs
for each, set its docId to true in a local object.
go to step 1 if BATCH_SIZE docs were retrieved (there's more to read in)
for each message_key in local object, add to /count_likes a doc w/ fields request_time and message_key.
Function B
Triggered onCreate of count_likes/{request_id}
Delete created docs message_key from /messages_updated.
let delta_likes = 0
query /likes for docs where message_key == created docs message_key and where counted == false.
for each, try to update counted to true (in parallel, not atomically)
if successful, increment delta_likes by 1.
query /likes for docs where message_key == created docs message_key, where delete == true and where counted == true.
for each doc, try to delete it (in parallel, not atomically)
if successful, decrement delta_likes by 1
if delta_likes != 0, transact the total likes for this message under
/messages by delta_likes.
delete this doc from /count_likes.
Function C (optional)
Triggered every Y minutes to delete /count_likes requests that were never met.
query docs under /count_likes that have request_time older than Z.
for each doc, delete it.
On the client
to see if you liked a message, query under /likes for a doc where liker_uid equals your uid, where message_key equals the message's key and where delete == false. if a doc exists, you have liked it.
to like a message, batch.set a like under /likes and batch.set a /messages_updated. if this batch fails, try a batch_two.update on the like by updating its delete field to false and batch_two.set its /messages_updated.
to unlike a message, batch.update on the like by updating its delete field to true and batch.set its /messages_updated.
Pros of this approach
this can be extended to counters for other things, not just messages.
a user can see if they've liked something.
a user can only like something once.
a user can spam toggle a like button and this still works.
any user can see who's liked what message by querying /likes by message_key.
any user can see all the messages any user has liked by querying /likes by liker_uid.
only a cloud function admin updates your like counts.
if a function is fired multiple times for the same event, this function is safe, meaning like counts will not be incremented multiple times for the same like.
if a function is not fired for some event, this approach still works. It just means that the count will not update until the next time someone else likes the same message.
likes are denormalized to only one root level collection, instead of the two that would be required if you had the like under the the message's likes subcollection and under the liker's messages_liked subcollection.
like counts for each message are updated in batches, ie if something has been liked 100 times, only 1 transaction of 100 is required, not 100 transactions of 1. this reduces write rate conflicts due to like counter transactions significantly.
Cons of this approach
Counts are only updated however often your cron job fires.
Relies on a cron service to fire and in general there's just more to set up.
Requires the function to authenticate with limited privileges to perform secure writes under /likes. In the Realtime Database this is possible. In Firestore, it's possible, but a bit hacky. If you can wait and don't want to use the hacky approach, use the regular unrestricted admin in development until Firestore supports authenticating with limited privileges.
May be costly depending on your standpoint. There are function invocations and read/write counts you should think about.
Things to consider
When you transact the count in Function B, you may want to consider trying this multiple times in case the max write rate of 1/sec is exceeded and the transaction fails.
In Function B, you may want to implement batch reading like in Function A if you expect to be counting a lot of likes per message.
If you need to update anything else periodically for the message (in another cron job), you may want to consider merging that function into Function B so the write rate of 1/sec isn't exceeded.
I want to build a realtime quiz game which randomly matches two players (according to their winning rate if they are logged in). I've read through the book Discover Meteor and have a basic understanding of the framework, but I just have no idea of how to implement the matching part. Anyone know how to do that?
if you want to match users who have scores close to each other, you can do something like this : mongodb - Find document with closest integer value
The Meteor code for those Mongo queries is very similar, but there are some subtle differences that are kind of tricky. In Meteor, it would look something like this :
SP // "selected player" = the User you want to match someone up with
var score = SP.score; // selected player's score
var queryLow = {score: {$lte:score},_id:{$ne:SP._id}};
var queryHigh = {score:{$gte:score},_id:{$ne:SP._id}};
// "L" is the player with the closest lower score
var L=Players.findOne(queryLow,{sort:{score:-1},limit:1});
// "H" is the player with the closest higher score
var H=Players.findOne(queryHigh,{sort:{score:1},limit:1});
so, now you have references to the players with scores right above and right below the 'selected player'. In terms of making it random, perhaps start with a simple algorithm like "match me with the next available player who's score is closest" , then if it's too predictable and boring you can throw some randomness into the algorithm.
you can view the above Meteor code working live here http://meteorpad.com/pad/4umMP4iY8AkB9ct2d/ClosestScore
and you can Fork it and mess about with the queries to see how it works.
good luck! Meteor is great, I really like it.
If you add the package peppelg:random-opponent-matcher to your application, you can match together opponents like this:
On the server, you need to have an instance of RandomOpponentMatcher like this:
new RandomOpponentMatcher('my-matcher', {name: 'fifo'}, function(user1, user2){
// Create the match/game they should play.
})
The function you pass to RandomOpponentMatcher will get called when two users been matched to play against each other. In it, you'll probably want to create the match the users should play against each other (this package does only match opponents together, it does not contain any functionality for playing games/matches).
On the client, you need to create an instance of RandomOpponentMatcher as well, but you only pass the name to it (the same name as you used on the server):
myMatcher = new RandomOpponentMatcher('my-matcher')
Then when the users is logged in and which to be matched with a random opponent, all you need to do is to call the add method. For example:
<template name="myTemplate">
<button class="clickMatchesWithOpponent">Match me with someone!</button>
</template>
Template.myTemplate.events({
'click .clickMatchesWithOpponent': function(event, template){
myMatcher.add()
}
})
When two different logged in users has clicked on the button, the function you passed to RandomOpponentMatcher on the server will get called.
One implementation might be as follows:
A user somehow triggers a 'looking for game' event that sets an attribute on user.profile.lookingForGame to true. The event then makes a call to a server side Meteor method which queries for all other online users looking for games.
From there you it really depends on how you want to handle users once they 'match'.
To determine all online users, try using the User Status package:
https://github.com/mizzao/meteor-user-status
Once added, any online user will have an attribute in the profile object of 'online'. You can use this to query for all online users.
Tryng to get a simple result using "Where" style in firebase but get null althe time, anyone can help with that?
http://jsfiddle.net/vQEmt/68/
new Firebase("https://examples-sql-queries.firebaseio.com/messages")
.startAt('Inigo Montoya')
.endAt('Inigo Montoya')
.once('value', show);
function show(snap) {
$('pre').text(JSON.stringify(snap.val(), null, 2));
}
Looking at the applicable records, I see that the .priority is set to the timestamp, not the username.
Thus, you can't startAt/endAt the user's name as you've attempted here. Those are only applicable to the .priority field. These capabilities will be expanding significantly over the next year, as enhancements to the Firebase API continue to roll out.
For now, your best option for arbitrary search of fields is use a search engine. It's wicked-easy to spin one up and have the full power of a search engine at your fingertips, rather than mucking with glacial SQL-esque queries. It looks like you've already stumbled on the appropriate blog posts for that topic.
You can, of course, use an index which lists users by name and stores the keys of all their post ids. And, considering this is a very small data set--less than 100k--could even just grab the whole thing and search it on the client (larger data sets could use endAt/startAt/limit to grab a recent subset of messages):
new Firebase("https://examples-sql-queries.firebaseio.com/messages").once('value', function(snapshot) {
var messages = [];
snapshot.forEach(function(ss) {
if( ss.val().name === "Inigo Montoya" ) {
messages.push(ss.val());
}
});
console.log(messages);
});
Also see: Database-style queries with Firebase