Heeey all hopefully you can help me out with the following.
I have build a small hangman game, well at least I've build the backend. It's event sourced and I use eventstore as an event store. I've got some events in the store and I created a projection which unfortunately doesn't return any results.
What I want to achieve with this projection is very simple. It should show the word to guess per game that is going on right now. I'll listen to one event ("HangmanBackend.Domain.GameStarted").
fromCategory('hangman')
.foreachStream()
.when({
$init: function(){
return {
game_status: "in progress",
word_to_guess: ""
};
},
"HangmanBackend.Domain.GameStarted": function(state, event) {
state.word_to_guess = event.word;
return state;
}
});
All streams (1 stream per game) are prefixed with hangman, that why I can use "fromCategory" in the projection.
Is there any clear reason why I don't see any results?
Already many thanks in advance!
Related
I'm trying to get a dataset of messages out of my firebase database and want the messages sorted by added/timestamp. But for some reason no orderby I put in the code is actually used. I tried doing these 2 things.
_messagesRef = FirebaseDatabase.instance.reference().child('messages/'+key);
_membersSubscription = _messagesRef
.orderByChild('timestamp')
.onValue//On valuechange
.listen((Event event) => _messagesSubscriptionCallback(event));
_messagesRef = FirebaseDatabase.instance.reference().child('messages/'+key);
_membersSubscription = _messagesRef
.orderByKey()
.onValue//On valuechange
.listen((Event event) => _messagesSubscriptionCallback(event));
Both give me back the same dataset that is not ordered by timestamp or key in the callback. I've added the output underneath
{
-LA-Aw6crEAV53LxdujP:{
sender:1508,
message:test s9 2,
timestamp:1523642778089
},
-LA-Arby61T1UG5URMn6:{
sender:1508,
message:test s9,
timestamp:1523642759679
},
-LA-AyC7F8KAqceZBE3j:{
sender:1509,
message:test emu 1,
timestamp:1523642786632
},
-LA22WiUfL2tbh7-OjtM:{
sender:1508,
message:Blaj,
timestamp:1523690904480
},
-LA-B29RRXbdOPT1mG7m:{
sender:1508,
message:tesy3,
timestamp:1523642806940
}
}
This is how the data should be.
I hope someone can help me with this issue. I think I might misunderstand how ordering data works with Firebase
Kind regards,
Namanix
The result you show is a JSON object with other objects in there. JSON objects are never ordered as far as I know, only retrievable by key. JSON Arrays would be, but it doesn't look like you get that. When you would change this to an array the document IDs would have to be inside the document instead of being the object key. My guess would be that 'orderBy' is meant to be used for example to limit the number of items you get for pagination. Than you can order by timestamp, limit the number of items to 20 and search from the last timestamp you got.
I think if you want to order them I would put them in a new list of objects which can be ordered.
Most likely (it's hard to be sure without seeing _messagesSubscriptionCallback) you're throwing the ordering information away when you convert the data from Firebase into a plain JSON object, which (as Rene also says) doesn't have any defined order.
But the data your request from Firebase does have ordering information, you just have to be sure to not drop it.
The usual way to do this is to listen for onChildAdded instead of onValue. With that Firebase will invoke onChildAdded for each child in turn, and it will do so in the order you requested.
Also see this example and for example what FirebaseAnimatedList does here.
I now temporarily fixed it by doing this in my callback. But this feels like a very bad way to fix it. I hope to get some thoughts on this.
static void _messagesSubscriptionCallback(Event event) {
_messagesList.clear();
_messages.clear();
_messages = event.snapshot.value;
_messagesList = _messages.keys.toList();
_messagesList.sort((a, b) {
return b.compareTo(a) ;
});
onMessagesChange();
}
I am building a recommender system where I use Firebase to store and retrieve data about movies and user preferences.
Each movie can have several attributes, and the data looks as follows:
{
"titanic":
{"1997": 1, "english": 1, "dicaprio": 1, "romance": 1, "drama": 1 },
"inception":
{ "2010": 1, "english": 1, "dicaprio": 1, "adventure": 1, "scifi": 1}
...
}
To make the recommendations, my algorithm requires as input all the data (movies) and is matched against an user profile.
However, in production mode I need to retrieve over >10,000 movies. While the algorithm can handle this relatively fast, it takes a lot of time to load this data from Firebase.
I retrieve the data as follows:
firebase.database().ref(moviesRef).on('value', function(snapshot) {
// snapshot.val();
}, function(error){
console.log(error)
});
I am there wondering if you have any thoughts on how to speed things up? Are there any plugins or techniques known to solve this?
I am aware that denormalization could help split the data up, but the problem is really that I need ALL movies and ALL the corresponding attributes.
My suggestion would be to use Cloud Functions to handle this.
Solution 1 (Ideally)
If you can calculate suggestions every hour / day / week
You can use a Cloud Functions Cron to fire up daily / weekly and calculate recommendations per users every week / day. This way you can achieve a result more or less similar to what Spotify does with their weekly playlists / recommendations.
The main advantage of this is that your users wouldn't have to wait for all 10,000 movies to be downloaded, as this would happen in a cloud function, every Sunday night, compile a list of 25 recommendations, and save into your user's data node, which you can download when the user accesses their profile.
Your cloud functions code would look like this :
var movies, allUsers;
exports.weekly_job = functions.pubsub.topic('weekly-tick').onPublish((event) => {
getMoviesAndUsers();
});
function getMoviesAndUsers () {
firebase.database().ref(moviesRef).on('value', function(snapshot) {
movies = snapshot.val();
firebase.database().ref(allUsersRef).on('value', function(snapshot) {
allUsers = snapshot.val();
createRecommendations();
});
});
}
function createRecommendations () {
// do something magical with movies and allUsers here.
// then write the recommendations to each user's profiles kind of like
userRef.update({"userRecommendations" : {"reco1" : "Her", "reco2", "Black Mirror"}});
// etc.
}
Forgive the pseudo-code. I hope this gives an idea though.
Then on your frontend you would have to get only the userRecommendations for each user. This way you can shift the bandwidth & computing from the users device to a cloud function. And in terms of efficiency, without knowing how you calculate recommendations, I can't make any suggestions.
Solution 2
If you can't calculate suggestions every hour / day / week, and you have to do it each time user accesses their recommendations panel
Then you can trigger a cloud function every time the user visits their recommendations page. A quick cheat solution I use for this is to write a value into the user's profile like : {getRecommendations:true}, once on pageload, and then in cloud functions listen for changes in getRecommendations. As long as you have a structure like this :
userID > getRecommendations : true
And if you have proper security rules so that each user can only write to their path, this method would get you the correct userID making the request as well. So you will know which user to calculate recommendations for. A cloud function could most likely pull 10,000 records faster and save the user bandwidth, and finally would write only the recommendations to the users profile. (similar to Solution 1 above) Your setup would like this :
[Frontend Code]
//on pageload
userProfileRef.update({"getRecommendations" : true});
userRecommendationsRef.on('value', function(snapshot) { gotUserRecos(snapshot.val()); });
[Cloud Functions (Backend Code)]
exports.userRequestedRecommendations = functions.database.ref('/users/{uid}/getRecommendations').onWrite(event => {
const uid = event.params.uid;
firebase.database().ref(moviesRef).on('value', function(snapshot) {
movies = snapshot.val();
firebase.database().ref(userRefFromUID).on('value', function(snapshot) {
usersMovieTasteInformation = snapshot.val();
// do something magical with movies and user's preferences here.
// then
return userRecommendationsRef.update({"getRecommendations" : {"reco1" : "Her", "reco2", "Black Mirror"}});
});
});
});
Since your frontend will be listening for changes at userRecommendationsRef, as soon as your cloud function is done, your user will see the results. This might take a few seconds, so consider using a loading indicator.
P.S 1: I ended up using more pseudo-code than originally intended, and removed error handling etc. hoping that this generally gets the point across. If there's anything unclear, comment and I'll be happy to clarify.
P.S. 2: I'm using a very similar flow for a mini-internal-service I built for one of my clients, and it's been happily operating for longer than a month now.
Firebase NoSQL JSON structure best practice is to "Avoid nesting data", but you said, you don't want to change your data. So, for your condition, you can have REST call to any particular node (node of your each movie) of the firebase.
Solution 1) You can create some fixed number of Threads via ThreadPoolExecutors. From each worker thread, you can do HTTP (REST call request) as below. Based on your device performance and memory power, you can decide how many worker threads you want to manipulate via ThreadPoolExecutors. You can have code snippet something like below:
/* creates threads on demand */
ThreadFactory threadFactory = Executors.defaultThreadFactory();
/* Creates a thread pool that creates new threads as needed, but will reuse previously constructed threads when they are available */
ExecutorService threadPoolExecutor = Executors.newFixedThreadPool(10); /* you have 10 different worker threads */
for(int i = 0; i<100; i++) { /* you can load first 100 movies */
/* you can use your 10 different threads to read first 10 movies */
threadPoolExecutor.execute(() -> {
/* OkHttp Reqeust */
/* urlStr can be something like "https://earthquakesenotifications.firebaseio.com/movies?print=pretty" */
Request request = new Request.Builder().url(urlStr+"/i").build();
/* Note: Firebase, by default, store index for every array.
Since you are storing all your movies in movies JSON array,
it would be easier, you read first (0) from the first worker thread,
second (1) from the second worker thread and so on. */
try {
Response response = new OkHttpClient().newCall(request).execute();
/* OkHttpClient is HTTP client to request */
String str = response.body().string();
} catch (IOException e) {
e.printStackTrace();
}
return myStr;
});
}
threadPoolExecutor.shutdown();
Solution 2) Solution 1 is not based on the Listener-Observer pattern. Actually, Firebase has PUSH technology. Means, whenever something particular node changes in Firebase NoSQL JSON, the corresponding client, who has connection listener for particular node of the JSON, will get new data via onDataChange(DataSnapshot dataSnapshot) { }. For this you can create an array of DatabaseReferences like below:
Iterable<DataSnapshot> databaseReferenceList = FirebaseDatabase.getInstance().getReference().getRoot().child("movies").getChildren();
for(DataSnapshot o : databaseReferenceList) {
#Override
public void onDataChange(DataSnapshot o) {
/* show your ith movie in ListView. But even you use RecyclerView, showing each Movie in your RecyclerView's item is still show. */
/* so you can store movie in Movies ArrayList. When everything completes, then you can update RecyclerView */
}
#Override
public void onCancelled(DatabaseError databaseError) {
}
}
Although you stated your algorithm needs all the movies and all attributes, it does not mean that it processes them all at once. Any computation unit has its limits, and within your algorithm, you probably chunk the data into smaller parts that your computation unit can handle.
Having said that, if you want to speed things up, you can modify your algorithm to parallelize fetching and processing of the data/movies:
| fetch | -> |process | -> | fetch | ...
|chunk(1)| |chunk(1)| |chunk(3)|
(in parallel) | fetch | -> |process | ...
|chunk(2)| |chunk(2)|
With this approach, you can spare almost the whole processing time (but the last chunk) if processing is really faster than fetching (but you have not said how "relatively fast" your algorithm run, compared to fetching all the movies)
This "high level" approach of your problem is probably your better chance if fetching the movies is really slow although it requires more work than simply activating a hypothetic "speed up" button of a Library. Though it is a sound approach when dealing with large chunk of data.
EDIT: as the original question was too vague I have updated it to make it more concrete
I'd like to create a dashboard in Meteor that shows some statistics about my collections (e.g. how many docs, how many users...). I have been trying the past days but can't seem to find a good/intelligent way.
I initially just did the following:
Template.dashboard.helpers({
getProductsCount: function() {
return Products.find().count();
}
});
This did not work. I think because it counts the number of products from minimongo, but not sure.
Then I did the following:
In the template helper, call a method and get the value to show on the dashboard page (does not work)
Was told not to use pub/sub mechanism for this type of metric
Worked via Session variables (did work, but feels a bit strange to store this kind of metric data in Session variables
So then I read in another SO response about Reactive Variables and tried the following:
Template.dashboard.helpers({
getProductsCount: function() {
return Template.instance().myAsyncValue.get();
}
});
Template.dashboard.created = function() {
var self = this;
self.myAsyncValue = new ReactiveVar("Waiting for response from server");
Meteor.call('getProductsCount', function(error, asyncValue){
if (error)
console.log(error);
else
self.myAsyncValue.set(asyncValue);
});
};
This works, but I find this extremely difficult for something as simple as showing a product count (or any other metric). Not sure I understand the reason why I should use sth as reactive variables?
Then -out of curiosity- I tried the following (using meteor add simple:reactive-method) and it works:
Template.customerDashboard.helpers({
getProductsCount: function () {
return ReactiveMethod.call("getProductsCount");
}
});
So the question really is why having to use Reactive variables and methods for sth as simple as this. Can someone explain?
If you want to show the count only in the view, the best way is to return the count number only. you do not need publish/subscribe at all. you can use server methods. and if you want to show data also, you can go for pub-sub. and your approach is correct.
I'm learning Meteor and I was trying to pass the result of a Collection.find() into and array (using a variable) and the simpler code I have is (in a file that is in the root):
CalEvents = new Mongo.Collection('calevents'); //creating a collection
/*------------------------- Populating the database with dummy data-------*/
if (Meteor.isServer) {
Meteor.startup(function () {
if (CalEvents.find().count() === 0) {
CalEvents.insert({
title: "Initial room",
start: '2010-02-02'
});
}
});
}
/*--------------- Creating an array from the collection-----------------*/
events = [];
calEvents = CalEvents.find({});
calEvents.forEach(function(evt){
events.push({
title: evt.title,
start: evt.start,
})
});
The page has nothing to show but using the console I can see (CalEvents.find().fetch()) that I have data in my database but the "events" variable is empty...
I can't understand why because I tried several other things such as changing file names and moving code to guarantee the proper order.
And I already tried to use CalEvents.find().fetch() to create an array an put the result into a variable but I'm not able to do it...
Does anyone know what's so simple that I'm missing?...
Do you use autosubscribe?
You probably need to make sure the sbscription is ready. See Meteor: How can I tell when the database is ready? and Displaying loader while meteor collection loads.
The reason you do see CalEvents.find().fetch() returning items in the console is that by the time you make that call, the subscription is ready. But in your events = []; ... code (which I assume is in a file under the client directory, you might have assumed that the subscription data has arrived when in fact it has not.
A useful debugging tool is Chrome's device mode ("phone" icon near the search icon in DevTools), which lets you simulate slow networks (e.g. GPRS, with 500ms delay for every request).
Say I have two view models that each have an observable property that represents different, but similar data.
function site1Model(username) {
this.username = ko.observable(username);
....
}
function site2Model(username) = {
this.username = ko.observable(username);
....
}
These view models are independent and not necessarily linked to each other, but in some cases, a third view model creates a link between them.
function site3Model(username) = {
this.site1 = new site1Model(username);
this.site2 = new site2Model(username);
// we now need to ensure that the usernames are kept the same between site1/2
...
}
Here are some options that I've come up with.
Use a computed observable that reads one and writes to both:
site3Model.username = ko.computed({
read: function() {
return this.site1.username(); // assume they are always the same
},
write: function(value) {
this.site1.username(value);
this.site2.username(value);
},
owner: site3Model
}
This will keep the values in sync as long as changes always come through the computed. But if an underlying observable is changed directly, it won't do so.
Use the subscribe method to update each from the other:
site3Model.site1.username.subscribe(function(value) {
this.site2.username(value);
}, site3Model);
site3Model.site2.username.subscribe(function(value) {
this.site1.username(value);
}, site3Model);
This works as long as the observables suppress notifications when the values are the same; otherwise you'd end up with an infinite loop. You could also do the check earlier: if (this.site1.username() !== value) this.site1.username(value); This also has a problem that the observables have to be simple (it won't work right if site1 and site2 themselves are observables).
Use computed to do the subscribe and updates:
site3Model.username1Updater = ko.computed(function() {
this.site1.username(this.site2.username());
}, site3Model);
site3Model.username2Updater = ko.computed(function() {
this.site2.username(this.site1.username());
}, site3Model);
This format allows us to have other dependencies. For example, we could make site1 and site2 observables and then use this.site1().username(this.site2().username()); This method also requires a check for equality to avoid an infinite loop. If we can't depend on the observable to do it, we could check within the computed, but would add another dependency on the observable we're updating (until something like observable.peek is available).
This method also has the downside of running the update code once initially to set up the dependencies (since that's how computed works).
Since I feel that all of these methods have a downside, is there another way to do this that would be simple (less than 10 lines of code), efficient (not run unnecessary code or updates), and flexible (handle multiple levels of observables)?
It is not exactly 10 lines of code (although you could strip it down to your liking), but I use pub/sub messages between view models for this situation.
Here is a small library that I wrote for it: https://github.com/rniemeyer/knockout-postbox
The basic idea is just to create a ko.subscribable and use topic-based subscriptions. The library extends subscribables to add subscribeTo, publishOn and syncWith (both publish and subscribe on a topic). These methods will set up the proper subscriptions for an observable to automatically participate in this messaging and stay synchronized with the topic.
Now your view models do not need to have direct references to each other and can communicate through the pubsub system. You can refactor your view models without breaking anything.
Like I said you could strip it down to less than 10 lines of code. The library just adds some extras like being able to unsubscribe, being able to have control over when publishing actually happens (equalityComparer), and you can specify a transform to run on incoming values.
Feel free to post any feedback.
Here is a basic sample: http://jsfiddle.net/rniemeyer/mg3hj/
Ryan and John, Thank you both for your answers. Unfortunately, I really don't want to introduce a global naming system that the pub/sub systems require.
Ryan, I agree that the subscribe method is probably the best. I've put together a set of functions to handle the subscription. I'm not using an extension because I also want to handle the case where the observables themselves might be dynamic. These functions accept either observables or functions that return observables. If the source observable is dynamic, I wrap the accessor function call in a computed observable to have a fixed observable to subscribe to.
function subscribeObservables(source, target, dontSetInitially) {
var sourceObservable = ko.isObservable(source)
? source
: ko.computed(function(){ return source()(); }),
isTargetObservable = ko.isObservable(target),
callback = function(value) {
var targetObservable = isTargetObservable ? target : target();
if (targetObservable() !== value)
targetObservable(value);
};
if (!dontSetInitially)
callback(sourceObservable());
return sourceObservable.subscribe(callback);
}
function syncObservables(primary, secondary) {
subscribeObservables(primary, secondary);
subscribeObservables(secondary, primary, true);
}
This is about 20 lines, so maybe my target of less than 10 lines was a bit unreasonable. :-)
I modified Ryan's postbox example to demonstrate the above functions: http://jsfiddle.net/mbest/vcLFt/
Another option is to create an isolated datacontext that maintains the models of observables. the viewmodels all look to the datacontext for their data and refer to the same objects, so when one updates, they all do. The VM's dependency is on the datacontext, but not on other VMs. I've been doing this lately and it has worked well. Although, it is much more complex than using pub/sub.
If you want simple pub/sub, you can use Ryan Niemyer's library that he mentioned or use amplify.js which has pub/sub messaging (basically a messenger or event aggregator) built in. Both are lightweight and decoupled.
In case anyone needed.
Another option is to create a reference object/observable.
This also handle object that contains multiple observable.
(function(){
var subscriptions = [];
ko.helper = {
syncObject: function (topic, obj) {
if(subscriptions[topic]){
return subscriptions[topic];
} else {
return subscriptions[topic] = obj;
}
}
};
})();
In your view models.
function site1Model(username) {
this.username = syncObject('username', ko.observable());
this.username(username);
....
}
function site2Model(username) = {
this.username = syncObject('username', ko.observable());
this.username(username);
....
}