I have a fairly simple Meteor application.
I tried to send newsletter to about 3000 users in my list and things went wrong. A random set of users got multiple emails (between 41 to 1).
I shut the server down as soon I noticed this behavior. around 1300 emails were sent to 210 users. I am trying to figure out what happened and why.
Here is the code flow:
SendNow (client clode) --> SendNow (server method) --> populateQue (server function) --> processQue(server function) --> sendEmails (server method)
Client side code :
'click .sendNow': function(){
/* code that forms data object */
Meteor.call('sendNow',data);
}
Server code : server/method.js
Meteor.methods({
'sendNow' : function(data){
if(userWithPermission()){
var done = populateQue(data);
if(done)
processQue();
return {'method':'sendNow','status':'ok'}
},
'sendEmails': function(data){
this.unblock();
var result = Mandrill.messages('send', data);// using external library
SentEmails.insert(data);//Save sent emails in a collection
}
});
Function on server : server/utils.js
populateQue = function(data) {
/* code to get all users in to array */
MessageQue.remove();//Remove all documents from the Que
for (var i=0; i<users.length; i++) {
MessageQue.insert({userId: users[i]._id});
}
return true;
}
processQue = function(){
var messageQue = MessageQue.find({}).fetch();
for(i=0; i < messageQue.length; i++){
Meteor.call('sendEmails', data);
MessageQue.remove({_id: messageQue[i]._id});//Remove sent emails from the Que
}
}
My first hunch was MessageQue got messed up as I am removing items while processQue is using it but i was wrong. I am unable to simulate this behavior again after few tests
Test 1 : replaced Mandrill.message('send',data) with Meteor._sleepForMs(1000); - Only one email/person was seen in SentEmails collection.
Test 2 : Put Mandrill in Test mode (had to use different API key) and re ran the code with couple of log statements. - Only one email/person was seen in SentEMails and also in Mandrill's interface.
It's definitely not external library. its somewhere in my code or in the way I understood meteor to work.
Only one thing I noticed is an error occurred while accessing SentEmails collection through another view code. I have a view that displays SentEmails on the client with date as filter.
Here is the error :
Exception from sub sentEmailDocs id 9LTq6mMD4xNcre4YX Error:
Exception while polling query
{
"collectionName":"sent_emails",
"selector":{"date":{"$gt":"2015-07-09T05:00:00.000Z","$lt":"2015-07-11T05:00:00.000Z"}},
"options":{"transform":null,"sort":{"date":-1}}
}:
Runner error: Overflow sort stage buffered data usage of 33565660 bytes exceeds internal limit of 33554432 bytes
Is this the smoking gun? Would this have caused the random behavior?
I have put couple checks to prevent this from happening but I am puzzled on what might have caused and why? I will be happy to provide more information. Thanks in advance to who ever is willing to spend few mins on this.
Shot in the dark here, but the remove method takes an object, otherwise it doesn't do anything. MessageQue.remove() probably didn't clear the queue. You need MessageQue.remove({}). Test the theory by doing an if (MessageQue.find().count() > 0)... after the remove.
If you're set on having a separate collection for the queue, and I'm not saying that's a bad thing, I'd set the _id to be the userId. That way you can't possibly send someone the same message twice.
Related
I have been struggling to find a solution for this and it seems that i'm doing something in the wrong way due to my limited knowladge, so here is the breakdown of the problem:
public void RegisterNewUser()
{
FetchRegisterInputValues();
if (CheckRegisterDataIntegrity())
{
_auth.CreateUserWithEmailAndPasswordAsync(_email, _password).ContinueWith(task => {
if (task.IsCanceled) {
Debug.LogError("CreateUserWithEmailAndPasswordAsync was canceled.");
return;
}
if (task.IsFaulted)
{
HandleRegistrationErrors(task.Exception);
return;
}
// Firebase user has been created.
Firebase.Auth.FirebaseUser newUser = task.Result;
Debug.LogFormat("Firebase user created successfully: {0} ({1})",
newUser.DisplayName, newUser.UserId);
});
}
else if (!CheckRegisterDataIntegrity())
{
HandleRegistrationErrors(new AggregateException("passwords do not match"));
}
}
above is the Registration function that I got straight from Firebase docs, it's very straightforward
the FetchRegisterInputValues(); function gets the email and passwords, the CheckRegisterDataIntegrity() compares the password with the password conformation in the form, and finally HandleRegistrationErrors(task.Exception); is meant to fire a popup panel to show the error,
this is how HandleRegistrationErrors(task.Exception); looks
private void HandleRegistrationErrors(AggregateException errMsg)
{
print("its here from the errors method " + errMsg.Message);
registerErrorPopup.OpenNotification();
registerErrorPopup.description = errMsg.Message;
}
it's using a UI asset from the asset store, the .OpenNotification(); starts the animation and pops it up, and then im just showing the message.
Now, I got two problems, the first is when there is an error encountered by Firebase and the if (task.IsFaulted) Condition is true, the HandleRegistrationErrors function should be called, right?. well that's exactly what happens, except only the print("it's here from the errors method " + errMsg.Message); line gets called and the rest of the function does not execute, I thought at first that its a problem with asset, but I tried doing it manually (created a native UI with unity and used SetActive() method to start the popUp), but again only print method executed, I think its because of the
CreateUserWithEmailAndPasswordAsync is Asynchronous and I should handle errors accordingly, but I really don't know how to go about it and there is no documentation that I could find.
The second problem is how to get the correct Error Message because of the task.Exception.Message always returns me a "One or more errors occurred". while the task.Exception itself gives the right message but it's not formatted correctly.
The first question is the easiest. To update your code with the minimal amount of effort, just replace ContinueWith with ContinueWithOnMainThread will force logic onto the main thread. Also, you should avoid calling task.Result if task.Exception is non-null as it will just raise the exception (see the related documentation).
For the threading related stuff: I go into much more detail about threading with Firebase and Unity here and you can read about the ContinueWithOnMainThread extension here.
For your second issue, the issue you're running into is that task.Exception is an AggregateException. I typically just attach a debugger and inspect this when debugging (or let Crashlytics analyze it in the field), and my UI state is only concerned about success or failure. If you want to inspect the error, the documentation I linked for AggregateException recommends doing something like:
task.Exception.Handle((e) => Debug.LogError($"Failed because {e}"));
Although I would play with .Flatten() or .GetBaseException() to see if those are easier to deal with.
I hope this helps!
--Patrick
I have a method that checks for all unread messages belonging to a user. When the app loads, this number appears next to the "Messages" drop down. In Meteor, how would I update this count or variable for when a new message comes in or when the user reads an unread message? Pretty much I needs the method to send down the new count anytime a message status changes without refreshing the app itself.
I'm familiar with the Tracker.autorun functionality but I don't think it'll help with this situation. What's the best practice for approaching this?
Use Publish/Subscribe. It is always reactive. If you do not want to have all unread messages sent to the client straight away and counted there, you create a custom collection that justs count the number of unread messages and publishes that count. Look at the example a bit down in the linked page that starts with
// server: publish the current size of a collection
This is exactly your use case.
I have exactly this setup for new messages. In my header I have:
<li>Messages <span class="counter">{{Messages.count}}</span></li>
And then I have a helper that returns the cursor:
Template.header.helpers({
Messages: function(){ return Messages.find(); }
});
In the old days, before David Weldon set me straight I used to have a helper to return the count, now I just refer to the count directly in the blaze html template.
Now, in this approach I'm subscribing to the Messages collection so that new messages are transmitted to the client and can then be counted locally. This is on the assumption that they are going to be read soon. If you want to avoid this step then you should probably publish a Stats collection or include a stats key in the user object so that just the count itself can be synced via pub-sub.
You can just have a field like read, and update like:
Method for marking one message as read:
markRead: function(messageId){
Messages.update(messageId, {
$set: {
read: true //this needs to be set to false when its inserted
}
})
}
Bulk update method (assuming all messages have receiverId saved):
markAllRead: function(){
Messages.update({receiver: Meteor.userId(), read:false}, {
$set: {
read: true
}
}, {multi: true})
}
You can count read:false ones to retrieve count and you don't have to write anything else
Helper:
count: function(){
//even if your publish/subscribe is correct, the count we want is from messages that are not read and the receiver is current user.
return Messages.find({receiver: Meteor.userId(), read: false }).count();
}
Event:
'click .elementClass': function(){
//both users see the messages and they can both click. We want to update the right message for the right user. Otherwise, the other user can mark the message as read when the receiver is the other user which they shouldn't be able to do. You can do a simple check on the client side, and another check in the method if necessary.
if(this.receiver === Meteor.userId()){
Meteor.call('markAsRead', this._id)
}
}
Let me know if it solves your problem/answers all your questions.
I'm learning Meteor and I was trying to pass the result of a Collection.find() into and array (using a variable) and the simpler code I have is (in a file that is in the root):
CalEvents = new Mongo.Collection('calevents'); //creating a collection
/*------------------------- Populating the database with dummy data-------*/
if (Meteor.isServer) {
Meteor.startup(function () {
if (CalEvents.find().count() === 0) {
CalEvents.insert({
title: "Initial room",
start: '2010-02-02'
});
}
});
}
/*--------------- Creating an array from the collection-----------------*/
events = [];
calEvents = CalEvents.find({});
calEvents.forEach(function(evt){
events.push({
title: evt.title,
start: evt.start,
})
});
The page has nothing to show but using the console I can see (CalEvents.find().fetch()) that I have data in my database but the "events" variable is empty...
I can't understand why because I tried several other things such as changing file names and moving code to guarantee the proper order.
And I already tried to use CalEvents.find().fetch() to create an array an put the result into a variable but I'm not able to do it...
Does anyone know what's so simple that I'm missing?...
Do you use autosubscribe?
You probably need to make sure the sbscription is ready. See Meteor: How can I tell when the database is ready? and Displaying loader while meteor collection loads.
The reason you do see CalEvents.find().fetch() returning items in the console is that by the time you make that call, the subscription is ready. But in your events = []; ... code (which I assume is in a file under the client directory, you might have assumed that the subscription data has arrived when in fact it has not.
A useful debugging tool is Chrome's device mode ("phone" icon near the search icon in DevTools), which lets you simulate slow networks (e.g. GPRS, with 500ms delay for every request).
I'm having issues with writing to a socket (with http) from a sqlite (sqlite3) libary in Node.JS
https://gist.github.com/RyanCopley/6004c3ce372e060bbf18
Lines 68 to 75, I have 4 attempts to write. Outside of the db.each, everything works in any context. INSIDE of it, it crashes miserably. I'm not entirely sure why, but I feel like there is a conflict between the two libraries
BTW I already know concatenating the SQL statement is bad :3
This is because the callback function in db.each is called asynchronously. This means that line 79: res.end() will be called before res.write("Found row!");, triggering an error.
I think what you want to do is something like this:
db.serialize(function() {
that.res.write("["); // works
db.each("SELECT * FROM messages WHERE channel = '"+q.chan+"' AND id > "+q.since+" ORDER BY id", function(err, row) {
res.write("Found row!"); //does not work
that.res.write("Found row!"); //does not work
console.log("Found row!");
});
res.write("]");//works
});
I'm writing an application based on ember-data, it loads up all of its data asynchronously. However, the didLoad function does not get called until find is used. For example:
App = Ember.Application.create();
App.Store = DS.Store.create({revision: 3});
App.Thing = DS.Model.extend({
didLoad: function(){
alert("I loaded " + this.get('id'));
}
});
App.Store.load(App.Thing,{id: "foo"});
...will not trigger the alert, and findAll will not return the model. However, when I run:
App.Store.find(App.Thing,"foo");
The didLoad function will trigger, and it can be found with App.Store.findAll(App.Thing).
What's going on?
The ember-data source code explains it well:
// A record enters this state when the store askes
// the adapter for its data. It remains in this state
// until the adapter provides the requested data.
//
// Usually, this process is asynchronous, using an
// XHR to retrieve the data.
loading: DS.State.create({
// TRANSITIONS
exit: function(manager) {
var record = get(manager, 'record');
record.fire('didLoad');
},
// EVENTS
didChangeData: function(manager, data) {
didChangeData(manager);
manager.send('loadedData');
},
loadedData: function(manager) {
manager.goToState('loaded');
}
}),
this means that 'didLoad' will only be triggered when the record was loaded via the adapter.
The 'find' method asks the adapter for the data - this looks it up in the pool of currently available data hashes and in your case finds it, because you already provided it. In other cases however the data maybe does not exist locally in the browser but remain on the server, which would trigger an ajax request in the adapter to fetch it.
So 'didLoad' currently only works in combination with an adapter (e.g: find)
But I totally agree with you that this should be changed since triggering 'didLoad' on models that are loaded vai Store.load seems pretty obvious ;-)