casperjs and a/b testing - automated-tests

i have a signup.js test that automates signing up for my web app (obviously). we're currently a/b testing a new flow that takes you to a different page ('.com/signupa' vs '.com/signupb') and i'm wondering what the best way to reflect this in my test.
options:
use evaluateOrDie and make it die at .com/signupb (this seems dumb)
flesh out test for .com/signupb and make it go that route if it hits that test (is this possible?) something like..
casper.waitForResource("classic.png",
function success() {
this.echo('on the old signup flow ');
<continue with regular signup test>
},
function fail() {
this.test.assertExists("classic.png");
<do something else>
});
any other ideas greatly appreciated!

My preference would be to hide some information in each of your pages, so you can cleanly switch on them. E.g.
<span id="version1"></span>
vs.
<span id="version2"></span>
Then, after submitting the form:
casper.then(function(){
if (this.exists('#version2')) {
testNewSite(this);
} else {
testOldSite(this);
}
});
But detecting on something you already know is only in one of the pages, like the "classic.png" you show in your question, is also fine. (It just feels a little more brittle: the web development team can break your tests by renaming that image, or putting an image with that name in the new version, etc., etc.)

Related

How to analyze weird Goal Flow results? Is it possible to view specific users' journeys?

To help analyzing a site's user flow, I wrote a test bed in JavaScript, creating a new fake tracker, sending a few fake pageviews in like 1-2 second intervals according to a pretty extensive, randomized graph of expected views. I then set up goals in GA, containing funnels through certain pages. Then I left it running for a few hours, accumulating a few hundreds fake users.
Raw goal conversion percentages look good, page content flows too, BUT... when I'm looking at Goal Flow, some of the connections in the funnels don't make sense. For example 80% of my fake users seem to go from step "about" directly to step "success", skipping "product" and "payment", while - according to my test setup - that can't ever happen. (Step names used are examples to match the code below.)
I'd like to see exactly what paths did these particular users take - did some pages just not register, or did they register out of order, or what? Is there any way I can view RAW user journeys in GA, page by page?
For those interested, the testing code is basically like this:
ga("create","UA-0000000-2", "auto", "testtracker", {
'cookieName':"_ga_test_"+Date.now(),
'cookieExpires':120,
'clientId': 'cid-'+Date.now(),
});
var spd=1500;
var delay=0;
function pageview(page) {
setTimeout(function() {
ga("testtracker.send","pageview",page);
console.log("Sending: "+page);
},delay+=spd);
}
pageview("start");
if (Math.random()<.60) pageview("about");
if (Math.random()<.30) {
pageview("product");
if (Math.random()<.20) {
pageview("payment");
if (Math.random()<.70) {
pageview("success");
}
}
}
Apparently, I still have a lot to learn about GA.
The solution is to use Audience -> User Explorer and defining a very specific Section based on the suspicious sequence of pages does show individual users matching that pattern. And indeed somehow GA logged a pretty large number of such odd users for me, perhaps failing to register pages visited in too narrow time intervals.
I'm leaving the question for posterity, and perhaps for those who can find my code snippet useful.

Meteor: send message to user at hot code push

How can I let the user know when they are getting a hot code push?
At the moment the screen will go blank during the push, and the user will feel it's rather weird. I want to reassure them the app is updating.
Is there a hook or something which I can use?
Here's the shortest solution I've found so far that doesn't require external packages:
var ALERT_DELAY = 3000;
var needToShowAlert = true;
Reload._onMigrate(function (retry) {
if (needToShowAlert) {
console.log('going to reload in 3 seconds...');
needToShowAlert = false;
_.delay(retry, ALERT_DELAY);
return [false];
} else {
return [true];
}
});
You can just copy that into the client code of your app and change two things:
Replace the console.log with an alert modal or something informing the user that the screen is about to reload.
Replace ALERT_DELAY with some number of milliseconds that you think are appropriate for the user to read the modal from (1).
Other notes
I'd recommend watching this video on Evented Mind, which explains what's going on in a little more detail.
You can also read the comments in the reload source for further enlightenment.
I can image more complex reload logic, especially around deciding when to allow a reload. Also see this pacakge for one possible implementation.
You could send something on Meteor.startup() in your client-side code. I personally use Bert to toast messages.

Meteor utilities:avatar data

I'd like to use the utilities:avatar package, but I'm having some major reservations.
The docs tell me that I should publish my user data, like this:
Meteor.publish("otherUserData", function() {
var data = Meteor.users.find({
}, {
fields : {
"services.twitter.profile_image_url_https" : true,
"services.facebook.id" : true,
"services.google.picture" : true,
"services.github.username" : true,
"services.instagram.profile_picture" : true
}
});
return data;
});
If I understand Meteor's publish/subscribe mechanism correctly, this would push these fields for the entire user database to every client! Clearly, this is not a sustainable solution. Equally clearly, however, either I am doing something wrong, or I am understanding something wrong.
Also: This unscalable solution works fine in a browser, but no avatar icons are visible when the app is deployed to a mobile device, for some reason.
Any ideas?
Separate the issue of which fields to publish from which users you want to publish data on.
Presumably you want to show avatars for other users that the current user is interacting with. You need to decide what query to use in
Meteor.users.find(query,{fields: {...}});
so that you narrow down the list from all users to just pertinent ones.
In my app I end up using reywood:publish-composite to publish the users that are related to the current user via an intermediate collection.
The unscalability of utilities:avatar seems, as far as I can tell, to be a real issue, and there isn't much to be done about it except to remove utilities:avatar and rewrite the avatar URL-fetching code by hand.
As for the avatars not appearing on mobile devices, the answer was simply that we needed to grant permission to access remote URLs in mobile-config.js, like this:
App.accessRule("http://*");
App.accessRule("https://*");

Accounts.OnCreateUser() not firing

I'm following this fantastic tutorial on customizing login found as the answer on this post - http://goo.gl/VLO34 - but it's not working for me.
I copied all the files verbatim, and even added supplementary smart packages such as underscore _.pick, and http Meteor.http.get just in case it was required, and mistakenly left out.
Anyhoo - I get github to authorize my app, but Meteor.users in the web console and db.users.findOne() on the local mongo instance show that Accounts.OnCreateUser() didn't add any new information to my user profile [that I'm pulling in from github]. In other words, {{currentUser.profile.avatar_url}} and {{currentUser.profile.login}} won't reveal anything following that tutorial. So I get blank info on the screen.
I tried that screencasts first attempt, and noticed that {loginButtons}} returns values for {{currentUser.profile.login}}. I've reviewed the code many times for typos, but feel that something is quite off with Accounts.onCreateUser(fn)...
I'm using Meteor 0.5.7, and if anyone else has experienced this problem following that screencast, please let me know. Thanks,
EDIT: I've deployed the project to - http://rptest-customlogin.meteor.com/.
Author of the screencast here. And as of a few seconds ago, a new user on your site :-). So, it looks like login is working on your site. But I'm guessing what's happening in your app is the login info isn't available yet at the time of rendering or at the time you're printing to the console. The reason is that all of that info is being populated asynchronously. It takes a few seconds for the process to complete. If you're relying on Meteor.user().profile data in your templates you need to check first if the login process is still underway.
To do that, you can use either the Meteor.loggingIn() javascript function or the {{#if loggingIn}} handlebars block helper. That function is "reactive" which means once the result changes from true to false your UI will update. So the template might look something like this:
<template name="loginDependentWidget">
{{#if loggingIn}}
Logging In
{{else}}
{{currentUser.profile.avatar_url}}
{{/if}}
</template>
Does this help?
Might be a typo but Accounts.OnCreateUser(fn); should be Accounts.onCreateUser(fn);
Meteor docs: http://docs.meteor.com/#accounts_oncreateuser
And then another post on the same subject:
Meteor login with external service: how to get profile information?
EDIT:
Posting as edit due the formatting of the below piece of code.
In the meantime I have got it running on my own project with this piece of code:
Accounts.onCreateUser(function(options, user) {
if(!options || !user) {
console.log('error creating user');
return;
} else {
if(options.profile) {
user.profile = options.profile;
}
}
return user;
});
Which is working just fine. Have you placed the Accounts.onCreateUser(); on the server?

Run Javascript on the body of a Gmail message

I want to display LaTeX math in the gmail messages that I receive, so that for example $\mathbb P^2$ would show as a nice formula. Now, there are several Javascripts available (for example, this one, or MathJax which would do the job, I just need to call them at the right time to manipulate the gmail message.
I know that this is possible to do in "basic HTML" and "print" views. Is it possible to do in the standard Gmail view? I tried to insert a call to the javascript right before the "canvas_frame" iframe, but that did not work.
My suspicion is that manipulating a Gmail message by any Javascript would be a major security flaw (think of all the malicious links one could insert) and that Google does everything to prevent this. And so the answer to my question is probably 'no'. Am I right in this?
Of course, it would be very easy for Google to implement viewing of LaTeX and MathML math simply by using MathJax on their servers. I made the corresponding Gmail Lab request, but no answer, and no interest from Google apparently.
So, again: is this possible to do without Google's cooperation, on the client side?
I think one of the better ways to do this might be to embed images using the Google Charts API.
<img src="http://chart.apis.google.com/chart?cht=tx&chl=x=\frac{-b%20\pm%20\sqrt{b^2-4ac}}{2a}">
To Learn more: https://developers.google.com/chart/image/ [note, the API has been officially deprecated, but will work until April 2015]
If you really must use LaTeX and some js library, I think one way you could accomplish this is by injecting a script tag into the iframe.
I hope this is a good starting point.
Example:
// ==UserScript==
// #name Test Gmail Alterations
// #version 1
// #author Justen
// #description Test Alter Email
// #include https://mail.google.com/mail/*
// #include http://mail.google.com/mail/*
// #license GPL version 3 or any later version; http://www.gnu.org/copyleft/gpl.html
// ==/UserScript==
(function GmailIframeInject() {
GM_log('Starting GMail iFrame Injection');
var GmailCode = function() {
// Your code here;
// The ':pd' (div id) changes, so you might have to do some extra work
var mail = document.getElementById(':pd');
mail.innerHTML = '<h1>Hello, World!</h1>';
};
var iframe = document.getElementById('canvas_frame');
var doc = null;
if( iframe ) {
GM_log('Got iFrame');
doc = iframe.contentDocument;
} else {
GM_log('ERROR: Could not get iframe with id canvas_frame');
return
}
if( doc ) {
GM_log('Injecting GmailCode');
var code = "(" + GmailCode + ")();"
doc.body.appendChild(doc.createElement('script')).innerHTML=code;
} else {
GM_log('ERROR: Could not get iframe content document');
return;
}
})();
Well, there are already greasemonkey scripts that do things to GMail as far as i know (like this one). Is this a possible security hole? Of course, anything you'd do with executable code has that risk. Google seems to move a glacial speeds on things they're not interested in. They really do seem to function based on internal championing of ideas, so best way forward is to go find sympathetic googlers, if you want them to include something into GMail. Otherwise stick to Greasemonkey, at least you'll have an easy install path for other people who'd like to see the same functionality.

Resources