We are working on a Silverstripe project which requires multiple versions of the same dataobjects.
The same data structure & (roughly) the same data content will be used for A-B testing so the versions would be something like this:
Stage / Draft
A-B Test
B-A Test
Live / Production
I'm guessing the Versioned class can be extended to include multiple stages.
But the behaviour we require is non-linear with A-B/B-A being alternate content rather than a linear progression.
Does anyone have an approach that would work in this situation?
Related
Couple of questions about Form Recognizer (FR) model management:
Background:
I'm using FR Labeling tool to train models and C# Function app to interact with FA service and analyze forms.
Each time a model is trained - a new instance is created. New version does not hold any reference to previous versions and there is no way of selecting a model by name in code. Latest model might be queried using TrainingCompletedOn property but it's not failproof and cannot be used if FR has more than one project. Question: if continuous development is being done and model is constantly improved - is there a way (or best practice) to manage which model has to be targeted.
In connection to 1st Q - since FR always creates a new model - it ends up with a big list of not used models that are still active. and since there is no connection between them - there is no safe way of performing a cleanup. old Models can be removed using API but it's a manual process. Any recommendations on how old model versions can be managed?
Can a model be exported added to version control and deployed to other environments from version control? There is an API endpoint to copy models between FR instances, but I would like to keep it in version control and deploy to environments from there.
In connection to Q3 - What is the recommended practice for managing FR project in DevOps? how can work be versioned and deployed across different environments?
Thank you
each model is unique and independent. it's immutable, you need to pick the model with best accuracy based on your test data set.
you can call DELETE api to delete a model.
not such support at this point. as each model can't be changed after it's created, I don't think there is much value in version control for a model.
please see #1, you could use a test data set to measure model performance. If the model does poorly on one test file, you could label that test file and add it to the training set, and train a new (better) model.
-xin (MS Form Recognizer Team)
Suppose I have two SCORM 2004 modules - instruction.zip and test.zip. The first contains instructional web pages and the second contains an interactive quiz. Each package was authored separately. I want to combine them to create a single course of study in which students read the web pages and then test their knowledge. (I will leave to one side the significant issues of sequencing and navigation.)
What is the recommended way of combining the two? I have tried (i) merging the two (did not work due to differences in file structure and dependencies) and (ii) adding test.zip to instruction.zip as a complete package and adding links (issues with reporting of test results).
I realise that most people author their courses using Captivate or other software to produce an single integrated package. For reasons that need not be discussed here, that is not an option in my case: the test assets will be developed separately and need to be combined with the instructional assets.
Grant,
I've got a packager on my site https://cybercussion.com which may be able to help you. If there is any advanced features your using though I haven't built out support for that yet. There is a 30 day trial for it.
You'd just need to expand the content into something like:
Multi-SCO/
SCO Title 1/ [all SCO 1 files]
SCO Title 2/ [all SCO 2 files]
You can also do this by hand by merging the imsmanifest organization markup together which if your friendly with XML is a option. You'll just need to manage the organization and resource elements. You also may have DTD/XSDs apart of both packages.
Manually zipping this yourself could result in a error importing on the LMS. Some platforms expect the imsmanifest.xml to be in the root of the zip and if its inside a folder it could error. So watch out for that.
We have some great SCORM 2004 samples on our site that may serve as a guide for you as far as sequencing and navigation goes. Check out the golf samples here
If you have any questions, please let us know!
Joe Donnelly
support#scorm.com
I created some new test cases on a couple of new features I would like to request for addition to Flyway. These features are mainly changes to the Flyway class. However, there seems to be two unit test categories, SmallTests and MediumTests. Is there some sort of criteria for determining which one my test cases should fall under?
Yes. These are based on the test categories described by the Google Testing Blog: http://googletesting.blogspot.de/2010/12/test-sizes.html
But make sure raise issues for your fixes and discuss them first to avoid doing work for nothing.
I've got the interesting task to build complex workflow tests for an existing web application which has never had unit or integration tests (all testing by developers / users without structure or guidelines).
Setup
The target is an ASP.NET (no MVC) web application, that has been build over several years. There is no clean MVC or any other pattern, so the HTML output is very bad (generated IDs, not much css classes, in line css styles
--> hard to test).
The application is data-centric, so there is a lot to test and the status of the database is very important for the tests.
I'm thinking of the following workflow:
1. Reset Database to test start data
2. Run Tests which create and test data by simulation user interactions
Tools
I was playing around with Selenium IDE, but it does not feel structured enough.
I'm dreaming of a toolset, where I can write tests in (literate) coffeescript / javascript, that can be executed in the browser (no need for headless testing), but all tools I can find aim to test javascript functions and not user interactions.
My test steps need to be:
page loads
test if page has loaded correctly by searching for text A or counting elements in ul B
click on button C, wait for popup D, type text in field E
click save button and check if text from field E is shown in element F
Is jasmine able to test these kind of user interactions? I can only find jasmine tests for javascript functions or the existance of HTML elements, but not for complex workflows with test steps that rely on each other.
Thanks in advance!
Selenium IDE is not the way to go for this.
Selenium is built on the WebDriver JSON Wire Protocol. Having this 'base' means it has been very easily plugged into many many different languages, all using the same kind of API.
One of them being JavaScript:
https://code.google.com/p/selenium/wiki/WebDriverJs
Disclaimer: the JS Bindings are very new!
I'm not sure I understand why you must do it in JavaScript, especially as I can see this severely limits your options.
I suggest IBM Rational Functional Tester. It's java based, making it very extensible. It's not Javascript, so be warned that test are written in java code. For most of it there's a recorder: you simply click around and it records your actions.
Regarding automated tests with it, here's my opinions .
It is a commercial product. I'm not affiliated with IBM but I work with RFT a lot.
You may want to look at RIATest for cross-platform cross-browser testing of web applications. It uses ECMAScript based script language which is very similar to JavaScript.
It works on Windows and Mac, supported browsers are Firefox, IE and Chrome. Automated testing scripts written on one platform/browser can be run against all other supported platforms/browsers.
It is certainly possible to automate the steps that you described. Dynamically generated IDs are not good but you should be able to use other properties (such as text, element type, etc.) to identify the HTML objects that you want to automate and test for.
(Disclaimer: I am a RIATest team member).
Having toyed with the concept in the past, I am interested in using multivariate testing on my companies Sitecore website. There are a number of places where I feel we can definitely improve sales through the use of A/B testing in:
Running two entirely different templates to see what layouts work better for users
Running a number of different Sublayouts (forms) on the site to see which ones people are more likely to fill out
Trialling different content - Running two different sets of copy to see if users are more likely to stay on the page
I want to use the Marketing Suite within Sitecore, and I want to be able to measure who visits pages more and count, out of two or more sublayout forms, which form is used the most. Sadly, I have no experience with the OMS and am struggling to see how one actually implements these things.
Let's say I have a content item, with a bunch of sublayouts attached to it within its template. Can someone help guide me towards a way of achiving the three things I want to run multivariate testing on?
EDIT: On the subject of the two sublayouts I want to test on a template; I have two sublayouts, which are both simple ASP.NET email forms. Once a user fills in the form the contents of the form are written to a database and an email (using Sitecore.Context.Item to get an "Email From" field from the content item that runs the form).
This is where I get stuck. A number of the sublayouts I have don't seem to have any "content" that needs pulling from a data source. The only content I can see in the case of the two forms I want to test is the "Email To" fields. So, if I were to abstract those away into their own data templates, and then added those as data sources I assume that I would then have to change my code for these to stop using Sitecore.Context.Item?
The point where I get stuck is with the data sources for the Multivariate Test Variables and the data sources for the Sublayouts. If I have two data templates containing the Email fields for each, two sublayouts that contain the forms that need testing and two multivariate variables, what goes where?
I believe you can read about it in the Analytics Configuration Reference (PDF link) under section 2.2.
You essentially create a MV test that wraps over potential data sources of a sublayout. The test then randomly assigns a DataSource, so your sublayouts need to be written to work with a DataSource.
With Sitecore 8 released Multivariate Testing is now supported out of the box as well as AB Testing.
You can run two entirely different templates to see which Layout works best for the user by Page Test in Sitecore's Optimization Tool on the Launch Pad. Creating a Page Test you can select the current version of the Item then create a new Version of the Item with the different Layout. This can also be done for Content on the Page
After that you need to decide how a winner can be chosen e.g. most goals completed by users, registrations etc then Sitecore will automatically run the test for you showing A and B to various users and ultimately choose a winner based on the Test Objective. You can choose a winner mannually or let Sitecore automatically choose after a set Duration.
Creating a Mulitvariate Test on number of different Sublayouts as well as imagery, personalisation, content etc is a little more interesting. To create a Multivariate Test is done via Workflow Actions, I've posted a blog recently how to add Maultivariate Testing to workflow.
Approving with a Test will prompt Sitecore to create a Multivariate Test for all variables (Sublayouts, Content, Personalization etc). It creates an 'Experience' for every possible combination of these variables and tests them against each other.
For a more in-depth explination and guide I have recently posted a tutorial to create a Multivariate Test in Sitecore.
There are two trainings that you (and a developer on your team) should really consider attending: OMS Certified Marketer and OMS .NET Developer.
Working with a Sitecore Certified OMS .NET Developer, you will be able to accomplish your marketing objectives. This is what Sitecore Training is for!
Please see the following and regsiter for the next available trainings:
http://www.sitecore.net/Training/Course-Overview/OMS-11-Certified-Marketer.aspx
http://www.sitecore.net/Training/Course-Overview/OMS-11-NET-Developer.aspx