Web Analytics & Multi Variable Testing (MVT)


It’s been almost over a year that I’ve been looking into A/B & Multivariable Testing (MVT) and was thinking of integrating these kinds of capabilities inside our content management systems.  As we have been building CMS since more than 6 years, for us building up the different pages and combinations was really a simple thing.  The questions began to emerge when we wanted to report.

The following questions were raised by the IT and Web Analytics business units. 

  • Was a Web Analytics solution the best tool to report on a multivariable test? 
  • Wasn’t it to complicated to set-up? 
  • Was it necessary for these kind of activities to use a second administration console (on top of the CMS admin)?
  • How was a WA tool going to report easily on multiple variables being tested at the same time? 
  • Could a Web Analytics solution provide our clients with tips about statistical relevancy of the results?

Faced with these questions we decided to make our own reporting system within OniSystem.  This in theory would allow us to ease our clients work.  We wanted to make a tool that would allow to report on different situations as newsletter landing pages, website homepages, checkout processes, lead generation (contact forms, downloaded documents, …), … 

Any piece of content, from a site layout, to a sentence or an image should be testable through this module.  This would ensure that our clients could use our module whatever their business model or strategy.  We also wanted our clients to see the effectiveness against any objective and not report only on one objective.  The MVT module allows even once you have launched it, to measure the data against any objective imaginable (newsletter subscription, contact, purchase, visit of a specific page, …).  Our clients can then measure the data against their different objectives and then decide which combination works best for them.  Aurélie has helped us with her statistical knowledge to include the necessary formulas to ensure the relevancy of the results.  The system thus alerts you when the data is relevant!

OniSystem A/B Testing module

Here’s a screenshot of a simple A/B testing on our WebAnalytics.be website’s Home Page. We tested two different HP versions and measured the data against the visit of our contact page as the purpose of our website is lead generation. 

As you can see OniSystem presented more or less evenly both versions to the visitors.  The change we made was rather light (including or not a drop down with a sector segmentation) and the conversion rate is rather similar, even though we see that the first version of the Home Page was 15% more effective than the second.  Imagine using this kind of technology with several different tests?  At OX2 we are very excited with this new module that will really help our customers getting the best return of their online initiatives.


%d bloggers like this: