What Is Multivariate Testing

What Is Multivariate Testing image.

What is Multivariate Testing

Unlike A/B Testing, which allows two distinct versions of a page layout to compete head to head with each other, multivariate (or multi variable as it’s sometimes known) can simultaneously pitch multiple elements against each other.  Using scientific formulas it is possible to test multiple aspects of a page at the same time.  Not only are you able to measure which elements work and which don’t, you are also able to see which combination of elements perform better than others. Often with interesting and unexpected results.
Using the dynamic properties of multivariate testing you can rapidly learn the best combination of page elements – things like the page header, choice of colours, call to action phrase, the best special offer or find out if the form with detailed instructions works better than the short version. Where previously you had to rely on pure guesswork and experience to design a page you are now able to let your visitors design it for you.  You may find that the version you started with was the best one all along, but without testing how will you know?

Following a process that mimics natural evolution a multivariate test begins with a random set of properties (page elements).  These properties are pitched against one another and the ‘fittest’ individual elements are chosen for inclusion in subsequent stages of the test.  This process continues until the traits of the best elements are common throughout the testing ‘population’.

Performing a multivariate test requires specialist software, such as Google’s excellent Site Optimizer tool. To initiate a test the following steps are followed:

  1. Decide which page elements are to be testing.
    In many way this is the most difficult stage of the test, as you need to decide which parts of the site your visitors are happy with and which you think need improving; and not just you own personal preferences.
  2. Create variations of each element.
  3. Add the tracking code
    Usually in the form of on-page JavaScript code.
  4. Specify a success metric
    How do you know if a combination has worked? This is measured by tracking a specific conversion page, such as the checkout, form completion etc.  Any visitors arriving at this location have achieved the goal you have set them.
  5. Run the test
    Which may last for a few days or even months; depending on the number of combinations being trialed and the number of daily visits your site received.
  6. Measure the results
    Seen via the tracking reports of the testing tool you are using.
  7. Either end the trial, or select the winning combination and run this as an additional A/B test with the original