• Conversion Rate Optimisation

18th Jan 2017

3 min

What is A/B Testing?

A/B testing (sometimes referred to as split testing) allows you to determine the impact of content, design and functionality changes to your website. By creating multiple versions of a web page and collecting data & statistics for a control group (usually A) and at least one variation (B) we can learn about how your ideas impact user behaviour and ultimately your key goals and objectives.

What is the difference between A/B and multivariate testing?

Multivariate testing (MVT) is a technique in which multiple variables are modified. So instead of running one A/B test (or a batch test), multivariate testing runs multiple tests, each with a different combination of element variations. The ultimate goal of MVT is to see which combination of element changes delivers the biggest uplift against the control.

The number of tests will always be [# of Variations on Element A] X [# of Variations on Element B] (and so on) = [Total # of Variations]. So for example, if you have two variations of a header image (A) and two variations of a headline (B) you will have four separate tests running.

Due to a higher number of variations being run at one time, MVT often requires a high volume of site traffic (to compensate for the split of traffic to each variation and how long it would take for each combination to reach statistical significance) to ensure each variation in the test reaches validity. This means that for many businesses, multivariate tests are not the most efficient way of testing and improving their online experience.

Why should you perform A/B split tests?

In too many businesses, changes to the website are made on a whim or without solid evidence to support them. Even when you have great insights from data and user research, it is still difficult to predict how users will behave when you roll out changes. AB testing takes the guesswork out of the process and allows you to be confident that you are making the right decisions.

Beyond that, we have seen across all of our optimisation clients that A/B testing is consistently one of their greatest levers for growth. Our growth methodology™ consistently delivers a 77% test success rate (much better than the in-house industry average of 33%). By working with an AB testing agency like PRWD, you will become more data-driven and more likely to meet your growth forecasts.

Key A/B testing tools