Phil Williams
Head of UX and Optimisation
View posts

  • Conversion Rate Optimisation

2nd Jul 2015

5 min

A/B testing is quite a scientific process. Hours of user research is conducted to compile significant amounts of data. From the data, a hypothesis of how a site can be improved is finalised and set against the control. Once the test has concluded and has gathered enough data, there is a rigorous analysis of the results through an analytics tool and a winner is awarded. Methodical, calculated and as many believe, unimaginative.

This, of course, is far from the truth. That’s because the decision to A/B test your site raises the most creative question of all. ‘What if…?’

What if we changed the size?

What if we changed the layout?

Following the ‘if’ you have the ‘why’ and ‘how’. ‘Why’ should we change the layout and ‘how’ should we do it?

These questions allow creative thought to permeate every stage of the Optimisation process.

As a child, one could ask ‘if’ ‘where’ and ‘how’ without fear, judgement and risk. They were the fundamental basis for how we discovered the world around us. As a working adult in the real world, such questions can seem daunting.

This is why questioning your own website or questioning a certain process will inevitably lead you on a path of change and discovery.

Are your ideas just Pink Elephants?

 

giphy pink elephants

 

Here’s a little game.

If I say to you don’t think about pink elephants, what’s the first thing that pops into your head? Even though I asked you not to, I bet it was a pink elephant. Now your brain is scrambling around trying to think of something else. This is creative thought on a basic level; lateral thinking in it’s simplest form.

When faced with an obstacle or a preconceived idea, your brain has to find a way around it. When A/B testing, the control is  the pink elephant. When testing, you can play it safe and just change the colour of the ‘elephant’ (control) to green (variant). You can think of another animal entirely, like a mouse but these ideas are still related in some way to the original. Trying to think of something entirely different can be challenging when you have a preconceived idea of how something should be.

As a designer, it is my job to think more laterally and to avoid just changing the elephant’s colour. Being creative in this industry doesn’t consist of making the most audacious website or the most beautiful graphics, it can be completely the opposite. For example, take the Architect Mies van der Rohe, one of the 20th century’s most creative minds. Famous for his pioneering modern architecture and part of the Bauhaus movement, he was passionate about rational thinking and simple ideas. He adopted the phrase ‘less is more’.

This ethos is against simply designing for design’s sake; it’s looking at a problem from a new perspective and delivering a minimal viable product.  A truly creative way of thinking.

This creative process is still as important (if not more so) in today’s world as it was in the early 1900’s. Now we are bombarded with information from all over the world, 24 hours a day, from multiple sources and devices.

If you can deliver your content/solution/product and cut through the noise of today’s information overload, you could be onto a winning formula.

Example from EA Games

Below is an A/B test for the Sim City 5 game. The goal was to increase the game’s pre-orders.  The control offered $20 off your next purchase in the form of an attention grabbing banner above the fold.

 

Sim City A/B Testing Control

 

The variation is a much simpler page, just the product and with no mention of the offer at all.

 

simcity2

 

The creative process here is obvious – highlight the offer and more people will be enticed to buy the game. The results however, were more surprising. The simpler variation drove 43.3% more purchases.

Initially, this seems counter intuitive. What it actually tells us is customers (who had been anticipating the game’s release for months) didn’t need any incentive to buy the game; they just wanted the simplest way to buy the product.

Some may argue the simpler variant is the less creative option but look at the scenario differently. Instead of sticking with an offer banner and simply testing different copy, the decision was made to remove the banner completely. This is quite a radical undertaking that goes against the initial marketing plan for the product. Less (on the page) was definitely more (conversions).

Conclusion

The EA example of removing the offer banner (the pink elephant) was just one solution to one problem. What about all the other possible scenarios? After deducing the problem areas of your site, creating new hypotheses to test is where creative thought can flourish, as long as you keep your mind open to new possibilities. A/B testing doesn’t kill creativity, it stimulates it. All A/B testing does is have an audience validate one creative idea over another.