2013 was a great year for optimisation testing. The industry as a whole made leaps and bounds in terms of getting optimisation as a line in many companies budgets, as the business case for Conversion Rate Optimisation starts to get through to the right people in the right places.
We saw it ourselves with one of our clients, The Student Room, who at the beginning of 2013 were just starting out in the world of testing and optimisation. Fast forward twelve months and one of their key internal champions took on a new role within the business as Director of Optimisation. They’ve also just signed up to another 12 months working with us! We were so proud of the quality of tests we achieved with The Student Room last year, that we entered one of the tests into the Which Test Won Awards 2014 and won!
You can hear what the judges had to say about the test hypothesis and why they liked it (skip to 39:00) along with a range of different tests to give you an idea of what others were testing in 2013. Below you can read the full case study of the award winning Which Test Won entry.
How to: Increase conversion of paying subscribers by 185.3%
AT A GLANCE
Client: Get Revising
About: Get Revising is an online community of over 1 million students and teachers. The site offers over 123,000 resources as well as tools to assist learners from GCSE to University level.
Goals: Increase subscriptions of premium users while not deterring non-paying users from signing-up
Test: Sign Up Page
Results: Increased conversion of premium subscribers by 185.3%
“We desperately needed to discover what combination of our products and offering was the right one to tip users into the paid subscription. We also needed to do this as leanly as possible, with development resource being tight. I believe that the technique used ticked both of these boxes. “
Pete Taylor, Director of Optimisation at The Student Room Group
Get Revising offers students a multitude of study resources, tools and online forums. They offered a free subscription which gave access to most of the sites features. They also offered a premium subscription costing £4.99 per month, which gave access to two additional features not available to free subscription users. The challenge was to increase the number of users subscribing to the premium service, while not deterring users from signing up to the free version.
It was therefore important to understand which features and messages were the most compelling to encourage users to sign-up for a premium subscription rather than settling for the free subscription.
From previous user-testing on Get Revising we had observed that the majority of users didn’t perceive added value in the premium subscription because most of the features were given away in the free subscription. We also found that users didn’t understand what the premium subscription really offered them and that this information needed to be presented in a clearer way.
The main hypothesis was that by presenting the information in a “features” table, it would be more obvious what the user was missing out on by opting for the free registration rather than the premium subscription. The second hypothesis was that by including a number of variations with differing features included, we would be able to determine where the ‘tipping point’ is between free and premium and which features are the most influential in user’s decision making, allowing us to increase the number of premium subscription conversions.
We also needed to be mindful of not discouraging users that engage with Get Revising as free users. We needed to balance the proposition so that it was persuasive to premium subscribers while still offering value to non-paying users. Therefore our final hypothesis was that by doing this test we will find the right balance of free resources and paid resources, without causing an unmanageable decline of registrations.
The concept for this test came from the Chairman of the Student Room Group (long live the Hippo!). We went from concept to live in less than 24 hours.
This test was fairly straightforward to set up in Optimizely. We planned four different configurations of features for the free subscription, designed to appeal to specific user personas. We then used the Optimizely editor to vary the ticks and crosses in the free column.
Our primary goal for this test was premium subscriptions which we tracked using a page view goal using the “thank you” page after subscribing. We then used targeting by URL and traffic allocation features to set up the test for the 5 variations (including the control).
The winning variation increased conversion of premium subscribers by 185.3%. It also created a 14.8% decrease in users opting for the free registration and therefore was an acceptable trade off with the big increase in conversions and additional revenue generated.
- This test allowed us to better understand what makes the most compelling feature set for the premium package
- Gaining access to the database of resources and the study groups made the biggest positive impact on users opting to buy the premium subscription
- The decreased features on offer to free subscribers had a small negative impact on the number of free sign-ups however the increase in premium subscriptions negated this.
A massive congratulations to Pete and his team at The Student Room and Thanks to Which Test Won for our badge. Here’s to many more winning tests in 2014