What is data analysis?
Data analysis, in its widest sense, can be defined as the process of collecting, interpreting and modelling data in order to identify insights that lead to valuable learning and aid decision making. This is a helpful introduction, but in a conversion rate optimisation (CRO) context we can be more specific and go in to greater detail about how data analysis is performed and why it’s so important.
Data analysis for CRO
Data is a key cornerstone of CRO, so collecting the right data and carrying out effective analysis is absolutely vital. There are many ways that we can use data throughout our CRO process, but we will look in to three key areas here:
- Customer data analysis
- Audience segmentation
- AB test data analysis
Customer data analysis
Customer data analysis is the process of using all data and information available to a business in order to better understand its customers. When conducting a conversion rate optimisation campaign, before even thinking about what we might A/B test we start off with customer data analysis in order to better understand how people currently use the website in terms of onsite behaviour, to map out the customer journey, and to identify any issues that may be a barrier to conversions. We will typically carry out a range of quantitative research using a range of analytics tools as well as qualitative research, which in some cases will provide us with other forms of data.
The quantitative research is usually undertaken using our client’s primary analytics tool such as Google Analytics. Provided that their tools are well configured, there will be a range of custom reports, segments and goals alongside the standard reports that we can use to observe user behaviour and identify potential areas for improvement. If a client’s analytics configuration is sub-optimal, we will often carry out an analytics and data configuration audit to deduce how the configuration can be improved to collect the most valuable insights possible – so that you can place trust in the data when making important business decisions.
We may also use secondary analytics tools such as heat maps or form analytics, which add a richness to our picture of user behaviour.
Qualitative research methods such as onsite or offline surveys may also produce data that can be used to identify opportunities to improve your onsite experience.
A key part of our data analysis process is to understand the different types of users that visit your website and whether behaviour changes for different segments. For example, do visitors from different marketing channels behave differently? It is quite likely that new visitors arriving via paid advertising for the first time will behave differently to users returning to your site from your email campaigns. There are many varied factors that can be used to build helps segments. For many clients these become important when we start A/B testing, as we can use these segments to run more advanced experiments.
A/B test data analysis
Once we start split testing, our testing tool will start to collect data about every user that visits one of our test control or variation pages. Early on we need to check that the correct data is being captured correctly. This may include standard metrics and custom goals that you have determined during your test design phase to capture insights on specific actions or behaviours.
Once you have run your experiments for a suitable length of time (when the test becomes statistically significant) and collected sufficient data, you will be able to draw valid conclusions. Once a test is concluded we can carry out detailed analysis to provide results for key metrics and interactions. As for collecting data in your testing tools, we recommend, whenever possible, that you send data to your primary analytics tool to increase the scope for detailed analysis with a greater range of metrics and dimensions. Beyond key goals and metrics, it may be valuable to carry out some deeper analysis to look for patterns or trends in the experiment data. In some cases this will lead to a conclusive observation or drive new test hypotheses for subsequent experiments.
For more details, check out the dedicated A/B test analysis page.
- Primary Testing Tool – Google Analytics, Adobe Omniture, Coremetrics, WebTrends
- Secondary Analytics tools – Hotjar, Formisimo, Clicktale, Decibel Insights
- Survey Tools – iPerceptions, Kampyle, Qualaroo
- Testing Tool – Qubit, Optimizely, VWO,