Paul Rouke
Founder & Director of Optimisation
View posts

  • User Research

21st Dec 2017

20 min

In an industry devoted to the people who use our products, services, and applications, research is paramount. We ask questions. We take notes. We learn everything we can about the target audience, and then iteratively test our work throughout the design process.

User research serves many purposes throughout the design process. It helps us prove or disprove our assumptions, find commonalities across our target audience, and recognise users needs, goals and mental models. Overall, research improves our understanding, informs our work and validates our decisions.

In this complete beginners guide, we’ll look at the many elements of user research, from interviews and studies to heat mapping and surveys. Readers will get a head start on how to use these design research techniques in their work, and improve their experiences for all users.

What is user research?

User research encompasses a variety of investigative methods used to add context and insight to the UX design process. UX practitioners have borrowed many research techniques from academics, scientists, marketers, and other professions. However, there are still types of research that are fairly unique to the UX world.

The main goal of user research is to inform the design or A/B testing process from the perspective of the end user. It is research that prevents us from designing for one user: ourselves. It is now well accepted that the purpose of UX is to design with the end-user in mind, but it’s research that tells us who that person is, in what context they’ll use this product or service, and what they need from us.

User research incorporates a group of methodologies that include user testing, interviews, surveys and others. Every project is different, and the tasks that one researcher takes on will differ from those appropriate in another setting. These methodologies can be split into two camps: attitudinal and behavioural.

Attitudinal User Research Techniques

Attitudinal methods gather qualitative insights into user’s thoughts, feelings, needs, attitudes and motivations. From these methods you can expect to gather powerful insights in the form of quotations and anecdotes.

Some of the most popular attitudinal user research techniques include:

1. Focus Groups

Focus groups are a small group of participants representative of your consumer market to hold a short interview session with, where people can talk freely and answer your questions.

This is particularly valuable to gain the thoughts and experiences of a representative group who have used your product or service. As people can speak freely, they may highlight insights that you hadn’t prepared questions for or weren’t aware of,

However, this method isn’t without some downfalls, your results could be biased dependent on the dynamics of the group and it may be difficult to get individual opinions if people don’t feel comfortable sharing them. They can also be quite time consuming and costly to implement; especially if you need to hire the space to hold a focus group or provide payment/incentives to secure participants.

2. Surveys

Surveys avoid the problem of a group dynamic, as you can hand out a list of questions for your participants pertaining to their preferences, attitudes, characteristics, and opinions on a given product or website, allowing them to answer anonymously. As a research method, surveys and polls allow you to count or quantify concepts – a sample or sunset of the broader audience is used, the learnings from which can be applied to the broader population.

The benefit of this is that people tend to be more open about their experience with the site. It is also necessary to add a question at the end asking the user to add any additional comments not covered in your survey.

This is a useful tool to gain a basic insight into the key areas you want to understand about user experience on your site. However, obtaining in-depth insights using a survey is difficult to achieve and may require some further incentive for people to complete. Once people have answered your survey it is also difficult to gain any further information/ ask follow up questions.

3. 1-1 moderated user research

This method allows users to carry out scenarios and tasks whilst talking through their needs, thoughts and frustrations. The moderator probes the participant with open questions in order to further explore and gain additional feedback about the user experience. This is particularly valuable to understand the opinions and recommendations from the people that matter most.

Gaining the most valuable and meaningful insights isn’t always straightforward, particularly when the person you are speaking to has never previously been asked for their opinions and ideas. This is especially true when speaking to staff within an organisation, whether you are looking to improve production processes, improve efficiency on the shop floor, or as is often the case with ourselves, reviewing a company intranet.

4. Remote moderated user research

1-1 research where the research and participants are based in separate locations, sometimes this is conducted over a video call system with screen sharing such as Skype.

5. Remote unmoderated user research

Remote testing tools such as UserZoom or WhatUsersDo allow you to set tasks for users to read and do. Their screen will be recorded and they will think aloud whilst completing the tasks. Video recordings of the sessions will be available for replay after.

6. Diary study

Participants are given a mechanism (diary or camera) to record and describe aspects of their lives that are relevant to a product or service. Diary studies are typically longitudinal and can only be done for data that is easily recorded by participants. They allow you to explore how attitudes and perceptions change over time.

Behavioural User Research techniques

Behavioural research methods on the other hand, aim to measure what users actually do, providing quantitative data about how users actually interact with your website. Behavioural research methods can provide answers to questions such as where the majority of users click after first arriving at your homepage, or if users fail to notice the key proposition messaging displaying in your USP bar.

Some of the most popular attitudinal user research techniques include:

  1. Card sorting

In a card sort, a user is provided with a set of terms, and asked to categorise them. In a closed card sort, the user is also given the category names; in an open card sort the user creates whatever categories he or she feels are most appropriate. The goal of a card sort is to explore relationships between content, and better understand the hierarchies that a user perceives. Many content strategists and information architects rely on card sorts to test out hierarchy theories, or kick start work on a site map by exposing users’ models.

2. Tree testing

Just as card sorts are a great way to gather information before a websites architecture has been created, tree tests are helpful in validating that architecture. In a tree test,  users are given as task and shown the top level of a site map. Then, much like in a usability test, they are asked to talk through where they would go to accomplish the task. However, unlike in a usability test, the user doesn’t see a screen when they choose a site section. Instead, they will see the next level of the architecture. The goal is to identify whether information is categorised correctly.

3. Concept testing

A researcher shares an approximation of a product or service that captures the key essence (the value proposition) of a new concept or product in order to determine if it meets the needs of the target audience. Concept testing can be done one-on-one or with larger numbers of participants, and either in person or online.

4. The 5 second test

This method involves showing users a single content page for 5 seconds. The aim of a 5 second test is to gather a participant’s initial impressions and assess the screen’s clarity and conciseness using simple questions like “What is the most important information on the page?” or “How would you go about achieving your goal on this screen?”. The setup is similar to a standard usability test, and may sometimes be combined with one.

5. Field study

Research activities that take place in the user’s context rather than in your office or lab. The range of possible study methods and activities is very wide. Field studies also vary a lot in terms of how the researcher interacts (or doesn’t) with participants. Some field studies are purely observational, some are interviews in which the questions evolve as understanding increases, and some involve prototype feature exploration or demonstration of pain point in existing products.

6. Heatmap

These colourful tools provide more than just data – they offer incredible insight on how visitors are absorbing and interacting with your website. Simply put, a heatmap is a form of visual analytics tool that uses colour to instantly convey information about the activity of users on your website. The colours generated by a heatmap where users are scanning and clicking and, perhaps more revealing, where they aren’t scanning and clicking. Heatmaps will show you where your visitors are clicking and what areas of your website they are paying the most attention to. They can also show how effective your design is at encouraging your users to explore your entire page (scrolling).

7. Data Analysis

Of course we must also talk about data analysis. Setting up the right software (e.g. Google Analytics) properly for your site will enable you to track pre-determined goals, such as enquiries made. Use this software to see the user journey between landing on your site and converting to a goal (or not converting). Crucial behavioural indicators on here, such as bounce rate, average time spent on a page, number of sessions and so on, provide an insight into how much value your site has to its users, is your site easy to navigate? Are people finding what they came on the site for etc. a low average time per session and a high bounce rate would suggest that they are not. While this method can be less informative on actual user experience, it is a key insight that you should be testing and monitoring regularly as a foundation for your research.

8. Guerrilla testing

A modern, lightweight take on traditional test. Instead of renting a lab, guerrilla research is typically done out in the community; users are found at coffee shops or stations and asked to complete basic tasks with a website or service, in exchange for a small incentive. While guerrilla testing is a great option, particularly on a budget, it is best used only for products or services with a large user base. More niche products will struggle to find reliable information from the random selection acquired in guerrilla testing.

So, to summarise…

Both attitudinal and behavioural research methods have their place in user research and the most valuable and actionable insights will come from using techniques from both camps in conjunction with one another, essentially to back up what users say with what they actually do.

Humans, as a species are naturally self-conscious, constantly worried about how others perceive us. This desire to be accepted and to fit in, can influence what users say they think and prevent them from being completely honest. This is one of the main downfalls of traditional methods of attitudinal research as users are influenced by others opinions and are keen to agree with the majority. This phenomenon is known as herd behaviour and works in the same way as using social proof on websites to increase sales. This may not be so prevalent in all attitudinal user research methods, but it’s worth being aware of.

And despite this downfall, user’s opinions are incredibly valuable, they are the people interacting with your website, experiencing your brand and ultimately converting. Conducting research with real users, recruited based on your target audience, provides insights into their needs and motivations, something that cannot be predicted by practitioners and that will be void of bias or ulterior motive.

 

 

User Research

Photo by Joris Louwes Flickr

 

 

On the other hand, behavioural methods have the advantage of gathering data on how users actually interact with your website. This can be done in a lab setting, or through trend analysis of data from real-life users. These users are making a genuine investment on your website, in terms of time and also potentially in terms of money.

Despite the value of gathering data about what users do and how they interact with your website, quantitative data is open to individual interpretation, which is why it is important to provide context by also asking users why. A heat map produced from an eye tracking study may show heavy areas of attention spread across the website’s homepage. This could be interpreted as a positive thing, showing that users are engaged and interested in the homepage content. However, attention spread across the homepage could also suggest that there is no clear route for users to take and therefore indicate a struggle. Similarly, analytics data may show that a large percentage of users leave your website from a particular page, but it will not tell you why they left your website and if they left satisfied or frustrated.

To gain the most rounded picture of your users, conduct attitudinal and behavioural methods simultaneously. Use exit surveys alongside analysis of analytics data to find out why users leave your website before converting. Conduct moderated user testing session facilitated with eye tracking technology to help prompt users to articulate why they did what they did. Taking on a dual approach research methodology will provide strong and robust evidence to facilitate effective changes to your website.

5 Tips for Conducting One-to-One User Research

A number of the methods outlined above involve some form of one-to-one interaction, whether it’s guerilla testing, moderated user research, or an interview. To get the most from these interactions, take a look at the following tips.

1) Give the person you are speaking to the confidence that their input is truly valued

Many people, especially those who have never taken part in this type of research before, think that there input won’t be valuable, for any number of reasons. This could include:

  • they don’t use the website or system often
  • they have never done this type of activity before
  • they have never been asked for their opinion before
  • they don’t feel their role is very important, so why would their opinions be important

We tend to give people confidence in these situations by:

  • explaining to them the approach that the user research is taking
  • re-iterate during the meeting how valuable their views and insights are
  • explain that this isn’t just a one-off experience and that gaining input from people such as themselves is a fundamental part to the website or intranet improvement process going forward
  • ask whether they would like to be involved in further elements of the research and design process

It can be quite remarkable experiencing a person’s confidence grow, from at the start where they are very apprehensive and introverted to becoming very passionate and open to sharing their views.

2) Give the person your full attention – keep your head up!

Rather than try and carry out the face-to-face meeting as well as making notes throughout on the key points being made, simply voice record the meeting (asking permission first of course) to allow you to give the person your full attention.

Benefits of voice recording the meeting include:

  • the conversation is much more natural
  • it ensures you will never miss a vital piece of feedback whilst you are making a note of a previous comment
  • you are paying much more attention to what the person is saying, which in turn helps you think of which questions and prompts should follow next
  • the person feels fully engaged rather than looking at the top of your head for parts of the meeting
  • if you choose to quote the person in the research report and findings work you do after, you ensure the quote is 100% accurate and note based on your notes

A few things to consider on voice recording meetings:

  • if like us, due to the sheet amount of valuable insights and comments that each meeting contains, you choose to get full transcriptions of each meeting, this is a very time consuming exercise and one which we definitely recommend you outsource
  • you may also be thinking of video recording the meetings, especially if (as I will come on to later) you also use these meetings to carry out user testing. You need to be aware that this will usually heighten the sense of apprehension for the person which can affect their confidence in fully engaging with the process

3) Ask open, probing questions

Being asked to share your views on a particular website or company intranet is very much a one-off scenario (unless of course you are adopting a true user experience design approach and will be engaging with that person throughout the design and development process). However familiar with a system the person is, it takes certain types of questions to get people to open up and share the types of insights that will be of most value in your research.

Typical questions we might ask include:

  • Can you describe the reasons why you typically [add in different user scenarios]?
  • Can you explain the approach you tend to take when [add in different user scenarios]?
  • What improvements do you think would make your job/experience easier and more effective?
  • At [add in various areas of the website or intranet], what information are you looking for and why?
  • Can you explain a situation where you haven’t been able to find what you are looking for, and what have been your next steps?

It is vital that your questioning doesn’t lead the person down a particular route on purpose – the more natural and open ended you make the questioning the more likely you are of gaining really valuable insights.

4) Bring the user research to life by incorporating user testing

Providing the research you are doing is for an existing website or company intranet, bring a greater degree of engagement and value to the research by incorporating user testing into the meeting.

Website testing or system testing, not user testing

It’s vital to stress at this point that when you explain this to the person, you are clear that you are wanting to test the usability and effectiveness of the website or system, not their capabilities or ability to remember where certain information is.

Key points when incorporating testing to user research:

  • avoid the temptation to use a lab testing environment – let the user work in their own comfortable surroundings, whether this is at home or at their desk at work
  • don’t let the person blame themselves if they are having trouble with a particular task for a scenario
  • plan ahead with typical scenarios, and be as descriptive as possible
  • ensure that the scenarios you are asking them to experience have some correlation to their potential wants and needs
  • explain the importance of them adopting the think out loud approach, which you need for both your own benefit during the meeting but more importantly for when the voice recording is transcribed
  • once it is clear they are unable to complete a task at a certain point, prompt them in the right direction and then let them continue on with their think out loud approach

5) Don’t rely on first impressions

Quite often the person that you are speaking to immediately makes it clear that they know their input won’t be very useful. This can be for a number of reasons:

  • they only ever use a very small element of the website or intranet, and that can be explained in a matter of seconds
  • they hardly ever use the website or intranet
  • they are against change within their organisation and so they aren’t wanting to disrupt anything
  • they dismiss the very fact that you are asking for their feedback, and are apprehensive that what they say will make any difference or actually be used to actually make improvements

This where the first four tips all come into play – by giving confidence to the person, providing your full attention, asking probing questions and bringing the session to life by letting them use the website/intranet for themselves, you will often find that the people who on first impressions seem to be offering the least amount of valuable feedback actually turn out the complete opposite.

 

Tools of the trade

The following tools are excellent for assisting in the planning, set up, and running of user research. Below is a short description of each tool and what it can be used for.

 

Ethnio Ethnio was the first moderated remote research software when it launched, and it’s still going strong. Ethnio finds users who are currently using a site or app, and (with their permission) allows interviewers to ask them questions about their experience as they go. It automates many elements of the typical in-person test, including real-time notifications, and paying participants with Amazon gift cards.

 

Optimal WorkshopOptimal Workshop has everything! The full Workshop is a bundle of four research tools, all of which are also available sold separately (and very affordably). Treejack is great for remotely testing information architecture, either to test the nomenclature or the hierarchies themselves. Optimal Sort provides online card sorting, to see how users choose to organise content. Chalkmark offers heat maps of click patterns across a site, and Reframer is a tool for taking notes and identifying themes easily. All come highly recommended.

 

SurveyMonkey Surveys and questionnaires are great ways to gather information, but they’re most useful when hundreds of responses can be seen at once. Enter SurveyMonkey, an online survey-creation and reporting tool, which allows people to customise and brand their own surveys and then send them out via social media, embed them into websites, or integrate with mass mailings. SurveyMonkey also allows for easy analysis and reporting when the results come in.

 

UserTestingWhen it’s not possible to schedule a real-time test with users, UserTesting is a great way to see how people use a site. Researchers can create a series of tasks, and then receive videos from participants—either pre-chosen, or randomly selected. Researchers are able to see a video of the participant using the site, and speaking aloud to explain what they’re doing.

 

UserZoom The good news is, whatever you need, UserZoom has it. Usability testing, both moderated and unmoderated, remote testing for mobile and desktop, benchmarking, card sorting, tree testing, surveys, and rankings: they’ve got it!

 

What Users Do WhatUsersDo is another remote user testing platform. You’ll get access to a global panel of 30,000 UX testers – real people, from a variety of backgrounds – who will complete any task you set for them. They’ll give you their spoken aloud thoughts and feelings, via video recordings of the test, which you can use to observe their behaviour, analyse and build a business case for improving the usability of your digital product.

 

Hotjar – Hotjar is a suite of analytics tools that will help you gather qualitative data such as heatmaps, funnel tracking, user polls, surveys and more.