Starter Guide to Usability Testing
Usability studies are the foundation of user research. Usability testing is the most common type of testing method that user researchers or people who do research (PWDR) utilize. For anyone looking to get started with UX research, usability testing is one of the best starting points.
What is Usability Testing?
A usability test usually works by having a person observe and guide another person through using a product. The observer/guide is usually the researcher or someone else designated as the moderator. The other person participating in the usability test would be an actual user of the product.
The researcher would give the participant a task to go through. For example, if it’s an e-commerce website, the researcher might ask the participant to find out how much a certain pair of shoes would cost. As the participant goes through the task, the researcher would be observing what the participant does, the comments they provide, and see if they’re able to do the task successfully.
Usability testing evaluates the effectiveness of a product’s design. It’s an evaluative user research method that’s intended to provide product teams with tangible data that can be used to inform product decisions. It helps teams discover if a product works the way they intended it to.
In short, a usability test is accomplished by getting real users to interact with a product, while the researcher observes their behavior and reactions to it.
What is a user?
Because finding real users is crucial to usability testing, it’s important to define what a real user actually means.
To put it in a basic way, a user is someone who uses the product on which you’re looking to conduct user research. That’s pretty broad, right? It sounds like that could be anyone.
In reality, it’s rarely that simple.
If you’re a business, you typically have a target audience that you’re looking to market to. You might have multiple target audiences, but typically you’re focusing on certain demographics rather than trying to capture everyone.
When it comes to usability testing, it’s important to understand what you’re looking to test. That might help determine who the right user actually means.
How do I run a usability test?
In order to successfully conduct usability testing, there are a number of different stages and steps involved that you’ll need to follow. Your way may differ, but the following steps highlight the basic approach to planning usability testing.
1. Define the problem (and have initial team alignments as needed)
Before doing any form of research, it’s vital to get a firm grasp of the problem you want to research. When we learn the scientific method in school, we learn that everything starts with a hypothesis. It’s the same with usability testing methods.
What are you going to test?
You can’t have user testing without something to test. Sometimes the answer might be straightforward because you’re on a specific product team and need to do research on that product. Sometimes, it might not be super straightforward and you’ll need to consult with others.
You need to be very specific about what you’re going to test. For instance, if your product has an app and a website, you’ll need to decide which one you’re going to test. After that, you’ll need to decide what aspects of the user experience you’re going to test.
Let’s pretend that we’re an e-commerce site and we want to test usability of the website. This is the first time we’re doing website usability testing, so we want to test out the basic functions of our website. We want to evaluate how well the user finds and buys a product. As we get further along in this process, we’ll get to more specific tasks, but for now, this is a sufficient level of detail to know.
2. Align with the product team
Aligning with the product team doesn’t happen just once. It should happen on an ongoing basis throughout the research lifecycle. However, in the beginning, it’s crucial to work with the product team to inform your research plan.
Once you’ve got an idea of what you’re going to test, it’s important to get buy-in and information from the product team. You want to make sure you’re doing research that provides value to them. Therefore, you want to make sure you’re getting the information that they need from the usability tests that you run.
You might choose to have a meeting with the entire team to discuss the usability testing plan, or you might decide it would be more effective to conduct stakeholder interviews, which would give you time to go more in-depth with each member of the team.
Find your hypothesis
As someone doing the research, it might be hard to find the hypothesis. By the nature of the researcher’s role, they might not be the ones doing the day-to-day work. As such, finding the hypothesis might involve working with different members of the product team to find out some hypothesis options. It’s good to get as specific as possible here.
Continuing our e-commerce example, we might hypothesize that it takes the user, at most, 3 clicks to find the product they’re looking for. We might hypothesize that it shouldn’t take them longer than 10 seconds to find the “add to cart” button. These specific hypotheses can later help inform the tasks given to the participants.
All in all, we might hypothesize that our website is easy to use, which can be validated through certain metrics like the System Usability Scale (SUS) and Standard Ease Question (SEQ).
Determine Tasks
A key to usability testing is determining the right tasks to give the research participants. If you’re already working on the product in question, it might not be hard for you to figure those out. It’s always good to work with your teammates to make sure you’re selecting the right tasks.
In our e-commerce example, we’d create separate tasks for almost every action. One task could be finding X product, another task could be finding the product reviews, another task could be adding the item to the cart, and so on.
Find the “Happy Path”
In the design and development process, the team would have determined something called the happy path. This happy path would be the ideal way a user would complete tasks. In the design process, a designer would have envisioned the perfect way to do something. However, in reality, real users might not do that perfect action.
Thus, you should always find out the happy path so that you can compare the ideal with reality.
3. Plan your study
A lot of the time, the study planning happens in conjunction with the alignment stage. However, in this step, you’re formalizing your research plan and documenting it.
Common components of a research plan include:
- Research Goals – what you want the research to accomplish
- Research Questions – what questions you want the study to find out (not to be confused with the questions you ask participants
- Timeline – details on how long the research would take, including dates for the different stages
- Target users – information regarding the job titles or personas you’re looking for
- Metrics – noting down what metrics you plan to collect during the study
Sometimes researchers will also include their protocol into their research plan. The protocol would include the script that the person moderating the research would read.
The protocol script might include:
- Introduction – introducing you to the participant
- Disclaimers – where you’d remind the participant of any NDAs they signed and confirm they’re okay with you recording the session
- Setting the stage – if there’s a certain scenario the participant should have in mind
- General questions – you might have certain questions you’d like to know about the participant
- Ex: How often do you use this product?
- Tasks – the specific research tasks for them to go through
- Metrics – questions relating to the success of the task or ease of use of the system
- Ex: Rate on a scale of 1-5 your confidence that you did this task successfully.
You have to get as specific as possible with your research plan. Once you’ve written your research plan, it’s a good idea to get your stakeholders to review and approve it.
4. Identify the right participants
Identifying the right participants is one of the most important aspects of user testing. Products are rarely designed to work for absolutely every single person on the planet. Even if you’re working on one of those products, you might be trying to test how a certain group of people uses that particular product.
For instance, I might be doing usability tests for an e-commerce site that serves everyone (like Walmart or Amazon), but then my team might want to know how well retired people can complete tasks. In that case, I would focus my efforts on finding retired people. I’d want to get firm on the characteristics of these types of participants.
Many products have a specific target audience. A database tool would target database administrators and other data workers. A tennis racquet retailer would target people who play tennis. And so on.
Getting clear on the participants you need will enable you to then be able to find them.
5. Recruit participants
Recruiting can be a complex topic because it’s not always easy to find the right participants. However, for the purposes of this article, we’ll talk about a common step needing in almost any recruiting scenario: creating a screener.
What’s a screener?
A screener typically takes the form of a survey or questionnaire designed to filter out participants who don’t fit within your target audience. On almost any recruiting platform, you’ll run into professional testers who do user testing tests to make money. They’re often not the people you’re looking for.
Creating your screener
To find the right users, you want to ask questions that only they can answer. It’s challenging because you want to avoid questions and answers that they can search for online. It’s a good idea to mix multiple-choice and open-ended questions so that you can see how genuine they are. You also might want to ask them to include a link to their LinkedIn profile or some other professional profile.
6. Schedule usability testing sessions
Depending on how you recruit participants, you might have the option to invite them directly within the same for usability testing with you.
Regardless, when scheduling, you’ll want to be sure you’re ready to go with usability testing. Only give them date options that you know you’re ready for conducting usability testing. It may seem basic, but inevitably a participant books for a time sooner than you wanted them to.
Inviting Observers
A great way to get people invested in your research is to let observers watch your usability testing sessions. They get to hear from real users first hand what’s going on with the product. When you’re figuring out your schedule, try to see what times work for your teammates. Once a usability testing session has been scheduled, invite your team to set in. Remote usability testing makes this really easy.
7. Conduct usability studies
At last! We’re at the stage where we actually do the usability testing. A lot goes into preparing a usability test, but it pays off.
There are a number of best practices to bear in mind while conducting usability testing.
Be quiet!
Of course, you’ll need to talk during your usability testing session to give the participants tasks, but you also want to make sure they’re doing as much talking as possible. You want to hear their thoughts.
Invite Participants to Think Aloud
Since you want to hear participants’ thoughts during usability testing, invite them to share their thoughts verbally as they go through your tasks.
Remind the participants to do this often. It might not feel natural to some people.
You can say things like, “I notice you haven’t said anything in a moment. What’s on your mind?” or “This task has made you pause. What are you seeing?”
8. Synthesize data
After you have your data collected, it’s time to synthesize the data. There are many different methods for analysis, but all in all, your goal is to discover trends and findings to share with your team. At this stage, it’s helpful to go back to your research plan and see what your research goals and questions were.
If you collected certain metrics during your usability test, this is where you’d calculate the data collected and figure out any scores you need.
9. Report findings
Now that the usability testing is complete, you need to share the findings with your stakeholders by creating a usability testing report. It’s important to do this so that they’ll see the results of the research and know what to do with the data collected. Getting buy-in early on during your research helps make this process much easier.
It’s important to hold your team accountable for implementing your research. It’s always good to include recommendations for what they can do next to improve the product. Luckily, through usability testing, the participants often give good ideas if you give them the space to do so!
When do I run usability studies?
Usability testing methods work best when you have something specific to evaluate. It’s best to run usability tests when your product is live and you can test how it functions. There are plenty of other methods you can utilize at different stages of the product development lifecycle.
You might consider running the same usability test plan over time. For example, you might want to run the same test every quarter or after every product release so that you can see how things change over time. These benchmarking usability tests help businesses track progress.
How is usability testing different than other forms of user testing?
Usability testing focuses specifically on evaluating the usability of a product. Usability testing methods focus on specific tasks and measurements to gather feedback and measurements on how well a product performs.
User testing is a term that includes usability testing along with other research methods. Usability testing is a type of user testing, but user testing is not a type of usability testing.
Final thoughts
Usability testing is an excellent method to evaluate the success of a product. It enables you to see how well your product is performing at a certain point in time and gives you data that you can use to improve your product. Usability testing is the foundation of research. For anyone looking into starting out with UX research, trying out usability testing is a great first step.
