Moderated usability testing

In moderated usability testing, a moderator observes test participants and interacts with them to ask questions and prompt feedback. The moderator can ask the participant to think out loud, as well as ask follow-up questions and clarify responses to gain a deeper understanding of the participant’s experience. 

During a moderated usability test, the moderator guides participants through a series of tasks and provides support or clarification as needed. They can also provide a comfortable and supportive environment for participants to feel at ease and share their honest feedback.

Moderated usability testing is useful for testing complex or sensitive tasks, as well as for gathering in-depth feedback from participants. It can be conducted in-person or remotely using online testing or video conferencing tools.

In-person moderated usability testing

In-person moderated usability testing involves bringing participants into a physical location, such as a lab or conference room, to test a product or prototype. It involves a moderator guiding participants and observing their behavior in real-time. 

In-person testing can be particularly useful when you’re testing physical products or gathering detailed qualitative feedback. It allows you to observe nonverbal cues and body language, which can provide useful insights into user behavior. Additionally, participants can ask questions and receive immediate feedback, which can enhance their overall testing experience.

In-person testing is more time-consuming and expensive than remote testing methods. It requires a physical location and may require the support of a Research Ops team to manage logistics such as scheduling, recruitment, and participant incentives. Additionally, participants may feel more pressure to perform in an in-person setting, which can affect their behavior and feedback.

Unmoderated usability testing

Unmoderated usability testing is conducted remotely and asynchronously, without the presence of a moderator. Participants are given a set of tasks to complete, and their interactions are recorded and analyzed for usability issues. 

This type of testing is often conducted using a specialized tool like Lyssna, which allows you to set up and distribute tests to a large number of participants. 

Unmoderated usability testing offers a number of advantages over moderated testing, including greater scalability, cost-effectiveness, and the ability to reach a more diverse range of participants. However, it also has some limitations, such as the lack of opportunity to ask participants questions in real-time.

What is remote usability testing?

Remote usability testing is a form of usability testing where test participants are located in a different location from the researcher or moderator. Sessions can be moderated or unmoderated. It’s typically conducted using online tools or software that allows you to observe and record the participant's interactions with a website, app, or prototype. 

What are the advantages of remote usability testing? 

Here are some of the advantages of running remote usability testing:

• Convenience: Remote usability testing allows you to test participants from anywhere, which can save time and money on travel expenses.

• Flexibility: For remote moderated tests, you can schedule sessions at a time that’s convenient for you and the participant, which can help increase participation rates. For remote unmoderated tests, participants can complete the test at a time that works best for them.

• Large sample sizes: You can recruit a large number of participants and from a wide range of demographics, which can help increase the statistical power of your results.

• Improved participant comfort: Participants can complete tests in the comfort of their own environment, which can lead to more natural behavior.

• Collaboration: Multiple stakeholders can observe testing sessions remotely, which can facilitate collaboration and feedback.

What are the disadvantages of remote usability testing?

Despite its pros, there are some drawbacks to remote usability testing, such as:

• Technical issues: Remote testing requires a stable internet connection and access to the software you’re using, which can be a barrier if you or your participants have any technical issues.

• Limited observation: With remote testing, you might not be able to observe nonverbal cues or the physical environment of the participant, which could limit your insights.

• Limited control: You have less control over the testing environment and might not be able to provide immediate assistance to participants if they encounter issues.

• Limited engagement: Remote testing may lack the same level of engagement as in-person testing, which can impact participant motivation and attention.

When should you choose remote vs in-person usability testing?

You can run a remote usability test at any time during the design process. It can be a good option in various situations, such as:

• When you want to test with participants who are geographically dispersed and can’t easily come to a physical location for testing.

• When you have a limited budget. Remote testing can be more cost-effective and easier to organize than in-person testing.

• When you want to get quick feedback on a product or design. Remote testing can be set up and run relatively quickly.

• When you want to test a product with a large number of participants. Remote testing can often accommodate more participants than in-person testing.

• When you want to test a product with participants who are more comfortable in their natural environment. Remote testing allows participants to use their own devices in their own space.

Top tips for remote unmoderated usability testing

If you’ve decided that remote is the way to go, here are some things to bear in mind.

Choose the right remote testing tool

There are many remote usability testing tools available, so make sure to choose one that suits your needs and budget. Consider factors such as ease of use, features, and support.

Prepare your test participants

If you’re running a remote moderated test, send clear instructions to your participants ahead of time. Double check they have the necessary equipment and software to participate in the test.

If you’re running a remote unmoderated test, be sure to share your goals and include clear instructions and an explanation of the scenario and test task.

Set up your testing environment

For remote moderated tests, ensure that you have a quiet and distraction-free testing environment, with a reliable internet connection. Consider appropriate lighting, headphones, and a microphone, if needed.

Monitor the test

In a remote moderated test, keep an eye on the test as it’s happening, and be available to answer any questions that participants may have.

Record the session

Record the session so you can review it later and make note of any insights.

Debrief your test participants

After the test, debrief your participants and thank them for their time. Ask for feedback on the testing process and any areas for improvement.

You can still do this if you’re running a remote unmoderated test. For example, in Lyssna you can add a short or long-text question at the end of the test asking for feedback, or a linear scale question such as “How easy did you find this task?”, with 1 being not at all easy and 5 being very easy. You can also personalize the end screen with a thankyou message.

Analyze and report the results

Finally, analyze the results and report your findings to your team and other stakeholders. Use these insights to make improvements to your product or service, and repeat the usability testing process with your next iteration. We go into more detail about this in Analyzing usability test results.

Exploring the diversity of usability testing methods

In this chapter, we reviewed different types of usability testing, from moderated to unmoderated, in-person to remote. Each type of usability test offers distinct advantages and considerations. 

Here are some key takeaways from this chapter:

• Moderated usability testing: This method, whether conducted in-person or remotely, enables real-time interaction with participants, allowing for in-depth insights into their experiences. It’s particularly useful for complex or sensitive tasks.

• Unmoderated usability testing: This remote and asynchronous approach offers scalability and cost-effectiveness. While it lacks real-time interaction, it's ideal for reaching diverse participant groups.

• Remote usability testing: Conducted with participants in different locations, remote usability testing offers flexibility, cost savings, and the ability to test with geographically dispersed users. It’s particularly useful for quick feedback and accommodating large sample sizes.

By understanding these usability testing variations and their strengths, you can make informed decisions to align your testing approach with your study’s objectives and constraints.