Carry out usability tests correctly: methods, process and practical examples

Usability tests give you the answers that numbers alone can't provide, and you don't need a huge budget or weeks of preparation. In this article, you'll learn how to find out with simple means where your users are really stuck and how to systematically solve these problems.

The most important thing in brief:

• 5 users are enough for the usability test, they reveal approx. 85% of all problems
• Observe instead of asking! Real usage behavior shows you the most
• Use the think-aloud method: thinking out loud brings many insights
• The tests are possible remotely & with little technology: via Zoom and Google Forms
• Test regularly: Test small more often than rarely large

What is a usability test?

A usability test is an empirical method for systematically evaluating the usability of digital programs. It shows how users interact with software and where there are difficulties in understanding or use barriers.

During a usability test, you watch test people use your website, app, or product. While they do typical tasks, you watch where they get stuck, what confuses them and where everything runs smoothly.

Simple example of a usability test
You are launching an online shop for outdoor equipment. In the usability test, you give a test taker the task:

“Find a waterproof backpack for a three-day hike and order it.”

After that, you only observe the behavior. You might see him searching for a long time for the “waterproof” filter. Or how he aborts the checkout process because the shipping costs are only displayed at the very end and therefore the total price rises unexpectedly.

How many test subjects does a usability test need?

Jakob Nielsen found out, that five test subjects have already uncovered around 85% of all usability problems. His most important finding was:

“Complex usability tests are simply a waste of resources. For the best results, let a maximum of five users test and run as many small tests as possible.” Jakob Nielsen, originally translated from English.

Difference between usability tests and other methods

Usability testing is often confused with other methods or lumped together. This is understandable, because many approaches deal with users and their behavior and all point to the higher-level Conversion rate optimization one. This series also includes the UX optimization. However, the differences are important because they determine what kind of insights you get:

• A/B testing: In A/B testing, you show different user groups two versions of your website and measure which one works better. However, you don't know why that is the case.
• Focus groups: People discuss your product and share their opinions. But that doesn't show you how they would actually use it.
• Surveys: You ask what people think or want. (However, it is only during the usability test that you can see what they are actually doing.)
• Heatmaps: They show you where users are clicking and scrolling. But not why they're doing it or whether they've achieved their goal in the process.

Types of usability testing

Usability tests differ primarily in four dimensions: moderated vs. unmoderated, remote vs. on-site, with or without thinking out loud, and in a controlled vs. natural environment. With 5 to 8 participants, moderated tests already reveal 85 to 90% of the problems; unmoderated tests would require around 30+ participants.

Moderated vs. unmoderated usability tests

Moderated testing: You or a colleague will guide you through the test, ask questions and can follow up. It's like having a conversation: You see what's happening and You can also find out what the user is thinking. Moderated testing identifies 85 to 90% of issues Even with 5 to 8 participants.

Unmoderated testing: The participants receive tasks and work through them independently, while software records everything. It's cheaper and faster, but you miss out on the thoughts and reactions. To do this, you need around 30 or more participants for meaningful results.

Usability testing remotely vs. on-site

Remote testing: The participants sit at their own computer at home or at work, and you connect via video call. This is practical, inexpensive and users are in their familiar environment, so they may also behave more naturally there. You see around 80 to 85% of the problems that you would also see locally.

Tests on site: You're in the same room or usability lab, with controlled conditions, professional recording, and no distractions. You see facial expressions and body language and can immediately help with technical problems. This reveals 90 to 95% of all problems, but costs more time and money.

Guerrilla Testing

You go to a coffee shop or other busy place with your laptop, talk to people and ask them for 10 minutes of their time for a quick test. That works well for first impressions and obvious problems. The advantage is that the test subjects are completely unbiased.

Usability test: think-aloud vs. silent observation

Think Aloud: The participants should say out loud what they are thinking and doing. This gives you insights into their thought processes. Studies show that think-aloud reveals 4.8 times more individual problems than subsequent surveys.

Silent observation: You just watch and ask afterwards. This shows more natural behavior, but you learn less about user motivation.

Phases of usability testing

A usability test consists of three main phases: preparation, execution and follow-up. Preparation is decisive for the success of the entire test and usually takes more time than actually carrying out the test.

1st preparatory phase

• Define goals: What exactly do you want to find out? “Test the website” is not a goal. “Find out why 60% of users cancel at checkout.” The more precise your goal, the more targeted you can set up the test.
• Define test object: What are you testing specifically? A live website, a prototype, an app? This also determines which tools you need and how realistic the test results will be.
• Formulate tasks: Develop realistic scenarios that your target group would also have in real life. Instead of “click on the red button,” better “You're looking for a gift for your sister for her birthday. Find something suitable and order it.”
• Recruit participants: Find people who match your real audience. That means not only demographic characteristics, but also behavior and experience with similar products.

2nd implementation phase

The following scheme is often used to carry out usability tests:

• Greeting: Explain the process, alleviate the nervousness of the participants and make it clear that you're testing the product, not them. People tend to feel bad when they don't understand something.
• Perform tasks: Let the participants work and hold back. Your job is to watch, not to help. If they get stuck, that's an important insight.
• think-aloud method: Ask participants to think out loud. “What's going through your head right now?” or “What are you looking for?” are good questions to get them talking.
• Documenting: Write down everything: where they hesitate, what confuses them, what words they use. The video recordings are valuable documentation, but you should still create notes in parallel. Tip: Keep a timer running during the test so that you can timestamp individual notes and assign them to the right time in the video.

3rd follow-up phase

• Evaluation: Go through your notes and videos. Search for patterns: Did several participants have problems in the same place? Which tasks were not solved at all or were only solved in a very cumbersome way?
• Prepare results: Gather your insights in a way that others can understand and use. Screenshots of problem areas, quotes from participants and specific suggestions for improvement help.
• presentation: Share the results with your team. Video excerpts are often more convincing than long reports. When the development team sees a user searching for a button for minutes, they understand the problem better than through any description.
• Plan implementation: The best findings are of no use if they end up in a drawer. Prioritize the problems found and plan concrete steps to fix them.

Rule of thumb: For each day of testing, you should plan two to three days for preparation and follow-up.

Usability testing tools

The tools for usability testing range from free basic solutions to professional platforms. You often need less to get started than you think, because sometimes just zoom and a smartphone are enough.

Free and inexpensive tools for usability testing

• Zoom or teams: Absolutely sufficient for remote testing. You can share the screen, record, and take notes at the same time.
• OBS Studio: Free screen recording software if you're doing local testing. A bit more complicated to use but very powerful.
• Google Forms: For preliminary questionnaires and post-test feedback. It's free and easy to use.
• Smart phone: Often underrated, but perfect for guerrilla testing, for example. You can create video screen recordings and then evaluate the testing sessions in peace and quiet.

usability tools

• Hotjar: Heatmaps, session recordings and simple surveys in one tool.
• Maze: Focus on prototype testing with good Figma integration. Particularly strong with unmoderated tests.
• Lyssna (formerly UsabilityHub): Good for specific tests such as first-click or 5-second tests. 300 free answers per year.

Professional usability test platforms

• UserTesting: The market leader with a large pool of participants and AI-based evaluation. Higher price range.
• Lookback: Especially for moderated tests with good video quality. Mid-range price range but very easy to use.
• UsabilityHub (now part of Lyssna): Various testing methods under one roof, from card sorting to prototype testing.

Tip: Here you can find A/B testing tools.

Key facts: What you should take with you to usability tests

You need less than you think:

Five test subjects have already identified 85% of all usability problems. A simple test with Zoom, Google Forms and your smartphone is all you need to get started and costs less than 50 francs per month.

Observing begs questions:

Usability tests show you what people really do. This makes them valuable and different from surveys or focus groups, which only collect opinions.

Preparation is important:

Most of the time is spent on planning and evaluation, not on the actual execution of tests. For each test day, you plan two to three days for preparation and follow-up. This investment makes the difference between useful insights and wasted time.

Limits of usability testing that you should keep in mind:
1. The test situation is not realistic
In the test, users know that they are being watched. They are more concentrated and motivated than in everyday life. This leads to a higher Involvement. Many errors that occur in real use remain undetected as a result.
2. The behavior is often not typical
Users approach tasks more specifically than outside of testing. They think more consciously and try to do everything right. This does not correspond to actual use, which often takes place on the side or under time pressure.
3. Usability tests only check the operation
They show whether something is working, but not whether it is needed at all. This does not answer strategic questions about the product or market.

Stuck in your testing?

We at Axisbits know from the experience of numerous website and software projects that both testing and conversion rate optimization can be very detailed and time-consuming.

When the performance of your project Always driving below your expectations And if you are sure that more needs to be done there, a neutral view from outside will probably help you.

Your shop should implement more, your landing page deliver more leads and is there still too much room in your pipeline?
Maybe it's time for a neutral look at your content, your software, and the entire website. Together, we can develop a roadmap that allows your sales to rise again. Get in touch with us and we'll show you how we would approach conversion rate optimization including usability testing.

{{fs-btn-cta}}