UX and research · Beginner · 15 min
Run a five-person usability test
By Padding · Updated 18 Nov 2026
A laptop or phone running the live product, the observation spreadsheet from your test plan and a whiteboard or spreadsheet for the review · November 2026
You will run five in-person think-aloud sessions on a live product in one day, hold back from helping when people struggle, and finish with a list of observed problems showing how many of the five hit each one.
On this page
A usability test fails quietly when the moderator helps. Someone hesitates over a button, the moderator says "it's the one at the top", the person finishes the task, and the notes record a success. The problem is still in the product, and now nobody knows it is there.
This tutorial is about running the sessions, not planning them. It assumes you already have the plan, session script, success rules, stop rule and observation sheet from Plan a usability test with tasks and success criteria. Here you put them to work: five one-to-one sessions in a single day on the live product, each person thinking aloud, the moderator saying as little as possible, and the note-taker filling the same sheet for every session. At the end of the day you sit down with the team and turn the notes into a list of problems, each with a count of how many participants hit it.
Five people from one user group are likely to surface the problems that many of that group run into, especially the common ones. The often-quoted five-user figure is an average across studies, not a guarantee: you will miss some problems, rarer ones especially, and you will not get statistics. Treat the counts as "how widespread did this look", not as percentages.
The running example is Northline Buses, a fictional regional bus company with a live mobile ticket app. Its plan has three tasks: buy a single ticket for tomorrow, show the ticket to the driver once bought, and get a refund for a ticket bought by mistake. Swap in your own product and plan.
Step 1 · Set the day out and prepare the kit
Book one room for the whole day and set the schedule before anything else. Your plan's script runs to about 45 minutes, so give each slot an hour: the session, then 15 minutes for notes, a reset and a break.
09:30 Set-up, device check, moderator reads the script aloud once
10:00 P1
11:00 P2
12:00 Lunch
13:00 P3
14:00 P4
15:00 P5
16:00 Break
16:30 Team review (90 minutes)The pilot in your plan already tested the tasks and the timing, so the morning slot is not another dry run. It is for the moderator to hear the script in their own voice once and for both of you to check the kit.
Prepare the device the participants will use. A live product needs care that a prototype does not: use a test account with a payment method that can be refunded, or a sandbox if the product has one, so nobody spends their own money. Write down the exact starting state: which screen is open, whether the user is signed in, what is in their basket. You reset to this state before every session, so each person starts from the same place.
Print each scenario from the plan on its own card, word for word, one set per participant. Put the plan's stop rule and facilitator rules on a single card the moderator can see during the session.
Check it worked
You have a written schedule with five slots and a 90-minute review at the end, a written starting state for the device, five sets of scenario cards that match the plan word for word, and the stop rule and facilitator rules on one card.
Step 2 · Adapt the observation sheet for live note-taking
The plan's observation sheet (one tab per participant, one row per task) was designed for summarising. Taking notes live, while someone talks, needs three small additions. Make them in the plan's spreadsheet itself, so there is still only one sheet.
- A Flags column next to Outcome, for two flags:
hintwhen the participant got any help, from you or by accident, andstoppedwhen the stop rule ended the task. Flags explain an outcome code; they do not replace it. - Timestamps in Notes. Each observation starts with the recording clock, so it can be found again: "02:10 taps Tickets tab, scrolls, back to Home".
- Two lines below the task rows: "Debrief answers" and "Biggest problem / surprise", filled in after the session.
Keep the plan's outcome codes and their meaning. Read the success, partial and fail rule for each task once more with the note-taker before P1, and add these two rules to the top of each tab so they are applied the same way all day:
- Finished after a hint: the task was done only because someone helped, so it is Partial at best, flagged
hint, unless the plan's row for that task says otherwise. A hinted finish is never Success. - Fail covers any wrong or missing end state: the person did not finish, finished the wrong thing whether or not they realised it, gave up, or was stopped under the stop rule (flag
stopped).
Observations are things you saw or heard. They are not explanations ("didn't understand the menu"). Explanations come in the team review.
Name each tab with a participant code, not a name, and leave time on task out, as the plan says: people thinking aloud are slower than at home, so the times would mislead.
Check it worked
The plan's spreadsheet has a Flags column on all five participant tabs, the two lines below the task rows, and the hint and fail rules at the top of each tab. The note-taker can tell you, without looking, how they would code a task finished after one hint.
Step 3 · Deliver the opening and the warm-up
The first five minutes decide whether the person talks for the next 35. Read the plan's welcome, consent and warm-up word for word. Reading is fine and does not sound cold; paraphrasing is where moderators drift into "just show me how you'd do it" and give away the task.
A few things change when the words are spoken rather than written:
- Slow down on the line that you are testing the app, not them. It is the line people most need to hear, and it is easy to rush.
- Ask for consent before you start recording, then start the recording and ask the person to confirm on the recording that they agree. If you have not prepared the consent form, see Get informed consent for user research.
- In the warm-up, if the person goes quiet, model thinking aloud once yourself ("I'd be saying something like: I'm looking for a search box, I can't see one...") and let them try again. Do not take notes on the warm-up.
- Keep the background questions to the plan's two or three. If the person starts telling a long story, thank them and move on to the first task.
Check it worked
Your recording starts with the person agreeing to be recorded, and during the warm-up they spoke at least a few sentences about what they were doing without you asking.
Step 4 · Give each task and stay quiet
Hand over one scenario card at a time. Ask the person to read it aloud, then say "Start whenever you're ready" and stop talking. When the task ends, ask the plan's ease question (1 to 7) before the next card goes down.
This is the hardest part of the job. When someone struggles, you will feel a strong urge to rescue them, and they will often look at you for help. Every rescue removes evidence. The plan's facilitator rules tell you what not to do; these habits make the rules easier to keep in the room:
- Sit slightly behind and to the side, so they look at the screen rather than at you.
- Keep still. Nodding and "mm" read as "correct" as much as "good" does.
- When they say "I'm being stupid", answer: "You're not. This is the kind of thing we need to see. Keep going as you would at home."
- When they finish but do not say so, ask: "Where are you up to?"
- Keep an eye on the clock for the stop rule, not on the person's face. When it triggers, say "Thanks, that's really useful. Let's move on to the next one." and do not show the answer until the debrief.
If you slip and help, say nothing more about it. Tell the note-taker after the task so they flag it hint.
Check it worked
Listen back to two minutes of the recording from the task where the person struggled most. Every sentence you said is either reading a scenario card, a neutral prompt from the plan, or the stop-rule wording. There is no "try the top", "nearly", "good" or pointing at the screen. If there is, the task is flagged hint on the sheet and coded Partial at best, not Success.
Step 5 · Take notes in the sheet while it happens
While the moderator stays quiet, the note-taker fills the current task's row. Three things matter.
Write what happened, with a timestamp: "04:35 opens Account, then Help, then back to Tickets". Add a tally to Wrong turns each time the person moves away from the goal. Write exact words in quotes when the person says something revealing: "I'd have expected it under My tickets." Fill in the outcome code, any flags and the ease rating at the end of each task, before the next card goes down.
The moderator jots only short markers on a separate sheet, such as "T3 refund buried under Help" or "check 05:40", so they can stay focused on the person. Those markers point the note-taker to moments worth checking against the recording later.
Between sessions, use the 15 spare minutes. The note-taker tidies the tab while it is fresh. The moderator resets the device to the written starting state: signed in or out, empty basket, Home screen, any purchase from the last session refunded.
Check it worked
The tab for the session has an outcome code and an ease rating for every task, a flag wherever the moderator hinted or the stop rule triggered, at least one timestamped observation per task, and at least two direct quotes. The device is back on the written starting state before the next participant arrives.
Step 6 · Run the debrief and close
After the last task, ask the plan's debrief questions and nothing new. The aim is consistency: the same questions to all five people, so the answers can be compared.
The note-taker writes the answers on the "Debrief answers" line. They tell you how the person experienced the session, which is useful alongside, but separate from, what you saw. When people say it was easy after visibly struggling, record both and trust the observation.
Now you may show them anything they could not find, if they ask. Then follow the plan's close: thanks, the incentive, stop the recording. Before the next person arrives, fill in the "Biggest problem / surprise" line together, not at the end of the day.
Check it worked
Each completed tab has answers to the plan's debrief questions and the biggest problem and surprise filled in. Five tabs carry these lines by 16:00.
Step 7 · Run the team review the same day
At 16:30, get the product people who need to act on the results in one room: at least a designer, a developer and whoever owns the product. Anyone who watched a live session is especially useful here, because they saw it happen.
Work in this order, and keep to 90 minutes:
- Outcomes first (15 minutes). Open the plan's Summary tab: participants as columns, tasks as rows, one outcome code per cell, with flags beside the code. Put the ease ratings next to it. Everyone looks at both before anyone discusses causes. A task with mostly successes and low ease ratings is still worth a problem group.
- Observations onto the wall (30 minutes). Each person takes one or two participant tabs and writes each observation that shows a problem on its own sticky note or spreadsheet row, starting with the participant code: "P3: looked for refund under My tickets, found it under Help after several wrong turns".
- Group into problems (30 minutes). Move notes that describe the same underlying problem together. Name each group as a problem with the product, not with the people: "Refund option is under Help, where people do not look", not "users couldn't find refunds".
- Count (15 minutes). For each problem, count how many different participants hit it. Two notes from P3 still count as one participant.
Check each problem against the sheet, not memory. If nobody can point to an observation in a tab or a moment in a recording, it is an opinion, and it goes on a separate "ideas" list.
Check it worked
The Summary tab has 15 cells (five participants by three tasks, or as many tasks as you ran), every cell filled with Success, Partial or Fail, flags where they apply, and a set of named problem groups where each note starts with a participant code.
Step 8 · Write the problem list with counts
Before everyone leaves, turn the groups into one list. Sort it by how many participants hit each problem, highest first. Keep one row per problem:
# Problem Task Participants Seen in
1 Refund option sits under Help, not on the ticket 3 4 of 5 P1 P2 P3 P5
2 "Single" and "Day saver" look identical until tapped 1 3 of 5 P2 P4 P5
3 Bought ticket is not on Home; people reopen Buy 2 3 of 5 P1 P3 P4
4 Date picker defaults to today with no warning 1 2 of 5 P1 P4
5 "Activate" step not explained before showing driver 2 1 of 5 P2Keep the problems found by one person. A single observation can still point to something serious, such as buying the wrong ticket without knowing. The list records how widespread each problem looked; how bad each one is comes next, in Rate usability problems by severity.
Add three lines above the table: the date, the five participant codes and the user group they came from, and the tasks you tested. Share the list with the team that evening, with links to two or three short recording clips for the top problems. Clips persuade people who were not in the room far more than a table does.
Check it worked
Your list has one row per problem, each with the task it came from, a count written as "n of 5", and the participant codes that back it up. Every participant code on the list matches a tab in the sheet. It is sorted with the highest count at the top, and it has been sent to the team on the same day as the sessions.
Next steps
- Rate each problem by severity so the team fixes the right things first: Rate usability problems by severity.
- Tighten the tasks and success rules for your next round: Plan a usability test with tasks and success criteria.
- Find five better-matched participants for that round: Recruit participants for user research.
- If you collected a lot of quotes, group them by theme with Build an affinity map from interview notes.
Part of these paths
Related tutorials
- Plan a usability test with tasks and success criteriaYou will have a written usability test plan for a booking flow, with scenario tasks that do not give away the answer, a success rule for each task and a session script another person could run without you.
- Rate and prioritise usability problems by severityYou will have a severity table for the problems from a usability test, rated on a three-factor scale that two people apply the same way, and a ranked fix list the team can plan a release from.
Need this done for you?
Padding does this work for growing companies, from the first diagnosis to a system that keeps working.
Get new tutorials