The Big Budget Myth
When organizations think about "market research," they often imagine focus groups with 50 people, huge surveys with 1,000 respondents, and expensive laboratories with two-way mirrors.
This perception is dangerous. It stops teams from doing any research at all because they believe they lack the budget or time.
The truth is that Usability Testing is qualitatively different from Market Research. You are not trying to determine if people want to buy your product (which requires large sample sizes). You are trying to determine if people can use your product. For that, you need a much smaller group.
The Magic Number: 5
Jakob Nielsen, a principal of the Nielsen Norman Group and a pioneer of usability engineering, proved decades ago that testing with just 5 users is enough to uncover 85% of usability problems.
The Insight: You learn a lot from the first user. You learn a little less from the second. By the time you get to the sixth user, you are mostly just seeing the same problems over and over again.
The Curve of Diminishing Returns
Imagine you are testing a website where the "Checkout" button is broken.
- User 1 tries to click it and fails. You now know there is a bug.
- User 2 tries to click it and fails. You confirm the bug is not a fluke.
- User 3 tries to click it and fails. You are now very bored of watching people fail to click this button.
If you test with 10 more people, you will not learn anything new about that broken button. You are just wasting money. This is the curve of diminishing returns.
Zero Users: You know nothing. Your design is a guess.
One User: You gain an infinite increase in knowledge. You will likely find a third of all major issues.
Five Users: You have found the vast majority of issues. The cost of recruiting more users is now higher than the value of the new insights you might find.
Why Iteration Wins
Let's say you have the budget to test 15 users.
The Bad Strategy: Run one massive test with all 15 users at once. You will find all the obvious bugs, but you will also waste time watching 15 people struggle with the same login error.
The Winning Strategy: Run three separate tests with 5 users each.
- Test 1 (5 Users): Find the glaring issues (like the broken login). Stop testing and fix them.
- Test 2 (5 Users): Now that the login works, users can get deeper into the app. They will find new issues in the dashboard that the first group never even saw. Fix those.
- Test 3 (5 Users): Now you are polishing. You catch the subtle issues that were hidden behind the bigger problems.
When Do You Need More?
There is one major exception to the "5 User Rule."
If your product has distinct groups of users who behave very differently, you need to test 5 users from each group.
For example, if you are building a medical app used by both Doctors and Patients, you cannot just test 5 people total. You need to test 5 Doctors (to check the medical interface) and 5 Patients (to check the booking interface). Their behaviors and needs are completely different, so the overlap in their results will be small.
Conclusion
Do not let the pursuit of a "perfect" study stop you from doing a "good" one.
Testing with one person is better than testing with zero. Testing with five is better than testing with fifty, provided you use the saved budget to fix the problems and test again.
