Beyond the five-user assumption: Benefits of increased sample sizes in usability testing

By L. Faulkner. Behavoir research methods, instruments & computers, 2003

1,046 citations recorded by the source corpus.

Subfields: Information Systems, Human-Computer Interaction

View Paper: https://www.semanticscholar.org/paper/7369703e67a6812b4476fc95f8085c4d20ebe2ff

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Information Systems

1. Qian Yang
2. J.D. Zamfirescu-Pereira
3. Ehud Reiter
4. Judith Masthoff
5. Robert Biddle
6. Sonia Chiasson
7. P. V. Oorschot
8. Bjoern Hartmann

In Human-Computer Interaction

1. Qian Yang
2. J.D. Zamfirescu-Pereira
3. Judith Masthoff
4. Robert Biddle
5. Sonia Chiasson
6. P. V. Oorschot
7. Bjoern Hartmann
8. Richmond Y. Wong

Citations in the corpus

• A Usability Study and Critique of Two Password Managers (2006, background): “Faulkner justifies that twenty users “can allow the practitioner to approach increasing levels of certaint y that high percentages of existing usability problems have been found in the testing” [9].”
• Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts (2023, methodology): “Our sample size was chosen in line with prior work around formative testing for usability [19, 43]: our goal is to explore the frst, “easy-to-fnd” problems users encounter when engaged in our task, and our experience with a pilot user suggested that problem discoverability was likely to be high.”
• Providing Adaptive Health Updates Across the Personal Social Network (2014, background): “Although there has long been discussion of the number of participants needed to conduct an effective usability test, Faulkner found that 10 users found at least 80% of usability problems (Faulkner, 2003; Woolrych & Cockton, 2001).”
• A user-centered model for designing consumer mobile health application (apps) (2016, neutral): “Ten participants were selected because past research has shown that the minimum percentage of problems identified rose from 55% to 82% and the mean percentage of problems rose from 85% to 95% when the number of users was increased from 5 to 10 [25].”
• Graphical passwords: Learning from the first twelve years (2012, background): “Recent work revisiting this suggestion highlights that this is often not enough and that in some cases, severe usability prob­lems are only discovered after running a larger group of participants [Faulkner 2003; Perfetti and Landesman 2001; Spool and Schroeder 2001].”