Finding usability problems through heuristic evaluation

ABSTRACT

Usability specialists were better than non-specialists at performing heuristic evaluation, and “double experts” with specific expertise in the kind of interface being evaluated performed even better. Major usability problems have a higher probability than minor problems of being found in a heuristic evaluation, but more minor problems are found in absolute numbers. Usability heuristics relating to exits and user errors were more difficult to apply than the rest, and additional measures should be taken to find problems relating to these heuristics. Usability problems that relate to missing interface elements that ought to be introduced were more difficult to find by heuristic evaluation in interfaces implemented as paper prototypes but were as easy as other problems to find in running systems.

References

1. 1.Bias, R. Walkthroughs: Efficient collaborative testing. IEEE Software 8, 5 (September 1991), 94-95.
2. 2.Carroll, J.M. Infinite detail and emulation in an ontologically minimized HCI. Proc. ACM CHI'90 (Seattle, WA, 1-5 April 1990), 321-327.
3. 3.Carroll, .I.M., Kellogg, W.A., and Rosson, M.B. The task-artifact cycle. In Carroll, J.M. (Ed.), Designing Interaction: Psychology at the Human-Computer Interface. Cambridge University Press, Cambridge, U.K., 1991.74-102.
4. 4.D esurvire, H., Lawrence, D., and Atwood, M. Empiricism versus judgement: Comparing user interface evaluation methods on a new telephone-based interface. A CM S1GCHI Bulletin 23, 4 (October 1991), 58-59.
5. 5.Engelbeck, G., and Roberts, T.L. The effect of several voicemenu characteristics on menu selection performance. Behaviour & Information Technology in press.
6. 6.Gould, J.D., and Boles, S.J. Speech filingmAn office system for principals. IBM Systems Journal 23, I (1984), 65-81.
7. 7.Halstead-Nussloch, R. The design of phone-based interfaces for consumers. Proc. ACM CHI'89 (Austin, TX, 30 April--4 May 1989), 347-352.
8. 8.Jeffries, R., Miller, J.R., Wharton, C., and Uyeda, K.M. User interface evaluation in the real world: A comparison of four techniques. Proc. ACM CHI'91 (New Orleans, LA, 27 April- 2 May 1991), 119-124.
9. 9.Karat, C.-M., Campbell, R., Fiegel, T. Comparisons of empirical testing and walkthrough methods in user interface evaluation. Proc. ACM CHI'92 (Monterey, CA, 3-7 May 1992).
10. 10.Kellogg, W.A. Qualitative artifact analysis. Proc. INTER- ACT' 90 3rd IFIP Conf. Human-Computer Interaction (Cambridge, U.K., 27-31 August 1990), 193-198.
11. 11.Lewis, C., Poison, P., Wharton, C., and Rieman, J. Testing a walkthrough methodology for theory-based design of walkup-and-use interfaces. Proc. ACM CHI'90 (Seattle, WA, 1-5 April 1990), 235-241.
12. 12.Molich, R., and Nielsen, J. Improving a human-computer dialogue. Communications of the ACM 33, 3 (March 1990), 338- 348.
13. 13.Nielsen, J. Usability engineering at a discount. In Salvendy, G., and Smith, M.J. (Eds.), Designing and Using Human- Computer interfaces and Knowledge Based Systems, Elsevier Science Publishers, Amsterdam, 1989. 394-401.
14. 14.Nielsen, J. Paper versus computer implementations as mockup scenarios for heuristic evaluation. Proc. INTER- ACT" 90 3rd IFIP Conf. Human-Computer interaction (Cambridge, U.K., 27-31 August 1990), 315-320.
15. 15.Nielsen, J. Applying heuristic evaluation to a highly domainspecific interface. Manuscript submitted for publication.
16. 16.Nielsen, j. Usability Engineering. Academic Press, San Diego, CA, 1992.
17. 17.Nielsen, J., and Molich, R. Heuristic evaluation of user interfaces. Proc. ACM CHI'90 (Seattle, WA, 1-5 April 1990), 249-256.
18. 18.Teitelbaum, R.C., and Granda, R.E. The effects of positional constancy on searching menus for information. Proc. ACM CHI'83 (Boston, MA, 12-15 December 1983), 150-153.
19. 19.Wharton, C., Bradford, J., Jeffries, R., and Franzke, M. Applying cognitive walkthroughs to more complex interfaces: Experiences, issues, and recommendations. Proc. ACM CHI' 92 (Monterey, CA, 3-7 May 1992).

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

    • Interaction devices

      • Touch screens

    • Interaction paradigms