skip to main content
10.1145/1028014.1028016acmotherconferencesArticle/Chapter ViewAbstractPublication PagesnordichiConference Proceedingsconference-collections
Article

Two psychology-based usability inspection techniques studied in a diary experiment

Published: 23 October 2004 Publication History

Abstract

Inspection techniques are widely used during systems design as a supplement to empirical evaluations of usability. Psychology-based inspection techniques could give important insights into how thinking shapes interaction, yet most inspection techniques do not explicitly consider users' thinking. We present an experiment comparing two psychology-based inspection techniques, cognitive walkthrough (CW) and metaphors of human thinking (MOT). Twenty participants evaluated web sites for e-commerce while keeping diaries of insights and problems experienced with the techniques. Using MOT, participants identified 30% more usability problems and in a reference collection of problems achieved a broader coverage. Participants preferred using the metaphors, finding them broader in scope. An analysis of the diaries shows that participants find it hard to understand MOT, while CW limits the scope of their search for usability problems. Participants identified problems in many ways, not only through the techniques, reflecting large differences in individual working styles.

References

[1]
Bansler, J. & Bøødker, K. A. reappraisal of structured analysis: design in an organizational context. ACM Transactions on Information Systems, 11, 2 (1993), 165--193.
[2]
Blackmon, M., Polson, P., Kitajima, M., & Lewis, C. Cognitive walkthrough for the web. Proc. CHI 2002, CHI Letters 4(1), 463--470.
[3]
Dreyfus, H. & Dreyfus, S. Mind Over Machine, The Free Press, New York, NY, 1986.
[4]
Frøkjær, E. & Hornbææk, K. Metaphors of human thinking in HCI: Habit, stream of thought, awareness, utterance, and knowing. Proc. HF/OzCHI 2002.
[5]
Furnas, G., Landauer, T., Gomez, L., & Dumais, S. The vocabulary problem in human-system communication. Communications of the ACM, 30, 11 (1987), 964--971.
[6]
Hornbæk, K. & Frøøkjær, E. Evaluating user interfaces with metaphors of human thinking. Proc. User Interfaces for All, Lecture Notes in Computer Science 2615, Springer-Verlag (2002), 486--507.
[7]
Hornbæk, K. & Frøøkjær, E. Usability Inspection by Metaphors of Human Thinking Compared to Heuristic Evaluation. To appear in International Journal of Human-Computer Interaction (2004).
[8]
Jacobsen, N. & John, B. Two case studies in using cognitive walkthroughs for interface evaluation. Technical report CMU-CS-00-132 (2000).
[9]
James, W. The Principles of Psychology. Henry Holt & Co., 1890.
[10]
Jefferies, R., Miller, J., Wharton, C. & Uyeda, K. User interface evaluation in the real world. Proc. CHI'91, ACM Press (1991), 119--124.
[11]
John, B. & Packer, H. Learning and using the cognitive walkthrough method: a case study approach. Proc. CHI'95, ACM Press (1995), 429--436.
[12]
Lewis, C., Polson, P., Wharton, C., & Rieman, J. Testing a walkthrough methodology for theory-based design of walk-up-and-use interfaces. Proc. CHI'90, ACM Press (1990), 235--242.
[13]
Molich, R. Brugervenlige Edb-Systemer (in Danish). Teknisk Forlag, 1994.
[14]
Naur P. Program development studies based on diaries. Green, T., Payne, S., & van der Veer, G. Psychology of Computer Use. Academic Press, 1983, 159--170.
[15]
Naur P. Intuition in software development. Ehrig, H., Floyd, C., Nivat, M., & Thatcher, J. Formal Methods and Software Development, Vol. 2, Lecture Notes in Computer Science 186, Springer Verlag, 1985, 60--79.
[16]
Naur, P. Knowing and the Mystique of Logic and Rules. Kluwer Academic Publishers, Dordrecht, 1995.
[17]
Nielsen, J. & Mack, R. L. Usability Inspection Methods. Wiley and Sons Inc., 1994.
[18]
Nielsen, J. & Molich, R. Heuristic evaluation of user interfaces. Proc. CHI'90, ACM Press (1990), 249--256.
[19]
Nielsen, J., Molich, R., Snyder, C., & Farrell, S. E-Commerce User Experience. Nielsen Norman Group, 2001.
[20]
Norman D. Some observations on mental models. Gentner, D. & Stevens, A. Mental Models, Erlbaum, 1983, 7--14.
[21]
Palen, L. & Salzman, M. Voice-mail diary studies for naturalistic data capture under mobile conditions. Proc. CSCW 2002, CHI Letters 4(3), 87--95.
[22]
Pinelle, D. & Gutwin, C. Groupware walkthrough: adding context to groupware usability evaluation. Proc. CHI 2002, CHI Letters 4(1), 455--462.
[23]
Raskin, J. The Humane Interface: New Directions for Designing Interactive Systems. Addison-Wesley, 2000.
[24]
Rieman, J. A field study of exploratory learning strategies. ACM Transactions on Computer-Human Interaction, 3, 3 (1996), 189--218.
[25]
Sellen, A. & Harper, R. Paper as an analytic resource for the design of new technologies, Proc. CHI'97, ACM Press (1997), 319--326.
[26]
Smith, S. L. & Mosier, J. N. Guidelines for designing user interface software, ESD-TR-86-278 (1986).
[27]
Somberg, B. L. A. comparison of rule-based and positionally constant arrangements of computer menu items. Proc. CHI+GI'87, ACM Press (1987), 255--260.
[28]
Spencer, R. The streamlined cognitive walkthrough method, working around social constraints encountered in a software development company. Proc. CHI 2000, CHI Letters 2(1), 353--359.
[29]
Toms, E. G. & Duff, W. "I spent 1 1/2 hours sifting through one large box. ..": diaries as information behavior of the archives user: lessons learned. Journal of the American Society for Information Science and Technology, 53, 14 (2002), 1232--1238.
[30]
Wharton C., Rieman, J., Lewis, C. & Polson, P. The cognitive walkthrough method: a practitioner's guide. Nielsen, J. & Mack, R. L. Usability Inspection Methods. John Wiley & Sons, 1994, 105--140.

Cited By

View all
  • (2020)MagellanCommunications of the ACM10.1145/340547663:8(83-91)Online publication date: 22-Jul-2020
  • (2020)The Impact of Thinking-Aloud on Usability InspectionProceedings of the ACM on Human-Computer Interaction10.1145/33978764:EICS(1-22)Online publication date: 18-Jun-2020
  • (2020)Digital creativity support for original journalismCommunications of the ACM10.1145/338652663:8(46-53)Online publication date: 22-Jul-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
NordiCHI '04: Proceedings of the third Nordic conference on Human-computer interaction
October 2004
472 pages
ISBN:1581138571
DOI:10.1145/1028014
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2004

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. cognitive walkthrough
  2. individual differences
  3. inspection techniques
  4. metaphors of human thinking
  5. psychology
  6. usability evaluation techniques

Qualifiers

  • Article

Conference

NordiCHI04
NordiCHI04: NordiCHI 2004
October 23 - 27, 2004
Tampere, Finland

Acceptance Rates

Overall Acceptance Rate 379 of 1,572 submissions, 24%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)8
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2020)MagellanCommunications of the ACM10.1145/340547663:8(83-91)Online publication date: 22-Jul-2020
  • (2020)The Impact of Thinking-Aloud on Usability InspectionProceedings of the ACM on Human-Computer Interaction10.1145/33978764:EICS(1-22)Online publication date: 18-Jun-2020
  • (2020)Digital creativity support for original journalismCommunications of the ACM10.1145/338652663:8(46-53)Online publication date: 22-Jul-2020
  • (2020)Why computing belongs within the social sciencesCommunications of the ACM10.1145/338344463:8(54-59)Online publication date: 22-Jul-2020
  • (2020)Examining undergraduate computer science participation in North CarolinaCommunications of the ACM10.1145/337212263:8(60-68)Online publication date: 22-Jul-2020
  • (2020)Threats of a replication crisis in empirical computer scienceCommunications of the ACM10.1145/336031163:8(70-79)Online publication date: 22-Jul-2020
  • (2016)Knuckle Print Biometrics and Fusion Schemes -- Overview, Challenges, and SolutionsACM Computing Surveys10.1145/293872749:2(1-46)Online publication date: 11-Nov-2016
  • (2015)Data-Driven Retrospective Interviewing (DDRI): A proposed methodology for formative evaluation of pervasive gamesEntertainment Computing10.1016/j.entcom.2015.07.00211(1-19)Online publication date: Nov-2015
  • (2014)Neighbor Selection and Weighting in User-Based Collaborative FilteringACM Transactions on the Web10.1145/25799938:2(1-30)Online publication date: 1-Mar-2014
  • (2014)Textual and Content-Based Search in Repositories of Web Application ModelsACM Transactions on the Web10.1145/25799918:2(1-47)Online publication date: 1-Mar-2014
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media