skip to main content
10.1145/1054972.1055027acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
Article

Comparing usability problems and redesign proposals as input to practical systems development

Published: 02 April 2005 Publication History

Abstract

Usability problems predicted by evaluation techniques are useful input to systems development; it is uncertain whether redesign proposals aimed at alleviating those problems are likewise useful. We present a study of how developers of a large web application assess usability problems and redesign proposals as input to their systems development. Problems and redesign proposals were generated by 43 evaluators using an inspection technique and think aloud testing. Developers assessed redesign proposals to have higher utility in their work than usability problems. In interviews they explained how redesign proposals gave them new ideas for tackling well known problems. Redesign proposals were also seen as constructive and concrete input. Few usability problems were new to developers, but the problems supported prioritizing ongoing development of the application and taking design decisions. No developers, however, wanted to receive only problems or redesigns. We suggest developing and using redesign proposals as an integral part of usability evaluation.

References

[1]
Bailey, R. W., Allan, R. W., & Raiello, P. Usability Testing Vs. Heuristic Evaluation: a Head-to-Head Comparison, Proc. Human Factors Society 36th Annual Meeting, (1992), 409--413.
[2]
Cockton G., Lavery, D., & Woolrych, A. Inspection-Based Evaluations, in Jacko, J. A. & Sears, A. The Human-Computer Interaction Handbook, Lawrence Erlbaum Associates, 2003, 1118--1138.
[3]
Cockton, G., Woolrych, A., Hall, L., & Hidemarch, M. Changing Analysts' Tunes: The Surprising Impact of a New Instrument for Usability Inspection Method Assessment, Proc. HCI 2003, Springer Verlag (2003), 145--162.
[4]
Doubleday, A., Ryan, A., & Sutcliffe, A. A Comparison of Usability Techniques for Evaluating Design, Proc. DIS'97, ACM Press (1997), 101--110.
[5]
Dumas, J., Molich, R., & Jefferies, R. Describing Usability Problems: Are We Sending the Right Message?, interactions, 4 (2004), 24--29.
[6]
Dumas J. User-Based Evaluations, in Jacko, J. A. & Sears, A. The Human-Computer Interaction Handbook, Lawrence Erlbaum Associates, 2003, 1093--1117.
[7]
Dutt, A., Johnson, H., & Johnson, P. Evaluating Evaluation Methods, Proc. HCI 1994, Cambridge University Press (1994), 109--121.
[8]
Frøkjær, E. & Hornbææk, K. Metaphors of Human Thinking in HCI: Habit, Stream of Thought, Awareness, Utterance, and Knowing, Proc. HF/OzCHI 2002 (2002).
[9]
Fu, L. & Salvendy, G. Effectiveness of User-Testing and Heuristic Evaluation as a Function of Performance Classification, Behaviour and Information Technology, 21, 2 (2002), 137--143.
[10]
Gray, W. D. & Salzman, M. C. Damaged Merchandise? A Review of Experiments That Compare Usability Evaluation Methods, Human-Computer Interaction, 13, 3 (1998), 203--261.
[11]
Hartson, H. R., Andre, T. S., & Williges, R. C. Criteria for Evaluating Usability Evaluation Methods, International Journal of Human-Computer Interaction, 13, 4 (2001), 373--410.
[12]
Helms Jørgensen, A. Thinking-Aloud in User Interface Design: a Method Promoting Cognitive Ergonomics, Ergonomics, 33, 4 (1990), 501--507.
[13]
Hornbæk, K. & Frøøkjær, E. Evaluating User Interfaces with Metaphors of Human Thinking, Lecture Notes in Computer Science 2615, Springer (2002), 486--507.
[14]
Hornbæk, K. & Frøøkjær, E. Two Psychology-Based Usability Inspection Techniques Studied in a Diary Experiment, Proc. NordiCHI 2004, ACM Press (2004).
[15]
Hornbæk, K. & Frøøkjær, E. Usability Inspection by Metaphors of Human Thinking Compared to Heuristic Evaluation, International Journal of Human-Computer Interaction, 17, 3 (2004), 357--374.
[16]
Jacobsen, N. E. & John, B. E. Two Case Studies in Using Cognitive Walkthroughs for Interface Evaluation, CMU-CS-00-132 (2000).
[17]
Jeffries, R., Miller, J., Wharton, C., & Uyeda, K. User Interface Evaluation in the Real World: A Comparison of Four Techniques, Proc. CHI'91, (1991), 119--124.
[18]
Jeffries R., Usability Problem Reports: Helping Evaluators Communicate Effectively With Developers, in Nielsen, J. & Mack, R. L. Usability Inspection Methods, John Wiley, 1994, 273--294.
[19]
John, B. E. & Mashyna, M. M. Evaluating a Multimedia Authoring Tool With Cognitive Walkthrough and Think-Aloud User Studies, CMU-HCII-95-105 / CMU-CS-95-189 (1995).
[20]
John, B. E. & Marks, S. J. Tracking the Effectiveness of Usability Evaluation Methods, Behaviour and Information Technology, 16, 4/5 (1997), 188--202.
[21]
Johnson H., Generating User Requirements From Discount Usability Evaluations, in Harris, D. Engineering Psychology and Cognitive Ergonomics, Vol. 2, Ashgate Publishing, 1997, 339--357.
[22]
Karat, C.-M., Campbell, R., & Fiegel, T. Comparison of Empirical Testing and Walkthrough Methods in Usability Interface Evaluation, Proc. CHI'92, ACM Press (1992), 397--404.
[23]
Lavery, D., Cockton, G., & Atkinson, M. P. Comparison of Evaluation Methods Using Structured Usability Problem Reports, Behaviour and Information Technology, 16, 4/5 (1997), 246--266.
[24]
Lewis, C. Using the "Thinking-Aloud" Method in Cognitive Interface Design, Research Report RC9265 (1982).
[25]
Mack R. L. & Montaniz, F., Observing, Predicting, and Analyzing Usability Problems, in Nielsen, J. & Mack, R. L. Usability Inspection Methods, John Wiley and sons, 1994, 295--339.
[26]
Molich, R. Brugervenlige Edb-Systemer (in Danish), Teknisk Forlag, 1994.
[27]
Molich, Rolf, Comparative Usability Evaluation, 2003, www.dialogdesign.dk/cue.html.
[28]
Molich, Rolf, User testing, Discount user testing, 2003, www.dialogdesign.dk.
[29]
Molich, R., Ede, M. R., Kaasgaard, K., & Karyukin, B. Comparative Usability Evaluation, Behaviour and Information Technology, 23, 1 (2004), 65--74.
[30]
Molich, R. & Nielsen, J. Improving a Human-Computer Dialogue, Communications of the ACM, 33, 3 (1990), 338--348.
[31]
Muller, M. J. & McClard, A. Validating an Extension to Participatory Heuristic Evaluation: Quality of Work and Quality of Work Life, Proc. CHI'95, ACM Press (1995), 115--116.
[32]
Nielsen, J. & Molich, R. Heuristic Evaluation of User Interfaces, Proc. CHI'90, ACM Press (1990), 249--256.
[33]
Rosenbaum, S., Rohn, J., & Humberg, J. A Toolkit for Strategic Usability: Results from Workshops, Panels, and Surveys, Proc. CHI 2000, ACM Press (2000), 337--344.
[34]
Rubin, J. Handbook of Usability Testing, John Wiley & Sons, New York, NY, 1994.
[35]
Sawyer, P., Flanders, A., & Wixon, D. Making a Difference - The Impact of Inspections, Proc. CHI'96, ACM Press (1996), 376--382.
[36]
Sears, A. & Hess, D. Cognitive Walkthroughs: Understanding the Effect of Task Description Detail on Evaluator Performance, International Journal of Human-Computer Interaction, 11 (1999), 185--200.
[37]
Smith, A. & Dunckley, L. Prototype Evaluation and Redesign: Structuring the Design Space Through Contextual Techniques, Interacting with Computers, 14 (2002), 821--843.
[38]
Vredenburg, K., Mao, J.-Y., Smith, P. W., & Carey, T. A Survey of User-Centered Design Practice, Proc. CHI 2002, ACM Press (2002), 472--478.
[39]
Wixon, D. Evaluating Usability Methods: Why the Current Literature Fails the Practitioner, interactions, 10, 4 (2003), 29--34.

Cited By

View all
  • (2023)Usability testing and redesign of a mobile application for real estateProceedings of the 2nd International Conference of the ACM Greek SIGCHI Chapter10.1145/3609987.3609996(1-6)Online publication date: 27-Sep-2023
  • (2020)Usability Testing: A Practitioner's Guide to Evaluating the User ExperienceSynthesis Lectures on Human-Centered Informatics10.2200/S00987ED1V01Y202001HCI0451:1(i-105)Online publication date: 9-Mar-2020
  • (2020)mHealth Authentication Approach Based 3D Touchscreen and Microphone Sensors for Real-Time Remote Healthcare Monitoring System: Comprehensive Review, Open Issues and Methodological AspectsComputer Science Review10.1016/j.cosrev.2020.10030038(100300)Online publication date: Nov-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '05: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
April 2005
928 pages
ISBN:1581139985
DOI:10.1145/1054972
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 April 2005

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. empirical study
  2. metaphors of human thinking
  3. redesign
  4. think aloud
  5. usability evaluation
  6. usability inspection

Qualifiers

  • Article

Conference

CHI05
Sponsor:

Acceptance Rates

CHI '05 Paper Acceptance Rate 93 of 372 submissions, 25%;
Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)22
  • Downloads (Last 6 weeks)1
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Usability testing and redesign of a mobile application for real estateProceedings of the 2nd International Conference of the ACM Greek SIGCHI Chapter10.1145/3609987.3609996(1-6)Online publication date: 27-Sep-2023
  • (2020)Usability Testing: A Practitioner's Guide to Evaluating the User ExperienceSynthesis Lectures on Human-Centered Informatics10.2200/S00987ED1V01Y202001HCI0451:1(i-105)Online publication date: 9-Mar-2020
  • (2020)mHealth Authentication Approach Based 3D Touchscreen and Microphone Sensors for Real-Time Remote Healthcare Monitoring System: Comprehensive Review, Open Issues and Methodological AspectsComputer Science Review10.1016/j.cosrev.2020.10030038(100300)Online publication date: Nov-2020
  • (2019) Usability Testing Online Questionnaires: Experiences at the U . S . C ensus B ureau Advances in Questionnaire Design, Development, Evaluation and Testing10.1002/9781119263685.ch13(315-348)Online publication date: 18-Dec-2019
  • (2018)User Evaluations of Virtually Experiencing Mount EverestHuman-Centered Software Engineering10.1007/978-3-030-05909-5_18(294-307)Online publication date: 31-Dec-2018
  • (2016)The trade-off between usability and security in the context of eGovernmentProceedings of the 30th International BCS Human Computer Interaction Conference: Fusion!10.14236/ewic/HCI2016.36(1-13)Online publication date: 11-Jul-2016
  • (2016)Facilitating redesign with design cardsProceedings of the 28th Australian Conference on Computer-Human Interaction10.1145/3010915.3010921(452-461)Online publication date: 29-Nov-2016
  • (2016)Reporting usability defectsProceedings of the 20th International Conference on Evaluation and Assessment in Software Engineering10.1145/2915970.2915995(1-10)Online publication date: 1-Jun-2016
  • (2016)Using the Kano Model to Balance Delight and Frustration for an Enterprise ApplicationProceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems10.1145/2851581.2892284(3021-3027)Online publication date: 7-May-2016
  • (2016)What Influences Usability Defect Reporting? — A Survey of Software Development Practitioners2016 23rd Asia-Pacific Software Engineering Conference (APSEC)10.1109/APSEC.2016.014(17-24)Online publication date: 2016
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media