Heuristic evaluation of ambient displays

ABSTRACT

We present a technique for evaluating the usability and effectiveness of ambient displays. Ambient displays are abstract and aesthetic peripheral displays portraying non-critical information on the periphery of a user's attention. Although many innovative displays have been published, little existing work has focused on their evaluation, in part because evaluation of ambient displays is difficult and costly. We adapted a low-cost evaluation technique, heuristic evaluation, for use with ambient displays. With the help of ambient display designers, we defined a modified set of heuristics. We compared the performance of Nielsen's heuristics and our heuristics on two ambient displays. Evaluators using our heuristics found more, severe problems than evaluators using Nielsen's heuristics. Additionally, when using our heuristics, 3-5 evaluators were able to identify 40--60% of known usability issues. This implies that heuristic evaluation is an effective technique for identifying usability issues with ambient displays.

REFERENCES

• 1: Kevin Baker , Saul Greenberg , Carl Gutwin, Empirical development of a heuristic evaluation methodology for shared workspace groupware, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA [doi>10.1145/587078.587093]
• 2: C. Chafe and G. Niemeyer. Oxygen flute. Installation by Artists, San Jose Museum of Art, October 13-2001 - June, 2002.
• 3: Angela Chang , Ben Resner , Brad Koerner , XingChen Wang , Hiroshi Ishii, LumiTouch: an emotional communication device, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634252]
• 4: C. Chewar and D. S. McCrickard. Adapting UEMS for notification systems. Accepted to Ubicomp 2002 Workshop 9, September 2002.
• 5: W. D. Gray and M. C. Salzman. Damaged merchandise? A review of experiments that compare usability evaluation methods. Human-Computer Interaction, 13(3):203--261, 1998.
• 6: Lars Hallnäs , Johan Redström, Slow Technology – Designing for Reflection, Personal and Ubiquitous Computing, v.5 n.3, p.201-212, August 2001 [doi>10.1007/PL00000019]
• 7: Jeremy M. Heiner , Scott E. Hudson , Kenichiro Tanaka, The information percolator: ambient information display in a decorative object, Proceedings of the 12th annual ACM symposium on User interface software and technology, p.141-148, November 07-10, 1999, Asheville, North Carolina, United States [doi>10.1145/320719.322595]
• 8: Hiroshi Ishii , Sandia Ren , Phil Frei, Pinwheels: visualizing information flow in an architectural space, CHI '01 extended abstracts on Human factors in computing systems, March 31-April 05, 2001, Seattle, Washington [doi>10.1145/634067.634135]
• 9: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 10: Craig Wisneski , Hiroshi Ishii , Andrew Dahley , Matthew G. Gorbet , Scott Brave , Brygg Ullmer , Paul Yarin, Ambient Displays: Turning Architectural Space into an Interface between People and Digital Information, Proceedings of the First International Workshop on Cooperative Buildings, Integrating Information, Organization, and Architecture, p.22-32, February 01, 1998
• 11: E. H. M. van Dantzich, et al. Scope: Providing awareness of multiple notifications at a glance. In Proc. of AVI 2002. ACM Press, 2002.
• 12: Blair MacIntyre , Elizabeth D. Mynatt , Stephen Voida , Klaus M. Hansen , Joe Tullio , Gregory M. Corso, Support for multitasking and background awareness using interactive peripheral displays, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502355]
• 13: Elizabeth D. Mynatt , Maribeth Back , Roy Want , Michael Baer , Jason B. Ellis, Designing audio aura, Proceedings of the SIGCHI conference on Human factors in computing systems, p.566-573, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274720]
• 14: Elizabeth D. Mynatt , Jim Rowan , Sarah Craighill , Annie Jacobs, Digital family portraits: supporting peace of mind for extended family members, Proceedings of the SIGCHI conference on Human factors in computing systems, p.333-340, March 2001, Seattle, Washington, United States [doi>10.1145/365024.365126]
• 15: J. Nielsen. Ten usability heuristics. http://www.useit.com/papers/heuristic/heuristic_list.html, accessed on August 20, 2002.
• 16: Jakob Nielsen, Noncommand user interfaces, Communications of the ACM, v.36 n.4, p.83-99, April 1993 [doi>10.1145/255950.153582]
• 17: Jakob Nielsen , Thomas K. Landauer, A mathematical model of the finding of usability problems, Proceedings of the SIGCHI conference on Human factors in computing systems, p.206-213, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169166]
• 18: Jakob Nielsen , Rolf Molich, Heuristic evaluation of user interfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.249-256, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97281]
• 19: J. Nielsen and the Nielsen Norman Group. Internet: http://www.useit.com/papers/heuristic/, 2002.
• 20: David Tessendorf , C. M. Chewar , Ali Ndiwalana , Jon Pryor , D. Scott McCrickard , Chris North, An ordering of secondary task display attributes, CHI '02 extended abstracts on Human factors in computing systems, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/506443.506503]
• 21: Karel Vredenburg , Ji-Ye Mao , Paul W. Smith , Tom Carey, A survey of user-centered design practice, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503460]
• 22: M. Weiser and J. S. Brown. Designing calm technology. Internet: http://www.ubiq.com/weiser/calmtech/calmtech.htm, 1995.
• 23: Tom White , David Small, An interactive poetic garden, CHI 98 conference summary on Human factors in computing systems, p.335-336, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/286498.286804]
• 24: A. Woolrych and G. Cockton. Why and when five test users aren't enough. In Proceedings of IHM-HCI 2001 Conference, volume 2, pp. 105--108, 2001.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.2 Design Tools and Techniques
          Subjects: User interfaces

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Evaluation/methodology

General Terms:
Design, Experimentation, Human Factors

Keywords:
ambient displays, heuristic evaluation