Investigating user tolerance for errors in vision-enabled gesture-based interactions

ABSTRACT

In this paper, we describe our investigation into user tolerance of recognition errors during hand gesture interactions with visual displays. The study is based on our proposed interaction model for investigating gesture based interactions, focusing on three elements: Interaction context, system performance and user goals. This Wizard of Oz experiment investigates how recognition system accuracy rates and task characteristics in both desktop and ubiquitous computing scenarios can influence user tolerance for gesture interactions. Results suggest that interaction context is a greater influence on user tolerance than system performance alone, where recognition error rates can potentially reach 40% before users will abandon gestures and use an alternate interaction mode in a ubiquitous computing scenario. Results also suggest that in a desktop scenario, traditional input methods are more appropriate than gestures.

REFERENCES

• 1: Michel Beaudouin-Lafon, Designing interaction, not interfaces, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989865]
• 2: Richard A. Bolt, “Put-that-there”: Voice and gesture at the graphics interface, Proceedings of the 7th annual conference on Computer graphics and interactive techniques, p.262-270, July 14-18, 1980, Seattle, Washington, United States
• 3: T. W. Butler, Computer response time and user performance., Proceedings of the SIGCHI conference on Human Factors in Computing Systems, p.58-62, December 12-15, 1983, Boston, Massachusetts, United States [doi>10.1145/800045.801581]
• 4: Xiang Cao , Ravin Balakrishnan, VisionWand: interaction techniques for large displays using a passive wand tracked in 3D, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.173-182, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964716]
• 5: James L. Crowley , Joëlle Coutaz , François Bérard, Perceptual user interfaces: things that see, Communications of the ACM, v.43 n.3, p.54-ff., March 2000 [doi>10.1145/330534.330540]
• 6: R. I. Damper , S. D. Wood, Speech versus keying in command and control applications, International Journal of Human-Computer Studies, v.42 n.3, p.289-305, March 1995 [doi>10.1006/ijhc.1995.1014 ]
• 7: Jerry Alan Fails , Dan Olsen Jr., Light widgets: interacting in every-day spaces, Proceedings of the 7th international conference on Intelligent user interfaces, January 13-16, 2002, San Francisco, California, USA [doi>10.1145/502716.502729]
• 8: William Freeman and Craig D. Weissman. Television control by hand gestures. Technical report, IEEE Intl. Wkshp. on Automatic Face and Gesture Recognition, June 1995.
• 9: Dennis F. Galletta, Raymond Henry, Scott McCoy, and Peter Polak. Web site delays: How tolerant are users? Journal of the Association for Information Systems, 5(1):1--28, 2004.
• 10: Tovi Grossman , Ravin Balakrishnan, The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1055012]
• 11: Jonathon Hare, Maria Karam, Paul Lewis, and m. c. schraefel. igesture: A platform for investigating multimodal, multimedia gesture-based interactions. Technical report, Electronics and Computer Science, University of Southampton, November 2005.
• 12: Paul Kabbash , I. Scott MacKenzie , William Buxton, Human performance using computer input devices in the preferred and non-preferred hands, Proceedings of the SIGCHI conference on Human factors in computing systems, p.474-481, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169414]
• 13: Maria Karam , m. c. schraefel, A study on the use of semaphoric gestures to support secondary task interactions, CHI '05 extended abstracts on Human factors in computing systems, April 02-07, 2005, Portland, OR, USA [doi>10.1145/1056808.1057067]
• 14: Maria Karam and m. c. schraefel. A taxonomy of gestures in human computer interactions. Technical report, Electronics and Computer Science, University of Southampton, November 2005.
• 15: I. Scott MacKenzie , Colin Ware, Lag as a determinant of human performance in interactive systems, Proceedings of the SIGCHI conference on Human factors in computing systems, p.488-493, April 24-29, 1993, Amsterdam, The Netherlands [doi>10.1145/169059.169431]
• 16: D. S. McCrickard and C. M. Chewar. Cognitive Systems: Human Cognitive Models in Systems Design., chapter Designing Attention-Centric Notification Systems: Five HCI Challenges, pages 67--89. Lawrence Earlbaum, 2005.
• 17: Brad A. Myers , Rishi Bhatnagar , Jeffrey Nichols , Choon Hong Peck , Dave Kong , Robert Miller , A. Chris Long, Interacting at a distance: measuring the performance of laser pointers and other devices, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503383]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Keywords:
Wizard of Oz, gestures, secondary tasks, semaphoric gestures, ubiquitous computing