The vocal joystick:: evaluation of voice-based cursor control techniques

ABSTRACT

Mouse control has become a crucial aspect of many modern day computer interactions. This poses a challenge for individuals with motor impairments or those whose use of hands are restricted due to situational constraints. We present a system called the Vocal Joystick which allows the user to continuously control the mouse cursor by varying vocal parameters such as vowel quality, loudness and pitch. A survey of existing cursor control methods is presented to highlight the key characteristics of the Vocal Joystick. Evaluations were conducted to characterize expert performance capability of the Vocal Joystick, and to compare novice user performance and preference for the Vocal Joystick and two other existing speech based cursor control methods. Our results show that Fitts' law is a good predictor of the speedaccuracy tradeoff for the Vocal Joystick, and suggests that the optimal performance of the Vocal Joystick may be comparable to that of a conventional hand-operated joystick. Novice user evaluations show that the Vocal Joystick can be used by people without extensive training, and that it presents a viable alternative to existing speech-based cursor control methods.

REFERENCES

• 1: J. A. Bilmes, X. Li, J. Malkin, K. Kilanski, R. Wright, K. Kirchhoff, A. Subramanya, S. Harada, J. A. Landay, P. Dowden, and H. Chizeck. The Vocal Joystick: A voice-based human-computer interface for individuals with motor impairments. In Human Language Technology Conf./Conf. on Empirical Methods in Natural Language Processing, October 2005.
• 2: S. K. Card , W. K. English , B. J. Burr, Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys, for text selection on a CRT, Human-computer interaction: a multidisciplinary approach, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1987
• 3: Thomas M. Cover , Joy A. Thomas, Elements of information theory, Wiley-Interscience, New York, NY, 1991
• 4: Liwei Dai , Rich Goldman , Andrew Sears , Jeremy Lozier, Speech-based cursor control: a study of grid-based solutions, ACM SIGACCESS Accessibility and Computing
• 5: C. de Mauro, M. Gori, M. Maggini, and E. Martinelli. Easy access to graphical interfaces by voice mouse. Technical report, Universit`a di Siena, 2001. Available from the author at: [e-mail].
• 6: B. Epps. Comparison of six cursor control devices based on Fitts' law models. In Proceedings of the 30th Annual Meeting of the Human Factors Society, pages 327--331. Human Factors Society, 1986.
• 7: P. M. Fitts. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47:381--391, 1954.
• 8: Anthony J. Hornof , Anna Cavender, EyeDraw: enabling children with severe motor impairments to draw with their eyes, Proceedings of the SIGCHI conference on Human factors in computing systems, April 02-07, 2005, Portland, Oregon, USA [doi>10.1145/1054972.1054995]
• 9: Takeo Igarashi , John F. Hughes, Voice as sound: using non-verbal voice input for interactive control, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502372]
• 10: Robert J. K. Jacob, What you look at is what you get: eye movement-based interaction techniques, Proceedings of the SIGCHI conference on Human factors in computing systems: Empowering people, p.11-18, April 01-05, 1990, Seattle, Washington, United States [doi>10.1145/97243.97246]
• 11: Hesham M. Kamel , James A. Landay, Sketching images eyes-free: a grid-based dynamic drawing tool for the blind, Proceedings of the fifth international ACM conference on Assistive technologies, July 08-10, 2002, Edinburgh, Scotland [doi>10.1145/638249.638258]
• 12: . S. Mackenzie. Fitts' law as a research and design tool in human-computer interaction. Human-Computer Interaction, 7:91--139, 1992.
• 13: J. Malkin, X. Li, and J. Bilmes. Energy and loudness for speed control in the Vocal Joystick. In IEEE Automatic Speech Recognition and Understanding Workshop, November 2005.
• 14: Bill Manaris , Renée McCauley , Valanne MacGyvers, An Intelligent Interface for Keyboard and Mouse Control -- Providing Full Access to PC Functionality via Speech, Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference, p.182-188, May 21-23, 2001
• 15: Yoshiyuki Mihara , Etsuya Shibayama , Shin Takahashi, The migratory cursor: accurate speech-based cursor movement by moving multiple ghost cursors using non-verbal vocalizations, Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, October 09-12, 2005, Baltimore, MD, USA [doi>10.1145/1090785.1090801]
• 16: Darius Miniotas, Application of Fitts' law to eye gaze interaction, CHI '00 extended abstracts on Human factors in computing systems, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/633292.633496]
• 17: Robert G. Radwin , Gregg C. Vanderheiden , Mei-Li Lin, A method for evaluating head-controlled computer input devices using Fitts law, Human Factors, v.32 n.4, p.423-438, Aug. 1990
• 18: Andrew Sears , Mark Young, Physical disabilities and computing technologies: an analysis of impairments, The human-computer interaction handbook: fundamentals, evolving technologies and emerging applications, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 2002
• 19: Claude E. Shannon , Warren Weaver, A Mathematical Theory of Communication, University of Illinois Press, Champaign, IL, 1963

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Voice I/O

Additional Classification:
  K. Computing Milieux
  K.4 COMPUTERS AND SOCIETY
      K.4.2 Social Issues
          Subjects: Assistive technologies for persons with disabilities

General Terms:
Human Factors, Measurement, Performance

Keywords:
Fitts' law, continuous input, cursor control, speech recognition, voice-based interface