The vocal joystick:: evaluation of voice-based cursor control techniques

ABSTRACT

Mouse control has become a crucial aspect of many modern day computer interactions. This poses a challenge for individuals with motor impairments or those whose use of hands are restricted due to situational constraints. We present a system called the Vocal Joystick which allows the user to continuously control the mouse cursor by varying vocal parameters such as vowel quality, loudness and pitch. A survey of existing cursor control methods is presented to highlight the key characteristics of the Vocal Joystick. Evaluations were conducted to characterize expert performance capability of the Vocal Joystick, and to compare novice user performance and preference for the Vocal Joystick and two other existing speech based cursor control methods. Our results show that Fitts' law is a good predictor of the speedaccuracy tradeoff for the Vocal Joystick, and suggests that the optimal performance of the Vocal Joystick may be comparable to that of a conventional hand-operated joystick. Novice user evaluations show that the Vocal Joystick can be used by people without extensive training, and that it presents a viable alternative to existing speech-based cursor control methods.

References

1. J. A. Bilmes, X. Li, J. Malkin, K. Kilanski, R. Wright, K. Kirchhoff, A. Subramanya, S. Harada, J. A. Landay, P. Dowden, and H. Chizeck. The Vocal Joystick: A voice-based human-computer interface for individuals with motor impairments. In Human Language Technology Conf./Conf. on Empirical Methods in Natural Language Processing, October 2005.
2. S. K. Card, W. K. English, and B. J. Burr. Evaluation of mouse, rate-controlled isometric joystick, step keys, and text keys, for text selection on a crt. pages 386--392, 1987.
3. T. Cover and J. Thomas. Elements of Information Theory. Wiley, 1991.
4. L. Dai, R. Goldman, A. Sears, and J. Lozier. Speech-based cursor control: a study of grid-based solutions. SIGACCESS Access. Comput., (77-78):94--101, 2004.
5. C. de Mauro, M. Gori, M. Maggini, and E. Martinelli. Easy access to graphical interfaces by voice mouse. Technical report, Universit`a di Siena, 2001. Available from the author at: [email protected].
6. B. Epps. Comparison of six cursor control devices based on Fitts' law models. In Proceedings of the 30th Annual Meeting of the Human Factors Society, pages 327--331. Human Factors Society, 1986.
7. P. M. Fitts. The information capacity of the human motor system in controlling the amplitude of movement. Journal of Experimental Psychology, 47:381--391, 1954.
8. A. J. Hornof and A. Cavender. EyeDraw: enabling children with severe motor impairments to draw with their eyes. In CHI '05: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 161--170, New York, NY, USA, 2005. ACM Press.
9. T. Igarashi and J. F. Hughes. Voice as sound: using non-verbal voice input for interactive control. In UIST '01: Proceedings of the 14th annual ACM symposium on User interface software and technology, pages 155--156, New York, NY, USA, 2001. ACM Press.
10. R. J. K. Jacob. What you look at is what you get: eye movement-based interaction techniques. In CHI '90: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 11--18, New York, NY, USA, 1990. ACM Press.
11. H. M. Kamel and J. A. Landay. Sketching images eyes-free: a grid-based dynamic drawing tool for the blind. In Assets '02: Proceedings of the fifth international ACM conference on Assistive technologies, pages 33--40, New York, NY, USA, 2002. ACM Press.
12. . S. Mackenzie. Fitts' law as a research and design tool in human-computer interaction. Human-Computer Interaction, 7:91--139, 1992.
13. J. Malkin, X. Li, and J. Bilmes. Energy and loudness for speed control in the Vocal Joystick. In IEEE Automatic Speech Recognition and Understanding Workshop, November 2005.
14. B. Manaris, R. McCauley, and V. MacGyvers. An intelligent interface for keyboard and mouse control - providing full access to PC functionality via speech. In Proceedings of the Fourteenth International Florida Artificial Intelligence Research Society Conference, pages 182--188. AAAI Press, 2001.
15. Y. Mihara, E. Shibayama, and S. Takahashi. The migratory cursor: accurate speech-based cursor movement by moving multiple ghost cursors using non-verbal vocalizations. In Assets '05: Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, pages 76--83, New York, NY, USA, 2005. ACM Press.
16. D. Miniotas. Application of fitts' law to eye gaze interaction. In CHI '00: CHI '00 extended abstracts on Human factors in computing systems, pages 339--340, New York, NY, USA, 2000. ACM Press.
17. R. G. Radwin, G. C. Vanderheiden, and M. L. Lin. A method for evaluating head-controlled computer input devices using fitts law. Hum. Factors, 32(4):423--438, 1990.
18. A. Sears and M. Young. Physical disabilities and computing technologies: an analysis of impairments. pages 482--503, 2003.
19. C. E. Shannon and W. Weaver. The mathematical theory of communicaton. Urbana University of Illinois Press, Urbana, Illinois, 1949.

Index Terms

• Hardware

  • Communication hardware, interfaces and storage

    • Sound-based input / output

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Sound-based input / output

• Social and professional topics

  • Professional topics

    • Computing profession

      • Assistive technologies

  • User characteristics

    • People with disabilities