skip to main content
10.1145/1344471.1344492acmconferencesArticle/Chapter ViewAbstractPublication PagesetraConference Proceedingsconference-collections
poster

Integrated speech and gaze control for realistic desktop environments

Published: 26 March 2008 Publication History

Abstract

Nowadays various are the situations in which people need to interact with a Personal Computer without having the possibility to use traditional pointing devices, such as a keyboard or a mouse. In the latest years, various alternatives to the classical input devices like keyboard and mouse and novel interaction paradigms have been proposed. Particularly, multimodal interactions have been proposed to overcome the limit of each input channel take alone. In this paper we propose a multimodal system based on the integration of speech- and gaze-based inputs for interaction with a real desktop environment. A real-time grammar is generated to limit the vocal vocabulary basing on the fixated area. A disambiguation method is used for inherently ambiguous vocal commands, and the performed tests show its efficiency.

References

[1]
Baljko, M. 2005. The information-theoretic analysis of unimodal interfaces and their multimodal counterparts. In Assets '05: Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility, ACM Press, New York, NY, USA, 28--35.
[2]
Bolt, R. A. 1980. "put-that-there": Voice and gesture at the graphics interface. In SIGGRAPH '80: Proceedings of the 7th annual conference on Computer graphics and interactive techniques, ACM Press, New York, NY, USA, 262--270.
[3]
Cohen, P. R., Johnston, M., McGee, D., Oviatt, S., Pittman, J., Smith, I., Chen, L., and Chow, J. 1997. Quickset: Multimodal interaction for distributed applications. In Proceedings of the Fifth ACM International Multimedia Conference, ACM Press, 31--40.
[4]
Karl, L., Pettey, M., and Shneiderman, B. 1993. Speech activated versus mouse-activated commands for word processing applications: An empirical evaluation. In Intl. J. Man-Machine Studies.
[5]
Miniotas, D., Spakov, O., Tugoy, I., and MacKenzie, I. S. 2006. Speech-augmented eye gaze interaction with small closely spaced targets. In Proceedings of the 2006 symposium on Eye tracking research and applications, ACM Press, 66--72.
[6]
Sharma, R., Pavlovic, V., and Huang, T. 1998. Toward multimodal human-computer interface. In Proceedings of the IEEE, vol. 86, 853--869.
[7]
Sibert, L. E., and Jacob, R. J. K. 2000. Evaluation of eye gaze interaction. In CHI, 281--288.
[8]
Zhang, Q., Imamiya, A., Go, K., and Gao, X. 2004. Overriding errors in speech and gaze multimodal architecture. In Proc. 9th International Conference on Intelligent UserInterfaces (2004), ACM Press, 346--348.

Cited By

View all
  • (2024)Improving Error Correction and Text Editing Using Voice and Mouse Multimodal InterfaceInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2352932(1-24)Online publication date: 22-May-2024
  • (2022)Evaluating the Performance of Machine Learning Algorithms in Gaze Gesture Recognition SystemsIEEE Access10.1109/ACCESS.2021.313615310(1020-1035)Online publication date: 2022
  • (2022)Gazing at GamesundefinedOnline publication date: 21-Mar-2022
  • Show More Cited By

Index Terms

  1. Integrated speech and gaze control for realistic desktop environments

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      ETRA '08: Proceedings of the 2008 symposium on Eye tracking research & applications
      March 2008
      285 pages
      ISBN:9781595939821
      DOI:10.1145/1344471
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 26 March 2008

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. eye tracking
      2. multimodal interaction
      3. speech recognition
      4. usability

      Qualifiers

      • Poster

      Conference

      ETRA '08
      ETRA '08: Eye Tracking Research and Applications
      March 26 - 28, 2008
      Georgia, Savannah

      Acceptance Rates

      Overall Acceptance Rate 69 of 137 submissions, 50%

      Upcoming Conference

      ETRA '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)9
      • Downloads (Last 6 weeks)1
      Reflects downloads up to 16 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Improving Error Correction and Text Editing Using Voice and Mouse Multimodal InterfaceInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2352932(1-24)Online publication date: 22-May-2024
      • (2022)Evaluating the Performance of Machine Learning Algorithms in Gaze Gesture Recognition SystemsIEEE Access10.1109/ACCESS.2021.313615310(1020-1035)Online publication date: 2022
      • (2022)Gazing at GamesundefinedOnline publication date: 21-Mar-2022
      • (2020)Voice as a Mouse Click: Usability and Effectiveness of Simplified Hands-Free Gaze-Voice SelectionApplied Sciences10.3390/app1024879110:24(8791)Online publication date: 9-Dec-2020
      • (2020)VoiceyeProceedings of the 2020 ACM Designing Interactive Systems Conference10.1145/3357236.3395553(21-33)Online publication date: 3-Jul-2020
      • (2020)Leveraging Error Correction in Voice-based Text Entry by Talk-and-GazeProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376579(1-11)Online publication date: 21-Apr-2020
      • (2020)Integration of eye tracking and lip motion for hands-free computer accessUniversal Access in the Information Society10.1007/s10209-020-00723-w20:2(405-416)Online publication date: 28-May-2020
      • (2013)Mutual disambiguation of eye gaze and speech for sight translation and readingProceedings of the 6th workshop on Eye gaze in intelligent human machine interaction: gaze in multimodal interaction10.1145/2535948.2535953(35-40)Online publication date: 13-Dec-2013
      • (2012)Gazing at Games: An Introduction to Eye Tracking ControlSynthesis Lectures on Computer Graphics and Animation10.2200/S00395ED1V01Y201111CGR0145:1(1-113)Online publication date: 5-Mar-2012
      • (2012)Anthropomorphic awareness of partner robot to user's situation based on gaze and speech detectionInternational Journal of Autonomous and Adaptive Communications Systems10.1504/IJAACS.2012.0447825:1(18-38)Online publication date: 1-Jan-2012
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media