Inferring intent in eye-based interfaces: tracing eye movements with process models

ABSTRACT

While current eye-based interfaces offer enormous potential for efficient human-computer interaction, they also manifest the difficulty of inferring intent from user eye movements. This paper describes how fixation tracing facilitates the interpretation of eye movements and improves the flexibility and usability of eye-based interfaces. Fixation tracing uses hidden Markov models to map user actions to the sequential predictions of a cognitive process model. In a study of eye typing, results show that fixation tracing generates significantly more accurate interpretations than simpler methods and allows for more flexibility in designing usable interfaces. Implications for future research in eye-based interfaces and multimodal interfaces are discussed.

REFERENCES

• 1: Antti Aaltonen , Aulikki Hyrskykari , Kari-Jouko Räihä, 101 spots, or how do users read menus?, Proceedings of the SIGCHI conference on Human factors in computing systems, p.132-139, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274664]
• 2: Anderson, J. R., Corbett, A. T., Koedinger, K., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4, 167-207.
• 3: Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Hillsdale, NJ: Erlbaum.
• 4: Stuart K. Card , Allen Newell , Thomas P. Moran, The Psychology of Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1983
• 5: Frey, L. A., White, K. P., & Hutchinson, T. E. (1990). Eye-gaze word processing. IEEE Transactions on Systems, Man, and Cybernetics, 20, 944-950.
• 6: Goldberg, J. H., & Schryver, J. C. (1995). Eye-gaze determination of user intent at the computer interface. In J. M. Findlay, R. Walker, & R. W. Kentridge (Eds.), Eye Movement Research." Mechanisms, Processes, and Applications (pp. 491-502). New York: Elsevier Science Publishing.
• 7: Hutchinson, T. E., White, K. P., Martin, W. N., Reichert, K. C., & Frey, L. A. (1989). Humancomputer interaction using eye-gaze input. 1EEE Transactions on Systems, Man, and Cybernetics, 19, 1527-1534.
• 8: Robert J. K. Jacob, Eye tracking in advanced interface design, Virtual environments and advanced interface design, Oxford University Press, Inc., New York, NY, 1995
• 9: Lohse, G. L., & Johnson, E. J. (1996). A comparison of two process tracing methods for choice tasks. Organizational Behavior and Human Decision Processes, 68, 28-43.
• 10: Rabiner, L. R. (1989). A tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77, 257-286.
• 11: Salvucci, D. D., & Anderson, J. R. (1998). Tracing eye movement protocols with cognitive process models. In Proceedings of the Twentieth Annual Meeting of the Cognitive Science Society (pp. 923-928). Hillsdale, NJ: Erlbaum.
• 12: Stampe, D. M., & Reingold, E. M. (1995). Selection by looking: A novel computer interface and its application to psychological research. In J. M. Fiiadlay, R. Walker, & R. W. Kentridge (Eds.), Eye Movement Research: Mechanisms, Processes, and Applications (pp. 467- 478). New York: Elsevier Science Publishing.
• 13: Jie Yang , Rainer Stiefelhagen , Uwe Meier , Alex Waibel, Visual tracking for multimodal human computer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, p.140-147, April 18-23, 1998, Los Angeles, California, United States [doi>10.1145/274644.274666]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human factors

General Terms:
Design, Human Factors, Theory

Keywords:
cognitive models, eye movements, eye-based interfaces, hidden Markov models, tracing, user models