Article

Perceptual user interfaces using vision-based eye tracking

Authors:

Ravikrishna Ruddarraju,

Elizabeth D. MynattAuthors Info & Claims

ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces

Pages 227 - 233

https://doi.org/10.1145/958432.958475

Published: 05 November 2003 Publication History

Abstract

We present a multi-camera vision-based eye tracking method to robustly locate and track user's eyes as they interact with an application. We propose enhancements to various vision-based eye-tracking approaches, which include (a) the use of multiple cameras to estimate head pose and increase coverage of the sensors and (b) the use of probabilistic measures incorporating Fisher's linear discriminant to robustly track the eyes under varying lighting conditions in real-time. We present experiments and quantitative results to demonstrate the robustness of our eye tracking in two application prototypes.

References

[1]

A. Aaltonen, A. Hyrskykari, and K. Raiha. 101 spots on how do users read menus? In Human Factors in Computing Systems: CHI 98, pages 132--139, New York, 1998. ACM Press.

Digital Library

[2]

Alan Allport. Visual Attention. MIT Press, 1993.

[3]

M. Argyle. Social Interaction. Methuen & Co., London, England, 1969.

[4]

M. Argyle and M. Cook. Gaze and Mutual Gaze. Cambridge University Press, Cambridge, UK, 1976. barber:perceptioninfo

[5]

P. Barber and D. Legge. Perception and Information. Methuen & Co., London, England, 1976.

[6]

P. Belhumeur, J. Hespanha, and D. Kriegman. Eigenfaces vs. fisherfaces: Recognition using class specific linear projection. In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 19, July 1997.

Digital Library

[7]

La Cascia, M. Sclaroff, and S. Athitso. Fast reliable head tracking under varying illumination: An approach based on robust registration of texture mapped 3-d models. In IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000.

Digital Library

[8]

J. H. Goldberg and J. C. Schryver. Eye-gaze determination of user intent at computer interface. Elsevier Science Publishing, New York, New York, 1995.

[9]

A. Haro, M. Flickner, and I. Essa. Detecting and tracking eyes by using their physiological properties, dynamics, and appearance. In IEEE Computer Vision and Pattern Recognition, pages 163--168, 2000.

[10]

M. Harville, A. Rahimi, T. Darrell, G. Gordon, and J. Woodfill. 3-d pose tracking with linear depth and brightness constraints. In International Conference on Computer Vision, 1999.

[11]

B. Jabrain, J. Wu, R. Vertegaal, and L. Grigorov. Establishing remote conversations through eye contact with physical awareness proxies. In Extended Abstracts of ACM CHI, 2003.

Digital Library

[12]

Y. Matsumoto, T. Ogasawara, and A. Zelinsky. Behavior recognition based on head pose and gaze direction measurement. In IEEE International Conference on Intelligent Robots and Systems, 2000.

[13]

C. H. Morimoto, D. Koons, A. Amir, and M. Flickner. Pupil detection and tracking using multiple light sources. Technical report RJ-10117, IBM Almaden Research Center, 1998.

[14]

E. Mynatt, J. Rowan, and A Jacobs. Digital family portraits: Providing peace of mind for extended family members. In ACM CHI, 2001.

Digital Library

[15]

K. Nagel, C. Kidd, T. O'Connell, S. Patil, and G. Abowd. The family intercom: Developing a context-aware audio communication system. In Ubicomp, 2001.

Digital Library

[16]

R. Ruddarraju, A. Haro, and I. Essa. Fast multiple camera head pose tracking. In International Conference on Vision Interfaces, Halifax, Canada, 2003.

[17]

A. Schoedl, A. Haro, and I. Essa. Head tracking using a textured polygonal model. In Proceedings Workshop on Perceptual User Interfaces, 1998.

[18]

Rainer Stiefelhagen. Tracking focus of attention in meetings. In International Conference on Multi-Modal Interfaces, 2002.

Digital Library

[19]

Q. Tran and E. Mynatt. Cook's collage: Two exploratory designs. In CHI 2002, Conference Proceedings, 2002.

[20]

A. L. Yarbus. Eye Movements during Perception of Complex Objects. Plenum Press, New York, New York, 1967.

Cited By

Hisadome YWu TQin JSugano Y(2024)Rotation-Constrained Cross-View Feature Fusion for Multi-View Appearance-based Gaze Estimation2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00588(5973-5982)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00588
Jagnade GSable SIkar M(2023)Advancing Multimodal Fusion in Human-Computer Interaction: Integrating Eye Tracking, Lips Detection, Speech Recognition, and Voice Synthesis for Intelligent Cursor Control and Auditory Feedback2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT56998.2023.10306457(1-7)Online publication date: 6-Jul-2023
https://doi.org/10.1109/ICCCNT56998.2023.10306457
Madaan HGupta S(2021)AI Improving the Lives of Physically DisabledProceedings of the 12th International Conference on Soft Computing and Pattern Recognition (SoCPaR 2020)10.1007/978-3-030-73689-7_11(103-112)Online publication date: 16-Apr-2021
https://doi.org/10.1007/978-3-030-73689-7_11
Show More Cited By

Index Terms

Perceptual user interfaces using vision-based eye tracking
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Graphics systems and interfaces
      1. Graphics processors
    2. Image manipulation
      1. Image processing
2. Mathematics of computing
  1. Probability and statistics
    1. Probabilistic algorithms
    2. Probabilistic reasoning algorithms
      1. Markov-chain Monte Carlo methods
      2. Sequential Monte Carlo methods

Recommendations

Eye and gaze tracking for interactive graphic display

This paper describes a computer vision system based on active IR illumination for real-time gaze tracking for interactive graphic display. Unlike most of the existing gaze tracking techniques, which often require assuming a static head to work well and ...
Complemental Use of Multiple Cameras for Stable Tracking of Multiple Markers
VMR '09: Proceedings of the 3rd International Conference on Virtual and Mixed Reality: Held as Part of HCI International 2009

In many applications of Augmented Reality (AR), rectangular markers are tracked in real time by capturing with cameras. In this paper, we consider the AR application in which virtual objects are displayed onto markers while the markers and the cameras ...
A general framework for extension of a tracking range of user-calibration-free remote eye-gaze tracking systems
ETRA '12: Proceedings of the Symposium on Eye Tracking Research and Applications

Stereo-camera Remote Eye-Gaze Tracking (REGT) systems can provide calibration-free estimation of gaze. However, such systems have a limited tracking range due to the requirement for the eye to be tracked in both cameras. This paper presents a general ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '03: Proceedings of the 5th international conference on Multimodal interfaces

November 2003

318 pages

ISBN:1581136218

DOI:10.1145/958432

Conference Chair:
Sharon Oviatt
Oregon Health & Science University
,
Program Chairs:
Trevor Darrell
Massachusetts Institute of Technology
,
Mark Maybury
MITRE
,
Wolfgang Wahlster
DFKI, Germany

Copyright © 2003 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 November 2003

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICMI-PUI03

Sponsor:

ICMI-PUI03: International Conference on Multimodal User Interfaces

November 5 - 7, 2003

British Columbia, Vancouver, Canada

Acceptance Rates

ICMI '03 Paper Acceptance Rate 45 of 130 submissions, 35%;

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

24
Total Citations
View Citations
1,330
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hisadome YWu TQin JSugano Y(2024)Rotation-Constrained Cross-View Feature Fusion for Multi-View Appearance-based Gaze Estimation2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00588(5973-5982)Online publication date: 3-Jan-2024
https://doi.org/10.1109/WACV57701.2024.00588
Jagnade GSable SIkar M(2023)Advancing Multimodal Fusion in Human-Computer Interaction: Integrating Eye Tracking, Lips Detection, Speech Recognition, and Voice Synthesis for Intelligent Cursor Control and Auditory Feedback2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT)10.1109/ICCCNT56998.2023.10306457(1-7)Online publication date: 6-Jul-2023
https://doi.org/10.1109/ICCCNT56998.2023.10306457
Madaan HGupta S(2021)AI Improving the Lives of Physically DisabledProceedings of the 12th International Conference on Soft Computing and Pattern Recognition (SoCPaR 2020)10.1007/978-3-030-73689-7_11(103-112)Online publication date: 16-Apr-2021
https://doi.org/10.1007/978-3-030-73689-7_11
Rahman LAdan JBillah QIslam MKamal AHossain SShahnaz CSubhana A(2019)A low cost Human Computer Interface for Disabled People based on Eye Blink detection using Brain SignalTENCON 2019 - 2019 IEEE Region 10 Conference (TENCON)10.1109/TENCON.2019.8929452(1937-1941)Online publication date: Oct-2019
https://doi.org/10.1109/TENCON.2019.8929452
Karray FAlemzadeh MSaleh JArab M(2017)Human-Computer Interaction: Overview on State of the ArtInternational Journal on Smart Sensing and Intelligent Systems10.21307/ijssis-2017-2831:1(137-159)Online publication date: 13-Dec-2017
https://doi.org/10.21307/ijssis-2017-283
Tonsen MSteil JSugano YBulling A(2017)InvisibleEyeProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/31309711:3(1-21)Online publication date: 11-Sep-2017
https://dl.acm.org/doi/10.1145/3130971
Nautiyal LMalik PRam M(2017)Computer Interfaces in Diagnostic Process of Industrial EngineeringDiagnostic Techniques in Industrial Engineering10.1007/978-3-319-65497-3_5(157-170)Online publication date: 21-Oct-2017
https://doi.org/10.1007/978-3-319-65497-3_5
Jalaliniya SMardanbegi DKaye JDruin ALampe CMorris DHourcade J(2016)EyeGripProceedings of the 2016 CHI Conference on Human Factors in Computing Systems10.1145/2858036.2858584(5801-5811)Online publication date: 7-May-2016
https://dl.acm.org/doi/10.1145/2858036.2858584
Naizhong ZJing WJun W(2015)Hand-free head mouse control based on mouth tracking2015 10th International Conference on Computer Science & Education (ICCSE)10.1109/ICCSE.2015.7250337(707-713)Online publication date: Jul-2015
https://doi.org/10.1109/ICCSE.2015.7250337
Okoe MAlam SJianu RChen MLaramee R(2014)A gaze-enabled graph visualization to improve graph reading tasksProceedings of the 16th Eurographics Conference on Visualization10.5555/2854210.2854245(251-260)Online publication date: 9-Jun-2014
https://dl.acm.org/doi/10.5555/2854210.2854245
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten