skip to main content
10.1145/2858036.2858117acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

A Model Relating Pupil Diameter to Mental Workload and Lighting Conditions

Published: 07 May 2016 Publication History

Abstract

In this paper, we present a proof-of-concept approach to estimating mental workload by measuring the user's pupil diameter under various controlled lighting conditions. Knowing the user's mental workload is desirable for many application scenarios, ranging from driving a car, to adaptive workplace setups. Typically, physiological sensors allow inferring mental workload, but these sensors might be rather uncomfortable to wear. Measuring pupil diameter through remote eye-tracking instead is an unobtrusive method. However, a practical eye-tracking-based system must also account for pupil changes due to variable lighting conditions. Based on the results of a study with tasks of varying mental demand and six different lighting conditions, we built a simple model that is able to infer the workload independently of the lighting condition in 75% of the tested conditions.

Supplementary Material

MP4 File (p5776-pfleging.mp4)

References

[1]
Ulf Ahlstrom and Ferne J. Friedman-Berg. 2006. Using eye movement activity as a correlate of cognitive workload. International Journal of Industrial Ergonomics 36, 7 (Jul. 2006), 623--636.
[2]
Amanda N. Baraldi and Craig K. Enders. 2010. An introduction to modern missing data analyses. Journal of School Psychology 48, 1 (Feb. 2010), 5--37.
[3]
Jackson Beatty. 1982. Task-evoked pupillary responses, processing load, and the structure of processing resources. Psychological Bulletin 91, 2 (Mar. 1982), 276--292.
[4]
Simone Benedetto, Marco Pedrotti, Luca Minin, Thierry Baccino, Alessandra Re, and Roberto Montanari. 2011. Driver workload and eye blink duration. Transportation Research Part F: Traffic Psychology and Behavior 14, 3 (May 2011), 199--208.
[5]
Anne-Marie Brouwer, Maarten A. Hogervorst1, Jan B. F. van Erp, Tobias Heffelaar, Patrick H. Zimmerman, and Robert Oostenveld. 2012. Estimating workload using EEG spectral power and ERPs in the n-back task. Journal of Neural Engineering 9, 4, Article 045008 (Jul. 2012), 045008, 14 pages.
[6]
Joseph F. Coughlin, Bryan Reimer, and Bruce Mehler. 2009. Driver Wellness, Safety & the Development of an AwareCar. MIT AgeLab White Paper. Massachusetts Institute of Technology, Cambridge, MA. http://web.mit.edu/reimer/www/pdfs/coughlin_wellness_2009.pdf
[7]
Andrew T. Duchowski. 2002. A breadth-first survey of eye-tracking applications. Behavior Research Methods, Instruments, & Computers 34, 4 (Nov. 2002), 455--470.
[8]
Christine Fogarty, and John A. Stern, 1989. Eye movements and blinks: Their relationship to higher cognitive processes. International Journal of Psychophysiology 8, 1 (Sep. 1983), 35--42. http://dx.doi.org/10.1016/0167--8760(89)90017--2
[9]
Thomas M. Gable, Andrew L. Kun, Bruce N. Walker, and Riley J. Winton. 2015. Comparing heart rate and pupil size as objective measures of workload in the driving context: initial look. In Adjunct Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '15). ACM, NY, NY, USA, 20--25.
[10]
Alan Gevins, and Michael E. Smith. 2003. Neurophysiological measures of cognitive workload during human-computer interaction. Theoretical Issues in Ergonomics Science 4, 1--2 (2003), 113--131.
[11]
Audrey Girouard and Erin Treacy Solovey and Robert J.K. Jacob. 2013. Designing a passive brain computer interface using real time classification of functional near-infrared spectroscopy. International Journal of Autonomous and Adaptive Communication Systems 6,1 (2013), 26--44.
[12]
D. M. A. Gronwall. 1977. Paced Auditory Serial-Addition Task: A Measure of Recovery From Concussion. Perceptual and Motor Skills 44 (1977), 367--373.
[13]
Jennifer A. Healey, and Rosalind W. Picard. 2005. Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans. Intelligent Transportation Systems 6, 2 (June 2005), 156--166.
[14]
Peter A. Heeman, Tomer Meshorer, Andrew L. Kun, Oskar Palinko, and Zeljko Medenica. 2013. Estimating cognitive load using pupil diameter during a spoken dialogue task. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '13). ACM, NY, NY, USA, 242--245.
[15]
Daniel Kahneman, D., and Jackson Beatty. 1966. Pupil Diameter and Load on Memory. Science 154, 3756 (Dec. 1966), 1583--1585.
[16]
Daniel Kahneman, Linda Onuska, L., and Ruth E. Wolman. 1968. Effects of pupillary response in a short-term memory task. The Quarterly Journal of Experimental Psychology 20, 3 (1968), 309--311.
[17]
W. K. Kirchner. Age differences in short-term retention of rapidly changing information. Journal of experimental psychology 55, 4 (Apr. 1958), 352--358.
[18]
Jeff Klingner, Rakshit Kumar, and Pat Hanrahan. 2008. Measuring the task-evoked pupillary response with a remote eye tracker. In Proceedings of the 2008 symposium on Eye tracking research & applications (ETRA '08). ACM, NY, NY, USA, 69--72.
[19]
Jeff Klingner, Barbara Tversky, and Pat Hanrahan. 2011. Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology 48, 3 (Mar. 2011), 323--332.
[20]
Andrew L. Kun, Susanne Boll, and Albrecht Schmidt. 2016. Shirting gears: User interfaces in the age of autonomous driving. IEEE Pervasive Computing 15, 1 (January-March 2016), 32--37.
[21]
Andrew L. Kun, and Zeljko Medenica. 2012. Video call or not, that is the question. In CHI '12 Extended Abstracts on Human Factors in Computing Systems. ACM, NY, NY, USA, 1631--1636.
[22]
Andrew L. Kun, Oskar Palinko, Zeljko Medenica, and Peter A. Heeman. 2013. On the Feasibility of Using Pupil Diameter to Estimate Cognitive Load Changes for In-Vehicle Spoken Dialogues. In INTERSPEECH, (2013), 3766--3770.
[23]
Andrew L. Kun, Oskar Palinko, and Razumeniø, I. Exploring the effects of size and luminance of visual targets on the pupillary light reflex. In Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '12). ACM, NY, NY, USA, 183--186.
[24]
Sandra P. Marshall, 2002. The Index of Cognitive Activity: measuring cognitive workload. In Proceedings of the 2002 IEEE 7th Conference on Human Factors and Power Plants. IEEE, 7--5 - 7--9.
[25]
Bruce Mehler, Bryan Reimer, Joseph F. Coughlin, and Jeffery A. Dusek. 2009. Impact of incremental increases in cognitive workload on physiological arousal and performance in young adult drivers. Transportation Research Record: Journal of the Transportation Research Board 2138, 1 (2009), 6--12.
[26]
Bruce Mehler, Bryan Reimer, and Joseph F. Coughlin. 2012. Sensitivity of physiological measures for detecting systematic variations in cognitive demand from a working memory task an on-road study across three age groups. Human Factors 54, 3 (2010), 396--412.
[27]
Bruce Mehler, Bryan Reimer and Jeffery A. Dusek. 2011. MIT AgeLab Delayed Digit Recall Task (n-back) Working Paper 2011--3B. Massachusetts Institute of Technology, Cambridge, MA. http://agelab.mit.edu/system/files/Mehler_et_al_n-back-white-paper_2011_B.pdf
[28]
W. Thomas Miller, and Andrew L. Kun. 2013. Using speech, GUIs and buttons in police vehicles: field data on user preferences for the Project54 system. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '13). ACM, NY, NY, USA, 108--113.
[29]
William F. Moroney, David W. Biers, Thomas Eggemeier, and Jennifer A. Mitchell. 1992. A comparison of two scoring procedures with the NASA Task Load Index in a simulated flight task. In Proceedings of the IEEE 1992 National Aerospace and Electronics Conference, NAECON 1992. IEEE, 734--740.
[30]
Nesrine Mezhoudi, Iyad Khaddam, and Jean Vanderdonckt. 2015. Toward Usable Intelligent User Interface. In: Proceedings of the 17th International Conference on Human-Computer Interaction (HCII '15), Part II, LNCS 9170, Springer International Publishing, 459--471.
[31]
Michael Myrtek, Doris Weber, Georg Brügner, and Wolfgang Müller. 1996. Occupational stress and strain of female students: results of physiological, behavioral, and psychological monitoring. Biological Psychology 42, 3 (Feb. 1996), 379--391.
[32]
Luis Nunes and Miguel A. Rescarte 2002. Cognitive demands of hands-free-phone conversation while driving. Transportation Research Part F 5, 2 (Jun. 2002), 133--144.
[33]
Julie Onton, Arnaud Delorme, and Scott Makeig. 2005. Frontal midline EEG dynamics during working memory. NeuroImage 27, 2 (Aug. 2005), 341--356.
[34]
Oskar Palinko and Andrew Kun. 2011. Exploring the influence of light and cognitive load on pupil diameter in driving simulator studies. In Proceedings of the Sixth International Driving Symposium on Human Factors in Driver Assessment, Training and Vehicle Design (Driving Assessment 2011), University of Iowa, Iowa City, IA, 329--336. http://drivingassessment.uiowa.edu/sites/default/files/DA2011/Papers/048_PalinkoKun.pdf
[35]
Oskar Palinko and Andrew Kun. 2012. Exploring the effects of visual cognitive load and illumination on pupil diameter in driving simulators. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '12), ACM, NY, NY, 413--416.
[36]
Oskar Palinko, Andrew L. Kun, Alexander Shyrokov, and Peter A. Heeman. 2010. Estimating cognitive load using remote eye tracking in a driving simulator. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '10), ACM, NY, NY, 141--144.
[37]
Bastian Pfleging and Albrecht Schmidt. 2015. (Non-) Driving-Related Activities in the Car: Defining Driver Activitgies for Manual and Automated Driving. In Workshop on Experiencing Autonomous Vehicles: Crossing the Boundaries between a Drive and a Ride at CHI '15. http://www.hcilab.org/wp-content/uploads/pfleging-2015-drivingrelatedactivities.pdf
[38]
Bastian Pfleging, Stefan Schneegass, and Albrecht Schmidt. 2013. Exploring user expectations for context and road video sharing while calling and driving. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '13). ACM, NY, NY, USA, 132--139.
[39]
Bryan Reimer, Bruce Mehler, Joseph F. Coughlin, Kathryn M. Godfrey, and Chauanzhong Tan. 2009. An on-road assessment of the impact of cognitive workload on physiological arousal in young adult drivers. In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Application (AutomotiveUI '09), ACM, NY, NY, 115--118.
[40]
A. H. Roscoe. 1992. Assessing pilot workload. Why measure heart rate, HRV and respiration? Biological Psychology 34, 2 (Nov. 1992), 259--287.
[41]
Dennis W. Rowe, John Sibert, and Don Irwin. 1998. Heart rate variability: Indicator of user state as an aid to human-computer interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '98), ACM, NY, NY, 480--487.
[42]
Kilseop Ryu and Rohae Myung. 2005. Evaluation of mental workload with a combined measure based on physiological indices during a dual task of tracking and mental arithmetic. International Journal of Industrial Ergonomics 35, 11 (Nov. 2005), 991--1009.
[43]
Stefan Schneegass, Bastian Pfleging, Nora Broy, Frederik Heinrich, and Albrecht Schmidt. A data set of real world driving to assess driver workload. In Proceedings of the 5th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI 2013). ACM, NY, NY, 150--157.
[44]
Erin T. Solovey, Paul Schermerhorn, Matthias Scheutz, Angelo Sassaroli, Sergio Fantini, and Robert J.K. Jacob. 2012. Brainput: Enhancing Interactive Systems with Streaming fNIRS Brain Input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, NY, NY, USA, 2193--2202.
[45]
Yi-Fang Tsai, Erik Viirre, Christopher Strychacz, Bradley Chase, and Tzyy-Ping Jung. 2007. Task performance and eye activity: Predicting behavior relating to cognitive workload. Aviation, Space, and Environmental Medicine 78, 5 (May 2007), B176-B185.
[46]
Karl F. Van Orden, Wendy Limbert, Scott Makeig, and Tzyy-Ping Jung. 2001. Eye activity correlates of workload during visuospatial memory task. Human Factors: The Journal of the Human Factors and Ergonomics Society 43, 1 (Spring 2001), 111--121.
[47]
Ying Wang, Bryan Reimer, Bruce Mehler, Jun Zhang, Alea Mehler, and Joseph F. Coughlin. 2010. The Impact of Repeated Cognitive Tasks on Driving Performance and Visual Attention. In Proceedings of the 3rd International Conference on Applied Human Factors and Ergonomics (AHFE '10), Miami, FL, USA.
[48]
Yan Yang, Bryan Reimer, Bruce Mehler,and Jonathan Dobres. 2013. A field study assessing driving performance, visual attention, heart rate and subjective ratings in response to two types of cognitive workload. In Proceedings of the 7th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design (Driving Assessment 2013). University of Iowa, Iowa City, IA, 397--403.
[49]
Yabo Yang, Keith Thompson, and Stephen A. Burns. 2002. Pupil location under mesopic, photopic, and pharmacologically dilated conditions. Investigative Ophthalmology & Visual Science 43, 7 (Jul. 2002), 2508--2512.
[50]
Zhiwei Zhu, Kikuo Fujimura, and Qiang Ji. 2002. Real-time eye detection and tracking under various light conditions. In Proceedings of the 2002 Symposium on Eye Tracking Research & Applications (ETRA 2002). ACM, NY, NY, 139--144.

Cited By

View all
  • (2025)The fundamentals of eye tracking part 2: From research question to operationalizationBehavior Research Methods10.3758/s13428-024-02590-257:2Online publication date: 24-Jan-2025
  • (2025)Prediction of intrinsic and extraneous cognitive load with oculometric and biometric indicatorsScientific Reports10.1038/s41598-025-89336-y15:1Online publication date: 12-Feb-2025
  • (2025)Optimal frequency bands for pupillography for maximal correlation with HRVScientific Reports10.1038/s41598-025-85663-215:1Online publication date: 27-Jan-2025
  • Show More Cited By

Index Terms

  1. A Model Relating Pupil Diameter to Mental Workload and Lighting Conditions

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '16: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems
    May 2016
    6108 pages
    ISBN:9781450333627
    DOI:10.1145/2858036
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 May 2016

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. adaptive user interfaces
    2. cognitive workload
    3. compensation of pupillary light reflex
    4. estimation of mental workload
    5. eye-tracking
    6. lighting
    7. psychophysiology
    8. task-evoked pupillary response

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI'16
    Sponsor:
    CHI'16: CHI Conference on Human Factors in Computing Systems
    May 7 - 12, 2016
    California, San Jose, USA

    Acceptance Rates

    CHI '16 Paper Acceptance Rate 565 of 2,435 submissions, 23%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)556
    • Downloads (Last 6 weeks)71
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)The fundamentals of eye tracking part 2: From research question to operationalizationBehavior Research Methods10.3758/s13428-024-02590-257:2Online publication date: 24-Jan-2025
    • (2025)Prediction of intrinsic and extraneous cognitive load with oculometric and biometric indicatorsScientific Reports10.1038/s41598-025-89336-y15:1Online publication date: 12-Feb-2025
    • (2025)Optimal frequency bands for pupillography for maximal correlation with HRVScientific Reports10.1038/s41598-025-85663-215:1Online publication date: 27-Jan-2025
    • (2025)Pre-AttentiveGaze: gaze-based authentication dataset with momentary visual interactionsScientific Data10.1038/s41597-025-04538-312:1Online publication date: 13-Feb-2025
    • (2024)Post-Takeover Proficiency in Conditionally Automated Driving: Understanding Stabilization Time with Driving and Physiological SignalsSensors10.3390/s2410319324:10(3193)Online publication date: 17-May-2024
    • (2024)Assessment of Drivers’ Mental Workload by Multimodal Measures during Auditory-Based Dual-Task Driving ScenariosSensors10.3390/s2403104124:3(1041)Online publication date: 5-Feb-2024
    • (2024)Changes in Non-Driving-Related Activities from Conditional to Full Automation and Their Implications for Interior Design: A Systematic Review and Meta-AnalysisApplied Sciences10.3390/app1420944214:20(9442)Online publication date: 16-Oct-2024
    • (2024)EEG as a potential ground truth for the assessment of cognitive state in software development activities: A multimodal imaging studyPLOS ONE10.1371/journal.pone.029910819:3(e0299108)Online publication date: 7-Mar-2024
    • (2024)Your Eyes on Speed: Using Pupil Dilation to Adaptively Select Speed-Reading Parameters in Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36765318:MHCI(1-17)Online publication date: 24-Sep-2024
    • (2024)VisRecall++: Analysing and Predicting Visualisation Recallability from Gaze BehaviourProceedings of the ACM on Human-Computer Interaction10.1145/36556138:ETRA(1-18)Online publication date: 28-May-2024
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media