skip to main content
10.1145/1620509.1620529acmotherconferencesArticle/Chapter ViewAbstractPublication PagesautomotiveuiConference Proceedingsconference-collections
research-article

Heart on the road: HRV analysis for monitoring a driver's affective state

Published: 21 September 2009 Publication History

Abstract

Driving a vehicle is a task affected by an increasing number and a rising complexity of Driver Assistance Systems (DAS) resulting in a raised cognitive load of the driver, and in consequence to the distraction from the main activity of driving. A number of potential solutions have been proposed so far, however, although these techniques broaden the perception horizon (e. g. the introduction of the sense of touch as additional information modality or the utilization of multimodal instead of unimodal interfaces), they demand the attention of the driver too. In order to cope with the issues of workload and/or distraction, it would be essential to find a non-distracting and noninvasive solution for the emergence of information.
In this work we have investigated the application of heart rate variability (HRV) analysis to electrocardiography (ECG) data for identifying driving situations of possible threat by monitoring and recording the autonomic arousal states of the driver. For verification we have collected ECG and global positioning system (GPS) data in more than 20 test journeys on two regularly driven routes during a period of two weeks.
The first results have shown that an indicated difference of the arousal state of the driver for a dedicated point on a route, compared to its usual state, can be interpreted as a warning sign and used to notify the driver about this, perhaps safety critical, change. To provide evidence for this hypothesis it would be essential in the next step to conduct a large number of journeys on different times of the day, using different drivers and various roadways.

References

[1]
A. Aleksandrowicz and S. Leonhardt. Wireless and Non-contact ECG Measurement System -- the "Aachen SmartChair". Acta Polytechnica, Journal of Advanced Engineering, 47(4--5):68--71 (4), 2007.
[2]
L. Barrett. Discrete emotions or dimensions? The role of valence focus and arousal focus. Cognition&Emotion, 12(4):579--599, 1998.
[3]
M. Benovoy, J. R. Cooperstock, and J. Deitcher. Biosignals analysis and its application in a performance setting -- towards the development of an emotional-imaging generator. In P. Encarnação and A. Veloso, editors, BIOSIGNALS (1), pages 253--258. INSTICC -- Institute for Systems and Technologies of Information, Control and Communication, 2008.
[4]
N. O. Bernsen. Defining a taxonomy of output modalities from an HCI perspective. Computer Standards&Interfaces, 18(6--7):537--553, December 1997.
[5]
P. Blobel, L. Zvanmi, and B. AKA. Estimation of Mental Stress Levels Based on Heart Rate Variability and Stress Factor in Mobile Settings. In EHealth: Combining Health Telematics, Telemedicine, Biomedical Engineering and Bioinformatics to the Edge: CeHR Conference Proceedings 2007. Akademische Verlagsgesellschaft, 2008.
[6]
G. Clifford, F. Azuaje, and P. McSharry. Advanced methods and tools for ECG data analysis. Artech House, 2006.
[7]
A. Ferscha and A. Riener. Pervasive Adaptation in Car Crowds. In First International Workshop on User-Centric Pervasive Adaptation (UCPA) at MOBILWARE 2009, Berlin, Germany, pages 1--6 (6). Springer Berlin, Heidelberg, April 2009.
[8]
A. Gluhak, M. Presser, L. Zhu, S. Esfandiyari, and S. Kupschick. Towards mood based mobile services and applications. In Proceedings of 2nd European Conference on Smart Sensing and Context, (EuroSSC 2007), Kendal, England, pages 159--174 (16). Springer, October 2007.
[9]
A. Goldberger and E. Goldberger. Clinical Electrocardiography, A Simplified Approach. Critical Care Medicine, 9(12):891, 1981.
[10]
P. Green. Crashes induced by driver information systems and what can be done to reduce them. In SAE CONFERENCE PROCEEDINGS P, pages 27--36. SAE; 1999, 2000.
[11]
M. Grimm, K. Kroschel, H. Harris, C. Nass, B. Schuller, G. Rigoll, and T. Moosmayr. On the Necessity and Feasibility of Detecting a Driver's Emotional State While Driving. In Proceedings of the 2nd Intern. Conference on Affective Computing and Intelligent Interaction (ACII '07), pages 126--138, Berlin, Heidelberg, 2007. Springer-Verlag.
[12]
A. Haag, S. Goronzy, P. Schaich, and J. Williams. Emotion recognition using bio-sensors: First steps towards an automatic system. Lecture Notes in Computer Science, 3068:36--48, 2004.
[13]
C. Jones and M. Jonsson. Automatic recognition of affective cues in the speech of car drivers to allow appropriate responses. In Proceedings of the 17th Australia Conference on Computer-Human Interaction, pages 1--10. CHISIG of Australia, Narrabundah, Australia, Australia, 2005.
[14]
I.-M. Jonsson, C. Nass, H. Harris, and L. Takayama. Matching in-car voice with driver state: Impact on attitude and driving performance. In Proceedings of the Third International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design, pages 173--180, Stevenson, Washington, 2005.
[15]
D. Kim, Y. Seo, S. Kim, and S. Jung. Short Term Analysis of Long Term Patterns of Heart Rate Variability in Subjects under Mental Stress. In BioMedical Engineering and Informatics, 2008. BMEI 2008. International Conference on, volume 2, 2008.
[16]
R. Lane, K. McRae, E. Reiman, K. Chen, G. Ahern, and J. Thayer. Neural correlates of heart rate variability during emotion. Neuroimage, 44(1):213--222, 2009.
[17]
L. Li and J. Chen. Emotion Recognition Using Physiological Signals. Lecture Notes in Computer Science, 4282:437, 2006.
[18]
I. B. Mauss and M. D. Robinson. Measures of emotion: A review. Cognition&Emotion, 23(2):209--237, February 2009.
[19]
G. Mauter and S. Katzki. The Application of Operational Haptics in Automotive Engineering. Business Briefing: Global Automotive Manufacturing&Technology 2003 pp. 78--80, Team for Operational Haptics, Audi AG, 2003.
[20]
W. B. Mendes. Autonomic nervous system. In Methods in the Neurobiology of Social and Personality Psychology. E. Harmon-Jones&J. Beer (Eds.) Guilford Press, in press.
[21]
C. Nass, M. Jonsson, H. Harris, B. Reaves, J. Endo, S. Brave, and L. Takayama. Improving automotive safety by pairing driver emotion and car voice emotion. In Conference on Human Factors in Computing Systems, pages 1973--1976. ACM New York, NY, USA, 2005.
[22]
M. Nygårds and L. Sörnmo. Delineation of the QRS complex using the envelope of the ECG. Medical and Biological Engineering and Computing, 21(5):538--547, 1983.
[23]
R. Picard. Affective Computing. Technical Report 321, Cambridge, Massachusetts, MIT Media Laboratory, Perceptual Computing Section, Cambridge, Massachusetts, 1995.
[24]
P. Rainville, A. Bechara, N. Naqvi, and A. Damasio. Basic emotions are associated with distinct patterns of cardiorespiratory activity. International Journal of Psychophysiology, 61(1):5--18, 2006.
[25]
T. A. Ranney, E. Mazzae, R. Garrott, and M. J. Goodman. NHTSA driver distraction research: Past, present, and future. In 40th Annual Proceedings of the Association for the Advancement of Automotive Medicine, Des Plaines, IL, 2000.
[26]
A. Riener. Sensor-Actuator Supported Implicit Interaction in Driver Assistance Systems. PhD Thesis, Johannes Kepler University, Linz, Austria, January 2009.
[27]
A. Riener and A. Ferscha. Simulation Driven Experiment Control in Driver Assistance Assessment. In D. Roberts, editor, Proceedings of the 12th IEEE International Symposium on Distributed Simulation and Real Time Applications (DS-RT 2008), number 3425, pages 217--226 (10), Vancouver, BC, Canada, October 2008. IEEE Computer Society Press.
[28]
G. Rigas, C. Katsis, G. Ganiatsas, and D. Fotiadis. A user independent, biosignal based, emotion recognition method. Lecture Notes in Computer Science, 4511:314, 2007.
[29]
S. Rohrmann and H. Hopp. Cardiovascular indicators of disgust. International Journal of Psychophysiology, 68(3):201--208, 2008.
[30]
J. Russell. A circumplex model of affect. Journal of personality and social psychology, 39(6):1161--1178, 1980.
[31]
A. Rydström, P. Bengtsson, C. Grane, R. Broström, J. Agardh, and J. Nielsson. Multifunctional systems in vehicles: a usability evaluation. In A. J. Thatcher and A. Todd, editors, Proceedings of the Fourth International Cyberspace Conference on Ergonomics (CybErg 2005), Johannesburg, pages 768--775 (8). Ergonomics Association Press, 2005.
[32]
N. Sebe, I. Cohen, and T. Huang. Handbook of Pattern Recognition and Computer Vision, chapter Multimodal Emotion Recognition. World Scientific, January 2005. ISBN: 981-256-105-6.
[33]
J. Wagner, J. Kim, and E. Andre. From physiological signals to emotions: Implementing and comparing selected methods for feature extraction and classification. In IEEE International Conference on Multimedia and Expo, 2005. ICME 2005, pages 940--943, 2005.
[34]
J. Wang and Y. Gong. Recognition of multiple drivers' emotional state. In ICPR '08: The 19th International Conference on Pattern Recognition, December 2008.
[35]
A. Zinck and A. Newen. Classifying emotion: a developmental account. Synthese, 161(1):1--25, March 2008.

Cited By

View all
  • (2024)IoT-Based Assessment of a Driver’s Stress LevelSensors10.3390/s2417547924:17(5479)Online publication date: 23-Aug-2024
  • (2024)Scenario-Function System for Automotive Intelligent Cockpits: Framework, Research Progress and PerspectivesIEEE Transactions on Intelligent Vehicles10.1109/TIV.2024.33829959:5(4890-4904)Online publication date: May-2024
  • (2024)Recent advances of ECG monitoring and webserver health monitoring applications: A reviewOptics & Laser Technology10.1016/j.optlastec.2024.111039177(111039)Online publication date: Oct-2024
  • Show More Cited By

Index Terms

  1. Heart on the road: HRV analysis for monitoring a driver's affective state

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    AutomotiveUI '09: Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications
    September 2009
    143 pages
    ISBN:9781605585710
    DOI:10.1145/1620509

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 September 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. HRV analysis
    2. affective state recognition
    3. driver-vehicle interface
    4. electrocardiography (ECG)
    5. emotional state recognition
    6. on-the-road studies
    7. user-centered design

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    AutomotiveUI '09

    Acceptance Rates

    Overall Acceptance Rate 248 of 566 submissions, 44%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)36
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 30 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)IoT-Based Assessment of a Driver’s Stress LevelSensors10.3390/s2417547924:17(5479)Online publication date: 23-Aug-2024
    • (2024)Scenario-Function System for Automotive Intelligent Cockpits: Framework, Research Progress and PerspectivesIEEE Transactions on Intelligent Vehicles10.1109/TIV.2024.33829959:5(4890-4904)Online publication date: May-2024
    • (2024)Recent advances of ECG monitoring and webserver health monitoring applications: A reviewOptics & Laser Technology10.1016/j.optlastec.2024.111039177(111039)Online publication date: Oct-2024
    • (2023)Physiological Indices to Predict Driver Situation Awareness in VRAdjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing10.1145/3594739.3610687(40-45)Online publication date: 8-Oct-2023
    • (2023)Exploring the Potential of Eye-Tracking Technology for Emotion Recognition: A Preliminary Investigation2023 IEEE International Conference on Metrology for eXtended Reality, Artificial Intelligence and Neural Engineering (MetroXRAINE)10.1109/MetroXRAINE58569.2023.10405633(763-768)Online publication date: 25-Oct-2023
    • (2023)Stress influence on real-world driving identified by monitoring heart rate variability and morphologic variability of ECG signals: The case of intercity roadsInternational Journal of Occupational Safety and Ergonomics10.1080/10803548.2023.2293391(1-35)Online publication date: 11-Dec-2023
    • (2022)Exploring Emotion Responses toward Pedestrian Crossing Actions for Designing In-vehicle Empathic InterfacesExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519764(1-6)Online publication date: 27-Apr-2022
    • (2022)The Visual Scanning Behavior and Mental Workload of Drivers at Prairie Highway Intersections With Different CharacteristicsIEEE Access10.1109/ACCESS.2022.322365310(123043-123056)Online publication date: 2022
    • (2022)Interactive effects of task load and music tempo on psychological, psychophysiological, and behavioural outcomes during simulated drivingErgonomics10.1080/00140139.2021.200387265:7(915-932)Online publication date: 17-Jan-2022
    • (2021)VEmotion: Using Driving Context for Indirect Emotion Prediction in Real-TimeThe 34th Annual ACM Symposium on User Interface Software and Technology10.1145/3472749.3474775(638-651)Online publication date: 10-Oct-2021
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media