skip to main content
10.1145/3122986.3122999acmconferencesArticle/Chapter ViewAbstractPublication PagesautomotiveuiConference Proceedingsconference-collections
research-article

The Effects of Situational Demands on Gaze, Speech and Gesture Input in the Vehicle

Published: 24 September 2017 Publication History

Abstract

Various on-the-road situations can make additional demands on the driver that go beyond the basic demands of driving. Thereby, they influence the appropriateness of in-vehicle input modalities to operate secondary tasks in the car. In this work, we assess the specific impacts of situational demands on gaze, gesture and speech input regarding driving performance, interaction efficiency and subjective ratings. An experiment with 29 participants in a driving simulator revealed significant interactions between situational demands and the input modality on secondary task completion times, perceived suitability and cognitive workload. Impairments were greatest when the situational demand addressed the same sensory channel as the used input modality. This was reflected differently in objective and subjective data depending on the used input modality. With this work, we explore the performance of natural input modalities across different situations and thereby support interaction designers that plan to integrate these modalities in automotive interaction concepts.

References

[1]
Jonathan F. Antin, Thomas A. Dingus, Melissa C. Hulse, and Walter W. Wierwille. 1990. An evaluation of the effectiveness and efficiency of an automobile moving-map navigational display. International Journal of Man-Machine Studies 33, 5 (1990), 581--594.
[2]
Jacob Cohen. 1988. Statistical power analysis for the behavioral sciences. (1988).
[3]
Johan Engström, Emma Johansson, and Joakim Östlund. 2005. Effects of visual and cognitive load in real and simulated motorway driving. Transportation Research Part F: Traffic Psychology and Behaviour 8, 2 SPEC. ISS. (2005), 97--120.
[4]
Anke Huckauf and Mario H. Urbina. 2011. On Object selection in gaze controlled systems. ACM Transactions on Applied Perception 8, 4 (2011), 1--14.
[5]
Dagmar Kern, Angela Mahr, Sandro Castronovo, Albrecht Schmidt, and Christian Müller. 2010. Making use of drivers' glances onto the screen for explicit gaze-based interaction. In Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications -- AutomotiveUI '10. ACM, New York, NY, USA, 110--116.
[6]
Sang Hun Lee, Se-One Yoon, and Jae Hoon Shin. 2015. On-wheel finger gesture control for in-vehicle systems on central consoles. In Adjunct Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI '15. ACM, New York, NY, USA, 94--99.
[7]
Keenan R. May, Thomas M. Gable, and Bruce N. Walker. 2014. A Multimodal Air Gesture Interface for In Vehicle Menu Navigation. In Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI '14. ACM, New York, NY, USA, 1--6.
[8]
Bruce Mehler, David Kidd, Bryan Reimer, Ian Reagan, Jonathan Dobres, and Anne McCartt. 2016. Multi-modal assessment of on-road demand of voice and manual phone calling and voice navigation entry across two embedded vehicle systems. Ergonomics 59, 3 (mar 2016), 344--367.
[9]
Christian Müller, Garrett Weinberg, and Anthony Vetro. 2011. Multimodal input in the car, today and tomorrow. IEEE Multimedia 18, 1 (2011), 98--103.
[10]
Sharon Oviatt. 1999. Ten myths of multimodal interaction. Commun. ACM 42, 11 (1999), 74--81.
[11]
Annie Pauzié. 2008. A method to assess the driver mental workload: The driving activity load index (DALI). IET Intelligent Transport Systems 2, 4 (2008), 315.
[12]
Bastian Pfleging, Stefan Schneegass, and Albrecht Schmidt. 2012. Multimodal interaction in the car - combining speech and gestures on the steering wheel. In Proceedings of the 4th International Conference on Automotive User Interfaces and Interactive Vehicular Applications - AutomotiveUI 12. ACM, New York, NY, USA, 155--162.
[13]
David L. Strayer, Jason M. Watson, and Frank A. Drews. 2011. Cognitive Distraction While Multitasking in the Automobile. Vol. 54. 29--58 pages.
[14]
Christopher D. Wickens. 1980. The Structure of Attentional Resources. Attention and Performance VIII 8 (1980), 239--257.
[15]
Christopher D. Wickens. 2002. Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science 3, 2 (2002), 159--177.
[16]
Christopher D. Wickens, Diane L. Sandry, and Michael Vidulich. 1983. Compatibility and resource competition between modalities of input, central processing, and output. Human factors 25, 2 (1983), 227--248.
[17]
Walter W. Wierwille. 1993. Demands on driver resources associated with introducing advanced technology into the vehicle. Transportation Research Part C: Emerging Technologies 1, 2 (jun 1993), 133--142.

Cited By

View all
  • (2024)‘Talking with your Car’: Design of Human-Centered Conversational AI in Autonomous VehiclesProceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3640792.3675713(338-349)Online publication date: 22-Sep-2024
  • (2024)Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual DriversProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645152(1-13)Online publication date: 18-Mar-2024
  • (2024)Can You Hazard a Guess?: Evaluating the Effect of Augmented Reality Cues on Driver Hazard PredictionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642300(1-28)Online publication date: 11-May-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
AutomotiveUI '17: Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications
September 2017
317 pages
ISBN:9781450351508
DOI:10.1145/3122986
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 24 September 2017

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Gaze interaction
  2. gesture interaction
  3. natural interaction
  4. resource competition
  5. situational demand
  6. speech interaction

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

AutomotiveUI '17
Sponsor:

Acceptance Rates

AutomotiveUI '17 Paper Acceptance Rate 29 of 85 submissions, 34%;
Overall Acceptance Rate 248 of 566 submissions, 44%

Upcoming Conference

AutomotiveUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)35
  • Downloads (Last 6 weeks)3
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)‘Talking with your Car’: Design of Human-Centered Conversational AI in Autonomous VehiclesProceedings of the 16th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3640792.3675713(338-349)Online publication date: 22-Sep-2024
  • (2024)Looking for a better fit? An Incremental Learning Multimodal Object Referencing Framework adapting to Individual DriversProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645152(1-13)Online publication date: 18-Mar-2024
  • (2024)Can You Hazard a Guess?: Evaluating the Effect of Augmented Reality Cues on Driver Hazard PredictionProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642300(1-28)Online publication date: 11-May-2024
  • (2024)Aircraft human‐machine interaction assistant design: A novel multimodal data processing and application frameworkIET Control Theory & Applications10.1049/cth2.1275418:18(2742-2765)Online publication date: 28-Oct-2024
  • (2023)Effects of Urgency and Cognitive Load on Interaction in Highly Automated VehiclesProceedings of the ACM on Human-Computer Interaction10.1145/36042547:MHCI(1-20)Online publication date: 13-Sep-2023
  • (2023)It’s all about you: Personalized in-Vehicle Gesture Recognition with a Time-of-Flight CameraProceedings of the 15th International Conference on Automotive User Interfaces and Interactive Vehicular Applications10.1145/3580585.3607153(234-243)Online publication date: 18-Sep-2023
  • (2023)Towards Adaptive User-centered Neuro-symbolic Learning for Multimodal Interaction with Autonomous SystemsProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3616121(689-694)Online publication date: 9-Oct-2023
  • (2023)A Qualitative Study on the Expectations and Concerns Around Voice and Gesture Interactions in VehiclesProceedings of the 2023 ACM Designing Interactive Systems Conference10.1145/3563657.3596040(2155-2171)Online publication date: 10-Jul-2023
  • (2023)Multimodal Gaze-Based Interaction in Cars: Are Mid-Air Gestures with Haptic Feedback Safer Than Buttons?Design, User Experience, and Usability10.1007/978-3-031-35702-2_24(333-352)Online publication date: 9-Jul-2023
  • (2022)Adaptive User-Centered Multimodal Interaction towards Reliable and Trusted Automotive InterfacesProceedings of the 2022 International Conference on Multimodal Interaction10.1145/3536221.3557034(690-695)Online publication date: 7-Nov-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media