Using EEG to Understand why Behavior to Auditory In-vehicle Notifications Differs Across Test Environments

ABSTRACT

In this study, we employ EEG methods to clarify why auditory notifications, which were designed for task management in highly automated trucks, resulted in different performance behavior, when deployed in two different test settings: (a) student volunteers in a lab environment, (b) professional truck drivers in a realistic vehicle simulator. Behavioral data showed that professional drivers were slower and less sensitive in identifying notifications compared to their counterparts. Such differences can be difficult to interpret and frustrates the deployment of implementations from the laboratory to more realistic settings. Our EEG recordings of brain activity reveal that these differences were not due to differences in the detection and recognition of the notifications. Instead, it was due to differences in EEG activity associated with response generation. Thus, we show how measuring brain activity can deliver insights into how notifications are processed, at a finer granularity than can be afforded by behavior alone.

References

1. L. Bainbridge. 1983. Ironies of automation. Automatica 19, 6 (1983), 775--779.
2. Y. Benjamini and Y. Hochberg. 1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the royal statistical society. Series B (Methodological) (1995), 289--300. http://www.jstor.org/stable/2346101
3. N. Bigdely-Shamlo, T. Mullen, C. Kothe, K-M. Su, and K. A. Robbins. 2015. The PREP pipeline: standardized preprocessing for large-scale EEG analysis. Frontiers in neuroinformatics 9 (2015), 16.
4. M. G Bleichner, B. Mirkovic, and S. Debener. 2016. Identifying auditory attention with ear-EEG: cEEGrid versus high-density cap-EEG comparison. Journal of Neural Engineering 13, 6 (2016), 066004. http://stacks.iop.org/1741-2552/13/i=6/a=066004
5. S. Borojeni, L. Chuang, W. Heuten, and S. Boll. 2016. Assisting Drivers with Ambient Take-Over Requests in Highly Automated Driving. In Proceedings of the 8th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Automotive'UI 16). ACM, New York, NY, USA, 237--244.
6. Y. Cao, F. van der Sluis, M. Theune, R. op den Akker, and A. Nijholt. 2010. Evaluating Informative Auditory and Tactile Cues for In-vehicle Information Systems. In Proceedings of the 2Nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '10). ACM, New York, NY, USA, 102--109.
7. S.M. Casner, E.L. Hutchins, and D. Norman. 2016. The Challenges of Partially Automated Driving. Commun. ACM 59, 5 (April 2016), 70--77.
8. M. Delacre, D. Lakens, and C. Leys. in press. Why Psychologists Should by Default Use Welch's t-test Instead of Student's t-test. International Review of Social Psychology (in press).
9. A. Delorme and S. Makeig. 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. Journal of neuroscience methods 134, 1 (2004), 9--21.
10. A. Delorme, J. Palmer, J. Onton, R. Oostenveld, and S. Makeig. 2012. Independent EEG sources are dipolar. PloS one 7, 2 (2012), e30135.
11. E. Donchin and M.G.H. Coles. 1988. Is the P300 component a manifestation of context updating. Behavioral and brain sciences 11, 3 (1988), 357--427.
12. J. Edworthy, S. Loxley, and I. Dennis. 1991. Improving auditory warning design: relationship between warning sound parameters and perceived urgency. Human factors 33, 2 (1991), 205--231.
13. Johan Fagerlönn, Stefan Lindberg, and Anna Sirkka. 2015. Combined Auditory Warnings For Driving-Related Information. In Proceedings of the Audio Mostly 2015 on Interaction With Sound (AM '15). ACM, New York, NY, USA, Article 11, 5 pages.
14. K. Gramann, D.P. Ferris, J. Gwin, and S. Makeig. 2014. Imaging natural cognition in action. International Journal of Psychophysiology 91, 1 (2014), 22--29.
15. R. Gray. 2011. Looming Auditory Collision Warnings for Driving. Human Factors 53, 1 (2011), 63--74.
16. P. Green, W. Levison, G. Paelke, and C. Serafin. 1995. Preliminary Human Factors Design Guidelines for Driver Information Systems. Technical Report FHWA-RD-94-087. US Department of Transportation, Federal Highway Administration. 1--111 pages.
17. D. M. Groppe, T. P. Urbach, and M. Kutas. 2011. Mass univariate analysis of event-related brain potentials/fields I: A critical tutorial review. Psychophysiology 48, 12 (2011), 1711--1725.
18. S. Haufe, J-W. Kim, I-H. Kim, A. Sonnleitner, M. Schrauf, G. Curio, and B. Blankertz. 2014. Electrophysiology-based detection of emergency braking intention in real-world driving. Journal of neural engineering 11, 5 (2014), 056011.
19. T-P. Jung, S. Makeig, C. Humphries, T-W. Lee, M. J. Mckeown, V. Iragui, and T. J. Sejnowski. 2000. Removing electroencephalographic artifacts by blind source separation. Psychophysiology 37, 2 (2000), 163--178.
20. H.H. Kornhuber and L. Deecke. 1964. Hirnpotentialanderungen beim Menschen vor und nach Willkurbewegungen dargestellt mit Magnetbandspeicherung und Ruckwartsanalyse. In Pflugers Archiv, Vol. 281. 52.
21. S. Krupenia, A. Selmarker, J. Fagerlönn, K. Delsing, A. Jansson, B. Sandblad, and C. Grane. 2014. The Methods for Designing Future Autonomous Systems' (MODAS) project: Developing the cab for a highly autonomous truck. In Proceedings of the 5th International Conference on Applied Human Factors and Ergonomics (AHFE2014) (Krakow, Poland). 19--23.
22. A. L. Kun, S. Boll, and A. Schmidt. 2016. Shifting Gears: User Interfaces in the Age of Autonomous Driving. IEEE Pervasive Computing 15, 1 (Jan 2016), 32--38.
23. B. Libet. 1985. Unconscious cerebral initiative and the role of conscious will in voluntary action. Behavioral and Brain Sciences 8, 4 (1985), 529--539.
24. D.E.J. Linden, D. Prvulovic, E. Formisano, M. Völlinger, F. E. Zanella, R. Goebel, and T. Dierks. 1999. The functional neuroanatomy of target detection: an fMRI study of visual and auditory oddball tasks. Cerebral Cortex 9, 8 (1999), 815--823.
25. A. Löcken, W. Heuten, and S. Boll. 2015. Supporting Lane Change Decisions with Ambient Light. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '15). ACM, New York, NY, USA, 204--211.
26. A. Löcken, S. Sadeghian Borojeni, H. Müller, T.M. Gable, S. Triberti, C. Diels, C. Glatz, I. Alvarez, L. Chuang, and S. Boll. 2017. Towards Adaptive Ambient In-Vehicle Displays and Interactions: Insights and Design Guidelines from the 2015 AutomotiveUI Dedicated Workshop. In Automotive User Interfaces: Creating Interactive Experiences in the Car, G. Meixner and C. Müller (Eds.). Springer International Publishing, Cham, 325--348.
27. S.A. Lu, C.D. Wickens, J.C. Prinet, S.D. Hutchins, N. Sarter, and A. Sebok. 2013. Supporting interruption management and multimodal interface design: three meta-analyses of task performance as a function of interrupting task modality. Human Factors 55, 4 (2013), 697--724.
28. N. A. Macmillan and C. D. Creelman. 1991. Detection theory: A user's guide (1st ed.). Lawrence Erlbaum Associates Inc., New Jersey.
29. D.C. Marshall, J.D. Lee, and R.A. Austria. 2007. Alerts for in-vehicle information systems: annoyance, urgency, and appropriateness. Human factors 49, 1 (2007), 145--57.
30. D. Scott McCrickard and C. M. Chewar. 2003. Attuning Notification Design to User Goals and Attention Costs. Commun. ACM 46, 3 (March 2003), 67--72.
31. D. McKeown and S. Isherwood. 2007. Mapping Candidate Within-Vehicle Auditory Displays to Their Referents. Human Factors 49, 3 (2007), 417--428.
32. M. A. Mollenhauer, J. Lee, K. Cho, M. C. Hulse, and T. A. Dingus. 1994. The Effects of Sensory Modality and Information Priority on In-Vehicle Signing and Information Systems. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 38. 1072--1076.
33. M.A. Nees and B.N. Walker. 2011. Auditory displays for in-vehicle technologies. Reviews of human factors and ergonomics 7, 1 (2011), 58--99.
34. C.L. Paul, A. Komlodi, and W. Lutters. 2015. Interruptive notifications in support of task management. International Journal of Human-Computer Studies 79 (2015), 20--34.
35. T. W. Picton. 2010. Human auditory evoked potentials. Plural Publishing.
36. Ioannis Politis, Stephen Brewster, and Frank Pollick. 2015. Language-based Multimodal Displays for the Handover of Control in Autonomous Cars. In Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (Automotive UI '15). ACM, New York, NY, USA, 3--10.
37. Z. Pousman and J. Stasko. 2006. A Taxonomy of Ambient Information Systems: Four Patterns of Design. In Proceedings of the Working Conference on Advanced Visual Interfaces (AVI '06). ACM, New York, NY, USA, 67--74.
38. P. Praamstra, D.F. Stegeman, M.W.I.M. Horstink, and A.R. Cools. 1996. Dipole source analysis suggests selective modulation of the supplementary motor area contribution to the readiness potential. Electroencephalography and clinical neurophysiology 98, 6 (1996), 468--477.
39. T. Ross, K. Midtland, M. Fuchs, A. Pauzié, A. Engert, B. Duncan, G. Vaughan, M. Vernet, H. Peters, G. Burnett, and A. May. 1996. HARDIE Design Guidelines Handbook: Human Factors Guidelines for Information Presentation by ATT Systems. Technical Report V2008 HARDIE, Deliverable No. 20. European Commission Host Organization, CORDIS, Luxembourg. 1--562 pages.
40. M. Scheer, H. H. Bülthoff, and L. L Chuang. 2016. Steering Demands Diminish the Early-P3, Late-P3 and RON Components of the Event-Related Potential of Task-Irrelevant Environmental Sounds. Frontiers in human neuroscience 10, March (2016), 73.
41. D. L. Schomer and F. L. Da Silva. 2012. Niedermeyer's electroencephalography: basic principles, clinical applications, and related fields. Lippincott Williams & Wilkins.
42. M. Schultze-Kraft, D. Birman, M. Rusconi, C. Allefeld, K. Görgen, S. Dähne, B. Blankertz, and J-D. Haynes. 2016. The point of no return in vetoing self-initiated movements. Proceedings of the National Academy of Sciences 113, 4 (2016), 1080--1085.
43. H. Shibasaki and M. Hallett. 2006. What is the Bereitschaftspotential? Clinical Neurophysiology 117, 11 (2006), 2341--2356.
44. A. Stevens, A. Quimby, A. Board, T. Kersloot, and P. Burns. 2002. Design guidelines for safety of in-vehicle information systems. Technical Report TRL-PA-3721/01. Department of Transport, Local Government and the Regions. 1--55 pages. http://www.transport-research.info/Upload/Documents/200607/20060728
45. O. Tsimhoni and M. Flannagan. 2006. Pedestrian detection with night vision systems enhanced by automatic warnings. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 50. Sage Publications, 2443--2447.
46. C. D. Wickens. 2002. Multiple resources and performance prediction. Theoretical Issues in Ergonomics Science 3, 2 (2002), 159--177.
47. C. D. Wickens. 2008. Applied attention theory. Ergonomics 1, 2 (2008), 222.
48. T.O. Zander, L.M. Andreessen, A. Berg, M. Bleuel, J. Pawlitzki, L. Zawallich, L.R. Krol, and K. Gramann. 2017. Evaluation of a Dry EEG System for Application of Passive Brain-Computer Interfaces in Autonomous Driving. Frontiers in Human Neuroscience 11 (2017).
49. S. Zhang, R. Benenson, M. Omran, J. Hosang, and B. Schiele. 2016. How Far are We from Solving Pedestrian Detection?. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1259--1267.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User studies