Investigation of fNIRS brain sensing as input to information filtering systems

ABSTRACT

Today's users interact with an increasing amount of information, demanding a similar increase in attention and cognition. To help cope with information overload, recommendation engines direct users' attention to content that is most relevant to them. We suggest that functional near-infrared spectroscopy (fNIRS) brain measures can be used as an additional channel to information filtering systems. Using fNIRS, we acquire an implicit measure that correlates with user preference, thus avoiding the cognitive interruption that accompanies explicit preference ratings. We explore the use of fNIRS in information filtering systems by building and evaluating a brain-computer movie recommender. We find that our system recommends movies that are rated higher than in a control condition, improves recommendations with increased interaction with the system, and provides recommendations that are unique to each individual.

References

1. H. Ayaz, P. A. Shewokis, S. Bunce, K. Izzetoglu, B. Willems, and B. Onaral. Optical brain monitoring for operator training and mental workload assessment. NeuroImage, 59(1):36--47, Jan. 2012.
2. B. Bailey, J. Konstan, and J. Carlis. The effects of interruptions on task performance, annoyance, and anxiety in the user interface. In INTERACT, volume 1, 2001.
3. B. P. Bailey and S. T. Iqbal. Understanding changes in mental workload during execution of goal-directed tasks and its application for interruption management. ACM Transactions on Computer-Human Interaction, 14(4):1--28, Jan. 2008.
4. G. J. Barnum and C. A. Mattson. A Computationally Assisted Methodology for Preference-Guided Conceptual Design. Journal of Mechanical Design, 132(12):121003, 2010.
5. A. Bechara, H. Damasio, A. R. Damasio, and G. P. Lee. Different contributions of the human amygdala and ventromedial prefrontal cortex to decision-making. The Journal of Neuroscience, 19(13):5473--81, July 1999.
6. C. Calcanis, V. Callaghan, M. Gardner, and M. Walker. Towards end-user physiological profiling for video recommendation engines. Intelligent Environments, pages 1C2--1C2, 2008.
7. I. Cantador, P. Brusilovsky, and T. Kuflik. Second workshop on information heterogeneity and fusion in recommender systems (HetRec2011). RecSys 2011, pages 387--388, 2011.
8. M. Claypool, D. Brown, and P. Le. Inferring user interest. IEEE Internet Computing, 2001.
9. E. Cutrell and D. Tan. BCI for passive input in HCI. In ACM CHI, volume 8, pages 1--3. Citeseer, 2007.
10. M. Deppe, W. Schwindt, H. Kugel, and H. Nonlinear responses within the medial prefrontal cortex reveal when specific implicit information influences economic decision making. Journal of Neuroimaging, 15:171--182, 2005.
11. S. H. Fairclough. Fundamentals of Physiological Computing. Interacting with Computers, 21:133--145, 2009.
12. X. Fu, J. Budzik, and K. Hammond. Mining navigation history for recommendation. In IUI '00, New York, New York, USA, 2000. ACM.
13. K. Gajos and D. S. Weld. Preference elicitation for interface optimization. UIST '05, 2005.
14. L. George and A. Lécuyer. An overview of research on 'passive' brain-computer interfaces for implicit human-computer interaction. In International Conference on Applied Bionics and Biomechanics, 2010.
15. A. Girouard, E. T. Solovey, L. M. Hirshfield, E. M. Peck, K. Chauncey, A. Sassaroli, S. Fantini, and R. J. Jacob. From Brain Signals to Adaptive Interfaces: Using fNIRS in HCI. In Brain-Computer Interfaces: Applying our Minds to Human-Computer Interaction. Springer, 2010.
16. J. Gluck, A. Bunt, and J. Mcgrenere. Matching Attentional Draw with Utility in Interruption. In CHI '07, 2007.
17. M. Hall, E. Frank, G. Holmes, and B. Pfahringer. The WEKA data mining software: an update. In ACM SIGKDD, volume 11, pages 10--18, 2009.
18. J. Healey, R. Picard, and F. Dabek. A New Affect-Perceiving Interface and Its Application to Personalized Music Selection The Affective DJ: Music Selection with the. Workshop on Perceptual User Interfaces, pages 2--5, 1998.
19. L. M. Hirshfield, R. Gulotta, S. Hirshfield, S. Hincks, M. Russel, R. Ward, T. Williams, and R. J. K. Jacob. This is Your Brain on Interfaces: Enhancing Usability Testing with Functional Near-Infrared Spectroscopy. In CHI '11, 2011.
20. L. M. Hirshfield, E. T. Solovey, A. Girouard, J. Kebinger, R. J. K. Jacob, A. Sassaroli, and S. Fantini. Brain Measurement for Usability Testing and Adaptive Interfaces: An Example of Uncovering Syntactic Workload with Functional Near Infrared Spectroscopy. In CHI '09, 2009.
21. E. Horvitz and J. Apacible. Learning and reasoning about interruption. In Multimodal Interfaces '03, pages 20--27. ACM, 2003.
22. J. Huber, D. Ariely, and G. Fischer. Expressing Preferences in a Principal-Agent Task: A Comparison of Choice, Rating and Matching. Organizational Behavior and Human Decision Processes, 87(1):66--90, 2002.
23. S. Koelstra, A. Yazdani, M. Soleymani, C. Muhl, J. Lee, A. Nijholt, T. Pun, T. Ebrahimi, and I. Patras. Single trial classification of EEG and peripheral physiological signals for recognition of emotions induced by music videos. Brain Informatics, pages 89--100, 2010.
24. J. Kohlmorgen, G. Dornhege, M. Braun, B. Blankertz, K.-R. Muller, G. Curio, K. Hagemann, A. Bruns, M. Schrauf, and W. E. Kincses. Improving human performance in a real operating environment through real-time mental workload detection. In Toward Brain-Computer Interfacing, pages 409--422. MIT Press, 2007.
25. S. Luu and T. Chau. Decoding subjective preference from single-trial near-infrared spectroscopy signals. Journal of Neural Engineering, 6, 2008.
26. G. Mark and D. Gudith. The cost of interrupted work: more speed and stress. In CHI '08, pages 8--11, 2008.
27. M. Marx and C. Schmandt. CLUES: dynamic personalized message filtering. In CSCW '96. ACM, 1996.
28. S. M. McClure, J. Li, D. Tomlin, K. S. Cypert, L. M. Montague, and P. R. Montague. Neural correlates of behavioral preference for culturally familiar drinks. Neuron, 44(2):379--87, Oct. 2004.
29. M. Morita and Y. Shinoda. Information filtering based on user behavior analysis and best match text retrieval. In SIGIR '94, 1994.
30. D. Nichols. Implicit rating and filtering. In DELOS Workshop on Filtering and Collaborative Filtering. Citeseer, 1997.
31. J. Parsons, P. Ralph, and K. Gallagher. Using viewing time to infer user preference in recommender systems. In AAAI Workshop on Semantic Web Personalization, 2004.
32. M. P. Paulus and L. R. Frank. Ventromedial prefrontal cortex activation is critical for preference judgments. Neuroreport, 14(10):1311--5, July 2003.
33. H. A. Simon. Designing Organizations for an Information-Rich World. Computers, Communications, and the Public Interest, pages 37--72, 1971.
34. E. Solovey, P. Schermerhorn, M. Scheutz, A. Sassaroli, S. Fantini, and R. Jacob. Brainput: Enhancing Interactive Systems with Streaming fNIRS Brain Input. In CHI '12. ACM Press, 2012.
35. E. T. Solovey, A. Girouard, K. Chauncey, L. M. Hirshfield, A. Sassaroli, F. Zheng, S. Fantini, and R. J. K. Jacob. Using fNIRS Brain Sensing in Realistic HCI Settings: Experiments and Guidelines. In UIST '09, 2009.
36. G. Strangman, J. P. Culver, J. H. Thompson, and D. A. Boas. A Quantitative Comparison of Simultaneous BOLD fMRI and NIRS Recordings during Functional Brain Activation. NeuroImage, 17(2):719--731, Oct. 2002.
37. J. Teevan, S. Dumais, and E. Horvitz. Characterizing the value of personalizing search. In Proc. of Research and Development in Information Retrieval 2007. ACM, 2007.
38. A. Villringer and B. Chance. Non-invasive optical spectroscopy and imaging of human brain function. Trends in Neurosciences, 20(10):435--42, Oct. 1997.
39. T.-L. Wu, Y.-P. Hung, Y.-Y. Chuang, H.-H. Chen, H. H. Chen, J.-H. Chen, S.-K. Jeng, H.-K. Wang, C.-C. Ho, Y.-P. Lin, T.-T. Hu, M.-F. Weng, L.-W. Chan, C.-H. Yang, and Y.-H. Yang. Interactive content presentation based on expressed emotion and physiological feedback. ACM Multimedia, 2008.
40. T. Zander, C. Kothe, S. Jatzev, and M. Gaertner. Enhancing human-computer interaction with input from active and passive brain-computer interfaces. Brain-Computer Interfaces, pages 181--199, 2010.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)