Brain measurement for usability testing and adaptive interfaces: an example of uncovering syntactic workload with functional near infrared spectroscopy

ABSTRACT

A well designed user interface (UI) should be transparent, allowing users to focus their mental workload on the task at hand. We hypothesize that the overall mental workload required to perform a task using a computer system is composed of a portion attributable to the difficulty of the underlying task plus a portion attributable to the complexity of operating the user interface. In this regard, we follow Shneiderman's theory of syntactic and semantic components of a UI. We present an experiment protocol that can be used to measure the workload experienced by users in their various cognitive resources while working with a computer. We then describe an experiment where we used the protocol to quantify the syntactic workload of two user interfaces. We use functional near infrared spectroscopy, a new brain imaging technology that is beginning to be used in HCI. We also discuss extensions of our techniques to adaptive interfaces.

References

1. Baddeley, A. and Della Sala, S. Working memory and executive control. Philosophical Transactions of the Royal Society of London, 351, (1996), 1397--403.
2. Boechler, P. How Spatial Is Hyperspace? Interacting with Hypertext Documents: Cognitive Processes and Concepts. CyberPsychology and Behavior, 4,1,(2001) 23--46.
3. Chance, B., Anday, E., Nioka, S., Zhou, S., Hong, L., Worden, K., Li, C. Murray, T., Ovetsky, Y., Thomas, R. A novel method for fast imaging of brain function, non-invasively, with light. Optics Express, 10, 2, (1988), 411--423.
4. Czerwinski, M. and Larson, K. Cognition and the Web: Moving from Theory to Web Design. in Human Factors and Web Development, Ratner, J. (Ed.), Erlbaum: NJ, (2002), 147--165.
5. Eckstrom, R., French, J., Harman, H. and Derman, D. Kit of factor-referenced cognitive tests. (1976).
6. Gevins, A., Smith, M., McEvoy, L. and Yu, D. High-Resolution EEG Mapping of Cortical Activation Related to Working Memory: Effects of Task Difficulty, Type of Processing, and Practice. Cerebral Cortex, (1997), 374--385.
7. Gratton, G., Fabiani, M., Friedman, D., Franceschini, M., Fantini, S., Corballis, P. and Gratton, E. Rapid Changes of Optical Parameters in the Human Brain During a Tapping Task. Journal of Cognitive Neuroscience, 7. (1995), 446--456.
8. Grimes, D., Tan, D., Hudson, S., Shenoy, P. and Rao, R., Feasibility and Pragmatics of Classifying Working Memory Load with an Electroencephalograph. in Proc CHI 2008. ACM Press (2008).
9. Hart, S.G. and Staveland, L.E. Development of NASA-TLX : Results of empirical and theoretical research. in Hancock, P., Meshkati, N. ed. Human Mental Workload, Amsterdam, (1988), pp 139 -- 183.
10. Izzetoglu, K., Bunce, S., Onaral, B., Pourrezaei, K. and Chance, B. Functional Optical Brain Imaging Using Near-Infrared During Cognitive Tasks. Int. Journal of Human-Computer Interaction, 17, 2. (2004). 211--231.
11. John, M.S., Kobus, D., Morrison, J. and Schmorrow, D. Overview of the DARPA Augmented Cognition Technical Integration Experiment. Int. Journal of Human-Computer Interaction, 17, 2. (2004), 131--149.
12. Keogh, E. and Pazzani, M., Scaling up dynamic time warping for datamining applications. in Proc. of the Sixth ACM SIGKDD, (2000).
13. Kohlmorgen, J., Dornhege, G., Braun, M, Blankertz, B., Muller, K., Curio, G., Hagemann, K., Bruns, A., Sharuf, M.,Kincses, W. Improving Human Performance in a Real Operating Environment through Real-Time Mental Workload Detection. Toward Brain Computer Interfacing, MIT Press, (2007), 409--422.
14. Larson, K. and Czerwinski, M., Web Page Design: Implications of Memory, Structure and Scent for Information Retrieval. in Proc.CHI 1998, ACM Press, (1998).
15. Lee, J.C. and Tan, D.S., Using a Low-Cost Electroencephalograph for Task Classification in HCI Research. in Proc. UIST 2006. ACM Press, (2006).
16. Leung, H., Oh, H., Ferri, J. and Yi, Y. Load Response Functions in the Human Spatial Working Memory Circuit During Location Memory Updating. NeuroImage, 35, (2007), 368--377.
17. Muller, K.T., M., Dornhege, G., Krauledat, M., Curio, G. and Blankertz, B. Machine learning for real-time single-trial EEG-analysis: From Brain-computer interfacing to mental state monitoring. Journal of Neuroscience Methods, 167, 1, (2008), 82--90.
18. Parasuraman, R. and Caggiano, D. Neural and Genetic Assays of Human Mental Workload. in Quantifying Human Information Processing, Lexington Books, (2005), 123--149.
19. Sassaroli, A., Zheng, F., Hirshfield, L.M., Girouard, A., Solovey, E.T., Jacob, R.J.K. and Fantini, S. Discrimination of mental workload levels in human subjects with functional near-infrared spectroscopy. in the Journal of Innovative Optical Health Sciences, (2009), 227--237.
20. Shneiderman, B. and Plaisant, C. Designing the User Interface: Strategies for Effective Human-Computer Interaction, Fourth Edition, Addison-Wesley, Reading, Mass. (2005), 86--88.
21. Smith, E. and Jonides, J. Storage and Executive Processes n the Frontal Lobes. Science, 283, (1999), 1657--1661.
22. Son, I.-Y., Guhe, M., Gray, W., Yazici, B. and Schoelles, M. Human performance assessment using fNIR. Proc. of SPIE The International Society for Optical Engineering, 5797. (2005), 158--169.
23. Wickens, C., Lee, J., Liu, Y., Becker, S. Introduction to Human Factors Engineering. Pearson, (2004).
24. Witten, I.H. and Frank, E. Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann, San Francisco, (2005), 196--197.
25. Zhang, Q., Brown, E. and Strangman, G. Adaptive filtering for global interference cancellation and real-time recovery of evoked brain activity: a Monte Carlo simulation study. Journal of biomedical optics, 12, (2007), 044014-01 - 044014-12.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Graphical user interfaces