Towards cooperative brain-computer interfaces for space navigation

ABSTRACT

We explored the possibility of controlling a spacecraft simulator using an analogue Brain-Computer Interface (BCI) for 2-D pointer control. This is a difficult task, for which no previous attempt has been reported in the literature. Our system relies on an active display which produces event-related potentials (ERPs) in the user's brain. These are analysed in real-time to produce control vectors for the user interface. In tests, users of the simulator were told to pass as close as possible to the Sun. Performance was very promising, on average users managing to satisfy the simulation success criterion in 67.5% of the runs. Furthermore, to study the potential of a collaborative approach to spacecraft navigation, we developed BCIs where the system is controlled via the integration of the ERPs of two users. Performance analysis indicates that collaborative BCIs produce trajectories that are statistically significantly superior to those obtained by single users.

References

1. Allison, B. Z., Wolpaw, E. W., and Wolpaw, J. R. Brain-computer interface systems: progress and prospects. Expert Review of Medical Devices 4, 4 (Jul 2007), 463--474.
2. Bell, C. J., Shenoy, P., Chalodhorn, R., and Rao, R. P. N. Control of a humanoid robot by a noninvasive braincomputer interface in humans. Journal of Neural Engineering 5, 2 (2008), 214.
3. Beverina, F., Palmas, G., Silvoni, S., Piccione, F., and Giove, S. User adaptive BCIs: SSVEP and P300 based interfaces. PsychNology Journal 1, 4 (2003), 331--354.
4. Birbaumer, N., Ghanayim, N., Hinterberger, T., Iversen, I., Kotchoubey, B., Kübler, A., Perelmouter, J., Taub, E., and Flor, H. A spelling device for the paralysed. Nature 398, 6725 (Mar 1999), 297--298.
5. Birbaumer, N., Hinterberger, T., Kübler, A., and Neumann, N. The thought-translation device (TTD): neurobehavioral mechanisms and clinical outcome. IEEE Transactions on Neural System and Rehabilitation Engineering 11, 2 (Jun 2003), 120--123.
6. Broschart, M., de Negueruela, C., Millán, J. d. R., and Menon, C. Augmenting astronaut's capabilities through brain-machine interfaces. In Proceedings of the 20th International Joint Conference on Artificial Intelligence, Workshop on Artificial Intelligence for Space Applications (1 2007).
7. Citi, L., Poli, R., Cinel, C., and Sepulveda, F. P300-based BCI mouse with genetically-optimized analogue control. IEEE Transactions on Neural Systems and Rehabilitation Engineering 16, 1 (Feb. 2008), 51--61.
8. Coffey, E. B., Brouwer, A.-M., Wilschut, E. S., and van Erp, J. B. Brainmachine interfaces in space: Using spontaneous rather than intentionally generated brain signals. Acta Astronautica 67, 1-2 (2010), 1--11.
9. Coyle, S. M., Ward, T. E., and Markham, C. M. Brain-computer interface using a simplified functional near-infrared spectroscopy system. Journal of Neural Engineering 4, 3 (Sep 2007), 219--226.
10. Curran, E. A., and Stokes, M. J. Learning to control brain activity: a review of the production and control of EEG components for driving brain-computer interface (BCI) systems. Brain Cognition 51, 3 (Apr 2003), 326--336.
11. Donoghue, J. Connecting cortex to machines: recent advances in brain interfaces. Nature Neuroscience 5 (2002), 1085--1088.
12. Eckstein, M. P., Das, K., Pham, B. T., Peterson, M. F., Abbey, C. K., Sy, J. L., and Giesbrecht, B. Neural decoding of collective wisdom with multi-brain computing. NeuroImage 59, 1 (2012), 94--108.
13. Farwell, L. A., and Donchin, E. Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalography and Clinical Neurophysiology 70, 6 (Dec 1988), 510--523.
14. Finke, A., Knoblauch, A., Koesling, H., and Ritter, H. J. A hybrid brain interface for a humanoid robot assistant. In 33rd Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC2011), IEEE, IEEE (Boston, MA, USA, 30.08.2011 2011).
15. Galán, F., Nuttin, M., Lew, E., Ferrez, P., Vanacker, G., Philips, J., and Millán, J. A brain-actuated wheelchair: Asynchronous and non-invasive brain-computer interfaces for continuous control of robots. Clinical Neurophysiololy (2008).
16. Georgopoulos, A., Langheim, F., Leuthold, A., and Merkle, A. Magnetoencephalographic signals predict movement trajectory in space. Experimental Brain Research (Jul 2005), 1--4.
17. Hand, D. J., and Till, R. J. A simple generalisation of the area under the ROC curve for multiple class classification problems. Machine Learning 45 (2001), 171--186.
18. Hilton, B. D. Space commander (version 0.4). http://spacecommander.sourceforge.net/, 2006.
19. Lebedev, M., and Nicolelis, M. Brain-machine interfaces: past, present and future. Trends in Neurosciences 29, 9 (September 2006), 536--546.
20. Luck, S. J. An introduction to the event-related potential technique. MIT Press, Cambridge, Massachusetts, 2005.
21. Menon, C., de Negueruela, C., del Millán, J., Tonet, O., Carpi, F., Broschart, M., Ferrez, P., Buttfield, A., Dario, P., Citi, L., Laschi, C., Tombini, M., Sepulveda, F., Poli, R., Palaniappan, R., Tecchio, F., Rossini, P. M., and Rossi, D. D. Prospective on brain-machine interfaces for space system control. In 57th Astronautical Congress (Spain, 2-6 Oct 2006). paper IAC-06-D1.1.05.
22. Menon, C., de Negueruela, C., del Millán, J., Tonet, O., Carpi, F., Broschart, M., Ferrez, P., Buttfield, A., Tecchio, F., Sepulveda, F., Citi, L., Laschi, C., Tombini, M., Dario, P., Rossini, M., and Rossi, D. D. Prospects of brain-machine interfaces for space system control. Acta Astronautica 64, 4 (2009), 448--456.
23. Millan, J. D. R., Renkens, F., Mourino, J., and Gerstner, W. Noninvasive brain-actuated control of a mobile robot by human EEG. IEEE Transactions on Biomedical Engineering 51 (2004), 1026--1033.
24. Müller, G. R., Scherer, R., Neuper, C., and Pfurtscheller, G. Steady-state somatosensori evoked potentials: Suitable brain signals for brain-computer interfaces? IEEE Transactions on Neural Systems and Rehabilitation Engineering 14, 1 (2006), 30--37.
25. Pfurtscheller, G., Flotzinger, D., and Kalcher, J. Brain-computer interface: a new communication device for handicapped persons. Journal of Microcomputer Applications 16, 3 (1993), 293--299.
26. Poli, R., Cinel, C., Sepulveda, F., and Stoica, A. Improving decision-making based on visual perception via a collaborative brain-computer interface. In IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA), IEEE (San Diego (CA), February 2013). (to appear).
27. Poli, R., Salvaris, M., and Cinel, C. Evolutionary synthesis of a trajectory integrator for an analogue brain-computer interface mouse. In Applications of Evolutionary Computing, EvoApplications 2011, C. Di Chio, S. Cagnoni, C. Cotta, M. Ebner, A. Ekart, A. I. Esparcia-Alcazar, J. J. Merelo, F. Neri, M. Preuss, H. Richter, J. Togelius, and G. N. Yannakakis, Eds., vol. 6624 of LNCS, Springer Verlag (Turin, Italy, 27-29 Apr. 2011), 214--223.
28. Polikoff, J. B., Bunnell, H. T., and Borkowski Jr, W. J. Toward a P300-based computer interface. In Rehabilitation Engineering and Assistive Technology Society of North America (RESNA'95), Resna Press (Arlington, Va, 1995), 178--180.
29. Reza, F.-R., Z, A. B., Christoph, G., W, S. E., C, K. S., and Andrea, K. P300 brain computer interface: current challenges and emerging trends. Frontiers in Neuroengineering 5, 14 (2012).
30. Salvaris, M., Cinel, C., Citi, L., and Poli, R. Novel protocols for P300-based brain-computer interfaces. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 1 (2012), 8--17.
31. Schwartz, A. B. Cortical neural prosthetics. Annual Review of Neuroscience 27 (2004), 487--507.
32. Sellers, E. W., and Donchin, E. A P300-based brain-computer interface: Initial tests by ALS patients. Clinical Neurophysiology 117, 3 (Mar 2006), 538--548.
33. Soon, C. S., Brass, M., Heinze, H.-J., and Haynes, J.-D. Unconscious determinants of free decisions in the human brain. Nature Neuroscience 11, 5 (April 2008), 543--545.
34. Stoica, A. Multimind: Multi-brain signal fusion to exceed the power of a single brain. In EST, A. Stoica, D. Zarzhitsky, G. Howells, C. D. Frowd, K. D. McDonald-Maier, A. T. Erdogan, and T. Arslan, Eds., IEEE Computer Society (2012), 94--98.
35. Trejo, L. J., Rosipal, R., and Matthews, B. Brain-computer interfaces for 1-D and 2-D cursor control: Designs using volitional control of the EEG spectrum or steady-state visual evoked potentials. IEEE Transactions on Neural Systems and Rehabilitation Engineering 14, 2 (June 2006), 225--229.
36. Tzovara, A., Murray, M. M., Bourdaud, N., Chavarriaga, R., del R. Millán, J., Lucia, M. D., and Lucia, M. D. The timing of exploratory decision-making revealed by single-trial topographic EEG analyses. NeuroImage (2012), 1959--1969.
37. Wang, Y., and Jung, T.-P. A Collaborative Brain-Computer Interface for Improving Human Performance. PLoS ONE 6, 5 (May 2011), e20422+.
38. Weiskopf, N., Mathiak, K., Bock, S. W., Scharnowski, F., Veit, R., Grodd, W., Goebel, R., and Birbaumer, N. Principles of a brain-computer interface (BCI) based on real-time functional magnetic resonance imaging (fMRI). IEEE Transactions on Biomedical Engineering 51, 6 (Jun 2004), 966--970.
39. Wilson, J. J., and Palaniappan, R. Analogue mouse pointer control via an online steady state visual evoked potential (SSVEP) brain-computer interface. Journal of Neural Engineering 8 (October 2011).
40. Wolpaw, J. R., Birbaumer, N., McFarland, D. J., Pfurtscheller, G., and Vaughan, T. M. Brain-computer interfaces for communication and control. Clinical Neurophysiology 113, 6 (June 2002), 767--91.
41. Wolpaw, J. R., and McFarland, D. J. Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. Proceedings of the National Academy of Sciences 101, 51 (2004), 17849--17854.
42. Wolpaw, J. R., McFarland, D. J., Neat, G. W., and Forneris, C. A. An EEG-based brain-computer interface for cursor control. Electroencephalography and Clinical Neurophysiology 78, 3 (Mar 1991), 252--259.
43. Yuan, P., Wang, Y., Wu, W., Xu, H., Gao, X., and Gao, S. Study on an online collaborative BCI to accelerate response to visual targets. In Proceedings of 34nd IEEE EMBS Conference (2012).

Index Terms

• Human-centered computing