Gaze-directed Immersive Visualization of Scientific Ensembles

ABSTRACT

The latest advances in head-mounted displays (HMDs) for augmented reality (AR) and mixed reality (MR) have produced commercialized devices that are gradually accepted by the public. These HMDs are generally equipped with head tracking, which provides an excellent input to explore immersive visualization and interaction techniques for various AR/MR applications. This paper explores the head tracking function on the latest Microsoft HoloLens -- where gaze is defined as the ray starting at the head location and points forward. We present a gaze-directed visualization approach to study ensembles of 2D oil spill simulations in mixed reality. Our approach allows users to place an ensemble as an image stack in a real environment and explore the ensemble with gaze tracking. The prototype system demonstrates the challenges and promising effects of gaze-based interaction in the state-of-the-art mixed reality.

References

1. Amer Al-Rahayfeh and Miad Faezipour. 2013. Eye Tracking and Head Movement Detection: A State-of-Art Survey. IEEE Journal of Translational Engineering in Health and Medicine 1 (2013).
2. Fabian Beck, Tanja Blascheck, Thomas Ertl, and Daniel Weiskopf. 2015. Exploring Word-Sized Graphics for Visualizing Eye Tracking Data within Transcribed Experiment Recordings. In ETVIS2015.
3. T. Blascheck, K. Kurzhals, M. Raschke, M. Burch, D. Weiskopf, and T. Ertl. 2014. State-of-the-Art of Visualization for Eye Tracking Data. In EuroVis - STARs, R. Borgo, R. Maciejewski, and I. Viola (Eds.). The Eurographics Association.
4. B. Brinkman. 2012. Willing to be fooled: Security and autoamputation in augmented reality. In 2012 IEEE International Symposium on Mixed and Augmented Reality - Arts, Media, and Humanities (ISMAR-AMH). 89--90.
5. T. Chandler, M. Cordeil, T. Czauderna, T. Dwyer, J. Glowacki, C. Goncu, M. Klapperstueck, K. Klein, K. Marriott, F. Schreiber, and E. Wilson. 2015. Immersive Analytics. In Big Data Visual Analytics (BDVA), 2015. 1--8.
6. Henry Chen, Austin S Lee, Mark Swift, and John C Tang. 2015. 3D Collaboration Method over HoloLens™ and Skype™ End Points. In Proceedings of the 3rd International Workshop on Immersive Media Experiences. ACM, 27--30.
7. Adrian James Chung, Fani Deligianni, Xiao-Peng Hu, and Guang-Zhong Yang. 2004. Visual Feature Extraction via Eye Tracking for Saliency Driven 2D/3D Registration. In Proceedings of the 2004 Symposium on Eye Tracking Research & Applications (ETRA '04). 49--54.
8. J.C. Dietrich, C.J. Trahan, and etc. 2012. Surface trajectories of oil transport along the Northern Coastline of the Gulf of Mexico. Continental Shelf Research 41 (2012), 17--47.
9. S. Hillaire, A. Lecuyer, T. Regia-Corte, R. Cozot, J. Royan, and G. Breton. 2012. Design and Application of Real-Time Visual Attention Model for the Exploration of 3D Virtual Environments. IEEE Transactions on Visualization and Computer Graphics 18, 3 (2012), 356--368.
10. M. Hlawatsch, M. Burch, F. Beck, J. Freire, C. Silva, and D. Weiskopf. 2015. Visualizing the Evolution of Module Workflows. In 2015 19th International Conference on Information Visualisation. 40--49.
11. Youngmin Kim, Amitabh Varshney, David W. Jacobs, and François Guimbretière. 2010. Mesh Saliency and Human Eye Fixations. ACM Trans. Appl. Percept. 7, 2 (2010), 12:1--12:13.
12. A. Kotranza, D. Scott Lind, C. M. Pugh, and B. Lok. 2009. Real-time in-situ visual feedback of task performance in mixed environments for learning joint psychomotor-cognitive tasks. In Mixed and Augmented Reality, 2009. ISMAR 2009. 8th IEEE International Symposium on. 125--134.
13. Chang Ha Lee, Amitabh Varshney, and David W. Jacobs. 2005. Mesh Saliency. ACM Trans. Graph. 24, 3 (2005), 659--666.
14. S. Lee, G. J. Kim, and S. Choi. 2009. Real-Time Tracking of Visually Attended Objects in Virtual Environments and Its Application to LOD. IEEE Transactions on Visualization and Computer Graphics 15, 1 (2009), 6--19.
15. Aidong Lu, Ross Maciejewski, and David S. Ebert. 2010. Volume Composition and Evaluation Using Eye-tracking Data. ACM Trans. Appl. Percept. 7, 1 (2010), 4:1--4:20.
16. J.F. Mackworth and N.H. Mackworth. 1958. Eye fixations recorded on changing visual scenes by the television eye-marker. Journal of the Optical Society of America 48, 7 (1958), 439--444.
17. Thies Pfeiffer and Patrick Renner. 2014. EyeSee3D: A Low-cost Approach for Analyzing Mobile 3D Eye Tracking Data Using Computer Vision and Augmented Reality Technology. In Proceedings of the Symposium on Eye Tracking Research and Applications (ETRA '14). 195--202.
18. Dariusz Zapala and Bibianna Balaj. 2012. Eye Tracking and Head Tracking -- The two approaches in assistive technologies. (2012), 2406--2415.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Graphics input devices

• Information systems

  • Information systems applications

    • Multimedia information systems