EyeSee360: designing a visualization technique for out-of-view objects in head-mounted augmented reality

ABSTRACT

Head-mounted displays allow user to augment reality or dive into a virtual one. However, these 3D spaces often come with problems due to objects that may be out of view. Visualizing these out-of-view objects is useful under certain scenarios, such as situation monitoring during ship docking. To address this, we designed a lo-fi prototype of our EyeSee360 system, and based on user feedback, subsequently implemented EyeSee360. We evaluate our technique against well-known 2D off-screen object visualization techniques (Arrow, Halo, Wedge) adapted for head-mounted Augmented Reality, and found that EyeSee360 results in lowest error for direction estimation of out-of-view objects. Based on our findings, we outline the limitations of our approach and discuss the usefulness of our developed lo-fi prototyping tool.

References

1. Patrick Baudisch and Ruth Rosenholtz. 2003. Halo: A Technique for Visualizing Off-screen Objects. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, New York, NY, USA, 481--488.
2. Frank Biocca, Arthur Tang, Charles Owen, and Fan Xiao. 2006. Attention Funnel: Omnidirectional 3D Cursor for Mobile Augmented Reality Platforms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 1115--1122.
3. John Brooke and others. 1996. SUS-A quick and dirty usability scale. Usability evaluation in industry 189, 194 (1996), 4--7.
4. Stefano Burigat and Luca Chittaro. 2007. Navigation in 3D virtual environments: Effects of user experience and location-pointing navigation aids. International Journal of Human-Computer Studies 65, 11 (2007), 945--958.
5. Stefano Burigat, Luca Chittaro, and Silvia Gabrielli. 2006. Visualizing Locations of Off-screen Objects on Mobile Devices: A Comparative Evaluation of Three Approaches. In Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '06). ACM, New York, NY, USA, 239--246.
6. Andy Cockburn, Amy Karlson, and Benjamin B. Bederson. 2009. A Review of Overview+Detail, Zooming, and Focus+Context Interfaces. ACM Comput. Surv. 41, 1, Article 2 (Jan. 2009), 31 pages.
7. Marco de Sá and Elizabeth Churchill. 2012. Mobile Augmented Reality: Exploring Design and Prototyping Techniques. In Proceedings of the 14th International Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '12). ACM, New York, NY, USA, 221--230.
8. Uwe Gruenefeld, Abdallah El Ali, Wilko Heuten, and Susanne Boll. 2017. Visualizing Out-of-view Objects in Head-mounted Augmented Reality. In Proceedings of the 19th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA.
9. Sean Gustafson, Patrick Baudisch, Carl Gutwin, and Pourang Irani. 2008. Wedge: Clutter-free Visualization of Off-screen Locations. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 787--796.
10. Sean G. Gustafson and Pourang P. Irani. 2007. Comparing Visualizations for Tracking Off-screen Moving Targets. In CHI '07 Extended Abstracts on Human Factors in Computing Systems (CHI EA '07). ACM, New York, NY, USA, 2399--2404.
11. C.L. Hardin. 2000. Red and yellow, green and blue, warm and cool: explaining colour appearance. Journal of Consciousness Studies 7, 8--9 (2000), 113--122. http://www.ingentaconnect.com/content/imp/jcs/2000/00000007/F0020008/1046
12. Sandra G. Hart and Lowell E. Staveland. 1988. Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research. Advances in Psychology 52 (1988), 139 -- 183. ) 62386-9
13. Niels Henze, Benjamin Poppinga, and Susanne Boll. 2010. Experiments in the Wild: Public Evaluation of Off-screen Visualizations in the Android Market. In Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending Boundaries (NordiCHI '10). ACM, New York, NY, USA, 675--678.
14. Hyungeun Jo, Sungjae Hwang, Hyunwoo Park, and Jung-hee Ryu. 2011. Around-plot: Focus+ context interface for off-screen objects in 3D environments. Computers & Graphics 35, 4 (2011), 841--853.
15. James Kalat. 2015. Biological psychology. Nelson Education.
16. Adam M. Larson and Lester C. Loschky. 2009. The contributions of central versus peripheral vision to scene gist recognition. Journal of Vision 9, 10 (2009), 6. arXiv:/data/journals/jov/932854/jov-9-10-6.pdf
17. Felix Lauber, Claudius Böttcher, and Andreas Butz. 2014. PapAR: Paper Prototyping for Augmented Reality. In Adjunct Proceedings of the 6th International Conference on Automotive User Interfaces and Interactive Vehicular Applications (AutomotiveUI '14). ACM, New York, NY, USA, 1--6.
18. Ji-Yong Lee, Joung-Huem Kwon, Sang-Hun Nam, Joong-Jae Lee, and Bum-Jae You. 2016. Coexistent Space: Collaborative Interaction in Shared 3D Space. In Proceedings of the 2016 Symposium on Spatial User Interaction (SUI '16). ACM, New York, NY, USA, 175--175.
19. Kris Luyten, Donald Degraen, Gustavo Rovelo Ruiz, Sven Coppers, and Davy Vanacken. 2016. Hidden in Plain Sight: An Exploration of a Visual Language for Near-Eye Out-of-Focus Displays in the Peripheral View. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 487--497.
20. Marie-Christin Ostendorp, Jan Charles Lenk, and Andreas LÃijdtke. 2015. Smart Glasses to Support Maritime Pilots in Harbor Maneuvers. Procedia Manufacturing 3 (2015), 2840 -- 2847.
21. Benjamin I. Outram, Yun Suen Pai, Kevin Fan, Kouta Minamizawa, and Kai Kunze. 2016. AnyOrbit: Fluid 6DOF Spatial Navigation of Virtual Environments Using Orbital Motion. In Proceedings of the 2016 Symposium on Spatial User Interaction (SUI '16). ACM, New York, NY, USA, 199--199.
22. James F Parker Jr and Vita R West. 1973. Bioastronautics Data Book: NASA SP-3006. NASA Special Publication 3006 (1973).
23. Martin Pielot, Oliver Krull, and Susanne Boll. 2010. Where is My Team: Supporting Situation Awareness with Tactile Displays. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 1705--1714.
24. David Rudi, Ioannis Giannopoulos, Peter Kiefer, Christian Peier, and Martin Raubal. 2016. Interacting with Maps on Optical Head-Mounted Displays. In Proceedings of the 2016 Symposium on Spatial User Interaction (SUI '16). ACM, New York, NY, USA, 3--12.
25. Manojit Sarkar and Marc H Brown. 1994. Graphical fisheye views. Commun. ACM 37, 12 (1994), 73--83.
26. Torben Schinke, Niels Henze, and Susanne Boll. 2010. Visualization of Off-screen Objects in Mobile Augmented Reality. In Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI '10). ACM, New York, NY, USA, 313--316.
27. Teresa Siu and Valeria Herskovic. 2013. SidebARs: Improving Awareness of Off-screen Elements in Mobile Augmented Reality. In Proceedings of the 2013 Chilean Conference on Human - Computer Interaction (ChileCHI '13). ACM, New York, NY, USA, 36--41.
28. Richard Stoakley, Matthew J. Conway, and Randy Pausch. 1995. Virtual Reality on a WIM: Interactive Worlds in Miniature. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '95). ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 265--272.
29. Hans Strasburger, Ingo Rentschler, and Martin Jüttner. 2011. Peripheral vision and pattern recognition: A review. Journal of vision 11, 5 (2011), 13--13.
30. Mengu Sukan, Carmine Elvezio, Ohan Oda, Steven Feiner, and Barbara Tversky. 2014. ParaFrustum: Visualization Techniques for Guiding a User to a Constrained Set of Viewing Positions and Orientations. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology (UIST '14). ACM, New York, NY, USA, 331--340.
31. Marcus Tonnis and Gudrun Klinker. 2006. Effective Control of a Car Driver's Attention for Visual and Acoustic Guidance Towards the Direction of Imminent Dangers. In Proceedings of the 5th IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR '06). IEEE Computer Society, Washington, DC, USA, 13--22.
32. Matthias Trapp, Lars Schneider, Christine Lehmann, Norman Holz, and Jürgen Döllner. 2011. Strategies for visualising 3D points-of-interest on mobile devices. Journal of Location Based Services 5, 2 (2011), 79--99.
33. Polle T. Zellweger, Jock D. Mackinlay, Lance Good, Mark Stefik, and Patrick Baudisch. 2003. City Lights: Contextual Views in Minimal Space. In CHI '03 Extended Abstracts on Human Factors in Computing Systems (CHI EA '03). ACM, New York, NY, USA, 838--839.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods

      • User studies

    • Interaction paradigms

      • Mixed / augmented reality

  • Visualization

    • Visualization application domains

      • Information visualization