Head-mounted displays as opera glasses: using mixed-reality to deliver an egalitarian user experience during live events

ABSTRACT

This paper explores the use of head-mounted displays (HMDs) as a way to deliver a front row experience to any audience member during a live event. To do so, it presents a two-part user study that compares participants reported sense of presence across three experimental conditions: front row, back row, and back row with HMD (displaying 360° video captured live from the front row). Data was collected using the Temple Presence Inventory (TPI), which measures presence across eight factors. The reported sense of presence in the HMD condition was significantly higher in five of these measures, including spatial presence, social presence, passive social presence, active social presence, and social richness. We argue that the non-significant differences found in the other three factors – engagement, social realism, and perceptual realism – are artefacts of participants’ personal taste for the song being performed, or the effects of using a mixed-reality approach. Finally, the paper describes a basic system for low-latency, 360° video live streaming using off-the-shelf, affordable equipment and software.

References

1. Mark Billinghurst, Ivan Poupyrev, Hirokazu Kato, and Richard May. 2000. Mixing realities in Shared Space: an augmented reality interface for collaborative computing. In 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532), 1641–1644 vol.3.
2. Matthew Botvinick and Jonathan Cohen. 1998. Rubber hands “feel” touch that eyes see. Nature 391, 6669: 756–756.
3. Lauren deLisa Coleman. How The VR Concert Industry Is Boldly Jockeying For A Slice Of A Projected $660M Pie. Forbes. Retrieved August 13, 2017 from https://www.forbes.com/sites/laurencoleman/2017/02/23/how-the-vr-concertindustry-is-boldly-jockeying-for-a-slice-of-a-projected-660m-pie/
4. Michael Dowdy. 2007. Live Hip Hop, Collective Agency, and “Acting in Concert.” Popular Music and Society 30, 1: 75–91.
5. H. Henrik Ehrsson. 2007. The Experimental Induction of Out-of-Body Experiences. Science 317, 5841: 1048–1048.
6. Wijnand A IJsselsteijn, Yvonne A. W de Kort, and Antal Haans. 2006. Is This My Hand I See Before Me? The Rubber Hand Illusion in Reality, Virtual Reality, and Mixed Reality. Presence: Teleoperators and Virtual Environments 15, 4: 455–464.
7. Konstantina Kilteni, Jean-Marie Normand, Maria V. Sanchez-Vives, and Mel Slater. 2012. Extending Body Space in Immersive Virtual Reality: A Very Long Arm Illusion. PLOS ONE 7, 7: e40867.
8. Jane Lessiter, Jonathan Freeman, Edmund Keogh, and Jules Davidoff. 2001. A Cross-Media Presence Questionnaire: The ITC-Sense of Presence Inventory. Presence: Teleoperators and Virtual Environments 10, 3: 282–297.
9. Matthew Lombard, Theresa B. Ditton, and Lisa Weinstein. 2009. Measuring Presence: The Temple Presence Inventory. In Proceedings of the 12th Annual International Workshop on Presence, 1–15.
10. Paul Milgram and Fumio Kishino. 1994. A Taxonomy of Mixed Reality Visual Displays (Special Issue on Networked Reality). IEICE transactions on information and systems 77, 12: 1321–1329.
11. North by Northwestern. 2014. Turn it Off: Cell Phones and Concert Culture. Huffington Post. Retrieved August 13, 2017 from http://www.huffingtonpost.com/north-by-northwestern/turn-it-off-cellphones-a_b_5432289.html
12. Francesco Pavani and Massimiliano Zampini. 2007. The Role of Hand Size in the Fake-Hand Illusion Paradigm. Perception 36, 10: 1547–1554.
13. Rodrigo M. A. Silva, Bruno Feijó, Pablo B. Gomes, Thiago Frensh, and Daniel Monteiro. 2016. Real Time 360° Video Stitching and Streaming. In ACM SIGGRAPH 2016 Posters (SIGGRAPH ’16), 70:1–70:2.
14. Adalberto L. Simeone, Eduardo Velloso, and Hans Gellersen. 2015. Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI ’15), 3307–3316.
15. Filip Škola and Fotis Liarokapis. 2016. Examining the effect of body ownership in immersive virtual and augmented reality environments. The Visual Comp. 32, 6–8: 761–770.
16. Ina Wagner, Wolfgang Broll, Giulio Jacucci, Kari Kuutii, Rod McCall, Ann Morrison, Dieter Schmalstieg, and Jean-Jacques Terrin. 2009. On the Role of Presence in Mixed Reality. Presence: Teleoperators and Virtual Environments 18, 4: 249–276.
17. NVIDIA VRWorks TM. NVIDIA Developer. Retrieved May 21, 2017 from https://developer.nvidia.com/vrworks
18. Max_Worldmaking_Package: A package for Max/MSP/Jitter to support computational worldmaking. Computational Worldmaking Lab. Retrieved May 15, 2017 from https://github.com/worldmaking/Max_Worldmaking_Package
19. Theatregoers. London Opera Glass Company. Retrieved May 20, 2017 from zhttp://operaglasses.co.uk/theatregoers/
20. Virtual Reality Studio Session. Retrieved May 20, 2017 from https://www.edef.co.uk/event/154/
21. Cycling ’74. Retrieved May 15, 2017 from https://cycling74.com/
22. RICOH Live Streaming Driver (THETA UVC Blender) with Equirectangular Output. Retrieved August 13, 2017 from http://theta360.guide/communitydocument/uvc-theta.html
23. Danny Mooney Music - Home. Retrieved August 13, 2017 from https://www.facebook.com/DannyMooneyMusic/
24. Murray Lunam Drums - Home. Retrieved August 13, 2017 from https://www.facebook.com/MurrayLunamDrums/

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Empirical studies in HCI

  • Ubiquitous and mobile computing

    • Empirical studies in ubiquitous and mobile computing