Multi-reality games: an experience across the entire reality-virtuality continuum

ABSTRACT

Interactive play can take very different forms, from playing with physical board games to fully digital video games. In recent years, new video game paradigms were introduced to connect real-world objects to virtual game characters. However, even these applications focus on a specific section of the Reality-Virtuality Continuum, where the visual embodiment of characters is either largely static toys in the real world or pre-animated within the virtual world according to a determined set of motions.

We introduce a novel concept, called Multi-Reality Games, that encompasses interactions with real and virtual objects to span the entire spectrum of the Reality-Virtuality Continuum, from the real world to digital and/or back. Our application on real-virtual game interaction makes an evolutionary step toward the convergence of real and virtual game characters. Rather than static toys or pre-built and unconfigurable virtual counterparts, we bring together technologies from the entire Reality-Virtuality Continuum to target new game experiences.

We showcase our framework by proposing a game application on a mobile device. Without the need to change the location or set, we enable intuitive and seamless interactions between physical, augmented and virtual elements. The experience brings both worlds closer, and enables the user to customize the virtual scenario according to physical references.

References

1. Raphael Anderagg, Loïc Ciccone, and Robert W. Sumner. 2018. PuppetPhone: Puppeteering Virtual Characters Using a Smartphone. In Proceedings of ACM SIGGRAPH Conference on Motion, Interaction and Games.
2. E. Andre. 2006. Engaging in a Conversation with Synthetic Agents along the Virtuality Continuum. In 2006 IEEE/WIC/ACM International Conference on Intelligent Agent Technology. 19--20.
3. B. Besbes, S. N. Collette, M. Tamaazousti, S. Bourgeois, and V. Gay-Bellile. 2012. An interactive Augmented Reality system: A prototype for industrial maintenance training applications. In 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). 269--270.
4. Llogari Casas, Matthias Fauconneau, Maggie Kosek, , Kieran Mclister, and Kenny Mitchell. 2018. Image Based Proximate Shadow Retargeting. In Proceedings of the Computer Graphics and Visual Computing (CGVC) Conference 2018. Swansea, Wales, United Kingdom.
5. Llogari Casas, Maggie Kosek, and Kenny Mitchell. 2017. Props Alive : A Framework for Augmented Reality Stop Motion Animation. In 2017 IEEE 10th Workshop on Software Engineering and Architectures for Realtime Interactive Systems.
6. Gokcen Cimen, Christoph Maurhofer, Bob Sumner, and Martin Guay. 2018. AR Poser: Automatically Augmenting Mobile Pictures with Digital Avatars Imitating Poses. In 12th International Conference on Computer Graphics, Visualization, Computer Vision and Image Processing 2018.
7. L. Davis, J. Rolland, F. Hamza-Lup, Yonggang Ha, J. Norfleet, and C. Imielinska. 2003. Enabling a continuum of virtual environment experiences. IEEE Computer Graphics and Applications 23, 2 (2003), 10--12.
8. Lucio Gutierrez, Eleni Stroulia, and Ioanis Nikolaidis. 2012. fAARS: A Platform for Location-Aware Trans-reality Games. In Entertainment Computing - ICEC 2012, Marc Herrlich, Rainer Malaka, and Maic Masuch (Eds.). 185--192.
9. K. P. Jantke, O. Arnold, and S. Spundflasch. 2013. Aliens on the Bus: A family of pervasive games. In 2013 IEEE 2nd Global Conference on Consumer Electronics (GCCE). 387--391.
10. Bergstrom Karl, Jonsson Staffan, and Bjork Staffan. 2010. Undercurrents: A Computer-Based Gameplay Tool to Support Tabletop Roleplaying. In DiGRA Nordic: Proceedings of the 2010 International DiGRA Nordic Conference: Experiencing Games: Games, Play, and Players.
11. Vlasios Kasapakis, Damianos Gavalas, and Nikos Bubaris. 2013. Pervasive Games Research: A Design Aspects-based State of the Art Report. In Proceedings of the 17th Panhellenic Conference on Informatics. 152--157.
12. A. R. Khairuddin, M. S. Talib, and H. Haron. 2015. Review on simultaneous localization and mapping (SLAM). In 2015 IEEE International Conference on Control System, Computing and Engineering (ICCSCE). 85--90.
13. Eric Klopfer and Kurt Squire. 2008. Environmental Detectives---the development of an augmented reality platform for environmental simulations. Educational Technology Research and Development 56, 2 (2008), 203--228.
14. L. Frank Baum. 1901. The Master Key: an Electrical Fairy Tale. Bowen-Merrill (1901), 102.
15. Miikka J. Lehtonen and J. Tuomas Harviainen. 2016. Mobile Games and Player Communities: Designing for and with Clans. Design Management Review 27, 3 (2016), 20--26.
16. Craig A. Lindley. 2005. Game Space Design Foundations for Trans-reality Games. In Proceedings of the 2005 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology. 397--404.
17. Stéphane Magnenat, Dat Tien Ngo, Fabio Zünd, Mattia Ryffel, Gioacchino Noris, Gerhard Rothlin, Alessia Marra, Maurizio Nitti, Pascal Fua, Markus Gross, and Robert W. Sumner. 2015. Live Texturing of Augmented Reality Characters from Colored Drawings. IEEE Transactions on Visualization and Computer Graphics 21, 11 (2015), 1201--1210.
18. Paul Milgram, H. Takemura, A. Utsumi, and F. Kishino. 1994. Mixed Reality (MR) Reality-Virtuality (RV) Continuum. Systems Research 2351 (1994), 282--292.
19. Niantic and The Pokemon Company. 2016. Pokemon GO.
20. A. Nijholt. 2005. Meetings in the virtuality continuum: send your avatar. In 2005 International Conference on Cyberworlds (CW'05). 8 pp.-82.
21. Max Rheiner. 2014. Birdly an Attempt to Fly. In ACM SIGGRAPH 2014 Emerging Technologies. Article 3, 3:1--3:1 pages.
22. Ivan E. Sutherland. 1968. A head-mounted three dimensional display. In Proceedings of the AFIPS '68 (Fall, part I). 757--764.
23. D. Wagner and D. Schmalstieg. 2003. First steps towards handheld augmented reality. Seventh IEEE International Symposium on Wearable Computers, 2003. Proceedings. (2003), 127--135.
24. Andrzej Zarzycki. 2012. Urban Games: Inhabiting Real and Virtual Cities. In Proceedings of the 30th eCAADe Conference - Volume 1 / ISBN 978-9-4912070-2-0, Czech Technical University in Prague, Faculty of Architecture (Czech Republic) 12-14 September 2012, pp. 755--764.

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Mixed / augmented reality

• Human-centered computing

  • Interaction design

    • Interaction design theory, concepts and paradigms