Design experiences of multimodal mixed reality interfaces

ABSTRACT

This paper presents an overview of the most significant issues when designing mixed reality interfaces including displays, tracking, interface design, interactivity and realism. Multimodal issues regarding visualization and interaction are integrated into a single interface. Three case studies in diverse areas including automotive, archaeology and navigation are presented illustrating the use of the above issues addressed. Furthermore, the experiences gained and lessons learned are discussed including our plans for future work.

REFERENCES

• 1: Milgram, P. and Kishino, F. A Taxonomy of Mixed Reality Visual Displays. IEICE Transactions on Information Systems, Vol E77-D, 12, (December, 1994), 1321--1329.
• 2: Anton Fuhrmann , Helwig Löffelmann , Dieter Schmalstieg , Michael Gervautz, Collaborative Visualization in Augmented Reality, IEEE Computer Graphics and Applications, v.18 n.4, p.54-59, July 1998 [doi>10.1109/38.689665 ]
• 3: Morten Fjeld, Usability and collaborative aspects of augmented reality, interactions, v.11 n.6, November + December 2004 [doi>10.1145/1029036.1029044]
• 4: Steve Benford , Chris Greenhalgh , Gail Reynard , Chris Brown , Boriana Koleva, Understanding and constructing shared spaces with mixed-reality boundaries, ACM Transactions on Computer-Human Interaction (TOCHI), v.5 n.3, p.185-223, Sept. 1998 [doi>10.1145/292834.292836]
• 5: Shichao Ou , Deepak R. Karuppiah , Andrew H. Fagg , Edward Riseman, An Augmented Virtual Reality Interface for Assistive Monitoring of Smart Spaces, Proceedings of the Second IEEE International Conference on Pervasive Computing and Communications (PerCom'04), p.33, March 14-17, 2004
• 6: Ronald Azuma , Yohan Baillot , Reinhold Behringer , Steven Feiner , Simon Julier , Blair MacIntyre, Recent Advances in Augmented Reality, IEEE Computer Graphics and Applications, v.21 n.6, p.34-47, November 2001 [doi>10.1109/38.963459 ]
• 7: Ed Lantz, The future of virtual reality: head mounted displays versus spatially immersive displays (panel), Proceedings of the 23rd annual conference on Computer graphics and interactive techniques, p.485-486, August 1996 [doi>10.1145/237170.237289]
• 8: Feiner, S. K. Augmented Reality: A New Way of Seeing. Scientific American, 286, 4, (April 24, 2002), 48--55.
• 9: Fuchs, H., and Ackerman, J. Displays for Augmented Reality: Historical Remarks and Future Prospects. Mixed Reality Merging Real and Virtual Worlds, Ohta Y and Tamura H, Ohmsha Ltd, Chapter 2, (1999), 31--40.
• 10: Ronald Azuma, Tracking requirements for augmented reality, Communications of the ACM, v.36 n.7, p.50-51, July 1993 [doi>10.1145/159544.159581]
• 11: Klinker, G., and Stricker D. Augmented Reality: A Balancing Act Between High Quality and Real-Time Constraints. Mixed Reality Merging Real and Virtual Worlds, Ohta Y and Tamura H, 1999 Ohmsha Ltd, Chapter 18, (1999), 325--346.
• 12: J. P. Mellor, Realtime Camera Calibration for Enhanced Reality Visualization, Proceedings of the First International Conference on Computer Vision, Virtual Reality and Robotics in Medicine, p.471-475, April 03-06, 1995
• 13: Michael Bajura , Ulrich Neumann, Dynamic Registration Correction in Video-Based Augmented Reality Systems, IEEE Computer Graphics and Applications, v.15 n.5, p.52-60, September 1995 [doi>10.1109/38.403828 ]
• 14: ARTag. http://www.artag.net/
• 15: Kato, H., Billinghurst, M., et al., Virtual Object Manipulation on a Table-Top AR Environment. In Proceedings of the International Symposium on Augmented Reality, IEEE Computer Society, Munich, 5-6, (October, 2000), 111--119.
• 16: Jun Rekimoto , Yuji Ayatsuka, CyberCode: designing augmented reality environments with visual tags, Proceedings of DARE 2000 on Designing augmented reality environments, p.1-10, April 2000, Elsinore, Denmark [doi>10.1145/354666.354667]
• 17: Y. Genc , S. Riedel , F. Souvannavong , C. Akinlar , N. Navab, Marker-less Tracking for AR: A Learning-Based Approach, Proceedings of the 1st International Symposium on Mixed and Augmented Reality, p.295, September 30-October 01, 2002
• 18: Fotis Liarokapis , Martin White , Paul Lister, Augmented Reality Interface Toolkit, Proceedings of the Information Visualisation, Eighth International Conference, p.761-767, July 14-16, 2004 [doi>10.1109/IV.2004.29]
• 19: Liarokapis, F. Sylaiou, S. et al. An Interactive Visualisation Interface for Virtual Museums. In Proceedings of the 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage (VAST), Eurographics, Brussels, (December, 2004), 47--56.
• 20: Liarokapis, F., Brujic-Okretic, V., Papakonstantinou, S., Exploring Urban Environments using Virtual and Augmented Reality. Journal of Virtual Reality and Broadcasting, GRAPP 2006 Special Issue, Digital Peer Publishing, (2006), 3(5): 1--13.
• 21: Billinghurst, M. Kato, H. and Poupyrev, I. The MagicBook: A Traditional AR Interface. Computer and Graphics, IEEE Press, (2001), 25, 745--753.
• 22: Botley, P., Porter, S. and Newman, R. DMU-A framework for concurrent design. In Proceedings of the 13th National Conference on Manufacturing Research, 1997, 33--37.
• 23: ARCO - Augmented Representations of Cultural Objects. http://www.arco-web.org/
• 24: LOCUS - Development of Location-context Tools for UMTS Mobile Information Services. http://www.locus.org.uk/
• 25: Fotis Liarokapis , Ian Greatbatch , David Mountain , Anil Gunesh , Vesna Brujic-Okretic , Jonathan Raper, Mobile Augmented Reality Techniques for GeoVisualisation, Proceedings of the Ninth International Conference on Information Visualisation, p.745-751, July 06-08, 2005 [doi>10.1109/IV.2005.79]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.3 COMPUTER GRAPHICS
      I.3.0 General

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.0 General

  J. Computer Applications
  J.0 GENERAL

General Terms:
Design, Experimentation, Human Factors

Keywords:
human-computer interaction, mixed reality, tangible interfaces, virtual and augmented reality