Moveable interactive projected displays using projector based tracking

ABSTRACT

Video projectors have typically been used to display images on surfaces whose geometric relationship to the projector remains constant, such as walls or pre-calibrated surfaces. In this paper, we present a technique for projecting content onto moveable surfaces that adapts to the motion and location of the surface to simulate an active display. This is accomplished using a projector based location tracking techinque. We use light sensors embedded into the moveable surface and project low-perceptibility Gray-coded patterns to first discover the sensor locations, and then incrementally track them at interactive rates. We describe how to reduce the perceptibility of tracking patterns, achieve interactive tracking rates, use motion modeling to improve tracking performance, and respond to sensor occlusions. A group of tracked sensors can define quadrangles for simulating moveable displays while single sensors can be used as control inputs. By unifying the tracking and display technology into a single mechanism, we can substantially reduce the cost and complexity of implementing applications that combine motion tracking and projected imagery.

REFERENCES

• 1: Dynamic Shader Lamps: Painting on Movable Objects, Proceedings of the IEEE and ACM International Symposium on Augmented Reality (ISAR'01), p.207, October 29-30, 2001
• 2: Patrick Baudisch , Nathaniel Good , Paul Stewart, Focus plus context screens: combining display technology with visualization techniques, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502354]
• 3: Ben-Joseph, E., Ishii, H., Underkoffler, J., Piper, B., and Yeung, L. "Urban Simulation and the Luminous Planning Table: Bridging the Gap between the Digital and the Tangible." In Journal of Planning in Education and Research, Volume 21, pp.195--202.
• 4: Eric A. Bier , Maureen C. Stone , Ken Pier , William Buxton , Tony D. DeRose, Toolglass and magic lenses: the see-through interface, Proceedings of the 20th annual conference on Computer graphics and interactive techniques, p.73-80, September 1993 [doi>10.1145/166117.166126]
• 5: Daniel Cotting , Martin Naef , Markus Gross , Henry Fuchs, Embedding Imperceptible Patterns into Projected Images for Simultaneous Acquisition and Display, Proceedings of the Third IEEE and ACM International Symposium on Mixed and Augmented Reality (ISMAR'04), p.100-109, November 02-05, 2004 [doi>10.1109/ISMAR.2004.30]
• 6: Hinckley, K., "Bumping Objects Together as a Semantically Rich Way of Forming Connections between Ubiquitous Devices." UbiComp 2003 formal video program.
• 7: Ken Hinckley , Gonzalo Ramos , Francois Guimbretiere , Patrick Baudisch , Marc Smith, Stitching: pen gestures that span multiple displays, Proceedings of the working conference on Advanced visual interfaces, May 25-28, 2004, Gallipoli, Italy [doi>10.1145/989863.989866]
• 8: Hiroshi Ishii , Brygg Ullmer, Tangible bits: towards seamless interfaces between people, bits and atoms, Proceedings of the SIGCHI conference on Human factors in computing systems, p.234-241, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258715]
• 9: Kjeldsen, R., Levas, A., Pinhanez, C., "Dynamically Reconfigurable Vision-Based User Interfaces" In 3rd International Conference on Vision Systems (ICVS'03). Graz, Austria. April 2003.
• 10: Johnny C. Lee , Paul H. Dietz , Dan Maynes-Aminzade , Ramesh Raskar , Scott E. Hudson, Automatic projector calibration with embedded light sensors, Proceedings of the 17th annual ACM symposium on User interface software and technology, October 24-27, 2004, Santa Fe, NM, USA [doi>10.1145/1029632.1029653]
• 11: Claudio S. Pinhanez, The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces, Proceedings of the 3rd international conference on Ubiquitous Computing, p.315-331, September 30-October 02, 2001, Atlanta, Georgia, USA
• 12: Raskar, R.; Beardsley, P.; van Baar, J.; Wang, Y.; Dietz, P.H.; Lee, J.; Leigh, D.; Willwacher, T., "RFIG Lamps: Interacting with a Self-Describing World via Photosensing Wireless Tags and Projectors." In proceedings of ACM SIGGRAPH. August 2004.
• 13: Ramesh Raskar , Greg Welch , Matt Cutts , Adam Lake , Lev Stesin , Henry Fuchs, The office of the future: a unified approach to image-based modeling and spatially immersive displays, Proceedings of the 25th annual conference on Computer graphics and interactive techniques, p.179-188, July 1998 [doi>10.1145/280814.280861]
• 14: Ramesh Raskar , Greg Welch , Kok-Lim Low , Deepak Bandyopadhyay, Shader Lamps: Animating Real Objects With Image-Based Illumination, Proceedings of the 12th Eurographics Workshop on Rendering Techniques, p.89-102, June 25-27, 2001
• 15: Jun Rekimoto , Masanori Saitoh, Augmented surfaces: a spatially continuous work space for hybrid computing environments, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.378-385, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303113]
• 16: Norbert A. Streitz , Jörg Geißler , Torsten Holmer , Shin'ichi Konomi , Christian Müller-Tomfelde , Wolfgang Reischl , Petra Rexroth , Peter Seitz , Ralf Steinmetz, i-LAND: an interactive landscape for creativity and innovation, Proceedings of the SIGCHI conference on Human factors in computing systems: the CHI is the limit, p.120-127, May 15-20, 1999, Pittsburgh, Pennsylvania, United States [doi>10.1145/302979.303010]
• 17: Summet, J., Sukthankar, R., "Tracking Locations of Moving Hand-held Displays Using Projected Light." In proceedings of Pervasive 2005. Munich, Germany. pp 37--46
• 18: Sukthankar, R., Cham, T., and Sukthankar, G. "Dynamic Shadow Elimination for Multi-Projector Displays." Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, 2001.
• 19: Brygg Ullmer , Hiroshi Ishii, The metaDESK: models and prototypes for tangible user interfaces, Proceedings of the 10th annual ACM symposium on User interface software and technology, p.223-232, October 14-17, 1997, Banff, Alberta, Canada [doi>10.1145/263407.263551]
• 20: Yotsukura, T. et al.,"Hyper Mask - Talking Head Projected onto Real Object". The Visual Computer, vol. 18(2):111--120, 2002
• 21: Yoder, L. "The Digital Display Technology of the Future." Texas Instrument white paper library, (www.dlp.com). Presented at Infocomm, 1997.
• 22: Zhengyou Zhang , Ying Wu , Ying Shan , Steven Shafer, Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper, Proceedings of the 2001 workshop on Perceptive user interfaces, November 15-16, 2001, Orlando, Florida [doi>10.1145/971478.971522]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Graphical user interfaces (GUI)

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Artificial, augmented, and virtual realities

General Terms:
Human Factors, Measurement

Keywords:
augmented reality, physical interaction, projector based tracking, simulated displays