Enabling beyond-surface interactions for interactive surface with an invisible projection

ABSTRACT

This paper presents a programmable infrared (IR) technique that utilizes invisible, programmable markers to support interaction beyond the surface of a diffused-illumination (DI) multi-touch system. We combine an IR projector and a standard color projector to simultaneously project visible content and invisible markers. Mobile devices outfitted with IR cameras can compute their 3D positions based on the markers perceived. Markers are selectively turned off to support multi-touch and direct on-surface tangible input. The proposed techniques enable a collaborative multi-display multi-touch tabletop system. We also present three interactive tools: i-m-View, i-m-Lamp, and i-m-Flashlight, which consist of a mobile tablet and projectors that users can freely interact with beyond the main display surface. Early user feedback shows that these interactive devices, combined with a large interactive display, allow more intuitive navigation and are reportedly enjoyable to use.

References

1. }}Michael Aron, Gilles Simon, and Marie-Odile Berger. Handling uncertain sensor data in vision-based camera tracking. In Proc. ISMAR '04, 58--67.
2. }}Patrick Baudisch, Nathaniel Good, Victoria Bellotti, and Pamela Schraedley. Keeping things in context: comparative evaluation of focus plus context screens, overviews, and zooming. In Proc. CHI '02, 259--266.
3. }}Hrvoje Benko, Edward W. Ishak, and Steven Feiner. Collaborative mixed reality visualization of an archaeological excavation. In ISMAR '04, 132--140.
4. }}Kar Wee Chia, Adrian David Cheok, and Simon J. D. Prince. Online 6 dof augmented reality registration from natural features. In Proc. ISMAR '02, 305--313.
5. }}Daniel Cotting, Martin Naef, Markus Gross, and Henry Fuchs. Embedding imperceptible patterns into projected images for simultaneous acquisition and display. In Proc. ISMAR '04, 100--109.
6. }}Paul Dietz and Darren Leigh. Diamondtouch: multi-user touch technology. In Proc. UIST '01, 219--226.
7. }}Mark Fiala. Automatic projector calibration using self-identifying patterns. In Workshops of CVPR '05. 113--113.
8. }}Anselm Grundhöfer, Manja Seeger, Ferry Hantsch, and Oliver Bimber. Dynamic adaptation of projected imperceptible codes. In Proc. ISMAR '07, 1--10.
9. }}Anselm Grundhöfer, Manja Seeger, Ferry Hantsch, and Oliver Bimber. Dynamic adaptation of projected imperceptible codes. In ISMAR '07, 1--10.
10. }}Jefferson Y. Han. Low-cost multi-touch sensing through frustrated total internal reflection. In Proc. UIST '05, 115--118.
11. }}Otmar Hilliges, Shahram Izadi, Andrew D. Wilson, Steve Hodges, Armando Garcia-Mendoza, and Andreas Butz. Interactions in the air: adding further depth to interactive tabletops. In Proc. UIST '09, 139--148.
12. }}Chuan-Heng Hsiao, Li-Wei Chan, Ting-Ting Hu, Mon-Chu Chen, Jane Hsu, and Yi-Ping Hung. To move or not to move: a comparison between steerable versus fixed focus region paradigms in multi-resolution tabletop display systems. In Proc. CHI '09, 153--162.
13. }}Ting-Ting Hu, Yi-Wei Chia, Li-Wei Chan, Yi-Ping Hung, and Jane Hsu. i-m-top: An interactive multi-resolution tabletop system accommodating to multi-resolution human vision. In Proc. Tabletop '08, 177--180.
14. }}Shahram Izadi, Steve Hodges, Stuart Taylor, Dan Rosenfeld, Nicolas Villar, Alex Butler, and Jonathan Westhues. Going beyond the display: a surface technology with an electronically switchable diffuser. In Proc. UIST '08, 269--278.
15. }}Yasuaki Kakehi and Takeshi Naemura. Ulteriorscape: optical superimposing on view-dependent tabletop display and its applications. In SIGGRAPH '08: ACM SIGGRAPH 2008 new tech demos.
16. }}Rudolph Emil Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, 82(Series D), 35--45.
17. }}Johnny Lee, Scott Hudson, and Pau Dietz. Hybrid infrared and visible light projection for location tracking. In Proc. UIST '07, 57--60.
18. }}Rubin G. Pelli D. Legge, G. and M. Schleske. Psychophyics of reading ii. low vision. Vision Research, 25, 2:253--266, 1985.
19. }}Alex Olwal. Lightsense: enabling spatially aware handheld interaction devices. In Proc. ISMAR '06, 119--122.
20. }}Alex Olwal and Steven Feiner. Spatially aware handhelds for high-precision tangible interaction with large displays. In Proc. TEI '09, 181--188.
21. }}Alex Olwal and Anders Henrysson. Lumar: A hybrid spatial display system for 2d and 3d handheld augmented reality. International Conference on Artificial Reality and Telexistence, 0, 63--70.
22. }}Johan Sanneblad and Lars Erik Holmquist. Ubiquitous graphics: combining hand-held and wall-size displays to interact with large images. In Proc. AVI '06, 373--377.
23. }}Garth B. D. Shoemaker and Kori M. Inkpen. Single display privacyware: augmenting public displays with private information. In Proc. CHI '01, 522--529.
24. }}Gilles Simon and Marie-Odile Berger. Reconstructing while registering: A novel approach for markerless augmented reality. In Proc. ISMAR '02, 285--285.
25. }}Hyunyoung Song, Tovi Grossman, George Fitzmaurice, François Guimbretiere, Azam Khan, Ramtin Attar, and Gordon Kurtenbach. Penlight: combining a mobile projector and a digital pen for dynamic visual overlay. In Proc. CHI '09, 143--152.
26. }}Hyunyoung Song, Francois Guimbretiere, Tovi Grossman, and George Fitzmaurice. Mouselight: bimanual interactions on digital paper using a pen and a spatially-aware mobile projector. In Proc. CHI '10, 2451--2460.
27. }}Michael Tsang, George W. Fitzmaurice, Gordon Kurtenbach, Azam Khan, and Bill Buxton. Boom chameleon: simultaneous capture of 3d viewpoint, voice and gesture annotations on a spatially-aware display. In Proc. UIST '02, 111--120.
28. }}Brygg Ullmer and Hiroshi Ishii. The metadesk: models and prototypes for tangible user interfaces. In Proc. UIST '97, 223--232.
29. }}Andrew D. Wilson. Playanywhere: a compactc interactive tabletop projection-vision system. In Proc. UIST '05, 83--92.
30. }}W. S. Yerazunis and M. Carbone. Privacy-enhanced displays by time-masking images. In Proc. OzCHI '01.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Graphical user interfaces