ABSTRACT
Although interactive surfaces have many unique and compelling qualities, the interactions they support are by their very nature bound to the display surface. In this paper we present a technique for users to seamlessly switch between interacting on the tabletop surface to above it. Our aim is to leverage the space above the surface in combination with the regular tabletop display to allow more intuitive manipulation of digital content in three-dimensions. Our goal is to design a technique that closely resembles the ways we manipulate physical objects in the real-world; conceptually, allowing virtual objects to be 'picked up' off the tabletop surface in order to manipulate their three dimensional position or orientation. We chart the evolution of this technique, implemented on two rear projection-vision tabletops. Both use special projection screen materials to allow sensing at significant depths beyond the display. Existing and new computer vision techniques are used to sense hand gestures and postures above the tabletop, which can be used alongside more familiar multi-touch interactions. Interacting above the surface in this way opens up many interesting challenges. In particular it breaks the direct interaction metaphor that most tabletops afford. We present a novel shadow-based technique to help alleviate this issue. We discuss the strengths and limitations of our technique based on our own observations and initial user feedback, and provide various insights from comparing, and contrasting, our tabletop implementations
- Agrawala, M., Beers, A.C., McDowall, I., Fröhlich, B., Bolas, M., and Hanrahan, P. The two-user Responsive Workbench: support for collaboration through individual views of a shared space. In SIGGRAPH. 1997. p. 327--332. Google ScholarDigital Library
- Bowman, D.A., Kruijff, E., LaViola, J.J.J., and Poupyrev, I., 3D User Interfaces: Theory and Practice. 1 ed. 2004: Addison-Wesley/Pearson Education. Google ScholarDigital Library
- Buchmann, V., Violich, S., Billinghurst, M., and Cockburn, A. FingARtips: gesture based direct manipulation in Augmented Reality. In GRAPHITE '04. 2004. Google ScholarDigital Library
- Chang, F., Chen, C.-J., and Lu, C.-J., A linear-time component labeling algorithm using contour tracing technique. Computer Vision and Image Understanding, 2004. 93(2): p. 206--220. Google ScholarDigital Library
- Codella, C., et al. Interactive simulation in a multiperson virtual world. In ACM CHI. 1992. p. 329--334. Google ScholarDigital Library
- Czernuszenko, M., Pape, D., Sandin, D., DeFanti, T., Dawe, G.L., and Brown, M.D., The ImmersaDesk and Infinity Wall projection-based virtual reality displays. Computer Graphics (1997). 31(2): p. 46--49. Google ScholarDigital Library
- Davidson, P.L. and Han, J.Y. Extending 2D object arrangement with pressure-sensitive layering cues. In ACM UIST'08. 2008. p. 87--90. 8. Dietz, P. and Leigh, D. DiamondTouch: a multi-user touch technology. In ACM UIST'01. 2001. p. (219--226). Google ScholarDigital Library
- Dietz, P. and Leigh, D. DiamondTouch: a multi-user touch technology. In ACM UIST'01. 2001. p. (219--226).). Google ScholarDigital Library
- Favalora, G.E., Volumetric 3D Displays and Application Infrastructure. Computer, 2005. 38(8 ): p. 37--44. Google ScholarDigital Library
- Fitzmaurice, G.W., Ishii, H., and Buxton, W.A.S. Bricks: laying the foundations for graspable user interfaces. In ACM CHI'95. 1995. p. 442--449. Google ScholarDigital Library
- Grossman, T. and Balakrishnan, R. Collaborative interaction with volumetric displays. In ACM CHI'08. 2008. Google ScholarDigital Library
- Grossman, T. and Wigdor, D. Going deeper: a taxonomy of 3D on the tabletop. In IEEE Tabletop '07. 2007. p. 137--144.Google ScholarCross Ref
- Grossman, T., Wigdor, D., and Balakrishnan, R. Multifinger gestural interaction with 3d volumetric displays. In ACM UIST'04. 2004. Google ScholarDigital Library
- Han, J.Y. Low-cost multi-touch sensing through frustrated total internal reflection. In ACM UIST'05. 2005. p. 115--118. Google ScholarDigital Library
- Hancock, M., Carpendale, S., and Cockburn, A. Shallow--depth 3d interaction: design and evaluation of one-, two- and three-touch techniques. In ACM CHI '07. 2007. p. 1147--1156. Google ScholarDigital Library
- Herndon, K.P., Zeleznik, R.C., Robbins, D.C., Brookshire, C.D., Snibbe, S.S., and van Dam, A. Interactive shadows. In ACM UIST'92. 1992. Google ScholarDigital Library
- Ishii, H., Ratti, C., Piper, B., Wang, Y., Biderman, A., and Ben-Joseph, E., Bringing Clay and Sand into Digital Design--Continuous Tangible User Interfaces. BT Technology Journal, 2004. 22(4): p. 287--299. Google ScholarDigital Library
- Ishii, H. and Ullmer, B. Tangible bits: towards seamless interfaces between people, bits and atoms. In ACM CHI'97. 1997. p. 234--241. Google ScholarDigital Library
- Izadi, S., et al. Going beyond the display: a surface technology with an electronically switchable diffuser. In ACM UIST'08. 2008. Google ScholarDigital Library
- Kakehi, Y. and Naemura, T. UlteriorScape: Interactive optical superimposition on a view-dependent tabletop display. In IEEE Tabletop '08. 2008. p. 189--192.Google ScholarCross Ref
- Kakehi, Y., Naemura, T., and Matsushita, M. Tablescape Plus: Interactive Small-sized Vertical Displays on a Horizontal Tabletop Display. In IEEE Tabletop '07. 2007. p. 155--162.Google ScholarCross Ref
- Lucero, A., Aliakseyeu, D., and Martens, J.-B. Augmenting Mood Boards: Flexible and Intuitive Interaction in the Context of the Design Studio. In IEEE Tabletop. 2007. p. 147--154.Google Scholar
- Malik, S. and Laszlo, J. Visual touchpad: a twohanded gestural input device. In ICMI '04. 2004. p. 289--296. Google ScholarDigital Library
- Matsushita, M., Iida, M., Ohguro, T., Shirai, Y., Kakehi, Y., and Naemura, T. Lumisight table: a face-to-face collaboration support system that optimizes direction of projected information to each stakeholder. In CSCW '04. 2004. p. 274--283. Google ScholarDigital Library
- Microsoft, Surface www.microsoft.com/surface/.Google Scholar
- Nakashima, K., Machida, T., Kiyokawa, K., and Takemura, H. A 2D-3D integrated environment for cooperative work. In ACM VRST'05. 2005. p. 16--22. Google ScholarDigital Library
- Perlin, K., Paxia, S., and Kollin, J.S. An autostereoscopic display. In ACM SIGGRAPH. 2000. p. 319--326. Google ScholarDigital Library
- ProfessionalPlastic, Acrylite® EndLighten.Google Scholar
- Raskar, R., Welch, G., and Chen, W.-C. Table-top spatially-augmented realty: bringing physical models to life with projected imagery. In ACM Augmented Reality, 1999. (IWAR '99). 1999. p. 64--71. Google ScholarDigital Library
- Rekimoto, J. SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. In ACM SIGCHI. 2002. p. 113--120. Google ScholarDigital Library
- Sandin, D.J., Margolis, T., Ge, J., Girado, J., Peterka, T., and DeFanti, T.A. The Varrier autostereoscopic virtual reality display. In ACM SIGGRAPH. 2005. p. 894--903. Google ScholarDigital Library
- Shoemaker, G., Tang, A., and Booth, K.S. Shadow reaching: a new perspective on interaction for large displays. In ACM UIST'07. 2007. p. 53--56. Google ScholarDigital Library
- Subramanian, S., Aliakseyeu, D., and Lucero, A. Multi-layer interaction for digital tables. In ACM UIST'06. 2006. Google ScholarDigital Library
- Tuddenham, P. Tabletop Interfaces for Remote Collaboration. PhD Thesis. 2008. University of CambridgeGoogle Scholar
- Ullmer, B. and Ishii, H. The metaDESK: models and prototypes for tangible user interfaces. In ACM UIST'97. 1997. Google ScholarDigital Library
- Underkoffler, J. and Ishii, H. Urp: a luminous-tangible workbench for urban planning and design. In ACM SIGCHI. 1999. p. 386--393. Google ScholarDigital Library
- Wilson, A.D. Depth-sensing video cameras for 3D Tangible Interaction. In 2nd IEEE Tabletop. 2007. p. 201--204.Google Scholar
- Wilson, A.D. PlayAnywhere: a compact interactive tabletop projection-vision system. In ACM UIST'05. 2005. p. (83--92). Google ScholarDigital Library
- Wilson, A.D. Robust computer vision-based detection of pinching for one and two-handed gesture input. In ACM UIST'06. 2006. Google ScholarDigital Library
- Wilson, A.D. TouchLight: an imaging touch screen and display for gesture-based interaction. In ICMI '04. 2004. p. 69--76. Google ScholarDigital Library
- Wilson, A.D., Izadi, S., Hilliges, O., Garcia-Mendoza, A., and Kirk, D. Bringing physics to the surface. In ACM UIST'08. 2008. p. 67--76. Google ScholarDigital Library
Index Terms
- Interactions in the air: adding further depth to interactive tabletops
Recommendations
Designing user-, hand-, and handpart-aware tabletop interactions with the TouchID toolkit
ITS '11: Proceedings of the ACM International Conference on Interactive Tabletops and SurfacesRecent work in multi-touch tabletop interaction introduced many novel techniques that let people manipulate digital content through touch. Yet most only detect touch blobs. This ignores richer interactions that would be possible if we could identify (1) ...
Tablets, tabletops, and smartphones: cross-platform comparisons of children’s touchscreen interactions
ICMI '17: Proceedings of the 19th ACM International Conference on Multimodal InteractionThe proliferation of smartphones and tablets has increased children’s access to and usage of touchscreen devices. Prior work on smartphones has shown that children’s touch interactions differ from adults’. However, larger screen devices like tablets ...
Beyond flat surface computing: challenges of depth-aware and curved interfaces
MM '09: Proceedings of the 17th ACM international conference on MultimediaIn the past decade, multi-touch-sensitive interactive surfaces have transitioned from pure research prototypes in the lab, to commercial products with wide-spread adoption. One of the longer term visions of this research follows the idea of ubiquitous ...
Comments