Pac-pac: pinching gesture recognition for tabletop entertainment system

ABSTRACT

We present our new interaction technique for tabletop system and video game application using it. This technique is for recognizing a pinching gesture performed with the thumb and forefinger and tapping gestures with them by a ceiling camera above the table. The positions and orientations of multiple gestures are recognized at 200Hz, and the users around the table are tracked by their arm positions. This technique is useful for large-sized tabletop system for multiple users because it does not require any additional equipments to them nor registration process before using the system. Some pilot studies demonstrated the robustness and accuracy of the proposed technique with displaying appropriate guide on the display. We also developed a video game application called "Pac-pac". A player can shoot a bullet along the orientation of his hand by tapping, from any side of the table. We exhibited Pac-pac several time and well-received.

References

1. X. Cao, A. D. Wilson, R. Balakrishnan, K. Hinckley, and S. E. Hudson. Shapetouch: Leveraging contact shape on interactive surfaces. In Proceedings of the 2008 IEEE International Workshop on Horizontal Interactive Human Computer System (TABLETOP), pages 139--146. IEEE, 2008.
2. P. H. Dietz and D. Leigh. Diamondtouch: A multi-user touch technology. In UIST '01: Proceedings of the 14th annual ACM symposium on User interface software and technology, pages 219--226, 2001.
3. K. Fukuchi. Concurrent Manipulation of Multiple Components on Graphical User Interface. PhD thesis, Tokyo Institute of Technology, 2006.
4. K. Fukuchi and J. Rekimoto. Marble market: Bimanual interactive game with a body shape sensor. In L. Ma, M. Rauterberg, and R. Nakatsu, editors, Entertainment Computing - ICEC2007, volume 4740 of Lecture Notes in Computer Science, pages 374--380. Springer, 2007.
5. M. Fukumoto and Y. Tonomura. Whisper: a wristwatch style wearable handset. In CHI '99: Proceedings of the SIGCHI conference on Human factors in computing systems, pages 112--119. ACM, 1999.
6. J. Y. Han. Low-cost multi-touch sensing through frustrated total internal reflection. In UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technology, pages 115--118. ACM Press, 2005.
7. G. Hollemans, T. Bergman, V. Buil, K. van Gelder, M. Groten, J. Hoonhout, T. Lashina, E. van Loenen, and S. van de Wijdeven. Entertaible: Multi-user multi-object concurrent input. In Adjunct Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST06), pages 55--56. ACM, 2006.
8. H. Mamiya, T. Sato, K. Fukuchi, and H. Koike. A tabletop entertainment system and finger tapping gesture recognition. In Proceedings of WISS 2007, pages 53--58. JSSST, 12 2007. (in Japanese).
9. N. Matsushita and J. Rekimoto. Holowall: Designing a finger, hand, body, and object sensitive wall. In UIST '97: Proceedings of the 10th annual ACM symposium on User interface software and technology, pages 209--210, 1997.
10. S. Noda, K. Yamamoto, M. Okumura, and I. Akahori. Yubi-nya: the ring detecting a ring by fingers. In Proceedings of INTERACTION 2009, pages 135--136, 2009. (in Japanese).
11. J. Rekimoto. Smartskin: An infrastructure for freehand manipulation on interactive surfaces. In Proceedings of CHI2002, pages 113--120, 2002.
12. T. Sato, K. Fukuchi, and H. Koike. Implementation and evaluations of vision-based finger flicking gesture recognition for tabletops. In Proceedings of the 2008 IEEE International Workshop on Horizontal Interactive Human Computer System (TABLETOP), pages 147--154. IEEE, 2008.
13. T. Sato, H. Mamiya, H. Koike, and K. Fukuchi. An augmented tabletop video game with pinching gesture recognition. In The 1st ACM SIGGRAPH Conference and Exhibition on Computer Graphics and Interactive TEchniques in Asia (SIGGRAPH Asia 2008): Art Gallery & Emerging Technologies. ACM, 2008.
14. Y. Sato, Y. Kobayashi, and H. Koike. Fast tracking of hands and fingertips in infrared images for augmented desk interface. In Proceedings of IEEE Automatic Face and Gesture Recognition, pages 462--467, 2000.
15. W. Westerman. Hand Tracking, Finger Identification and Chordic Manipulation on a Multi-Touch Surface. PhD thesis, University of Delaware, 1999.
16. A. Wilson. Robust computer vision-based detection of pinching for one and two-handed gesture input. In UIST '06: Proceedings of the 19th annual ACM symposium on User interface software and technology, pages 255--258. ACM, ACM Press, 2006.
17. T. G. Zimmerman, J. Lanier, C. Blanchard, S. Bryson, and Y. Harvill. A hand gesture interface device. SIGCHI Bulletin, 17(SI):189--192, 1987.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)