Tracking of deformable human hand in real time as continuous input for gesture-based interaction

ABSTRACT

Gesture input is a natural and effective interactive model. The tracking of deformable hand gesture is a very important task in gesture-based interaction. A novel real-time tracking approach is proposed to capture hand motion with single camera. It combines the characters of model-based and appearance-based method. The presented approach achieves auto-initialization by posture recognition and matching with image features. It solves the problem of interference among fingers successfully by the integration of K-Means clustering and Particle Filters. Moreover, tracking detection realizes resumption from tracking failure and automatic update of hand model. Experiments show that, the proposed method can achieve continuous real-time tracking of deformable hand gesture, and meets the requirements for gesture-based Human-Computer Interaction.

References

1. B. Stenger, P. R. Mendonca and R. Cipolla. Model based 3D tracking of an articulated hand. Proc. of IEEE conference of Computer Vision and Pattern Recognition, 2001.vol. 2. pp. 310--315.
2. C. Sminchisescu, B. Triggs. A robust multiple hypothesis approach to monocular human motion tracking. Technical Report RR-4208, INRIA, 2001.
3. Christian V. Hardenberg and Francois Berard. Bare-hand human-computer interaction. Proc. of ACM workshop on Perceptual User Interface, USA, 2001. pp.15--17.
4. G. R. Bradski. Real time face and object tracking as a component of a perceptual user interface. Proc. of IEEE 4th Workshop on Application of Computer Vision. 1998. pp. 214--219.
5. Ivan Laptev and Tony Lindeberg. Tracking of multi-state hand models using particle filtering and a hierarchy of multi-scale image features. Scale-Space 2001, LNCS 2106. pp. 63--74.
6. J. C. Cheng and Jose. Moura. Capture and representation of human walking in live video sequences. IEEE Trans.on Multimedia. 1999, 1(2), pp. 144--156.
7. Jessica JunLin Wang and Sammeer Singh. Video analysis of human dynamics-a survey. Real-time imaging. 2003, 9(5). pp. 321--346.
8. J. K. Aggarwal and Q. Cai, Human motion analysis: a review. Computer Vision and Image Understanding. 1999, 73(3). pp. 428--440.
9. Julier Letessier and F. Berard. Visual tracking of bare fingers for interactive surfaces. Proc. of 17th ACM symposium on User Interface Software and Technology. 2004. pp. 119--122.
10. James M. Rehg and Takeo Kanade. Visual tracking of high DOF articulated structures: an application to human hand tracking. Proc. of 3th European Conference on Computer Vision. 1994. pp. 35--46.
11. Jorg Zieren, Nils Unger and Suat Akyol. Hands tracking from frontal view for vision-based gesture recognition. Proc. of DAGM 2002, LNCS 2449. pp. 531--539.
12. Kenji Oka and Yoichi Sato. Real-time fingertip tracking and gesture recognition. Proc. of IEEE Computer Graphics and Applications. 2002, 22(6). pp. 64--71.
13. Lawrence Rabiner. A tutorial on Hidden Markov Models and selected applications in speech recognition. Proc. of the IEEE. 1989, 77(2). pp. 257--286.
14. Mathias Kolsch and Matthew Turk. Fast 2D hand tracking with flocks of features and multi-cue integration. Proc. of IEEE conference on Computer Vision and Pattern Recognition. 2004. pp. 158--164.
15. Matthieu Bray, Esther Koller-Meier and Luc Van Gool. Smart particle filtering for 3D hand tracking. Proc. of 6th IEEE Conference on Automatic Face and Gesture Recogntion. 2004. pp. 675--680.
16. M. H. Jaward, L. Mihaylova. A data association algorithm for multiple objects tracking in video sequence. The 2006 IEE Seminar on Target Tracking: Algorithms and Applications. 2006, Birmingham, UK.
17. Michael Isard and Andrew Blake. Contour tracking by stochastic propagation of conditional density. Proc. of European Conference of Computer Vision, 1996. pp. 343--356.
18. P. Robertson, R. Laddaga and M.V. Kleek. Virtual mouse vision based interface. In proceedings of IUI'2004. pp. 177--183.
19. R. Singh, B. Seth and U. B. Desai. Vision based GUI for interactive mobile robots. In proceedings of IUI'2005. pp. 254--256.
20. Shanon X. Ju, Micheal J. Black and Yaser Yacoob. Cardboard people: a parameterized model of articulated image motion. Second International Conference on Automatic Face and Gesture Recognition, 1996. pp. 38--44.
21. Yoichi Sato, Yoshinori Kobayashi and Hideki Koike. Fast tracking of hands and fingertips in infrared images for augmented desk interface. Proc. of IEEE International Conference on Automatic Face and Gesture Recognition. 2000. pp. 462--468.
22. Zhu JY, Wang XY, Wang WX, Dai GZ. Hand gesture recognition based on structure analysis. Accepted by ChinaGraph'2006 and will be published by Chinese Journal of Computers.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Tracking

      • Computer vision tasks

        • Scene understanding

  • Computer graphics

    • Graphics systems and interfaces

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction devices

      • Graphics input devices