Statistical segmentation and recognition of fingertip trajectories for a gesture interface

ABSTRACT

This paper presents a virtual push button interface created by drawing a shape or line in the air with a fingertip. As an example of such a gesture-based interface, we developed a four-button interface for entering multi-digit numbers by pushing gestures within an invisible 2x2 button matrix inside a square drawn by the user. Trajectories of fingertip movements entering randomly chosen multi-digit numbers are captured with a 3D position sensor mounted on the the forefinger's tip. We propose a statistical segmentation method for the trajectory of movements and a normalization method that is associated with the direction and size of gestures. The performance of the proposed method is evaluated in HMM-based gesture recognition. The recognition rate of 60.0% was improved to 91.3% after applying the normalization method.

REFERENCES

• 1: Vladimir I. Pavlovic , Rajeev Sharma , Thomas S. Huang, Visual Interpretation of Hand Gestures for Human-Computer Interaction: A Review, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.677-695, July 1997 [doi>10.1109/34.598226 ]
• 2: Jun Rekimoto, GestureWrist and GesturePad: Unobtrusive Wearable Interaction Devices, Proceedings of the 5th IEEE International Symposium on Wearable Computers, p.21, October 08-09, 2001
• 3: K. Tsukada and M. Yasumura, "Ubi-Finger: Gesture input device for mobile use," Proc. of 5th Asia Pacific Conference on Computer Human Interaction, vol.1, pp.388--400, Nov. 2002.
• 4: Tevfik Metin Sezgin , Thomas Stahovich , Randall Davis, Sketch based interfaces: early processing for sketch understanding, Proceedings of the 2001 workshop on Perceptive user interfaces, November 15-16, 2001, Orlando, Florida [doi>10.1145/971478.971487]
• 5: Xuedong Huang , Yasuo Ariki , Mervyn Jack, Hidden Markov Models for Speech Recognition, Columbia University Press, New York, NY, 1990
• 6: S. Young, D. Kershaw, J. Odell, D. Ollason, V. Valtchev, and P. Woodland, The HTK Book, Microsoft.
• 7: K. Morimoto, C. Miyajima, K. Itou, and K. Takeda, "A virtual button interface using fingertip movements," 2007 IEEE International Conference of Machine Learning and Cybernetics (ICMLC 2007), pp. 2089--2093, Hong Kong, Aug. 2007.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Additional Classification:
  H. Information Systems
  H.1 MODELS AND PRINCIPLES
      H.1.2 User/Machine Systems
          Subjects: Human information processing

General Terms:
Human Factors

Keywords:
3D position sensor, affine transformation, gesture interface, hidden Markov model, principal component analysis