Visual tracking of bare fingers for interactive surfaces

ABSTRACT

Visual tracking of bare fingers allows more direct manipulation of digital objects, multiple simultaneous users interacting with their two hands, and permits the interaction on large surfaces, using only commodity hardware. After presenting related work, we detail our implementation. Its design is based on our modeling of two classes of algorithms that are key to the tracker: Image Differencing Segmentation (IDS) and Fast Rejection Filters (FRF). We introduce a new chromatic distance for IDS and a FRF that is independent to finger rotation. The system runs at full frame rate (25 Hz) with an average total system latency of 80 ms, independently of the number of tracked fingers. When used in a controlled environment such as a meeting room, its robustness is satisfying for everyday use.

REFERENCES

• 1: Bérard, F. The Magic Table: Computer-Vision Based Augmentation of a Whiteboard for Creative Meetings. CD-ROM proceedings of the IEEE International Conference in Computer Vision, Workshop on Projector-Camera Systems (PROCAMS'03), Nice, France (2003).
• 2: Criminisi, A. Reid, I. Zisserman, A. A plane measuring device. British Machine Vision Conference, UK (1997).
• 3: Paul Dietz , Darren Leigh, DiamondTouch: a multi-user touch technology, Proceedings of the 14th annual ACM symposium on User interface software and technology, November 11-14, 2001, Orlando, Florida [doi>10.1145/502348.502389]
• 4: Hideki Koike , Yoichi Sato , Yoshinori Kobayashi , Hiroaki Tobita , Motoki Kobayashi, Interactive textbook and interactive Venn diagram: natural and intuitive interfaces on augmented desk system, Proceedings of the SIGCHI conference on Human factors in computing systems, p.121-128, April 01-06, 2000, The Hague, The Netherlands [doi>10.1145/332040.332415]
• 5: Jun Rekimoto, SmartSkin: an infrastructure for freehand manipulation on interactive surfaces, Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, April 20-25, 2002, Minneapolis, Minnesota, USA [doi>10.1145/503376.503397]
• 6: Christian von Hardenberg , François Bérard, Bare-hand human-computer interaction, Proceedings of the 2001 workshop on Perceptive user interfaces, November 15-16, 2001, Orlando, Florida [doi>10.1145/971478.971513]
• 7: Colin Ware , Ravin Balakrishnan, Reaching for objects in VR displays: lag and frame rate, ACM Transactions on Computer-Human Interaction (TOCHI), v.1 n.4, p.331-356, Dec. 1994 [doi>10.1145/198425.198426]
• 8: Christopher Richard Wren , Ali Azarbayejani , Trevor Darrell , Alex Paul Pentland, Pfinder: Real-Time Tracking of the Human Body, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.19 n.7, p.780-785, July 1997 [doi>10.1109/34.598236 ]
• 9: Mike Wu , Ravin Balakrishnan, Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays, Proceedings of the 16th annual ACM symposium on User interface software and technology, p.193-202, November 02-05, 2003, Vancouver, Canada [doi>10.1145/964696.964718]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)
          Subjects: Input devices and strategies (e.g., mouse, touchscreen)

Additional Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis
          Subjects: Tracking

Keywords:
finger tracking with computer vision, large interactive surface, multi-user multi-hand interaction