Synchronized real-time multi-sensor motion capture system

ABSTRACT

This work addresses the challenge of synchronizing multiple sources of visible and audible information from a variety of devices, while capturing human motion in realtime. Video and audio data will be used to augment and enrich a motion capture database that will be released to the research community. While other such augmented motion capture databases exist [Black and Sigal 2006], the goal of this work is to build on these previous works. Critical areas of improvement are in the synchronization between cameras and synchronization between devices. Adding an array of audio recording devices to the setup will greatly expand the research potential of the database, and the positioning of the cameras will be varied to give greater flexibility. The augmented database will facilitate the testing and validation of human pose estimation and motion tracking techniques, among other applications. This sketch briefly describes some of the interesting challenges faced in setting up the pipeline for capturing the synchronized data and the novel approaches proposed to solve them.

References

1. Black, M. J., and Sigal, L., 2006. HumanEva: Synchronized video and motion capture dataset for evaluation of articulated human motion, technical report CS-06-08. http://vision.cs.brown.edu/humaneva/index.html.

Index Terms

• Applied computing

  • Arts and humanities

    • Sound and music computing

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Reconstruction

      • Image and video acquisition

        • Motion capture

  • Computer graphics

    • Animation

      • Motion capture

      • Motion processing

• Information systems

  • Information retrieval

    • Specialized information retrieval

      • Multimedia and multimodal retrieval

        • Music retrieval