Unconventional approaches for facial animation and tracking

ABSTRACT

In recent years, depth cameras have become commodity hardware already in many households. That trend allows for an action-driven facial animator system to become commonly available. In this paper, we layout the methods involved in driving high quality CG facial animations by capturing and matching human actions at real-time speeds. We explore two important sub-problems for this issue: (i) head motion tracking and (ii) animation creation for virtual characters. For head motion tracking, we present a 3D template matching framework, achieving real-time performance and yielding results more accurate than current state-of-the-art motion tracking methods. For animation creation, we provide a 3D animation retrieval system that allows artists to easily retrieve and reuse desired animations from an existing database of already rigged facial animations.

References

1. Breitenstein, M., Kuettel, D., Weise, T., van Gool, L., and Pfister, H. 2008. Real-time face pose estimation from single range images. In Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on, 1--8.
2. Canny, J. 1986. A computational approach to edge detection. IEEE Trans. Pattern Anal. Mach. Intell. 8, 6 (June), 679--698.
3. Chai, J.-x., Xiao, J., and Hodgins, J. 2003. Vision-based control of 3d facial animation. In Proceedings of the 2003 ACM SIGGRAPH/Eurographics symposium on Computer animation, Eurographics Association, Aire-la-Ville, Switzerland, Switzerland, SCA '03, 193--206.
4. Fanelli, G., Gall, J., and Van Gool, L. 2011. Real time head pose estimation with random regression forests. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, 617--624.
5. Fanelli, G., Weise, T., Gall, J., and Gool, L. V. 2011. Real time head pose estimation from consumer depth cameras. In 33rd Annual Symposium of the German Association for Pattern Recognition (DAGM'11).
6. Gao, Y., and Leung, M. K. 2002. Line segment hausdorff distance on face matching. Pattern Recognition 35, 2, 361--371.
7. Gong, B., Wang, Y., Liu, J., and Tang, X. 2009. Automatic facial expression recognition on a single 3d face by exploring shape deformation. In Proceedings of the 17th ACM international conference on Multimedia, ACM, New York, NY, USA, MM '09, 569--572.
8. Keogh, E. 2002. Exact indexing of dynamic time warping. In Proceedings of the 28th international conference on Very Large Data Bases, VLDB Endowment, VLDB '02, 406--417.
9. Maalej, A., Ben Amor, B., Daoudi, M., Srivastava, A., and Berretti, S. 2010. Local 3d shape analysis for facial expression recognition. In Pattern Recognition (ICPR), 2010 20th International Conference on, 4129--4132.
10. Morency, L.-P., Rahimi, A., Checka, N., and Darrell, T. 2002. Fast stereo-based head tracking for interactive environments. In Proceedings of the Fifth IEEE International Conference on Automatic Face and Gesture Recognition, IEEE Computer Society, Washington, DC, USA, FGR '02, 390--.
11. Murphy-Chutorian, E., and Trivedi, M. 2009. Head pose estimation in computer vision: A survey. Pattern Analysis and Machine Intelligence, IEEE Transactions on 31, 4 (april), 607--626.
12. Soyel, H., and Demirel, H. 2007. Facial expression recognition using 3d facial feature distances. In ICIAR, 831--838.
13. Villagrasa, S., and Susin, A. 2009. Face! 3d facial animation system based on facs. In Proceedings of the IV Iberoamerican Symposium in Computer Graphics, DJ Editores, C. A., O. Rodríguez, F. Serón, R. Joan-Arinyo, and E. C. J. Madeiras, J. Rodríguez, Eds., Sociedad Venezolana de Computación Gráfica.
14. Wang, J., Yin, L., Wei, X., and Sun, Y. 2006. 3d facial expression recognition based on primitive surface feature distribution. In in Proc. Conf. Computer Vision and Pattern Recognition, 1399--1406.
15. Weise, T., Bouaziz, S., Li, H., and Pauly, M. 2011. Realtime performance-based facial animation. ACM Trans. Graph. 30, 4 (Aug.), 77:1--77:10.
16. Zhou, Y., Gu, L., and Zhang, H.-J. 2003. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proceedings of the 2003 IEEE computer society conference on Computer vision and pattern recognition, IEEE Computer Society, Washington, DC, USA, CVPR'03, 109--116.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Tracking

      • Computer vision tasks

        • Scene understanding

      • Image and video acquisition

  • Computer graphics

    • Animation