Example-based facial rigging

ABSTRACT

We introduce a method for generating facial blendshape rigs from a set of example poses of a CG character. Our system transfers controller semantics and expression dynamics from a generic template to the target blendshape model, while solving for an optimal reproduction of the training poses. This enables a scalable design process, where the user can iteratively add more training poses to refine the blendshape expression space. However, plausible animations can be obtained even with a single training pose. We show how formulating the optimization in gradient space yields superior results as compared to a direct optimization on blendshape vertices. We provide examples for both hand-crafted characters and 3D scans of a real actor and demonstrate the performance of our system in the context of markerless art-directable facial tracking.

References

1. Alexander, O., Rogers, M., Lambeth, W., Chiang, M., and Debevec, P. 2009. The digital emily project: photoreal facial modeling and animation. In SIGGRAPH '09 Courses.
2. Baran, I., and Popović, J. 2007. Automatic rigging and animation of 3d characters. ACM Trans. Graph. 26, 3, 72.
3. Bergeron, P., and Lachapelle, P. 1985. Controlling facial expressions and body movements in the computer generated animated short 'Tony de Peltrie'. In SIGGRAPH '85 Tutorial Notes, Advanced Computer Animation Course.
4. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In Proc. SIGGRAPH '99.
5. Blanz, V., Basso, C., Poggio, T., and Vetter, T. 2003. Reanimating faces in images and video. In EUROGRAPHICS '03.
6. Botsch, M., Sumner, R., Pauly, M., and Gross, M. 2006. Deformation transfer for detail-preserving surface editing. In Vision, Modeling, Visualization 2006, 357--364.
7. Choe, B., and Ko, H.-S. 2005. Analysis and synthesis of facial expressions with hand-generated muscle actuation basis. In SIGGRAPH '05 Courses.
8. Chuang, E. 2004. Analysis, Synthesis, and Retargeting of Facial Expressions. PhD thesis, Stanford University.
9. Coleman, T. F., and Li, Y. 1996. An interior trust region approach for nonlinear minimization subject to bounds. SIAM Journal on Optimization 6, 2, 418--445.
10. Ekman, P., and Friesen, W. 1978. Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists Press.
11. Kähler, K., Haber, J., and Peter Seidel, H. 2001. Geometry-based muscle modeling for facial animation. In In Proc. Graphics Interface 2001.
12. Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. ACM Transactions on Graphics (Proceedings SIGGRAPH Asia 2009) 28, 5.
13. Liu, X., Mao, T., Xia, S., Yu, Y., and Wang, Z. 2008. Facial animation by optimized blendshapes from motion capture data. Comput. Animat. Virtual Worlds 19, 3--4, 235--245.
14. Magnenat-Thalmann, N., Laperrière, R., and Thalmann, D. 1988. Joint-dependent local deformations for hand animation and object grasping. In Proceedings on Graphics interface '88, 26--33.
15. Noh, J.-Y., and Neumann, U. 2001. Expression cloning. In Proc. SIGGRAPH '01.
16. Orvalho, V. C. T., Zacur, E., and Susin, A. 2008. Transferring the rig and animations from a character to different face models. Comput. Graph. Forum 27, 8, 1997--2012.
17. Osipa, J. 2007. Stop Staring: Facial Modeling and Animation Done Right. Sybex, Second Edition.
18. Pighin, F., and Lewis, J. P. 2006. Facial motion retargeting. In SIGGRAPH '06 Courses.
19. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. H. 1998. Synthesizing realistic facial expressions from photographs. In Proc. SIGGRAPH '98.
20. Sifakis, E., Neverov, I., and Fedkiw, R. 2005. Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. 24, 3, 417--425.
21. Stahlberg, S., 2010. Nikita real-time character. Filmakademie Baden-Wuerttemberg/Institute of Animation's R&D Labs.
22. Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Transactions on Graphics (Proceedings SIGGRAPH 2004) 23, 3.
23. Terzopoulos, D., and Waters, K. 1990. Physically-based facial modeling, analysis and animation. Journal of Visualization and Computer Animation 1, 73--80.
24. Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24.
25. Waters, K. 1987. A muscle model for animation three-dimensional facial expression. In Proc. SIGGRAPH '87.
26. Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3d scanning with automatic motion compensation. In Proc. CVPR'07.
27. Weise, T., Li, H., Gool, L. V., and Pauly, M. 2009. Face/off: Live facial puppetry. In Proc. SCA'09.
28. Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: High-resolution capture for modeling and animation. In ACM Annual Conf. on Comp. Graphics.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Tracking

      • Computer vision tasks

        • Scene understanding

  • Computer graphics

    • Animation

    • Shape modeling

• Theory of computation

  • Randomness, geometry and discrete structures

    • Computational geometry