An example-based approach for facial expression cloning

ABSTRACT

In this paper, we present a novel example-based approach for cloning facial expressions of a source model to a target model while reflecting the characteristic features of the target model in the resulting animation. Our approach comprises three major parts: key-model construction, parameterization, and expression blending. We first present an effective scheme for constructing key-models. Given a set of source example key-models and their corresponding target key-models created by animators, we parameterize the target key-models using the source key-models and predefine the weight functions for the parameterized target key-models based on radial basis functions. In runtime, given an input model with some facial expression, we compute the parameter vector of the corresponding output model, to evaluate the weight values for the target key-models and obtain the output model by blending the target key-models with those weights. The resulting animation preserves the facial expressions of the input model as well as the characteristic features of the target model specified by animators. Our method is not only simple and accurate but also fast enough for various real-time applications such as video games or internet broadcasting.

References

1. B. Allen, B. Curless and Z. Popovic. "Articulated body deformation from range scan data", In Proceedings of SIGGRAPH 02, pp. 612--619, 2002.
2. C. Bregler, L. Loeb and E. Chuang and H. Deshpande. "Turning to the masters: Motion capturing cartoons", In Proceedings of SIGGRAPH 02, pp. 399--407, 2002.
3. I. Buck, A. Finkelstein, C. Jacobs, A. Klein, D. Salesin, J. Seims, R. Szeliski, and K. Toyama. "Performance-Driven Hand-Drawn Animation", In Proceedings of Symposium on Non-Photorealistic Animation and Rendering, 2000.
4. E. Chuang and C. Bregler. "Performance Driven Facial Animation using Blendshape Interpolation", Standford University Computer Science Technical Report, CSTR-2002-02, April 2002.
5. P. Ekman and W. V. Friesen. "Unmasking the face: A guide to recognizing emotions from facial clues", Prentice-Hall Inc., 1975.
6. D. Fidaleo, J-Y. Noh, T. Kim, R. Enciso and U. Neumann. "Classification and Volume Morphing for Performance-Driven Facial Animation", In Proceedings of Internatinoal Workshop on Digital and Computational Video, 2000.
7. C. G. Fisher. "Confusions among visually perceived consonants.", Jour. Speech and Hearing Research, 11, pp. 796--804, 1968.
8. M. Gleicher. "Retargetting motion to new Characters", ACM SIGGRAPH 98 Conference Proceedings, pp. 33--42, 1998.
9. B. Guenter, C. Grimm, D. Wood, H. Malvar, and F. Pighin. "Making Faces", ACM SIGGRAPH 98 Conference Proceedings, pp. 55--66, July 1998.
10. I. T. Jollife. "Principal components analysis", New York: Spinger, 1986.
11. P. Kalra and A. Mangili and N. M. Thalmann and D. Thalmann. "Simulation of facial muscle actions based on rational free from deformations", In Proceedings of Eurographics 92, pp. 59--69, 1992.
12. C. Kouadio and P. Poulin and P. Lachapelle. "Real-Time Facial Animation Based Upon a Bank of 3D Facial Expressions", Proc. Computer Animation, pp. 128--136, 1998.
13. G. A. Kalberer and L. V. Gool. "Face Animation Based on Observed 3D Speech Dynamics", Computer Animation 2001, pp. 20--27, November 2001.
14. J. Lee and S. Y. Shin. "A hierarchical approach to interactive motion editing for human-like figures", Proceedings of SIGGRAPH 99, pp. 39--48, 1999.
15. Y. C. Lee, D. Terzopoulos and K. Waters. "Realistic modeling for facial animation", In Proceedings of SIGGRAPH 95, pp. 55--62, 1995.
16. Marc Levoy and Pat Hanrahan. "Light Field Rendering", Proceedings of SIGGRAPH 96, pp. 31--42, 1996.
17. J. P. Lewis, M. Cordner, and N. Fong. "Pose Space Deformation: A Unified Approach to Shape Interpolation and Skeleton-Drive Deformation", ACM SIGGRAPH 2000 Conference Proceedings, pp. 165--172, July 2000.
18. J. Y. Noh and U. Neumann. "Expression cloning", In Proceedings of SIGGRAPH 01, pp. 277--288, 2001.
19. S. I. Park, H. J. Shin and S. Y. Shin. "On-line locomotion generation based on motion blending", In ACM SIGGRAPH Symposium on Computer Animation, pp. 105--111, 2002.
20. F. I. Parke and K. Waters. Computer Facial Animation. A K Peters, 289 Linden Street, Wellesley, MA 02181, 1996.
21. F. I. Parke. "Computer generated animation of faces", Master's thesis, University of Utah, 1972.
22. F. I. Parke. "Parameterized models for facial animation", In IEEE Computer Graphics and Applications, Vol. 2, No. 9, pp. 61--68, 1982.
23. F. Pighin, J. Hecker, D. Lischinski, R. Szeliski, and D. H. Salesin. "Synthesizing Realistic Facial Expressions from Photographs", In ACM SIGGRAPH 98 Conference Proceedings, pp. 75--84, July 1998.
24. F. Pighin, R. Szeliski and D. H. Salesin. "Resynthesizing facial animation through 3D model-based tracking", Proceedings of International Conference on Computer Vision 99, pp. 143--150, July 1999.
25. S. M. Platt and N. I. Badler. "Animating facial expressions", In Computer Graphics, Vol. 15(3), pp. 245--252, 1981.
26. Z. Popovic and A. Witkin. "Physically based motion transformation", In Proceedings of SIGGRAPH 99, pp. 11--20, 1999.
27. C. Rose, M. F. Cohen and B. Bodenheimer. "Verbs and adverbs: Multidimensional motion interpolation", In IEEE Computer Graphics and Applications, Vol. 18(5), pp. 32--40, 1998.
28. J. A. Russel. "A Circomplex Model of Affect", In J. Personality and Social Psychology, Vol. 39, pp. 1161--1178, 1980.
29. P.-P. Sloan and C. F. Rose and Michael F. Cohen. "Shape by example", In Proceedings of 2001 Symposium on Interactive 3D Graphics, pp. 135--144, 2001.
30. D. Terzopoulos and K. Waters. "Physically-based facial modeling, analysis, and animation", In Journal of Visualization and Computer Animation, Vol. 1, No. 4, pp. 73--80, 1990.
31. K. Waters. "A muscle model for animating three dimensional facial expressions", In Proceedings of SIGGRAPH 87, pp. 17--24, 1987.
32. Li-Yi Wei and Marc Levoy. "Fast Texture Synthesis Using Tree-Structured Vector Quantization", In Proceedings of SIGGRAPH 00, pp. 479--488, 2000.
33. L. Williams. "Performance driven facial animation", In Proceedings of SIGGRAPH 90, pp. 235--242, 1990.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation

    • Shape modeling

• Theory of computation

  • Randomness, geometry and discrete structures

    • Computational geometry