research-article

Pose-space animation and transfer of facial details

Authors:

Miguel A. Otaduy,

Markus GrossAuthors Info & Claims

SCA '08: Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation

Pages 57 - 66

Published: 07 July 2008 Publication History

Abstract

This paper presents a novel method for real-time animation of highly-detailed facial expressions based on a multi-scale decomposition of facial geometry into large-scale motion and fine-scale details, such as expression wrinkles. Our hybrid animation is tailored to the specific characteristics of large- and fine-scale facial deformations: Large-scale deformations are computed with a fast linear shell model, which is intuitively and accurately controlled through a sparse set of motion-capture markers or user-defined handle points. Fine-scale facial details are incorporated using a novel pose-space deformation technique, which learns the correspondence of sparse measurements of skin strain to wrinkle formation from a small set of example poses. Our hybrid method features real-time animation of highly-detailed faces with realistic wrinkle formation, and allows both large-scale deformations and fine-scale wrinkles to be edited intuitively. Furthermore, our pose-space representation enables the transfer of facial details to novel expressions or other facial models.

References

[1]

{BBA*07} Bickel B., Botsch M., Angst R., Matusik W., Otaduy M. A., Pfister H., Gross M.: Multi-scale capture of facial geometry and motion. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (2007), 33.1--33.10.

Digital Library

[2]

{BBPV03} Blanz V., Basso C., Poggio T., Vetter T.: Reanimating faces in images and video. Computer Graphics Forum 22, 3 (2003), 641--650.

[3]

{BMW*06} Borshukov G., Montgomery J., Werner W., Ruff B., Lau J., Thuriot P., Mooney P., Van Niekerk S., Raposo D., Duprat J.-L., Hable J., Kihlström H., Roizman D., Noone K., O'Connell J.: Playable universal capture. In ACM SIGGRAPH 06 Sketches&Applications (2006).

Digital Library

[4]

{BV99} Blanz V., Vetter T.: A morphable model for the synthesis of 3D faces. In Proc. SIGGRAPH 99 (1999), pp. 187--194.

Digital Library

[5]

{CBC*01} Carr J. C., Beatson R. K., Cherrie J. B., Mitchell T. J., Fright W. R., McCallum B. C., Evans T. R.: Reconstruction and representation of 3D objects with radial basis functions. In Proc. SIGGRAPH 01 (2001), pp. 67--76.

Digital Library

[6]

{CXH03} Chai J.-x., Xiao J., Hodgins J.: Vision-based control of 3D facial animation. In Proc. of ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2003), pp. 193--206.

Digital Library

[7]

{dLE07} d'Eon E., Luebke D., Enderton E.: Efficient rendering of human skin. In Eurographics Symposium on Rendering (2007), pp. 147--157.

Digital Library

[8]

{DM00} DeCarlo D., Metaxas D.: Optical flow constraints on deformable models with applications to face tracking. International Journal of Computer Vision 38, 2 (2000), 99--127.

Digital Library

[9]

{EBDP96} Essa I., Basu S., Darrell T., Pentland A.: Modeling, tracking and interactive animation of faces and heads: Using input from video. In Proc. of Computer Animation 96 (1996), pp. 68--79.

Digital Library

[10]

{For03} Fordham J.: Middle earth strikes back. Cinefex 92 (2003), 71--142.

[11]

{GMP*06} Golovinskiy A., Matusik W., Pfister H., Rusinkiewicz S., Funkhouser T.: A statistical model for synthesis of detailed facial geometry. ACM Trans. Graph. (Proc. SIGGRAPH) 25, 3 (2006), 1025--1034.

Digital Library

[12]

{JTDP03} Joshi P., Tien W. C., Desbrun M., Pighin F.: Learning controls for blend shape based realistic facial animation. In Proc. of ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2003), pp. 187--192.

Digital Library

[13]

{KGC*96} Koch R. M., Gross M. H., Carls F. R., von Büren D. F., Fankhauser G., Parish Y.: Simulating facial surgery using finite element methods. In Proc. SIGGRAPH 96 (1996), pp. 421--428.

Digital Library

[14]

{KJP02} Kry P., James D. L., Pai D. K.: EigenSkin: Real-time large deformation character skinning in hardware. In Proc. of ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2002), pp. 153--159.

Digital Library

[15]

{KM04} Kurihara T., Miyata N.: Modeling deformable human hands from medical images. In Proc. of ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2004), pp. 357--366.

Digital Library

[16]

{LCF00} Lewis J. P., Cordner M., Fong N.: Pose space deformations: A unified approach to shape interpolation and skeleton-driven deformation. In Proc. SIGGRAPH 00 (2000), pp. 165--172.

Digital Library

[17]

{MA07} Meyer M., Anderson J.: Key point subspace acceleration and soft caching. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (2007), 74.

Digital Library

[18]

{MDSB03} Meyer M., Desbrun M., Schröder P., Barr A. H.: Discrete differential-geometry operators for triangulated 2-manifolds. In Visualization and Mathematics III. Springer-Verlag, Heidelberg, 2003, pp. 35--57.

[19]

{MHP*07} Ma W.-C., Hawkins T., Peers P., Chabert C.-F., Weiss M., Debevec P.: Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination. In Proc. Eurographics Symposium on Rendering (2007), pp. 183--194.

Digital Library

[20]

{MTKL*02} Magnenat-Thalmann N., Kalra P., Lévêque J. L., Bazin R., Batisse D., Queleux B.: A computational skin model: fold and wrinkle formation. IEEE Trans. on Information Technology in Biomedicine 6, 4 (2002), 317--323.

Digital Library

[21]

{MTLT88} Magnenat-Thalmann N., Laperriere R., Thalmann D.: Joint-dependent local deformations for hand animation and object grasping. In Proc. of Graphics Interface (1988), pp. 317--323.

Digital Library

[22]

{NN99} Noh J.-Y., Neumann U.: A Survey of Facial Modeling and Animation Techniques. Tech. Rep. USC-TR-99-705, Univeristy of Southern Californina, 1999.

[23]

{NN01} Noh J.-y., Neumann U.: Expression cloning. In Proc. SIGGRAPH 01 (2001), pp. 277--288.

Digital Library

[24]

{NVI} NVIDIA: http://developer.nvidia.com/cuda.

[25]

{Par74} Parke F. I.: A parametric model for human faces. PhD thesis, University of Utah, Salt Lake City, Utah, Dec. 1974.

Digital Library

[26]

{PHL*98} Pighin F., Hecker J., Lischinski D., Szeliski R., Salesin D.: Synthesizing realistic facial expressions from photographs. In Computer Graphics (1998), pp. 75--84.

Digital Library

[27]

{PL06} Pighin F., Lewis J. P.: Performance-driven facial animation. In SIGGRAPH 2006 Course (2006).

Digital Library

[28]

{PSS99} Pighin F. H., Szeliski R., Salesin D.: Resynthe-sizing facial animation through 3D model-based tracking. In International Conference on Computer Vision (ICCV) (1999), pp. 143--150.

[29]

{RLN06} Rhee T., Lewis J. P., Neumann U.: Real-time weighted pose-space deformation on the GPU. Computer Graphics Forum (Proc. Eurographics) 25, 3 (2006), 439--448.

[30]

{SNF05} Sifakis E., Neverov I., Fedkiw R.: Automatic determination of facial muscle activations from sparse motion capture marker data. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3 (2005), 417--425.

Digital Library

[31]

{SP04} Sumner R. W., Popović J.: Deformation transfer for triangle meshes. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3 (2004), 399--405.

Digital Library

[32]

{TCR} Toledo S., Chen D., Rotkin V.: Taucs: A library of sparse linear solvers. http://www.tau.ac.il/~stoledo/taucs.

[33]

{VBPP05} Vlasic D., Brand M., Pfister H., Popović J.: Face transfer with multilinear models. ACM Trans. Graph. (Proc. SIGGRAPH) 24, 3 (2005), 426--433.

Digital Library

[34]

{VLR05} Venkataraman K., Lodha S., Raghavan R.: A kinematic-variational model for animating skin with wrinkles. Computers&Graphics 29, 5 (2005), 756--770.

Digital Library

[35]

{WKMT96} Wu Y., Kalra P., Magnenat-Thalmann N.: Simulation of static and dynamic wrinkles of skin. In Proc. of Computer Animation (1996), pp. 90--97.

Digital Library

[36]

{WPP07} Wang R. Y., Pulli K., Popović J.: Real-time enveloping with rotational regression. ACM Trans. Graph. (Proc. SIGGRAPH) 26, 3 (2007), 73.

Digital Library

[37]

{WSLG07} Weber O., Sorkine O., Lipman Y., Gotsman C.: Context-aware skeletal shape deformation. Computer Graphics Forum (Proc. Eurographics) 26, 3 (2007), 265--274.

[38]

{ZS05} Zhang Y., Sim T.: Realistic and efficient wrinkle simulation using an anatomy-based face model with adaptive refinement. In Computer Graphics International (2005), pp. 3--10.

Digital Library

[39]

{ZSCS04} Zhang L., Snavely N., Curless B., Seitz S. M.: Spacetime faces: High resolution capture for modeling and animation. ACM Trans. Graph. (Proc. SIGGRAPH) 23, 3 (2004), 548--558.

Digital Library

Cited By

Romero CCasas DPérez JOtaduy M(2021)Learning contact corrections for handle-based subspace dynamicsACM Transactions on Graphics10.1145/3450626.345987540:4(1-12)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459875
Bailey SOmens DDilorenzo PO'Brien J(2020)Fast and deep facial deformationsACM Transactions on Graphics10.1145/3386569.339239739:4(94:1-94:15)Online publication date: 12-Aug-2020
https://dl.acm.org/doi/10.1145/3386569.3392397
Hendler DMoser LBattulwar RCorral DCramer PMiller RCloudsdale RRoble D(2018)AvengersACM SIGGRAPH 2018 Talks10.1145/3214745.3214766(1-2)Online publication date: 12-Aug-2018
https://dl.acm.org/doi/10.1145/3214745.3214766
Show More Cited By

Index Terms

Pose-space animation and transfer of facial details
1. Computing methodologies

Recommendations

Weighted pose space editing for facial animation

Blendshapes are the most commonly used approach to realistic facial animation in production. A blendshape model typically begins with a relatively small number of blendshape targets reflecting major muscles or expressions. However, the majority of the ...
Artist friendly facial animation retargeting

This paper presents a novel facial animation retargeting system that is carefully designed to support the animator's workflow. Observation and analysis of the animators' often preferred process of key-frame animation with blendshape models informed our ...
Artist friendly facial animation retargeting
SA '11: Proceedings of the 2011 SIGGRAPH Asia Conference

This paper presents a novel facial animation retargeting system that is carefully designed to support the animator's workflow. Observation and analysis of the animators' often preferred process of key-frame animation with blendshape models informed our ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SCA '08: Proceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation

July 2008

230 pages

ISBN:9783905674101

Conference Chairs:
Eugene Fiume,
Jerry Tessendorf,
Program Chairs:
Markus Gross,
Doug James

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
EUROGRAPHICS: The European Association for Computer Graphics

Publisher

Eurographics Association

Goslar, Germany

Publication History

Published: 07 July 2008

Check for updates

Qualifiers

Research-article

Conference

SCA08

Sponsor:

SIGGRAPH
EUROGRAPHICS

SCA08: The ACM SIGGRAPH / Eurographics Symposium on Computer Animation

July 7 - 9, 2008

Dublin, Ireland

Acceptance Rates

SCA '08 Paper Acceptance Rate 24 of 60 submissions, 40%;

Overall Acceptance Rate 183 of 487 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

32
Total Citations
View Citations
743
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Romero CCasas DPérez JOtaduy M(2021)Learning contact corrections for handle-based subspace dynamicsACM Transactions on Graphics10.1145/3450626.345987540:4(1-12)Online publication date: 19-Jul-2021
https://dl.acm.org/doi/10.1145/3450626.3459875
Bailey SOmens DDilorenzo PO'Brien J(2020)Fast and deep facial deformationsACM Transactions on Graphics10.1145/3386569.339239739:4(94:1-94:15)Online publication date: 12-Aug-2020
https://dl.acm.org/doi/10.1145/3386569.3392397
Hendler DMoser LBattulwar RCorral DCramer PMiller RCloudsdale RRoble D(2018)AvengersACM SIGGRAPH 2018 Talks10.1145/3214745.3214766(1-2)Online publication date: 12-Aug-2018
https://dl.acm.org/doi/10.1145/3214745.3214766
Moser LHendler DRoble D(2017)MasqueradeACM SIGGRAPH 2017 Talks10.1145/3084363.3085086(1-2)Online publication date: 30-Jul-2017
https://dl.acm.org/doi/10.1145/3084363.3085086
Bertails-Descoubes FCoros S(2017)Guest Editor's IntroductionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2016.263854623:3(1165-1166)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1109/TVCG.2016.2638546
Holden DSaito JKomura T(2017)Learning Inverse Rig Mappings by Nonlinear RegressionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2016.262803623:3(1167-1178)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1109/TVCG.2016.2628036
Ma WRhee TYoshiyasu Y(2015)Making digital charactersSIGGRAPH Asia 2015 Courses10.1145/2818143.2818172(1-79)Online publication date: 2-Nov-2015
https://dl.acm.org/doi/10.1145/2818143.2818172
Holden DSaito JKomura TBarbič JDeng Z(2015)Learning an inverse rig mapping for character animationProceedings of the 14th ACM SIGGRAPH / Eurographics Symposium on Computer Animation10.1145/2786784.2786788(165-173)Online publication date: 7-Aug-2015
https://dl.acm.org/doi/10.1145/2786784.2786788
Ichim ABouaziz SPauly M(2015)Dynamic 3D avatar creation from hand-held video inputACM Transactions on Graphics10.1145/276697434:4(1-14)Online publication date: 27-Jul-2015
https://dl.acm.org/doi/10.1145/2766974
Bermano ABeeler TKozlov YBradley DBickel BGross M(2015)Detailed spatio-temporal reconstruction of eyelidsACM Transactions on Graphics10.1145/276692434:4(1-11)Online publication date: 27-Jul-2015
https://dl.acm.org/doi/10.1145/2766924
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten