skip to main content
article

Character animation from 2D pictures and 3D motion data

Published: 01 January 2007 Publication History

Abstract

This article presents a new method to animate photos of 2D characters using 3D motion capture data. Given a single image of a person or essentially human-like subject, our method transfers the motion of a 3D skeleton onto the subject's 2D shape in image space, generating the impression of a realistic movement. We present robust solutions to reconstruct a projective camera model and a 3D model pose which matches best to the given 2D image. Depending on the reconstructed view, a 2D shape template is selected which enables the proper handling of occlusions. After fitting the template to the character in the input image, it is deformed as-rigid-as-possible by taking the projected 3D motion data into account. Unlike previous work, our method thereby correctly handles projective shape distortion. It works for images from arbitrary views and requires only a small amount of user interaction. We present animations of a diverse set of human (and nonhuman) characters with different types of motions, such as walking, jumping, or dancing.

Supplementary Material

JPG File (a1-hornung_high.jpg)
JPG File (a1-hornung_low.jpg)
High Resolution (a1-hornung_high.mov)
Low Resolution (a1-hornung_low.mov)

References

[1]
Agarwal, A. and Triggs, B. 2006. Recovering 3d human pose from monocular images. IEEE Trans. Pattern Anal. Mach. Intel. 28, 1, 44--58.]]
[2]
Alexa, M., Cohen-Or, D., and Levin, D. 2000. As-Rigid-As-Possible shape interpolation. In Proceedings of the SIGGRAPH Conference. 157--164.]]
[3]
Barrett, W. A. and Cheney, A. S. 2002. Object-Based image editing. In Proceedings of the SIGGRAPH Conference. 777--784.]]
[4]
Bregler, C., Loeb, L., Chuang, E., and Deshpande, H. 2002. Turning to the masters: Motion capturing cartoons. In Proceedings of the SIGGRAPH Conference. 399--407.]]
[5]
Chuang, Y.-Y., Goldman, D. B., Zheng, K. C., Curless, B., Salesin, D., and Szeliski, R. 2005. Animating pictures with stochastic motion textures. ACM Trans. Graph. 24, 3, 853--860.]]
[6]
CMU Graphics Lab Motion Capture Database. 2007. http://mocap.cs.cmu.edu/.]]
[7]
Corrêa, W. T., Jensen, R. J., Thayer, C. E., and Finkelstein, A. 1998. Texture mapping for cel animation. In Proceedings of the SIGGRAPH Conference. 435--446.]]
[8]
Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-Based image completion. ACM Trans. Graph. 22, 3, 303--312.]]
[9]
Efros, A. A. and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In Proceedings of the 7th International Conference on Computer Vision. 1033--1038.]]
[10]
Gavrila, D. M. 1999. The visual analysis of human movement: A survey. Comput. Vision Image Understanding. 73, 1, 82--98.]]
[11]
Hartley, R. and Zisserman, A. 2003. Multiple View Geometry in Computer Vision, 2nd ed. Cambridge University Press, New York.]]
[12]
Horn, B. 1987. Closed form solutions of absolute orientation using unit quaternions. J. Optical Soci. America 4, 4 (Apr.), 629--642.]]
[13]
Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-Rigid-As-Possible shape manipulation. ACM Trans. Graph. 24, 3, 1134--1141.]]
[14]
Jojic, N. and Frey, B. J. 2001. Learning flexible sprites in video layers. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 199--206.]]
[15]
Kass, M., Witkin, A., and Terzopoulos, D. 1987. Snakes: Active contour models. Int. J. Comput. Vis. 321--331.]]
[16]
Lee, J., Chai, J., Reitsma, P. S. A., Hodgins, J. K., and Pollard, N. S. 2002. Interactive control of avatars animated with human motion data. In Proceedings of the SIGGRAPH Conference. 491--500.]]
[17]
Lewis, J. P., Cordner, M., and Fong, N. 2000. Pose space deformation: A unified approach to shape interpolation and skeleton-driven deformation. In Proceedings of the SIGGRAPH Conference. 165--172.]]
[18]
Liu, C., Torralba, A. B., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. Graph. 24, 3, 519--526.]]
[19]
Moeslund, T. B. and Granum, E. 2001. A survey of computer vision-based human motion capture. Comput. Vision Image Understanding. 81, 3, 231--268.]]
[20]
Mori, G., Berg, A., Efros, A., Eden, A., and Malik, J. 2004. Video based motion synthesis by splicing and morphing. Tech. Rep. UCB//CSD-04-1337, University of California, Berkeley. June.]]
[21]
Oh, B. M., Chen, M., Dorsey, J., and Durand, F. 2001. Image-Based modeling and photo editing. In Proceedings of the SIGGRAPH Conference. 433--442.]]
[22]
Ono, Y., Chen, B.-Y., and Nishita, T. 2004. 3D character model creation from cel animation. In IEEE Cyberworlds. 210--215.]]
[23]
Parameswaran, V. and Chellappa, R. 2004. View independent human body pose estimation from a single perspective image. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 16--22.]]
[24]
Pavić, D., Schönefeld, V., and Kobbelt, L. 2006. Interactive image completion with perspective correction. Vis. Comput. 22, 9, 671--681.]]
[25]
Rademacher, P. 1999. View-Dependent geometry. In Proceedings of the SIGGRAPH Conference. 439--446.]]
[26]
Sorkine, O., Cohen-Or, D., Lipman, Y., Alexa, M., Rössl, C., and Seidel, H.-P. 2004. Laplacian surface editing. In Proceedings of the Symposium on Geometry Processing. 179--188.]]
[27]
Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Trans. Graph. 23, 3, 315--321.]]
[28]
Sun, J., Yuan, L., Jia, J., and Shum, H.-Y. 2005. Image completion with structure propagation. ACM Trans. Graph. 24, 3, 861--868.]]
[29]
Taylor, C. J. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1677--1684.]]
[30]
Thorne, M., Burke, D., and van de Panne, M. 2004. Motion doodles: An interface for sketching character motion. ACM Trans. Graph. 23, 3, 424--431.]]
[31]
Triggs, B., McLauchlan, P., Hartley, R., and Fitzgibbon, A. 2000. Bundle adjustment---A modern synthesis. In Vision Algorithms: Theory and Practice, B. Triggs et al., Eds. Lecture Notes in Computer Science. Springer Verlag, 298--375.]]

Cited By

View all
  • (2024)Breathing Life Into Sketches Using Text-to-Video Priors2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00414(4325-4336)Online publication date: 16-Jun-2024
  • (2024)Make static person walk again via separating pose action from shapeGraphical Models10.1016/j.gmod.2024.101222134(101222)Online publication date: Aug-2024
  • (2024)AniClipart: Clipart Animation with Text-to-Video PriorsInternational Journal of Computer Vision10.1007/s11263-024-02306-1Online publication date: 27-Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 26, Issue 1
January 2007
96 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/1189762
Issue’s Table of Contents

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 January 2007
Published in TOG Volume 26, Issue 1

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. 2D character animation
  2. 3D motion data
  3. as-rigid-as-possible shape manipulation with perspective correction
  4. camera and model pose determination

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)123
  • Downloads (Last 6 weeks)9
Reflects downloads up to 13 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Breathing Life Into Sketches Using Text-to-Video Priors2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00414(4325-4336)Online publication date: 16-Jun-2024
  • (2024)Make static person walk again via separating pose action from shapeGraphical Models10.1016/j.gmod.2024.101222134(101222)Online publication date: Aug-2024
  • (2024)AniClipart: Clipart Animation with Text-to-Video PriorsInternational Journal of Computer Vision10.1007/s11263-024-02306-1Online publication date: 27-Dec-2024
  • (2023)A Method for Animating Children’s Drawings of the Human FigureACM Transactions on Graphics10.1145/359278842:3(1-15)Online publication date: 5-Jun-2023
  • (2023)3D Avatar Animation Optimization in Metaverse by Differential Evolution Algorithm2023 International Conference on Intelligent Metaverse Technologies & Applications (iMETA)10.1109/iMETA59369.2023.10294661(1-7)Online publication date: 18-Sep-2023
  • (2023)Vision-Based Human Pose Estimation via Deep Learning: A SurveyIEEE Transactions on Human-Machine Systems10.1109/THMS.2022.321924253:1(253-268)Online publication date: Feb-2023
  • (2023)Automatic Animation of Hair Blowing in Still Portrait Photos2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.02099(22906-22918)Online publication date: 1-Oct-2023
  • (2023)Exploring inbetween charts with trajectory-guided sliders for cutout animationMultimedia Tools and Applications10.1007/s11042-023-17354-x83:15(44581-44594)Online publication date: 18-Oct-2023
  • (2023)Turning Human into 2D Virtual Character Using Real-Time Motion CaptureCooperative Design, Visualization, and Engineering10.1007/978-3-031-43815-8_21(214-224)Online publication date: 1-Oct-2023
  • (2022)Action2video: Generating Videos of Human 3D ActionsInternational Journal of Computer Vision10.1007/s11263-021-01550-z130:2(285-315)Online publication date: 1-Feb-2022
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media