Article

Video-based character animation

Authors:

A. HiltonAuthors Info & Claims

SCA '05: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation

Pages 49 - 58

https://doi.org/10.1145/1073368.1073375

Published: 29 July 2005 Publication History

Abstract

In this paper we introduce a video-based representation for free viewpoint visualization and motion control of 3D character models created from multiple view video sequences of real people. Previous approaches to video-based rendering provide no control of scene dynamics to manipulate, retarget, and create new 3D content from captured scenes. Here we contribute a new approach, combining image based reconstruction and video-based animation to allow controlled animation of people from captured multiple view video sequences. We represent a character as a motion graph of free viewpoint video motions for animation control. We introduce the use of geometry videos to represent reconstructed scenes of people for free viewpoint video rendering. We describe a novel spherical matching algorithm to derive global surface to surface correspondence in spherical geometry images for motion blending and the construction of seamless transitions between motion sequences. Finally, we demonstrate interactive video-based character animation with real-time rendering and free viewpoint visualization. This approach synthesizes highly realistic character animations with dynamic surface shape and appearance captured from multiple view video of people.

References

[1]

{AFO03} Arikan O., Forsyth D. A., O'Brien J. F.: Motion synthesis from annotations. Proceedings ACM SIGGRAPH (2003), 402--408.]]

Digital Library

[2]

{BA83} Burt P., Adelson E.: A multiresolution spline with application to image mosaics. ACM Transactions on Graphics 2, 4 (1983), 217--236.]]

Digital Library

[3]

{BCS97} Bregler C., Covell M., Slaney M.: Video rewrite: Driving visual speech with audio. Proceedings ACM SIGGRAPH (1997), 1--8.]]

Digital Library

[4]

{Bou03} Bouguet J.-Y.: Camera Calibration Toolbox for Matlab: www.vision.caltech.edu/bouguetj/calib-doc. Tech. rep., MRL-INTEL, 2003.]]

[5]

{BSM*03} Briceño H., Sander P., McMillan L., Gortler S., Hoppe H.: Geometry videos: A new representation for 3d animations. ACM Symposium on Computer Animation (2003), 136--146.]]

Digital Library

[6]

{CG00} Cosatto E., Graf P.: Photo-realistic talking heads from image samples. IEEE Transaction on Multimedia 2, 3 (2000), 152--163.]]

Digital Library

[7]

{CMS99} Culbertson W. B., Malzbender T., Slabaugh G.: Generalized voxel coloring. Proceedings of the International Workshop on Vision Algorithms, Lecture Notes in Computer Science 1883 (1999), 100--115.]]

Digital Library

[8]

{CR03} Chui H., Rangarajan A.: A new point matching algorithm for non-rigid registration. Computer Vision and Image Understanding 89 (2003), 114--141.]]

Digital Library

[9]

{CTMS03} Carranza J., Theobalt C., Magnor M., Seidel H.-P.: Free-viewpoint video of human actors. Proceedings ACM SIGGRAPH 22, 3 (2003), 569--577.]]

Digital Library

[10]

{DYB98} Debevec P., Yu Y., Borshukov G.: Efficient view-dependent image-based rendering with projective texture-mapping. 9th Eurographics Rendering Workshop (1998), 105--116.]]

[11]

{EGP02} Ezzat T., Geiger G., Poggio T.: Trainable videorealistic speech animation. Proceedings ACM SIGGRAPH (2002), 388--398.]]

Digital Library

[12]

{GGSC96} Gortler S., Grzeszczuk R., Szeliski R., Cohen M.: The lumigraph. Proceedings ACM SIGGRAPH 30 (1996), 43--54.]]

Digital Library

[13]

{GM04} Goldluecke B., Marcus M.: Space-time isosurface evolution for temporally coherent 3d reconstruction. IEEE International Conference on Computer Vision and Pattern Recognition I (2004), 350--355.]]

[14]

{Hop96} Hoppe H.: Progressive meshes. Proceedings ACM SIGGRAPH (1996), 99--108.]]

Digital Library

[15]

{KGP02} Kovar L., Gleicher M., Pighin F.: Motion graphs. Proceedings ACM SIGGRAPH (2002), 473--482.]]

Digital Library

[16]

{KRN97} Kanade T., Rander P., Narayanan P.: Virtualized reality: Constructing virtual worlds from real scenes. IEEE Multimedia 4, 1 (1997), 34--47.]]

Digital Library

[17]

{KSE*03} Kwatra V., Schödl A., Essa I., Turk G., Bobick A.: Graphcut textures: Image and video synthesis using graph cuts. Proceedings ACM SIGGRAPH (2003), 277--286.]]

Digital Library

[18]

{LAS02} Li Y., Ang T., Shum H.-Y.: Motion texture: A two-level statistical model for character motion synthesis. Proceedings ACM SIGGRAPH (2002), 465--471.]]

Digital Library

[19]

{LC87} Lorenson W., Cline H.: Marching cubes: A high resolution 3d surface construction algorithm. Computer Graphics 21, 4 (1987), 163--169.]]

Digital Library

[20]

{LCR*02} Lee J., Chai J., Reitsma P. S., Hodgins J. K., Pollard N. S.: Interactive control avatars animated with human motion data. Proceedings ACM SIGGRAPH (2002), 491--500.]]

Digital Library

[21]

{LH96} Levoy M., Hanrahan P.: Light field rendering. Proceedings ACM SIGGRAPH 30 (1996), 31--42.]]

Digital Library

[22]

{MBR*00} Matusik W., Buehler C., Raskar R., Gortler S., McMillan L.: Image-based visual hulls. Proceedings of ACM SIGGRAPH (2000), 369--374.]]

Digital Library

[23]

{MTG97} Moezzi S., Tai L., Gerard P.: Virtual view generation for 3d digital video. IEEE Multimedia 4, 1 (1997), 18--25.]]

Digital Library

[24]

{PF01} Plankers R., Fua P.: Articulated soft objects for video-based body modeling. IEEE International Conference on Computer Vision (2001), 394--401.]]

[25]

{PH03} Praun E., Hoppe H.: Spherical parameterization and remeshing. Proceedings ACM SIGGRAPH (2003), 340--349.]]

Digital Library

[26]

{SE02} Schödl A., Essa I.: Controlled animation of video sprites. ACM Symposium on Computer Animation (2002), 121--127.]]

Digital Library

[27]

{SH03} Starck J., Hilton A.: Model-based multiple view reconstruction of people. IEEE International Conference on Computer Vision (2003), 915--922.]]

Digital Library

[28]

{SH05} Starck J., Hilton A.: Virtual view synthesis of people from multiple view video sequences. Graphical Models (to appear 2005).]]

Digital Library

[29]

{SSSE00} Schödl A., Szeliski R., Salesin D., Essa I.: Video textures. ACM Symposium on Computer Animation (2000), 489--498.]]

Digital Library

[30]

{VBK02} Vedula S., Baker S., Kanade T.: Spatiotemporal view interpolation. Eurographics Workshop on Rendering (2002), 1--11.]]

Digital Library

[31]

{VMR} www.ee.surrey.ac.uk/research/vssp/vmrg/vcphome.htm.]]

[32]

{ZBS04} Zhou K., Bao H., Shi J.: 3d surface filtering using spherical harmonics. Computer-Aided Design, Elsevier 36 (2004), 363--375.]]

[33]

{ZKU*04} Zitnick C., Kang S. B., Uyttendaele M., Winder S. A. J., Szeliski R.: High-quality video view interpolation using a layered representation. Proceedings ACM SIGGRAPH 23 (2004), 600--608.]]

Digital Library

Cited By

Weng CCurless BSrinivasan PBarron JKemelmacher-Shlizerman I(2022)HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.01573(16189-16199)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.01573
MacQuarrie ASteed A(2020)Investigating the Perceived Strengths and Limitations of Free-Viewpoint VideoFrontiers in Virtual Reality10.3389/frvir.2020.000111Online publication date: 21-Sep-2020
https://doi.org/10.3389/frvir.2020.00011
MacQuarrie ASteed A(2020)Exploring the Use of Skeletal Tracking for Cheaper Motion Graphs and On-Set Decision Making in Free-Viewpoint Video ProductionProceedings of the 17th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3429341.3429353(1-10)Online publication date: 7-Dec-2020
https://dl.acm.org/doi/10.1145/3429341.3429353
Show More Cited By

Index Terms

Video-based character animation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        3D imaging
  2. Computer graphics
    1. Animation

Recommendations

Layered acting for character animation
SIGGRAPH '03: ACM SIGGRAPH 2003 Papers

We introduce an acting-based animation system for creating and editing character animation at interactive speeds. Our system requires minimal training, typically under an hour, and is well suited for rapidly prototyping and creating expressive motion. A ...
Physics-based character animation with cascadeur
SIGGRAPH '19: ACM SIGGRAPH 2019 Studio

In this workshop we will create a realistic acrobatic 3D fighting animation using the animation software Cascadeur. We will learn the key features of physics-based character animation and will immediately apply the learned knowledge by creating an ...
Layered acting for character animation

We introduce an acting-based animation system for creating and editing character animation at interactive speeds. Our system requires minimal training, typically under an hour, and is well suited for rapidly prototyping and creating expressive motion. A ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SCA '05: Proceedings of the 2005 ACM SIGGRAPH/Eurographics symposium on Computer animation

July 2005

366 pages

ISBN:1595931988

DOI:10.1145/1073368

Conference Chairs:
Demetri Terzopoulos
New York University, USA
,
Victor Brian Zordan
University of California at Riverside, USA
,
Program Chairs:
Ken Anjyo
OLM Digital, Inc., Japan
,
Petros Faloutsos
University of California at Los Angeles, USA

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
EUROGRAPHICS: The European Association for Computer Graphics

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 July 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

SCA05

Sponsor:

SIGGRAPH
EUROGRAPHICS

SCA05: Symposium on Computer Animation

July 29 - 31, 2005

California, Los Angeles

Acceptance Rates

Overall Acceptance Rate 183 of 487 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

49
Total Citations
View Citations
1,222
Total Downloads

Downloads (Last 12 months)16
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Weng CCurless BSrinivasan PBarron JKemelmacher-Shlizerman I(2022)HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Video2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.01573(16189-16199)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.01573
MacQuarrie ASteed A(2020)Investigating the Perceived Strengths and Limitations of Free-Viewpoint VideoFrontiers in Virtual Reality10.3389/frvir.2020.000111Online publication date: 21-Sep-2020
https://doi.org/10.3389/frvir.2020.00011
MacQuarrie ASteed A(2020)Exploring the Use of Skeletal Tracking for Cheaper Motion Graphs and On-Set Decision Making in Free-Viewpoint Video ProductionProceedings of the 17th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3429341.3429353(1-10)Online publication date: 7-Dec-2020
https://dl.acm.org/doi/10.1145/3429341.3429353
de Bem RGhosh AAjanthan TMiksik OBoukhayma ASiddharth NTorr P(2020)DGPose: Deep Generative Models for Human Body AnalysisInternational Journal of Computer Vision10.1007/s11263-020-01306-1Online publication date: 24-Apr-2020
https://doi.org/10.1007/s11263-020-01306-1
Boukhayma ABoyer E(2019)Surface Motion Capture Animation SynthesisIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2018.283123325:6(2270-2283)Online publication date: 1-Jun-2019
https://doi.org/10.1109/TVCG.2018.2831233
Salazar GLuo XNavarro Newball AZuniga CLozano-Garzon C(2019)Multiple Character Motion Adaptation in Virtual Cities Using Procedural Animation2019 International Conference on Virtual Reality and Visualization (ICVRV)10.1109/ICVRV47840.2019.00053(223-226)Online publication date: Nov-2019
https://doi.org/10.1109/ICVRV47840.2019.00053
Regateiro JHilton AVolino M(2019)Dynamic Surface Animation using Generative Networks2019 International Conference on 3D Vision (3DV)10.1109/3DV.2019.00049(376-385)Online publication date: Sep-2019
https://doi.org/10.1109/3DV.2019.00049
(2018)A purely Bayesian approach for proportional visual data modellingInternational Journal of Intelligent Engineering Informatics10.1504/IJIEI.2018.0945136:5(491-508)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.1504/IJIEI.2018.094513
Savoye Y(2018)Cage-based performance captureACM SIGGRAPH 2018 Courses10.1145/3214834.3214836(1-72)Online publication date: 12-Aug-2018
https://dl.acm.org/doi/10.1145/3214834.3214836
Llobera JFraile INúñez JMalewski SArtigas XFernandez S(2018)Multi-device Content Based on Video. A Practical Toolset for Creation and DeliveryApplications and Usability of Interactive Television10.1007/978-3-319-90170-1_7(93-104)Online publication date: 20-Apr-2018
https://doi.org/10.1007/978-3-319-90170-1_7
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten