Content-aware exaggerated editing for life-like captured animations

ABSTRACT

Cutting-edge efforts have been invested in the automatic production of breath-taking visual effects involving time-varying data captured from real-actor performances. However, a challenges for computer-generated imagery is the puppetry of heterogeneous captured data, without the heavy use of artistic skills. Then, we focus on achieving desired exaggerated animations coherently while preserving life-life baked-in visual cues. In this paper, we propose a new method to generate content-aware exaggerated animations by melting motion, shape and appearance properties from captured data. In particular, our suggested approach explores two closely tools that serve the common theme of Animation-Cartoonization. The first one consists in realizing articulated-based stretchable cartoon editing from marker-based mocap clips. The second one generates video-based toon character from surface performance capture. Finally, we demonstrate the flexibility and stability of our approach on a variety of captured animations as input.

References

1. A. Baak, M. Müller, G. Bharaj, H.-P. Seidel, and C. Theobalt. A data-driven approach for real-time full body pose reconstruction from a depth camera. In IEEE 13th International Conference on Computer Vision (CVPR), Nov. 2011.
2. L. Ballan and G. M. Cortelazzo. Marker-less motion capture of skinned models in a four camera set-up using optical flow and silhouettes. In 3DPVT, Atlanta, GA, USA, June 2008.
3. I. Baran and J. Popović. Automatic rigging and animation of 3d characters. In ACM SIGGRAPH 2007 papers, SIGGRAPH '07, New York, NY, USA, 2007. ACM.
4. P. Barla, J. Thollot, and L. Markosian. X-toon: an extended toon shader. In Proceedings of NPAR'06, pages 127--132, New York, NY, USA, 2006. ACM.
5. M. Ben-Chen, O. Weber, and C. Gotsman. Spatial deformation transfer. In Proceedings of the SCA'09, pages 67--74, New York, NY, USA, 2009. ACM.
6. M. Ben-Chen, O. Weber, and C. Gotsman. Variational harmonic maps for space deformation. ACM Trans. Graph., 28:34:1--34:11, July 2009.
7. S. Bouvier-Zappa, V. Ostromoukhov, and P. Poulin. Motion cues for illustration of skeletal motion capture data. In Proceedings of NPAR'07, pages 133--140, New York, NY, USA, 2007. ACM.
8. C. Bregler, L. Loeb, E. Chuang, and H. Deshpande. Turning to the masters: motion capturing cartoons. In Proceedings of SIGGRAPH '02, 2002.
9. E. de Aguiar, C. Stoll, C. Theobalt, N. Ahmed, H.-P. Seidel, and S. Thrun. Performance capture from sparse multi-view video. In ACM SIGGRAPH 2008 papers, SIGGRAPH '08, pages 98:1--98:10, New York, NY, USA, 2008. ACM.
10. C. de Juan and B. Bodenheimer. Cartoon textures. In Proceedings of SCA'04, pages 267--276, Aire-la-Ville, Switzerland, Switzerland, 2004. Eurographics Association.
11. M. Eisemann, B. De Decker, M. Magnor, P. Bekaert, E. de Aguiar, N. Ahmed, C. Theobalt, and A. Sellent. Floating textures. Computer Graphics Forum (Proc. of Eurographics), 2008.
12. J. Gall, C. Stoll, E. D. Aguiar, C. Theobalt, B. Rosenhahn, and H. peter Seidel. Motion capture using joint skeleton tracking and surface estimation. In In IEEE Conf. on Computer Vision and Pattern Recognition, 2009.
13. K. Grochow, S. L. Martin, A. Hertzmann, and Z. Popović. Style-based inverse kinematics. In ACM SIGGRAPH 2004 Papers, SIGGRAPH '04, pages 522--531, New York, NY, USA, 2004. ACM.
14. P. Huang, C. Budd, and A. Hilton. Global temporal registration of multiple non-rigid surface sequences. In CVPR 2011, Colorado Springs, CO, USA, 20-25, pages 3473--3480, 2011.
15. A. Jacobson, I. Baran, J. Popović, and O. Sorkine. Bounded biharmonic weights for real-time deformation. ACM Transactions on Graphics (proceedings of ACM SIGGRAPH), 2011.
16. A. Jacobson and O. Sorkine. Stretchable and twistable bones for skeletal shape deformation. ACM Transactions on Graphics (proceedings of ACM SIGGRAPH ASIA), 30(6), 2011.
17. D. L. James and C. D. Twigg. Skinning mesh animations. In ACM SIGGRAPH 2005 Papers, SIGGRAPH '05, pages 399--407, New York, NY, USA, 2005. ACM.
18. P. Joshi, M. Meyer, T. DeRose, B. Green, and T. Sanocki. Harmonic coordinates for character articulation. ACM Trans. Graph., 26, July 2007.
19. T. Ju, Q.-Y. Zhou, M. van de Panne, D. Cohen-Or, and U. Neumann. Reusable skinning templates using cage-based deformations. ACM Trans. Graph., 27:122:1--122:10, December 2008.
20. D. Kang, J.-M. Chung, S.-H. Seo, J.-S. Choi, and K.-H. Yoon. Detail-adaptive toon shading using saliency. In Proceedings of VIZ'09, VIZ '09, pages 16--20, Washington, DC, USA, 2009.
21. L. Kavan, S. Collins, J. Žára, and C. O'Sullivan. Geometric skinning with approximate dual quaternion blending. ACM Trans. Graph., 27:105:1--105:23, November 2008.
22. L. Kavan, P.-P. Sloan, and C. O'Sullivan. Fast and efficient skinning of animated meshes. Computer Graphics Forum, 29(2), 2010.
23. A. G. Kirk, J. F. O'Brien, and D. A. Forsyth. Skeletal parameter estimation from optical motion capture data. In IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2005.
24. J. Kwon and I.-K. Lee. The squash-and-stretch stylization for character motions. IEEE Transactions on Visualization and Computer Graphics, 2011.
25. J.-y. Kwon and I.-K. Lee. Rubber-like exaggeration for character animation. In Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, pages 18--26, 2007.
26. C. Larboulette, M. paule Cani, and B. Arnaldi. Dynamic skinning: adding real-time dynamic effects to an existing character animation. In Spring Conference on Computer Graphics, 2005.
27. Y. Lipman, D. Levin, and D. Cohen-Or. Green coordinates. ACM Trans. Graph., 27:78:1--78:10, August 2008.
28. Y. Liu, C. Stoll, J. Gall, H.-P. Seidel, and C. Theobalt. Markerless motion capture of interacting characters using multi-view image segmentation. In CVPR'11, june 2011.
29. N. Magnenat-Thalmann, R. Laperrière, and D. Thalmann. Joint-dependent local deformations for hand animation and object grasping. In Proceedings on Graphics interface'88, pages 26--33, Toronto, Ont., Canada, Canada, 1988.
30. C. Miller, O. Arikan, and D. Fussell. Frankenrigs: building character rigs from multiple sources. In Proceedings of I3D'10, pages 31--38, New York, NY, USA, 2010.
31. S. Rusinkiewicz, M. Burns, and D. DeCarlo. Exaggerated shading for depicting shape and detail. ACM Transactions on Graphics (Proc. SIGGRAPH), 2006.
32. T. Shiratori, H. S. Park, L. Sigal, Y. Sheikh, and J. K. Hodgins. Motion capture from body-mounted cameras. ACM Transactions on Graphics, 30(4), 2011.
33. J. Starck and A. Hilton. Surface capture for performance-based animation. IEEE Comput. Graph. Appl., 27:21--31, May 2007.
34. C. Stoll, J. Gall, E. de Aguiar, S. Thrun, and C. Theobalt. Video-based reconstruction of animatable human characters. ACM Trans. Graph., 29:139:1--139:10, December 2010.
35. C. Stoll, N. Hasler, J. Gall, H.-P. Seidel, and C. Theobalt. Fast articulated motion tracking using a sums of gaussians body model. In ICCV'11, 2011.
36. D. Sýkora, M. Ben-Chen, M. Čadík, B. Whited, and M. Simmons. Textoons: Practical texture mapping for hand-drawn cartoon animations. In Proceedings of NPAR'11, 2011.
37. D. Vlasic, I. Baran, W. Matusik, and J. Popović. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph., 27:97:1--97:9, August 2008.
38. D. Vlasic, P. Peers, I. Baran, P. Debevec, J. Popović, S. Rusinkiewicz, and W. Matusik. Dynamic shape capture using multi-view photometric stereo. ACM Trans. Graph., 2009.
39. J. Wang, S. Drucker, M. Agrawala, and M. F. Cohen. The cartoon animation filter. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006), 23(3):1169--1173, July 2006.
40. O. Weber, O. Sorkine, Y. Lipman, and C. Gotsman. Context-aware skeletal shape deformation. Computer Graphics Forum (Proceedings of Eurographics), 26(3), 2007.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Video segmentation

      • Computer vision tasks

        • Video summarization

      • Image and video acquisition

  • Computer graphics

    • Animation

  • Modeling and simulation

    • Simulation types and techniques

      • Simulation by animation