Parametric reshaping of human bodies in images

ABSTRACT

We present an easy-to-use image retouching technique for realistic reshaping of human bodies in a single image. A model-based approach is taken by integrating a 3D whole-body morphable model into the reshaping process to achieve globally consistent editing effects. A novel body-aware image warping approach is introduced to reliably transfer the reshaping effects from the model to the image, even under moderate fitting errors. Thanks to the parametric nature of the model, our technique parameterizes the degree of reshaping by a small set of semantic attributes, such as weight and height. It allows easy creation of desired reshaping effects by changing the full-body attributes, while producing visually pleasing results even for loosely-dressed humans in casual photographs with a variety of poses and shapes.

References

1. Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: reconstruction and parameterization from range scans. ACM Trans. Graph. 22, 3, 587--594.
2. Allen, B., Curless, B., Popović, Z., and Hertzmann, A. 2006. Learning a correlated model of identity and pose-dependent body shape variation for real-time synthesis. In SCA '06, 147--156.
3. Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: shape completion and animation of people. ACM Trans. Graph. 24, 3, 408--416.
4. Arad, N., and Reisfeld, D. 1995. Image warping using few anchor points and radial functions. Computer Graphics Forum 14, 1, 35--46.
5. Balan, A., and Black, M. 2008. The naked truth: Estimating body shape under clothing. In ECCV '08.
6. Balan, A. O., Sigal, L., Black, M. J., Davis, J. E., and Haussecker, H. W. 2007. Detailed human shape and pose from images. In CVPR '07, 1--8.
7. Baran, I., and Popović, J. 2007. Automatic rigging and animation of 3D characters. ACM Trans. Graph. 26, 3, 72.
8. Barrett, W. A., and Cheney, A. S. 2002. Object-based image editing. ACM Trans. Graph. 21, 3, 777--784.
9. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3d faces. In SIGGRAPH '99, 187--194.
10. Carroll, R., Agrawala, M., and Agarwala, A. 2009. Optimizing content-preserving projections for wide-angle images. ACM Trans. Graph. 28, 3.
11. Davis, J., Agrawala, M., Chuang, E., Popović, Z., and Salesin, D. 2003. A sketching interface for articulated figure animation. In SCA '03, 320--328.
12. de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3, 98.
13. Eismann, K., and Palmer, W. 2006. Adobe Photoshop Restoration & Retouching. New Riders Press; 3 edition.
14. Guan, P., Weiss, A., Balan, A., and Black, M. J. 2009. Estimating human shape and pose from a single image. In ICCV '09.
15. Hasler, N., Stoll, C., Sunkel, M., Rosenhahn, B., and Seidel, H.-P. 2009. A statistical model of human pose and body shape. Computer Graphics Forum 28, 2, 337--346.
16. Hornung, A., Dekkers, E., and Kobbelt, L. 2007. Character animation from 2D pictures and 3D motion data. ACM Trans. Graph. 26, 1, 1.
17. Hua, Z., Wang, G., Lin, X., and Yan, H. 2009. Recovery of upper body poses in static images based on joints detection. Pattern Recognition Letters 30, 5, 503--512.
18. Igarashi, T., Moscovich, T., and Hughes, J. F. 2005. As-rigid-as-possible shape manipulation. ACM Trans. Graph. 24, 3, 1134--1141.
19. Kraevoy, V., Sheffer, A., and van de Panne, M. 2009. Modeling from contour drawings. In SBIM '09, 37--44.
20. Leyvand, T., Cohen-Or, D., Dror, G., and Lischinski, D. 2008. Data-driven enhancement of facial attractiveness. ACM Trans. Graph. 27, 3, 38.
21. Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Trans. Graph. 23, 3, 303--308.
22. Mortensen, E. N., and Barrett, W. A. 1995. Intelligent scissors for image composition. In SIGGRAPH '95, 191--198.
23. Parameswaran, V., and Chellappa, R. 2004. View independent human body pose estimation from a single perspective image. In CVPR '04, vol. 2.
24. Schaefer, S., McPhail, T., and Warren, J. 2006. Image deformation using moving least squares. ACM Trans. Graph. 25, 3, 533--540.
25. Seo, H., and Magnenat-Thalmann, N. 2004. An example-based approach to human body manipulation. Graphical Models 66, 1, 1--23.
26. Shamir, A., and Sorkine, O. 2009. Visual media retargeting. In SIGGRAPH Asia Course 2009.
27. Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Trans. Graph. 23, 3, 399--405.
28. Tan, G., Chen, W., and Liu, L. 2010. Image driven shape deformation using styles. Journal of Zhejiang University (SCIENCE C) 11, 1, 27--35.
29. Taylor, C. 2000. Reconstruction of articulated objects from point correspondences in a single uncalibrated image. Computer Vision and Image Understanding 80, 3, 349--363.
30. Tzur, Y., and Tal, A. 2009. FlexiStickers - photogrammetric texture mapping using casual images. ACM Trans. Graph. 28, 3.
31. Vlasic, D., Brand, M., Pfister, H., and Popović, J. 2005. Face transfer with multilinear models. ACM Trans. Graph. 24, 3, 426--433.
32. Vlasic, D., Baran, I., Matusik, W., and Popović, J. 2008. Articulated mesh animation from multi-view silhouettes. ACM Trans. Graph. 27, 3, 97.
33. Zhou, K., Huang, J., Snyder, J., Liu, X., Bao, H., Guo, B., and Shum, H.-Y. 2005. Large mesh deformation using the volumetric graph Laplacian. ACM Trans. Graph. 24, 3, 496--503.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Image and video acquisition

        • 3D imaging

  • Computer graphics

    • Animation

    • Graphics systems and interfaces

    • Shape modeling

• Theory of computation

  • Randomness, geometry and discrete structures

    • Computational geometry