Data-driven enhancement of facial attractiveness

Abstract

When human raters are presented with a collection of shapes and asked to rank them according to their aesthetic appeal, the results often indicate that there is a statistical consensus among the raters. Yet it might be difficult to define a succinct set of rules that capture the aesthetic preferences of the raters. In this work, we explore a data-driven approach to aesthetic enhancement of such shapes. Specifically, we focus on the challenging problem of enhancing the aesthetic appeal (or the attractiveness) of human faces in frontal photographs (portraits), while maintaining close similarity with the original.

The key component in our approach is an automatic facial attractiveness engine trained on datasets of faces with accompanying facial attractiveness ratings collected from groups of human raters. Given a new face, we extract a set of distances between a variety of facial feature locations, which define a point in a high-dimensional "face space". We then search the face space for a nearby point with a higher predicted attractiveness rating. Once such a point is found, the corresponding facial distances are embedded in the plane and serve as a target to define a 2D warp field which maps the original facial features to their adjusted locations. The effectiveness of our technique was experimentally validated by independent rating experiments, which indicate that it is indeed capable of increasing the facial attractiveness of most portraits that we have experimented with.

References

1. Alley, T. R., and Cunningham, M. R. 1991. Averaged faces are attractive, but very attractive faces are not average. Psychological Science 2, 123--125.
2. Beier, T., and Neely, S. 1992. Feature-based image metamorphosis. In Proc. ACM SIGGRAPH 92, ACM Press, 35--42.
3. Blanz, V., and Vetter, T. 1999. A morphable model for the synthesis of 3D faces. In Proc. ACM SIGGRAPH 99, ACM Press, 187--194.
4. Blanz, V., 2003. Manipulation of facial attractiveness. {web page} http://www.mpi-inf.mpg.de/~blanz/data/attractiveness/.
5. Bradski, G. 2000. The OpenCV library. Dr. Dobbs Journal 25, 11 (Nov.), 120, 122--125.
6. Cohen, J. D. 1997. Drawing graphs to convey proximity: an incremental arrangement method. ACM Trans. Comput.-Hum. Interact. 4, 3, 197--229.
7. Cootes, T. F., Taylor, C. J., Cooper, D. H., and Graham, J. 1995. Active shape models --- their training and their applications. Comput. Vis. Image Underst. 61, 1 (Jan.), 38--59.
8. Cunningham, M. R., Roberts, A. R., Wu, C.-H., Barbee, A. P., and Druen, P. B. 1995. Their ideas of beauty are, on the whole, the same as ours: Consistency and variability in the cross-cultural perception of female attractiveness. Journal of Personality and Social Psychology 68, 1, 261--279.
9. Eisenthal, Y., Dror, G., and Ruppin, E. 2006. Facial attractiveness: Beauty and the machine. Neural Computation 18, 1, 119--142.
10. Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In Proc. ACM SIGGRAPH 98, ACM Press, 55--66.
11. Joachims, T. 1999. Making large-scale SVM learning practical. In Advances in kernel methods: support vector learning. MIT Press, Cambridge, MA, USA, 169--184.
12. Jones, D. 1996. Physical Attractiveness and the Theory of Sexual Selection: Results from Five Populations. University of Michigan Press, Ann Arbor, Michigan.
13. Kagian, A., Dror, G., Leyvand, T., Cohen-Or, D., and Ruppin, E. 2007. A humanlike predictor of facial attractiveness. In Advances in Neural Information Processing Systems 19, B. Schölkopf, J. Platt, and T. Hoffman, Eds. MIT Press.
14. Langlois, J. H., Roggman, L. A., Casey, R. J., M. Ritter, J., Rieser-Danner, L. A., and Jenkins, V. Y. 1987. Infant preferences for attractive faces: Rudiments of a stereotype? Developmental Psychology 23, 5, 363--369.
15. Lanitis, A., Taylor, C. J., and Cootes, T. F. 2002. Toward automatic simulation of aging effects on face images. IEEE Trans. Pattern Anal. Mach. Intell. 24, 4, 442--455.
16. Lee, Y., Terzopoulos, D., and Waters, K. 1995. Realistic modeling for facial animation. In Proc. ACM SIGGRAPH 95, ACM Press, 55--62.
17. Lee, S.-Y., Wolberg, G., Chwa, K.-Y., and Shin, S. Y. 1996. Image metamorphosis with scattered feature constraints. IEEE Trans. Visualization and Computer Graphics 2, 4 (Dec.).
18. Lee, S.-Y., Wolberg, G., and Shin, S. Y. 1997. Scattered data interpolation with multilevel B-splines. IEEE Trans. Visualization and Computer Graphics 3, 3 (July--Sept.).
19. Levenberg, K. 1944. A method for the solution of certain problems in least squares. Quart. Appl. Math., 2, 164--168.
20. Lourakis, M., 2004. levmar: Levenberg-Marquardt non-linear least squares algorithms in C/C++. {web page} http://www.ics.forth.gr/~lourakis/levmar/.
21. Marquardt, D. 1963. An algorithm for least-squares estimation of nonlinear parameters. SIAM J. Appl. Math., 11, 431--441.
22. Martinez, A. M., and Benavente, R. 1998. The AR face database. CVC Technical Report 24.
23. O'Toole, A. J., Price, T., Vetter, T., Bartlett, J. C., and Blanz, V. 1999. 3D shape and 2D surface textures of human faces: the role of "averages" in attractiveness and age. Image Vision Comput. 18, 1, 9--19.
24. Parke, F. I., and Waters, K. 1996. Computer Facial Animation. A K Peters, Wellesley, Massachusetts.
25. Pérez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313--318.
26. Perrett, D. I., May, K. A., and Yoshikawa, S. 1994. Facial shape and judgements of female attractiveness. Nature 368, 239--242.
27. Perrett, D. I., Burt, D. M., Penton-Voak, I. S., Lee, K. J., Rowland, D. A., and Edwards, R. 1999. Symmetry and human facial attractiveness. Evolution and Human Behavior 20, 295--307.
28. Pighin, F., Hecker, J., Lischinski, D., Szeliski, R., and Salesin, D. H. 1998. Synthesizing realistic facial expressions from photographs. In Proc. ACM SIGGRAPH 98, 75--84.
29. Press, W. H., Flannery, B. P., Teukolsky, S. A., and Vetterling, W. T. 1992. Numerical Recipes: The Art of Scientific Computing, 2nd ed. Cambridge University Press.
30. Sammon, J. W. 1969. A nonlinear mapping for data structure analysis. IEEE Transactions on Computers C-18, 5, 401--409.
31. Slater, A., Der Schulenberg, C. V., Brown, E., Butter-worth, G., Badenoch, M., and Parsons, S. 1998. Newborn infants prefer attractive faces. Infant Behavior and Development, 345--354.
32. Vapnik, V. 1995. The nature of statistical learning theory. Springer, New York.
33. Yang, M.-H., Kriegman, D., and Ahuja, N. 2002. Detecting faces in images: A survey. IEEE Trans. PAMI 24, 1, 34--58.
34. Zhao, W., Chellappa, R., Phillips, P. J., and Rosenfeld, A. 2003. Face recognition: A literature survey. ACM Comput. Surv. 35, 4, 399--458.
35. Zhou, Y., Gu, L., and Zhang, H.-J. 2003. Bayesian tangent shape model: Estimating shape and pose parameters via bayesian inference. In Proc. CVPR 2003, 109--118.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision representations

        • Shape representations

      • Image and video acquisition

        • Epipolar geometry

  • Computer graphics

    • Image manipulation

  • Machine learning

  • Modeling and simulation

    • Model development and analysis

      • Model verification and validation