"Wow! you are so beautiful today!"

ABSTRACT

Beauty e-Experts, a fully automatic system for hairstyle and facial makeup recommendation and synthesis, is developed in this work. Given a user-provided frontal face image with short/bound hair and no/light makeup, the Beauty e-Experts system can not only recommend the most suitable hairdo and makeup, but also show the synthetic effects. To obtain enough knowledge for beauty modeling, we build the Beauty e-Experts Database, which contains 1,505 attractive female photos with a variety of beauty attributes and beauty-related attributes annotated. Based on this Beauty e-Experts Dataset, two problems are considered for the Beauty e-Experts system: what to recommend and how to wear, which describe a similar process of selecting hairstyle and cosmetics in our daily life. For the what-to-recommend problem, we propose a multiple tree-structured super-graphs model to explore the complex relationships among the high-level beauty attributes, mid-level beauty-related attributes and low-level image features, and then based on this model, the most compatible beauty attributes for a given facial image can be efficiently inferred. For the how-to-wear problem, an effective and efficient facial image synthesis module is designed to seamlessly synthesize the recommended hairstyle and makeup into the user facial image. Extensive experimental evaluations and analysis on testing images of various conditions well demonstrate the effectiveness of the proposed system.

References

1. T. Ahonen, A. Hadid, and M. Pietikainen. Face description with local binary patterns: application to face recognition. TPAMI, 2006.
2. S. Belongie, J. Malik, and J. Puzicha. Shape matching and object recognition using shape contexts. TPAMI, 2002.
3. F. Bookstein. Principal warps: thin-plate splines and the decomposition of deformations. TPAMI, 1989.
4. Y. Boykov and O. Veksler. Fast approximate energy minimization via graph cuts. TPAMI, 2001.
5. C. Chang and C. Lin. Libsvm: A library for support vector machines. In TIST, 2011.
6. F. Chen and D. Zhang. A benchmark for geometric facial beauty study. In Int. Conf. Medical Biometrics, 2010.
7. C. Chow and C. Liu. Approximating discrete probability distributions with dependence trees. TIT, 1968.
8. T. Cootes, C. Taylor, D. Cooper, and J. Graham. Active shape models-their training and application. CVIU, 1995.
9. A. Criminisi, P. Perez, and K. Toyama. Region filling and object removal by exemplar-based image inpainting. TIP, 2004.
10. N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In CVPR, 2005.
11. M. de Berg, O. Cheong, M. van Kreveld, and M. Overmars. Computational Geometry: Algorithms and Applications. Springer-Verlag, third edition, 2008.
12. P. Felzenszwalb, R. Girshick, D. McAllester, and D. Ramanan. Object detection with discriminatively trained part-based models. TPAMI, 2010.
13. R. Feris and L. Davis. Image ranking and retrieval based on multi-attribute queries. In CVPR, 2011.
14. A. Goshtasby. Piecewise linear mapping functions for image registration. PR, 1986.
15. D. Guo and T. Sim. Digital face makeup by example. In CVPR, 2009.
16. S. Haykin. Neural Networks. Prentice Hall, 1999.
17. K. He, J. Sun, and X. Tang. Guided image filtering. In ECCV, 2010.
18. G. Huang, M. Ramesh, T. Berg, and E. Learned. Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical report, University of Massachusetts, 2007.
19. T. Joachims. Optimizing search engines using clickthrough data. In ACM KDD, 2002.
20. I. Jolliffe. Principal component analysis. Encyclopedia of Statistics in Behavioral Science, 2002.
21. D. Koller and N. Friedman. Probabilistic Graphical Models: Principles and Techniques. MIT Press, 2009.
22. A. Levin, D. Lischinski, and Y. Weiss. A closed-form solution to natural image matting. TPAMI, 2008.
23. S. Liu, J. Feng, Z. Song, T. Zhang, H. Lu, C. Xu, and S. Yan. "Hi, magic closet, tell me what to wear". In ACM MM, 2012.
24. T. Mensink, J. Verbeek, and G. Csurka. Tree-structured crf models for interactive image labeling. TPAMI, 2013.
25. Y. Nagai, K. Ushiro, Y. Matsunami, T. Hashimoto, and Y. Kojima. Hairstyle suggesting system, hairstyle suggesting method, and computer program product. US Patent US20050251463 A1, 2005.
26. C. Rother, V. Kolmogorov, and A. Blake. Grabcut: Interactive foreground extraction using iterated graph cuts. TOG, 2004.
27. K. Scherbaum, T. Ritschel, M. Hullin, T. Thormahlen, V. Blanz, and H. Seidel. Computer-suggested facial makeup. CGF, 2011.
28. W. Tong, C. Tang, M. Brown, and Y. Xu. Example-based cosmetic transfer. In FG, 2007.
29. N. Wang, H. Ai, and F. Tang. What are good parts for hair shape modeling? In CVPR, 2012.
30. Y. Wang and G. Mori. A discriminative latent model of object classes and attributes. In ECCV, 2010.
31. Y. Yang and D. Ramanan. Articulated pose estimation with exible mixtures-of-parts. In CVPR, 2011.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning settings

• Information systems

  • Information retrieval

    • Retrieval models and ranking