Data-driven image color theme enhancement

ABSTRACT

It is often important for designers and photographers to convey or enhance desired color themes in their work. A color theme is typically defined as a template of colors and an associated verbal description. This paper presents a data-driven method for enhancing a desired color theme in an image. We formulate our goal as a unified optimization that simultaneously considers a desired color theme, texture-color relationships as well as automatic or user-specified color constraints. Quantifying the difference between an image and a color theme is made possible by color mood spaces and a generalization of an additivity relationship for two-color combinations. We incorporate prior knowledge, such as texture-color relationships, extracted from a database of photographs to maintain a natural look of the edited images. Experiments and a user study have confirmed the effectiveness of our method.

References

1. An, X., and Pellacini, F. 2008. Appprop: all-pairs appearance-space edit propagation. ACM Trans. Graph. 27, 3, 40.
2. Bae, S., Paris, S., and Durand, F. 2006. Two-scale tone management for photographic look. In SIGGRAPH '06: ACM SIGGRAPH 2006 Papers, ACM, New York, NY, USA, 637--645.
3. Chang, Y., Saito, S., Uchikawa, K., and Nakajima, M. 2005. Example-based color stylization of images. ACM Trans. Appl. Percept. 2, 3, 322--345.
4. Cohen-Or, D., Sorkine, O., Gal, R., Leyvand, T., and Xu, Y.-Q. 2006. Color harmonization. In SIGGRAPH '06: ACM SIGGRAPH 2006 Papers, ACM, New York, NY, USA, 624--630.
5. Felzenszwalb, P. F., and Huttenlocher, D. P. 2004. Efficient graph-based image segmentation. Int. J. Comput. Vision 59, 2, 167--181.
6. Freeman, W. T., Pasztor, E. C., and Carmichael, O. T. 2000. Learning low-level vision. Int. J. Comput. Vision 40, 1, 25--47.
7. Hogg, J. 1969. The prediction of semantic differential ratings of color combinations. J Gen Psychol 80, 141152.
8. Lawrence, C. T., and Tits, A. L. 1996. Nonlinear equality constraints in feasible sequential quadratic programming. Optimization Methods and Software 6, 265--282.
9. Lawrence, C. T., Zhou, J. L., and Tits, A. L. 1997. User's guide for cfsqp version 2.5: A c code for solving (large scale) constrained nonlinear (minimax) optimization problems, generating iterates satisfying all inequality constraints. Institute for Systems Research, University of Maryland, Technical Report TR-94-16r1 College Park, MD 20742.
10. Leung, T., and Malik, J. 2001. Representing and recognizing the visual appearance of materials using three-dimensional textons. Int. J. Comput. Vision 43, 1, 29--44.
11. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3, 689--694.
12. Lischinski, D., Farbman, Z., Uyttendaele, M., and Szeliski, R. 2006. Interactive local adjustment of tonal values. ACM Trans. Graph. 25, 3, 646--653.
13. Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y.-Q., and Shum, H.-Y. 2007. Natural Image Colorization. In Rendering Techniques 2007 (Proceedings Eurographics Symposium on Rendering), J. Kautz and S. Pattanaik, Eds., Eurographics.
14. Manjunath, B. S., and Ma, W. Y. 1996. Texture features for browsing and retrieval of image data. IEEE Trans. Pattern Anal. Mach. Intell. 18, 8, 837--842.
15. Ou, L.-C., Luo, M. R., Woodcock, A., and Wright, A. 2004. A study of colour emotion and colour preference. part i: Colour emotions for single colours. Color Research & Application 29, 3, 232--240.
16. Ou, L.-C., Luo, M. R., Woodcock, A., and Wright, A. 2004. A study of colour emotion and colour preference. part ii: Colour emotions for two-colour combinations. Color Research & Application 29, 4, 292--298.
17. Pellacini, F., and Lawrence, J. 2007. Appwand: editing measured materials using appearance-driven optimization. ACM Trans. Graph. 26, 3, 54.
18. Piti, F., and Kokaram, A. 2007. The linear monge-kantorovitch linear colour mapping for example-based colour transfer. Visual Media Production, 4th European Conference on Visual Media Production, London, UK, 1--9.
19. Qu, Y., Wong, T.-T., and Heng, P.-A. 2006. Manga colorization. ACM Transactions on Graphics (Proceedings of SIGGRAPH 2006) 25, 3, 1214--1220.
20. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Comput. Graph. Appl. 21, 5, 34--41.
21. Rubner, Y., Tomasi, C., and Guibas, L. J. 1998. A metric for distributions with applications to image databases. In ICCV '98: Proceedings of the Sixth International Conference on Computer Vision, IEEE Computer Society, Washington, DC, USA, 59.
22. Sato, T., Kajiwara, K., Hoshino, H., and Nakamura, T. 2000. Quantitative evaluation and categorising of human emotion induced by colour. Advances in Colour Science and Technology 3, 53--59.
23. Shapira, L., Shamir, A., and Cohen-Or, D. 2009. Image appearance exploration by model-based navigation. Comput. Graph. Forum 28, 2, 629--638.
24. Welsh, T., Ashikhmin, M., and Mueller, K. 2002. Transferring color to greyscale images. ACM Transactions on Graphics 21, 3, 277--280.
25. Xu, K., Li, Y., Ju, T., Hu, S.-M., and Liu, T.-Q. 2009. Efficient affinity-based edit propagation using k-d tree. In SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers, ACM, New York, NY, USA, 1--6.
26. Yedidia, J. S., Freeman, W. T., and Weiss, Y. 2003. Understanding belief propagation and its generalizations. 239--269.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Interest point and salient region detections

      • Computer vision representations

        • Image representations

  • Computer graphics

    • Image manipulation