Intrinsic colorization

Abstract

In this paper, we present an example-based colorization technique robust to illumination differences between grayscale target and color reference images. To achieve this goal, our method performs color transfer in an illumination-independent domain that is relatively free of shadows and highlights. It first recovers an illumination-independent intrinsic reflectance image of the target scene from multiple color references obtained by web search. The reference images from the web search may be taken from different vantage points, under different illumination conditions, and with different cameras. Grayscale versions of these reference images are then used in decomposing the grayscale target image into its intrinsic reflectance and illumination components. We transfer color from the color reflectance image to the grayscale reflectance image, and obtain the final result by relighting with the illumination component of the target image. We demonstrate via several examples that our method generates results with excellent color consistency.

References

1. Barrow, H., and Tenenbaum., J. 1978. Recovering intrinsic scene characteristics from images. Computer Vision Systems, 3--26.
2. Comaniciu, D., and Meer, P. 2002. Mean shift: A robust approach toward feature space analysis. IEEE Transactions on Pattern Analysis and Machine Intelligence 24, 5, 603--619.
3. Fischler, M. A., and Bolles, R. C. 1987. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Readings in computer vision: issues, problems, principles, and paradigms, 726--740.
4. Freeman, W. T., and Viola, P. A. 1998. Bayesian model of surface perception. In NIPS '97: Proceedings of the 1997 conference on Advances in neural information processing systems 10, MIT Press, Cambridge, MA, USA, 787--793.
5. Hays, J., and Efros, A. A. 2007. Scene completion using millions of photographs. ACM Trans. Graph. 26, 3, 4.
6. Hertzmann, A., Jacobs, C. E., Oliver, N., Curless, B., and Salesin, D. H. 2001. Image analogies. ACM Trans. Graph., 327--340.
7. Huang, Y.-C., Tung, Y.-S., Chen, J.-C., Wang, S.-W., and Wu, J.-L. 2005. An adaptive edge detection based colorization algorithm and its applications. In MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia, ACM, New York, NY, USA, 351--354.
8. Irony, R., Cohen-Or, D., and Lischinski, D. 2005. Colorization by example. In Rendering Techniques 2005, IEEE Computer Society Press, 201--210.
9. Lalonde, J.-F., Hoiem, D., Efros, A. A., Rother, C., Winn, J., and Criminisi, A. 2007. Photo clip art. ACM Trans. Graph. 26, 3, 3.
10. Land, E. H., and McCann, J. J. 1971. Lightness and retinex theory. Journal of the Optical Society of America (1917--1983) 61 (Jan.), 1.
11. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM Trans. Graph. 23, 3, 689--694.
12. Lowe, D. G. 2004. Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision 60, 2, 91--110.
13. Luan, Q., Wen, F., Cohen-Or, D., Liang, L., Xu, Y.-Q., and Shum, H.-Y. 2007. Natural image colorization. In Rendering Techniques 2007 (Proceedings Eurographics Symposium on Rendering), 309--320.
14. Portilla, J., Strela, V., Wainwright, M. J., and Simon-celli, E. P. 2002. Image denoising using gaussian scale mixtures in the wavelet domain. IEEE Transactions on Image Processing 12, 1338--1351.
15. Qu, Y., Wong, T.-T., and Heng, P.-A. 2006. Manga colorization. ACM Trans. Graph. 25, 3, 1214--1220.
16. Reinhard, E., Ashikhmin, M., Gooch, B., and Shirley, P. 2001. Color transfer between images. IEEE Computer Graphics and Applications 21, 5, 34--41.
17. Schnitman, Y., Caspi, Y., Cohen-Or, D., and Lischinski, D. 2006. Inducing semantic segmentation from an example. In Asian Conference on Computer Vision, 373--384.
18. Shewchuk, J. R. 1996. Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator. Applied Computational Geometry: Towards Geometric Engineering 1148 (May), 203--222. From the First ACM Workshop on Applied Computational Geometry.
19. Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph. 25, 3, 835--846.
20. Tappen, M. F., Freeman, W. T., and Adelson, E. H. 2005. Recovering intrinsic images from a single image. IEEE Transactions on Pattern Analysis and Machine Intelligence 27, 9, 1459--1472.
21. Weiss, Y. 2001. Deriving intrinsic images from image sequences. In Eighth IEEE International Conference on Computer Vision, vol. 2, 68--75.
22. Welsh, T., Ashikhmin, M., and Mueller, K. 2002. Transferring color to greyscale images. ACM Trans. Graph. 21, 3, 277--280.
23. Yatziv, L., and Sapiro, G. 2006. Fast image and video colorization using chrominance blending. IEEE Transactions on Image Processing 15, 5, 1120--1129.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

      • Computer vision tasks

        • Scene understanding

  • Computer graphics

    • Image manipulation