Colorization using optimization

ABSTRACT

Colorization is a computer-assisted process of adding color to a monochrome image or movie. The process typically involves segmenting images into regions and tracking these regions across image sequences. Neither of these tasks can be performed reliably in practice; consequently, colorization requires considerable user intervention and remains a tedious, time-consuming, and expensive task.In this paper we present a simple colorization method that requires neither precise image segmentation, nor accurate region tracking. Our method is based on a simple premise; neighboring pixels in space-time that have similar intensities should have similar colors. We formalize this premise using a quadratic cost function and obtain an optimization problem that can be solved efficiently using standard techniques. In our approach an artist only needs to annotate the image with a few color scribbles, and the indicated colors are automatically propagated in both space and time to produce a fully colorized image or sequence. We demonstrate that high quality colorizations of stills and movie clips may be obtained from a relatively modest amount of user input.

REFERENCES

• 1: BURNS, G. Colorization. Museum of Broadcast Communications: Encyclopedia of Television, http://www.museum.tv/archives/etv/index.html.
• 2: COOPER, R. 1991. Colorization and moral rights: Should the United States adopt unified protection for artists? Journalism Quarterly (Urbana, Illinois), Autumn.
• 3: JACK, K. 2001. Video Demystified, 3rd edition ed. Elsevier Science & Technology.
• 4: LUCAS, B., AND KANADE, T. 1981. An iterative image registration technique with an application to stereo vision. In Proc. Int. Joint Conf. AI 674--679.
• 5: MARKLE, W., AND HUNT, B., 1987. Coloring a black and white signal using motion detection. Canadian patent no. 1291260, Dec.
• 6: NEURALTEK, 2003. BlackMagic photo colorization software, version 2.8 http://www.timebrush.com/blackmagic.
• 7: PERONA, P., AND MALIK, J. 1989. Scale-space and edge detection using anisotropic diffusion. IEEE Transactions on PAMI 8, 5, 565--593.
• 8: William H. Press , Saul A. Teukolsky , William T. Vetterling , Brian P. Flannery, Numerical recipes in C (2nd ed.): the art of scientific computing, Cambridge University Press, New York, NY, 1992
• 9: Jianbo Shi , Jitendra Malik, Normalized Cuts and Image Segmentation, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.731, June 17-19, 1997
• 10: SILBERG, J., 1998. The Pleasantville post production team that focussed on the absence of color. Cinesite Press Article, http://www.cinesite.com/core/press/articles/1998/10_00_98-team.html.
• 11: TANG, B., SAPIRO, G., AND CASSELES, V. 2001. Color image enhancement via chromaticity diffusion. IEEE Transactions on Image Processing 10, 5, 701--708.
• 12: TORRALBA, A., AND FREEMAN, W. T. 2003. Properties and applications of shape recipes. In IEEE Computer Vision and Pattern Recognition (CVPR).
• 13: Yair Weiss, Segmentation Using Eigenvectors: A Unifying View, Proceedings of the International Conference on Computer Vision-Volume 2, p.975, September 20-25, 1999
• 14: Tomihisa Welsh , Michael Ashikhmin , Klaus Mueller, Transferring color to greyscale images, ACM Transactions on Graphics (TOG), v.21 n.3, July 2002
• 15: Assaf Zomet , Shmuel Peleg, Multi-sensor Super-Resolution, Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision, p.27, December 03-04, 2002

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.9 Applications

Keywords:
colorization, recoloring, segmentation