User-assisted intrinsic images

Abstract

For many computational photography applications, the lighting and materials in the scene are critical pieces of information. We seek to obtain intrinsic images, which decompose a photo into the product of an illumination component that represents lighting effects and a reflectance component that is the color of the observed material. This is an under-constrained problem and automatic methods are challenged by complex natural images. We describe a new approach that enables users to guide an optimization with simple indications such as regions of constant reflectance or illumination. Based on a simple assumption on local reflectance distributions, we derive a new propagation energy that enables a closed form solution using linear least-squares. We achieve fast performance by introducing a novel downsampling that preserves local color distributions. We demonstrate intrinsic image decomposition on a variety of images and show applications.

References

1. Agrawal, A., Raskar, R., and Chellappa, R. 2006. Edge suppression by gradient field transformation using cross-projection tensors. In CVPR, 2301--2308.
2. Barrow, H., and Tenenbaum, J. 1978. Recovering intrinsic scene characteristics from images. Computer Vision Systems.
3. Briggs, W. L., Henson, V. E., and McCormick, S. F. 2000. A multigrid tutorial (2nd ed.). Society for Industrial and Applied Mathematics.
4. Buatois, L., Caumon, G., and Lévy, B. 2007. Concurrent number cruncher: An efficient sparse linear solver on the gpu. In High Performance Computation Conference.
5. Chuang, Y.-Y., Curless, B., Salesin, D. H., and Szeliski, R. 2001. A bayesian approach to digital matting. In CVPR.
6. Fang, H., and Hart, J. C. 2004. Textureshop: Texture synthesis as a photograph editing tool. ACM TOG (proc. of SIGGRAPH 2004) 23, 3, 354--359.
7. Fattal, R. 2008. Single image dehazing. ACM TOG (proc. of SIGGRAPH 2008) 27, 3, 72.
8. Finlayson, G. D., Hordley, S. D., and Drew, M. S. 2002. Removing shadows from images. In ECCV.
9. Finlayson, G. D., Drew, M. S., and Lu, C. 2004. Intrinsic images by entropy minimization. In ECCV, 582--595.
10. Horn, B. K. 1986. Robot Vision. MIT Press.
11. Hsu, E., Mertens, T., Paris, S., Avidan, S., and Durand, F. 2008. Light mixture estimation for spatially varying white balance. ACM TOG (proc. of SIGGRAPH 2008) 27, 3, 70.
12. Khan, E., Reinhard, E., Fleming, R., and Bülthoff, H. 2005. Image-based material editing. ACM TOG (proc. of SIGGRAPH 2005) 24, 3, 654--663.
13. Land, E. H., and McCann, J. J. 1971. Lightness and retinex theory. Journal of the optical society of America 61, 1.
14. Levin, A., and Weiss, Y. 2007. User assisted separation of reflections from a single image using a sparsity prior. IEEE Trans. PAMI 29, 9, 1647--1654.
15. Levin, A., Lischinski, D., and Weiss, Y. 2004. Colorization using optimization. ACM TOG (proc. of SIGGRAPH 2004) 23, 689--694.
16. Levin, A., Lischinski, D., and Weiss, Y. 2008. A closed-form solution to natural image matting. IEEE Trans. PAMI.
17. Liu, X., Wan, L., Qu, Y., Wong, T.-T., Lin, S., Leung, C.-S., and Heng, P.-A. 2008. Intrinsic colorization. ACM TOG (proc. of SIGGRAPH Asia 2008) 27, 5, 152.
18. McCann, J., and Pollard, N. S. 2008. Real-time gradient-domain painting. ACM TOG (Proc. of SIGGRAPH) 27, 3, 93.
19. Mohan, A., Tumblin, J., and Choudhury, P. 2007. Editing soft shadows in a digital photograph. IEEE Computer Graphics and Applications 27, 2, 23--31.
20. Omer, I., and Werman, M. 2004. Color lines: Image specific color representation. In CVPR, 946--953.
21. Shen, L., Tan, P., and Lin, S. 2008. Intrinsic image decomposition with non-local texture cues. In CVPR.
22. Shor, Y., and Lischinski, D. 2008. The shadow meets the mask: Pyramid-based shadow removal. Computer Graphics Forum (Proc. of Eurographics) 27, 3.
23. Sinha, P., and Adelson, E. 1993. Recovering reflectance and illumination in a world of painted polyhedra. In ICCV, 156--163.
24. Tappen, M. F., Freeman, W. T., and Adelson, E. H. 2005. Recovering intrinsic images from a single image. IEEE Trans. PAMI 27, 9.
25. Weiss, Y. 2001. Deriving intrinsic images from image sequences. In ICCV, 68--75.
26. Wu, T.-P., Tang, C.-K., Brown, M. S., and Shum, H.-Y. 2007. Natural shadow matting. ACM TOG 26, 2, 8.
27. Yu, Y., and Malik, J. 1998. Recovering photometric properties of architectural scenes from photographs. In ACM SIGGRAPH 98, 207--217.

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision tasks

      • Image and video acquisition

  • Computer graphics

    • Shape modeling

• Theory of computation

  • Randomness, geometry and discrete structures

    • Computational geometry