Article

Removing photography artifacts using gradient projection and flash-exposure sampling

Authors:

Shree K. Nayar,

Yuanzhen LiAuthors Info & Claims

SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

Pages 828 - 835

https://doi.org/10.1145/1186822.1073269

Published: 01 July 2005 Publication History

Abstract

Flash images are known to suffer from several problems: saturation of nearby objects, poor illumination of distant objects, reflections of objects strongly lit by the flash and strong highlights due to the reflection of flash itself by glossy surfaces. We propose to use a flash and no-flash (ambient) image pair to produce better flash images. We present a novel gradient projection scheme based on a gradient coherence model that allows removal of reflections and highlights from flash images. We also present a brightness-ratio based algorithm that allows us to compensate for the falloff in the flash image brightness due to depth. In several practical scenarios, the quality of flash/no-flash images may be limited in terms of dynamic range. In such cases, we advocate using several images taken under different flash intensities and exposures. We analyze the flash intensity-exposure space and propose a method for adaptively sampling this space so as to minimize the number of captured images for any given scene. We present several experimental results that demonstrate the ability of our algorithms to produce improved flash images.

Supplementary Material

JPG File (pps052.jpg)

Download
16.95 KB

MP4 File (pps052.mp4)

Download
42.15 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Transactions on Graphics 23, 3 (Aug.), 294--302.

Digital Library

[2]

Canon. http://www.cps.canon-europe.com/kb/.

[3]

Chen, H., Belhumeur, P., and Jacobs, D. 2000. In search of illumination invariants. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 254--261.

[4]

Debevec, P. E., and Malik, J. 1997. Recovering high dynamic range radiance maps from photographs. In Proc. of the 24th annual conference on Computer graphics and interactive techniques, 369--378.

Digital Library

[5]

Eisemann, E., and Durand, F. 2004. Flash photography enhancement via intrinsic relighting. ACM Transactions on Graphics 23, 3 (Aug.). 673--678.

Digital Library

[6]

Farid, H., and Adelson, E. H. 1999. Separating reflections and lighting using independent components analysis. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 1262--1267.

[7]

Fattal, R., Lischinski, D., and Werman, M. 2002. Gradient domain high dynamic range compression. ACM Transactions on Graphics 21, 3, 249--256.

Digital Library

[8]

Finlayson, G., Hordley, S., and Drew, M. 2002. Removing shadows from images. In Proc. of European Conf. on Computer Vision, vol. 4, 823--836.

Digital Library

[9]

Finlayson, G. D., Drew, M. S., and Lu, C. 2004. Intrinsic images by entropy minimization. In Proc. of European Conf. on Computer Vision, vol. 3, 582--595.

[10]

Grossberg, M. D., and Nayar, S. K. 2003. High dynamic range from multiple images: which exposures to combine? In Proc. of ICCV Workshop on Color and Photometric Methods in Computer Vision.

[11]

Jahne, B. 1993. Spatio-temporal image processing, theory and scientific applications, vol. 751 of Lecture Notes in Computer Vision. Springer-Verlag.

Digital Library

[12]

Kodak. 2001. CCD image sensor noise sources. Application note MPT/PS-0233.

[13]

Levin, A., Zomet, A., and Weiss, Y. 2004. Separating reflections from a single image using local features. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 306--313.

[14]

Lichtenauer, J., Reinders, M., and Hendriks, E. 2004. Influence of the observation likelihood function on particle filtering performance in tracking applications. In Sixth IEEE Int'l. Conf. on Automatic Face and Gesture Recognition, 767--772.

Digital Library

[15]

Mann, S., and Picard, R. W. 1995. Being undigital with digital cameras: extending dynamic range by combining differently exposed pictures. In Proc. of IS&T 48th annual conference, 422--428.

[16]

Nayar, S. K., and Mitsunaga, T. 2000. High dynamic range imaging: spatially varying pixel exposures. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 472--479.

[17]

Nayar, S. K., Fang, X.-S., and Boult, T. 1997. Separation of reflection components using color and polarization. Int'l. Journal of Computer Vision 21, 3 (Feb.), 163--186.

Digital Library

[18]

Perez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Transactions on Graphics 22, 3, 313--318.

Digital Library

[19]

Petschnigg, G., Agrawala, M., Hoppe, H., Szeliski, R., Cohen, M., and Toyama, K. 2004. Digital photography with flash and no-flash image pairs. ACM Transactions on Graphics 23, 3 (Aug.), 664--672.

Digital Library

[20]

Press, W. H., Teukolsky, S., Vetterling, W. T., and Flannery, B. P. 1992. Numerical recipes in C: the art of scientific computing. Pearson Education.

Digital Library

[21]

Raskar, R., Tan, K., Feris, R., Yu, J., and Turk, M. 2004. Non-photorealistic camera: depth edge detection and stylized rendering using multi-flash imaging. ACM Transactions on Graphics 23, 3, 679--688.

Digital Library

[22]

Raskar, R., Ilie, A., and Yu, J. 2004. Image fusion for context enhancement and video surrealism. In Proc. of NPAR, 85--95.

Digital Library

[23]

Schechner, Y. Y., Kiryati, N., and Basri, R. 2000. Separation of transparent layers using focus. Int'l. Journal of Computer Vision 39, 1 (Aug.), 25--39.

Digital Library

[24]

Sun, J., Jia, J., Tang, C.-K., and Shum, H.-Y. 2004. Poisson matting. ACM Transactions on Graphics 23, 3, 315--321.

Digital Library

[25]

Szeliski, R., Avidan, S., and Anandan, P. 2000. Layer extraction from multiple images containing reflections and transparency. In Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, vol. 1, 246--243.

Cited By

Jo SJang OBhattacharyya CKim MLee TJang YSong HKwon HDo SKim S(2024)S-LIGHT: Synthetic Dataset for the Separation of Diffuse and Specular Reflection ImagesSensors10.3390/s2407228624:7(2286)Online publication date: 3-Apr-2024
https://doi.org/10.3390/s24072286
Rosh GB H PBoregowda LMitra KCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)R2SFD: Improving Single Image Reflection Removal using Semantic Feature DictionaryProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681450(10277-10286)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681450
Zou WLu XYi ZZhang LFu GLi PXiao C(2024)Eyeglass Reflection Removal With Joint Learning of Reflection Elimination and Content InpaintingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.340557634:10(10266-10280)Online publication date: Oct-2024
https://doi.org/10.1109/TCSVT.2024.3405576
Show More Cited By

Index Terms

Removing photography artifacts using gradient projection and flash-exposure sampling
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Image manipulation
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Random projections and metric embeddings

Recommendations

Removing photography artifacts using gradient projection and flash-exposure sampling

Flash images are known to suffer from several problems: saturation of nearby objects, poor illumination of distant objects, reflections of objects strongly lit by the flash and strong highlights due to the reflection of flash itself by glossy surfaces. ...
Using flash photography and image-based rendering to document cultural heritage artifacts
GCH '16: Proceedings of the 14th Eurographics Workshop on Graphics and Cultural Heritage

A novel image-based rendering system is proposed for documenting cultural heritage artifacts. The system utilizes backscattering photography to acquire the initial pictures and derives estimates for the object's diffuse albedo, surface normals, and the ...
Symmetric photography: exploiting data-sparseness in reflectance fields
EGSR '06: Proceedings of the 17th Eurographics conference on Rendering Techniques

We present a novel technique called symmetric photography to capture real world reflectance fields. The technique models the 8D reflectance field as a transport matrix between the 4D incident light field and the 4D exitant light field. It is a ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGGRAPH '05: ACM SIGGRAPH 2005 Papers

July 2005

826 pages

ISBN:9781450378253

DOI:10.1145/1186822

Editor:
Markus Gross
Eidgenössische Technische Hochschule Zürich

Copyright © 2005 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 July 2005

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGGRAPH05

Sponsor:

SIGGRAPH

SIGGRAPH05: Special Interest Group on Computer Graphics and Interactive Techniques Conference

July 31 - August 4, 2005

California, Los Angeles

Acceptance Rates

SIGGRAPH '05 Paper Acceptance Rate 98 of 461 submissions, 21%;

Overall Acceptance Rate 1,822 of 8,601 submissions, 21%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

57
Total Citations
View Citations
2,316
Total Downloads

Downloads (Last 12 months)26
Downloads (Last 6 weeks)2

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jo SJang OBhattacharyya CKim MLee TJang YSong HKwon HDo SKim S(2024)S-LIGHT: Synthetic Dataset for the Separation of Diffuse and Specular Reflection ImagesSensors10.3390/s2407228624:7(2286)Online publication date: 3-Apr-2024
https://doi.org/10.3390/s24072286
Rosh GB H PBoregowda LMitra KCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)R2SFD: Improving Single Image Reflection Removal using Semantic Feature DictionaryProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3681450(10277-10286)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3681450
Zou WLu XYi ZZhang LFu GLi PXiao C(2024)Eyeglass Reflection Removal With Joint Learning of Reflection Elimination and Content InpaintingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2024.340557634:10(10266-10280)Online publication date: Oct-2024
https://doi.org/10.1109/TCSVT.2024.3405576
Liu YWeng WGao RXiao ZZhang YXiong Z(2024)Disparity-Guided Multi-View Interaction Network for Light Field Reflection RemovalIEEE Transactions on Computational Imaging10.1109/TCI.2024.339477310(726-741)Online publication date: 2024
https://doi.org/10.1109/TCI.2024.3394773
Tang JZhou WLiu HMo YKe W(2024)Single Image Reflection Separation in Scenes with Light Source Only on One Side of the Camera2024 International Conference on Cloud and Network Computing (ICCNC)10.1109/ICCNC63989.2024.00015(36-41)Online publication date: 31-May-2024
https://doi.org/10.1109/ICCNC63989.2024.00015
Liu YDu DTu SYang WSuo SHan X(2024)Artifact-aware Digital Subtraction Angiogram Image Generation for Head and Neck Vessels2024 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM62325.2024.10822606(3527-3530)Online publication date: 3-Dec-2024
https://doi.org/10.1109/BIBM62325.2024.10822606
Feng YHan SWei QHui HCheng YLiu J(2024)Image Reflection Removal Method Based on Edge Clues and Convolutional Guided FiltersAdvancement in Computational Methods for Life Systems Modelling and Simulation10.1007/978-981-96-0188-2_27(340-350)Online publication date: 28-Dec-2024
https://doi.org/10.1007/978-981-96-0188-2_27
Wei AZhang HZhao E(2024)DereflectFormer: Vision Transformers for Single Image Reflection RemovalPattern Recognition10.1007/978-3-031-78305-0_17(257-274)Online publication date: 4-Dec-2024
https://doi.org/10.1007/978-3-031-78305-0_17
Lei CJiang XChen Q(2023)Robust Reflection Removal With Flash-Only Cues in the WildIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.331497245:12(15530-15545)Online publication date: Dec-2023
https://doi.org/10.1109/TPAMI.2023.3314972
Zhou CLyu ZRodrigues M(2023)BITS-Net: Blind Image Transparency Separation Network2023 IEEE International Conference on Image Processing (ICIP)10.1109/ICIP49359.2023.10222918(375-379)Online publication date: 8-Oct-2023
https://doi.org/10.1109/ICIP49359.2023.10222918
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten