article

Open access

Scene completion using millions of photographs

Authors:

Alexei A. EfrosAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 26, Issue 3

Pages 4 - es

https://doi.org/10.1145/1276377.1276382

Published: 29 July 2007 Publication History

Abstract

What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database that are not only seamless but also semantically valid. Our chief insight is that while the space of images is effectively infinite, the space of semantically differentiable scenes is actually not that large. For many image completion tasks we are able to find similar scenes which contain image fragments that will convincingly complete the image. Our algorithm is entirely data-driven, requiring no annotations or labelling by the user. Unlike existing image completion methods, our algorithm can generate a diverse set of results for each input image and we allow users to select among them. We demonstrate the superiority of our algorithm over existing image completion approaches.

Supplementary Material

MP4 File (pps004.mp4)

Download
67.39 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph. 23, 3, 294--302.

Digital Library

[2]

Agrawal, A., Raskar, R., and Chellappa, R. 2006. What is the range of surface reconstructions from a gradient field? In ECCV.

Digital Library

[3]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23, 11, 1222--1239.

Digital Library

[4]

Criminisi, A., Perez, P., and Toyama, K. 2003. Object removal by exemplar-based inpainting. CVPR 02, 721.

[5]

Diakopoulos, N., Essa, I., and Jain, R. 2004. Content based image synthesis. In Conference on Image and Video Retrieval.

[6]

Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based image completion. ACM Trans. Graph. 22, 3, 303--312.

Digital Library

[7]

Efros, A. A., and Freeman, W. T. 2001. Image quilting for texture synthesis and transfer. Proceedings of SIGGRAPH 2001 (August), 341--346.

Digital Library

[8]

Efros, A. A., and Leung, T. K. 1999. Texture synthesis by non-parametric sampling. In ICCV, 1033--1038.

Digital Library

[9]

Irani, M., Anandan, P., and Hsu, S. 1995. Mosaic based representations of video sequences and their applications.

[10]

Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-drop pasting. ACM Trans. Graph..

Digital Library

[11]

Johnson, M., Brostow, G. J., Shotton, J., Arandjelović, O., Kwatra, V., and Cipolla, R. 2006. Semantic photo synthesis. Computer Graphics Forum (Proc. Eurographics) 25, 3 (September), 407--413.

[12]

King, D. 1997. The Commissar Vanishes. Henry Holt and Co.

[13]

Komodakis, N. 2006. Image completion using global optimization. In CVPR, 442--452.

Digital Library

[14]

Kwatra, V., Schodl, A., Essa, I., Turk, G., and Bobick, A. 2003. Graphcut textures: Image and video synthesis using graph cuts. ACM Trans. Graph. 22, 3 (July), 277--286.

Digital Library

[15]

Kwatra, V., Essa, I., Bobick, A., and Kwatra, N. 2005. Texture optimization for example-based synthesis. In ACM Trans. Graph., 795--802.

Digital Library

[16]

Oliva, A., and Torralba, A. 2006. Building the gist of a scene: The role of global image features in recognition. In Visual Perception, Progress in Brain Research, vol. 155.

[17]

Perez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph. 22, 3, 313--318.

Digital Library

[18]

Russell, B. C., Torralba, A., Murphy, K. P., and Freeman, W. T. 2005. LabelMe: a database and web-based tool for image annotation. Tech. rep., MIT, 2005.

[19]

Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3d. ACM Trans. Graph. 25, 3, 835--846.

Digital Library

[20]

Sun, J., Yuan, L., Jia, J., and Shum, H.-Y. 2005. Image completion with structure propagation. ACM Trans. Graph. 24, 3, 861--868.

Digital Library

[21]

Torralba, A., Murphy, K. P., Freeman, W. T., and Rubin, M. A. 2003. Context-based vision system for place and object recognition. In ICCV.

Digital Library

[22]

Torralba, A., Fergus, R., and Freeman, W. T. 2007. Tiny images. Tech. Rep. MIT-CSAIL-TR-2007-024.

[23]

Wertheimer, M. 1938. Laws of organization in perceptual forms (partial translation). In A sourcebook of Gestalt Psychology, W. Ellis, Ed. Harcourt Brace and Company, 71--88.

[24]

Wexler, Y., Shechtman, E., and Irani, M. 2004. Space-time video completion. CVPR 01, 120--127.

[25]

Wilczkowiak, M., Brostow, G. J., Tordoff, B., and Cipolla, R. 2005. Hole filling through photomontage. In BMVC, 492--501.

Cited By

Xu YXia MHu KZhou SWeng L(2025)Style Transfer Review: Traditional Machine Learning to Deep LearningInformation10.3390/info1602015716:2(157)Online publication date: 19-Feb-2025
https://doi.org/10.3390/info16020157
Yeo WRyu H(2025)An Integration Framework for the Inpainting and Colorization of Arbitrary Masked Grayscale ImagesApplied Sciences10.3390/app1504197815:4(1978)Online publication date: 13-Feb-2025
https://doi.org/10.3390/app15041978
ren cxu y(2025)Based on structure and texture dual-stream network for ancient mural restorationFifth International Conference on Signal Processing and Computer Science (SPCS 2024)10.1117/12.3052938(19)Online publication date: 3-Jan-2025
https://doi.org/10.1117/12.3052938
Show More Cited By

Index Terms

Scene completion using millions of photographs
1. Applied computing
  1. Document management and text processing
    1. Document capture
      1. Graphics recognition and interpretation
2. Computing methodologies

Recommendations

Scene completion using millions of photographs

What can you do with a million images? In this paper, we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database ...
Scene Completion Using Millions of Photographs
Seminal Graphics Papers: Pushing the Boundaries, Volume 2

What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database ...
Scene completion using millions of photographs
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

What can you do with a million images? In this paper we present a new image completion algorithm powered by a huge database of photographs gathered from the Web. The algorithm patches up holes in images by finding similar image regions in the database ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 26, Issue 3

July 2007

976 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1276377

Issue’s Table of Contents

Seminal Graphics Papers: Pushing the Boundaries, Volume 2
August 2023
893 pages
ISBN:9798400708978
DOI:10.1145/3596711
Editor:
Mary C. Whitton
Department of Computer Science, UNC Chapel Hill, USA

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 July 2007

Published in TOG Volume 26, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Badges

Seminal Paper

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

488
Total Citations
View Citations
5,029
Total Downloads

Downloads (Last 12 months)441
Downloads (Last 6 weeks)86

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Xu YXia MHu KZhou SWeng L(2025)Style Transfer Review: Traditional Machine Learning to Deep LearningInformation10.3390/info1602015716:2(157)Online publication date: 19-Feb-2025
https://doi.org/10.3390/info16020157
Yeo WRyu H(2025)An Integration Framework for the Inpainting and Colorization of Arbitrary Masked Grayscale ImagesApplied Sciences10.3390/app1504197815:4(1978)Online publication date: 13-Feb-2025
https://doi.org/10.3390/app15041978
ren cxu y(2025)Based on structure and texture dual-stream network for ancient mural restorationFifth International Conference on Signal Processing and Computer Science (SPCS 2024)10.1117/12.3052938(19)Online publication date: 3-Jan-2025
https://doi.org/10.1117/12.3052938
Wang RYao YChen PFu LGao XZhang L(2025)Global Ionospheric VTEC Data Completion Method Based on Aggregated Contextual-Transformation Generative Adversarial NetsIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.351108363(1-23)Online publication date: 2025
https://doi.org/10.1109/TGRS.2024.3511083
Ko KWoo SKim C(2025)Error-Mask-Adaptive Dynamic Filtering for Image InpaintingIEEE Access10.1109/ACCESS.2025.353144013(18403-18417)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3531440
Jaiprakash SPrakash C(2025)Exploring text-to-image generation models: Applications and cloud resource utilizationComputers and Electrical Engineering10.1016/j.compeleceng.2025.110194123(110194)Online publication date: Apr-2025
https://doi.org/10.1016/j.compeleceng.2025.110194
Öztürk ÖBalaban Y(2024)ART IN PARTNERSHIP WITH HUMAN AND ARTIFICIAL INTELLIGENCE: CREATIVITY AND ALGORITHMSanat ve Tasarım Dergisi10.20488/sanattasarim.150608614:1(335-352)Online publication date: 28-Jun-2024
https://doi.org/10.20488/sanattasarim.1506086
Wan TFu CPeng JLu JLi PZhuo J(2024)Repairing the in situ hybridization missing data in the hippocampus region by using a 3D residual U-Net modelBiomedical Optics Express10.1364/BOE.52207815:6(3541)Online publication date: 1-May-2024
https://doi.org/10.1364/BOE.522078
Han XMa JCheng H(2024)Application Research on Image Recovery Technology Based on GANJournal of Sensors10.1155/2024/74981602024:1Online publication date: 21-Nov-2024
https://doi.org/10.1155/2024/7498160
Wang HCao TJin LLiu WWang RMa Z(2024)BlipFill: Completing Authentic Images with BLIP-enhanced Subject RepresentationProceedings of the 2024 8th International Conference on Computer Science and Artificial Intelligence10.1145/3709026.3709062(112-118)Online publication date: 6-Dec-2024
https://dl.acm.org/doi/10.1145/3709026.3709062
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

Figures

Tables

Media

View Issue’s Table of Contents