article

Photo clip art

Authors:

Jean-François Lalonde,

Alexei A. Efros,

Carsten Rother,

Antonio CriminisiAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 26, Issue 3

Pages 3 - es

https://doi.org/10.1145/1276377.1276381

Published: 29 July 2007 Publication History

Abstract

We present a system for inserting new objects into existing photographs by querying a vast image-based object library, pre-computed using a publicly available Internet object database. The central goal is to shield the user from all of the arduous tasks typically involved in image compositing. The user is only asked to do two simple things: 1) pick a 3D location in the scene to place a new object; 2) select an object to insert using a hierarchical menu. We pose the problem of object insertion as a data-driven, 3D-based, context-sensitive object retrieval task. Instead of trying to manipulate the object to change its orientation, color distribution, etc. to fit the new image, we simply retrieve an object of a specified class that has all the required properties (camera pose, lighting, resolution, etc) from our large object library. We present new automatic algorithms for improving object segmentation and blending, estimating true 3D object size and orientation, and estimating scene lighting conditions. We also present an intuitive user interface that makes object insertion fast and simple even for the artistically challenged.

Supplementary Material

JPG File (pps003.jpg)

Download
18.20 KB

MP4 File (pps003.mp4)

Download
38.08 MB

References

[1]

Agarwala, A., Dontcheva, M., Agrawala, M., Drucker, S., Colburn, A., Curless, B., Salesin, D., and Cohen, M. 2004. Interactive digital photomontage. ACM Trans. Graph. (SIGGRAPH 04) 23, 3, 294--302.

Digital Library

[2]

Berg, T. L., Berg, A. C., Edwards, J., Maire, M., White, R., Teh, Y.-W., Learned-Miller, E., and Forsyth, D. A. 2004. Names and faces in the news. In IEEE Computer Vision and Pattern Recognition (CVPR).

Digital Library

[3]

Boykov, Y., Veksler, O., and Zabih, R. 2001. Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Analysis and Machine Intelligence 23, 11.

Digital Library

[4]

Boykov, Y., Kolmogorov, V., Cremers, D., and Delong, A. 2006. An integral solution to surface evolution PDEs via Geo-Cuts. In European Conf. on Computer Vision (ECCV).

Digital Library

[5]

Cavanagh, P. 2005. The artist as neuroscientist. Nature 434 (March), 301--307.

[6]

Chuang, Y.-Y., Goldman, D. B., Curless, B., Salesin, D. H., and Szeliski, R. 2003. Shadow matting and compositing. ACM Transactions on Graphics (SIGGRAPH 03) 22, 3 (July), 494--500.

Digital Library

[7]

Criminisi, A., Reid, I., and Zisserman, A. 2000. Single view metrology. International Journal of Computer Vision 40, 2, 123--148.

Digital Library

[8]

Debevec, P. 1998. Rendering synthetic objects into real scenes: Bridging traditional and image-based graphics with global illumination and high dynamic range photography. In Proceedings of SIGGRAPH 98, 189--198.

Digital Library

[9]

Diakopoulos, N., Essa, I., and Jain, R. 2004. Content based image synthesis. In Conference on Image and Video Retrieval (CIVR).

[10]

Everingham, M., Zisserman, A., Williams, C., and Gool, L. V. 2006. The pascal visual object classes challenge 2006 results. Tech. rep., Oxford University.

[11]

Fei-Fei, L., Fergus, R., and Perona, P. 2004. Learning generative visual models from few training examples: An incremental bayesian approach tested on 101 object categories. In IEEE CVPR Workshop of Generative Model Based Vision.

Digital Library

[12]

Finlayson, G. D., Hordley, S. D., Lu, C., and Drew, M. S. 2006. On the removal of shadows from images. IEEE Trans. Pattern Analysis and Machine Intelligence 28, 1, 59--68.

Digital Library

[13]

Hoiem, D., Efros, A. A., and Hebert, M. 2005. Geometric context from a single image. In International Conference on Computer Vision (ICCV).

Digital Library

[14]

Hoiem, D., Efros, A. A., and Hebert, M. 2006. Putting objects in perspective. In IEEE Computer Vision and Pattern Recognition (CVPR).

Digital Library

[15]

Jia, J., Sun, J., Tang, C.-K., and Shum, H.-Y. 2006. Drag-and-drop pasting. ACM Transactions on Graphics (SIGGRAPH 06) 25, 3 (July), 631--637.

Digital Library

[16]

Johnson, M., Brostow, G. J., Shotton, J., Arandjelović, O., Kwatra, V., and Cipolla, R. 2006. Semantic photo synthesis. Computer Graphics Forum (Proc. Eurographics) 25, 3, 407--413.

[17]

Kersten, D., Knill, D., Mamassian, P., and Bulthoff, I. 1996. Illusory motion from shadows. Nature 379, 6560, 31--31.

[18]

Khan, E. A., Reinhard, E., Fleming, R. W., and Bülthoff, H. H. 2006. Image-based material editing. ACM Transactions on Graphics (SIGGRAPH 06) 25, 3 (July), 654--663.

Digital Library

[19]

Kolmogorov, V., and Boykov, Y. 2005. What metrics can be approximated by Geo-Cuts, or global optimization of length/area and flux. In International Conference on Computer Vision (ICCV).

Digital Library

[20]

Levin, A., Lischinski, D., and Weiss, Y. 2006. A closed form solution to natural image matting. In Proc IEEE Computer Vision and Pattern Recognition (extended Tech. Rep.).

Digital Library

[21]

Li, Y., Sun, J., Tang, C.-K., and Shum, H.-Y. 2004. Lazy snapping. ACM Transactions on Graphics (SIGGRAPH 04) 23, 3 (Aug.), 303--308.

Digital Library

[22]

Perez, P., Gangnet, M., and Blake, A. 2003. Poisson image editing. ACM Trans. Graph. (SIGGRAPH 03) 22, 3, 313--318.

Digital Library

[23]

Porter, T., and Duff, T. 1984. Compositing digital images. In Computer Graphics (Proceedings of SIGGRAPH 84), 253--259.

Digital Library

[24]

Quinlan, J. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers, Inc.

Digital Library

[25]

Rother, C., Kolmogorov, V., and Blake, A. 2004. Grab-Cut: interactive foreground extraction using iterated graph cuts. ACM Transactions on Graphics (SIGGRAPH 04) 23, 3 (Aug.), 309--314.

Digital Library

[26]

Rother, C., Bordeaux, L., Hamadi, Y., and Blake, A. 2006. Autocollage. ACM Transactions on Graphics (SIGGRAPH 06) 25, 3 (July), 847--852.

Digital Library

[27]

Rother, C. 2007. Cut-and-paste for photo clip art. Tech. Rep. MSR-TR-2007-45, Microsoft Research.

[28]

Russell, B. C., Torralba, A., Murphy, K. P., and Freeman, W. T. 2005. LabelMe: a database and web-based tool for image annotation. Tech. rep., MIT.

[29]

Russell, B. C., Efros, A. A., Sivic, J., Freeman, W. T., and Zisserman, A. 2006. Using multiple segmentations to discover objects and their extent in image collections. In IEEE Computer Vision and Pattern Recognition (CVPR).

Digital Library

[30]

Shotton, J., Winn, J., Rother, C., and Criminisi, A. 2006. Textonboost: Joint appearance, shape and context modeling for multi-class object recognition and segmentation. In European Conf. on Computer Vision (ECCV).

Digital Library

[31]

Snavely, N., Seitz, S. M., and Szeliski, R. 2006. Photo tourism: exploring photo collections in 3D. ACM Trans. Graph. (SIGGRAPH 06) 25, 3, 835--846.

Digital Library

[32]

Torralba, A., and Oliva, A. 2003. Statistics of natural image categories. Network: Computation in Neural Systems 14, 3 (August), 391--412.

[33]

von Ahn, L., Liu, R., and Blum, M. 2006. Peekaboom: A game for locating objects in images. In ACM CHI.

Digital Library

[34]

Wang, J., and Cohen, M. 2006. Simultaneous matting and compositing. Tech. Rep. MSR-TR-2006-63.

Cited By

Shahi MRajabi RMasoumzadeh F(2025)CNN-based classification of Persian miniature paintings from five renowned schoolsDigital Applications in Archaeology and Cultural Heritage10.1016/j.daach.2024.e0039736(e00397)Online publication date: Mar-2025
https://doi.org/10.1016/j.daach.2024.e00397
Zhou HMa RZhang LGao LMahdavi-Amiri AZhang H(2024)SAC-GAN: Structure-Aware Image CompositionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.322668930:7(3151-3165)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2022.3226689
Bell TLi D(2024)Diffusion Models for Cross-Domain Image-to-Image Translation with Paired and Partially Paired Datasets2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA61799.2024.10722775(1-8)Online publication date: 6-Oct-2024
https://doi.org/10.1109/DSAA61799.2024.10722775
Show More Cited By

Index Terms

Photo clip art
1. Computing methodologies

Recommendations

Photo clip art
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

We present a system for inserting new objects into existing photographs by querying a vast image-based object library, pre-computed using a publicly available Internet object database. The central goal is to shield the user from all of the arduous tasks ...
Webcam clip art: appearance and illuminant transfer from time-lapse sequences

Webcams placed all over the world observe and record the visual appearance of a variety of outdoor scenes over long periods of time. The recorded time-lapse image sequences cover a wide range of illumination and weather conditions -- a vast untapped ...
Webcam clip art: appearance and illuminant transfer from time-lapse sequences
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers

Webcams placed all over the world observe and record the visual appearance of a variety of outdoor scenes over long periods of time. The recorded time-lapse image sequences cover a wide range of illumination and weather conditions -- a vast untapped ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 26, Issue 3

July 2007

976 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1276377

Issue’s Table of Contents

Copyright © 2007 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 July 2007

Published in TOG Volume 26, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

156
Total Citations
View Citations
2,818
Total Downloads

Downloads (Last 12 months)178
Downloads (Last 6 weeks)7

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Shahi MRajabi RMasoumzadeh F(2025)CNN-based classification of Persian miniature paintings from five renowned schoolsDigital Applications in Archaeology and Cultural Heritage10.1016/j.daach.2024.e0039736(e00397)Online publication date: Mar-2025
https://doi.org/10.1016/j.daach.2024.e00397
Zhou HMa RZhang LGao LMahdavi-Amiri AZhang H(2024)SAC-GAN: Structure-Aware Image CompositionIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.322668930:7(3151-3165)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2022.3226689
Bell TLi D(2024)Diffusion Models for Cross-Domain Image-to-Image Translation with Paired and Partially Paired Datasets2024 IEEE 11th International Conference on Data Science and Advanced Analytics (DSAA)10.1109/DSAA61799.2024.10722775(1-8)Online publication date: 6-Oct-2024
https://doi.org/10.1109/DSAA61799.2024.10722775
Canet Tarrés GLin ZZhang ZZhang JSong YRuta DGilbert ACollomosse JKim S(2024)Thinking Outside the BBox: Unconstrained Generative Object CompositingComputer Vision – ECCV 202410.1007/978-3-031-73033-7_27(476-495)Online publication date: 31-Oct-2024
https://doi.org/10.1007/978-3-031-73033-7_27
Li YChen HDong SZhuang YLi L(2023)Multi-Temporal SamplePair Generation for Building Change Detection Promotion in Optical Remote Sensing Domain Based on Generative Adversarial NetworkRemote Sensing10.3390/rs1509247015:9(2470)Online publication date: 8-May-2023
https://doi.org/10.3390/rs15092470
Hold-Geoffroy YPiché-Meunier DSunkavalli KBazin JRameau FLalonde J(2023)A Perceptual Measure for Deep Single Image Camera and Lens CalibrationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.326964145:9(10603-10614)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3269641
Shen YChandaka BLin ZZhai ACui HForsyth DWang S(2023)Sim-on-Wheels: Physical World in the Loop Simulation for Self-DrivingIEEE Robotics and Automation Letters10.1109/LRA.2023.33256898:12(8192-8199)Online publication date: Dec-2023
https://doi.org/10.1109/LRA.2023.3325689
Zhang BSui JNiu L(2023)Foreground Object Search by Distilling Composite Image Feature2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.02101(22929-22938)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.02101
Kim HLee GChoi YKim JZhu J(2023)3D-aware Blending with Generative NeRFs2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.02094(22849-22861)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.02094
Jin LZhang JHold-Geoffroy YWang OBlackburn-Matzen KSticha MFouhey D(2023)Perspective Fields for Single Image Camera Calibration2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01660(17307-17316)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01660
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents