research-article

Free access

3D-aware image editing for out of bounds photography

Authors:

Antonio Criminisi,

Carsten Rother,

Gavin SmythAuthors Info & Claims

GI '09: Proceedings of Graphics Interface 2009

Pages 47 - 54

Published: 25 May 2009 Publication History

Abstract

In this paper, we propose algorithms to manipulate 2D images in a way that is consistent with the 3D geometry of the scene that they capture. We present these algorithms in the context of creating "Out of Bounds" (OOB) images - compelling, depth-rich images generated from single, conventional 2D photographs (fig. 1). Starting from a single image our tool enables rapid OOB prototyping; i.e. the ability to quickly create and experiment with many different variants of the OOB effect before deciding which one best expresses the users' artistic intentions. We achieve this with a flexible work-flow driven by an intuitive user interface.

The rich 3D perception of the final composition is achieved by exploiting two strong cues -- occlusions and shadows. A realistic-looking 3D frame is interactively inserted in the scene between segmented foreground objects and the background to generate novel occlusions and enhance the scene's perception of depth. This perception is further enhanced by adding new, realistic cast shadows. The key contributions of this paper are: (i) new algorithms for inserting simple 3D objects like frames in 2D images requiring minimal camera calibration, and (ii) new techniques for the realistic synthesis of cast shadows, even for complex 3D objects. These algorithms, although presented for OOB photography, may be directly used in general image composition tasks.

With our tool, untrained users can turn ordinary photos into compelling OOB images in seconds. In contrast with existing workflows, at any time the artist can modify any aspect of the composition while avoiding time-consuming pixel painting operations. Such a tool has important commercial applications, and is much more suitable for OOB prototyping than existing image editors.

References

[1]

J. Blinn. Me and my (fake) shadow. IEEE Comput. Graph. Appl., 8(1):82--86, 1988.

Digital Library

[2]

X. Cao, Y. Shen, M. Shah, and H. Foroosh. Single view compositing with shadows. The Visual Computer, 21(8--10):639--648, 2005.

[3]

P. Cavanaugh. The artist as neuroscientist. Nature, 434:301--307, 2005.

[4]

P. Cavanaugh and Y. Leclerc. Shape from shadows. J. Experimental Psychology, 15:3--27, 1989.

[5]

Y.-Y. Chuang, D. B. Goldman, K. C. Zheng, B. Curless, D. H. Salesin, and R. Szeliski. Animation pictures with stochastic motion textures. In SIGGRAPH, pages 853--860, 2005.

Digital Library

[6]

A. Criminisi. Accurate Visual Metrology from Single and Multiple Uncalibrated Images. Springer Verlag, 2001.

Digital Library

[7]

P. E. Debevec, C. J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: a hybrid geometry- and image-based approach. In SIGGRAPH, pages 11--20, 1996.

Digital Library

[8]

C. DeCoro, F. Cole, A. Finkelstein, and S. Rusinkiewicz. Stylized shadows. In Proc. NPAR, pages 77--83, 2007.

Digital Library

[9]

R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, ISBN: 0521540518, second edition, 2004.

[10]

D. Hoiem, A. A. Efros, and M. Hebert. Automatic photo pop-up. pages 577--584, 2005.

Digital Library

[11]

Y. Horry, K. ichi Anjyo, and K. Arai. Tour into the picture: using a spidery mesh interface to make animation from a single image. In SIGGRAPH, pages 225--232, 1997.

Digital Library

[12]

T. Igarashi, S. Matsuoka, and H. Tanaka. Teddy: a sketching interface for 3d freeform design. In SIGGRAPH, pages 409--416, 1999.

Digital Library

[13]

D. Kersten, D. Knill, P. Mamassian, and Bulthoff. Illusory motion from shadows. Nature, 379(6560):31, 1996.

[14]

J.-F. Lalonde, D. Hoiem, A. A. Efros, C. Rother, J. Winn, and A. Criminisi. Photo clip art. In SIGGRAPH, 2007.

Digital Library

[15]

D. Liebowitz, A. Criminisi, and A. Zisserman. Creating architectural models from images. In Proc. Eurographics, pages 39--50, 1999.

[16]

H. Lombaert, Y. Sun, L. Grady, and C. Xu. A multilevel banded graph cuts method for fast image segmentation. In Proc. ICCV, pages 259--265, 2005.

Digital Library

[17]

D. Marr. Vision: a computational investigation into the human representation and processing of visual information. W. H. Freeman, San Francisco, 1982.

Digital Library

[18]

A. Mohan, J. Tumblin, and P. Choudhury. Editing soft shadows in a digital photograph. IEEE Comput. Graph. Appl., 27(2):23--31, March 2007.

Digital Library

[19]

B. M. Oh, M. Chen, J. Dorsey, and F. Durand. Image-based modeling and photo editing. In SIGGRAPH, pages 433--442, 2001.

Digital Library

[20]

M. Prasad and A. Fitzgibbon. Single view reconstruction of curved surfaces. In Proc. CVPR, pages 1345--1354, 2006.

Digital Library

[21]

C. Rother, V. Kolmogorov, and A. Blake. Grabcut: interactive foreground extraction using iterated graph cuts. In SIGGRAPH, pages 309--314, 2004.

Digital Library

[22]

T.-P. Wu, C.-K. Tang, M. S. Brown, and H.-Y. Shum. Natural shadow matting. ACM Trans. Graph., 26(2):8, 2007.

Digital Library

Cited By

Iizuka SEndo YHirose MKanamori YMitani JFukui Y(2014)Object Repositioning Based on the Perspective in a Single ImageComputer Graphics Forum10.1111/cgf.1242333:8(157-166)Online publication date: 1-Dec-2014
https://dl.acm.org/doi/10.1111/cgf.12423
Kee EO'Brien JFarid H(2013)Exposing photo manipulation with inconsistent shadowsACM Transactions on Graphics10.1145/2487228.248723632:3(1-12)Online publication date: 4-Jul-2013
https://dl.acm.org/doi/10.1145/2487228.2487236
Fourquet E(2010)Learning about shadows from artistsProceedings of the Sixth international conference on Computational Aesthetics in Graphics, Visualization and Imaging10.5555/2381312.2381332(107-114)Online publication date: 14-Jun-2010
https://dl.acm.org/doi/10.5555/2381312.2381332

Index Terms

3D-aware image editing for out of bounds photography
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
  2. Computer graphics
    1. Image manipulation
    2. Rendering

Recommendations

Image-based 3D photography using opacity hulls

We have built a system for acquiring and displaying high quality graphical models of objects that are impossible to scan with traditional scanners. Our system can acquire highly specular and fuzzy materials, such as fur and feathers. The hardware set-up ...
Plenoptic Image Editing

This paper presents a new class of interactive image editing operations designed to maintain consistency between multiple images of a physical 3D scene. The distinguishing feature of these operations is that edits to any one image propagate ...
Image-based 3D photography using opacity hulls
SIGGRAPH '02: Proceedings of the 29th annual conference on Computer graphics and interactive techniques

We have built a system for acquiring and displaying high quality graphical models of objects that are impossible to scan with traditional scanners. Our system can acquire highly specular and fuzzy materials, such as fur and feathers. The hardware set-up ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

GI '09: Proceedings of Graphics Interface 2009

May 2009

257 pages

ISBN:9781568814704

Program Chairs:
Amy Gooch
University of Victoria, Canada
,
Melanie Tory
University of Victoria, Canada

Sponsors

The Canadian Human-Computer Communications Society / Société Canadienne du Dialogue Humaine Machine (CHCCS/SCDHM)

Publisher

Canadian Information Processing Society

Canada

Publication History

Published: 25 May 2009

Qualifiers

Research-article

Acceptance Rates

GI '09 Paper Acceptance Rate 28 of 77 submissions, 36%;

Overall Acceptance Rate 206 of 508 submissions, 41%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

3
Total Citations
View Citations
293
Total Downloads

Downloads (Last 12 months)25
Downloads (Last 6 weeks)10

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Iizuka SEndo YHirose MKanamori YMitani JFukui Y(2014)Object Repositioning Based on the Perspective in a Single ImageComputer Graphics Forum10.1111/cgf.1242333:8(157-166)Online publication date: 1-Dec-2014
https://dl.acm.org/doi/10.1111/cgf.12423
Kee EO'Brien JFarid H(2013)Exposing photo manipulation with inconsistent shadowsACM Transactions on Graphics10.1145/2487228.248723632:3(1-12)Online publication date: 4-Jul-2013
https://dl.acm.org/doi/10.1145/2487228.2487236
Fourquet E(2010)Learning about shadows from artistsProceedings of the Sixth international conference on Computational Aesthetics in Graphics, Visualization and Imaging10.5555/2381312.2381332(107-114)Online publication date: 14-Jun-2010
https://dl.acm.org/doi/10.5555/2381312.2381332

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten