skip to main content
10.5555/1555880.1555909guideproceedingsArticle/Chapter ViewAbstractPublication PagesgiConference Proceedingsconference-collections
research-article
Free access

Parallax photography: creating 3D cinematic effects from stills

Published: 25 May 2009 Publication History

Abstract

We present an approach to convert a small portion of a light field with extracted depth information into a cinematic effect with simulated, smooth camera motion that exhibits a sense of 3D parallax. We develop a taxonomy of the cinematic conventions of these effects, distilled from observations of documentary film footage and organized by the number of subjects of interest in the scene. We present an automatic, content-aware approach to apply these cinematic conventions to an input light field. A face detector identifies subjects of interest. We then optimize for a camera path that conforms to a cinematic convention, maximizes apparent parallax, and avoids missing information in the input. We describe a GPU-accelerated, temporally coherent rendering algorithm that allows users to create more complex camera moves interactively, while experimenting with effects such as focal length, depth of field, and selective, depth-based desaturation or brightening. We evaluate and demonstrate our approach on a wide variety of scenes and present a user study that compares our 3D cinematic effects to their 2D counterparts.

References

[1]
C. Buehler, M. Bosse, L. McMillan, S. J. Gortler, and M. F. Cohen. Unstructured lumigraph rendering. In Proc. of SIGGRAPH 2001, pages 425--432, August 2001.
[2]
A. Criminisi, P. Perez, and K. Toyama. Object removal by exemplar-based inpainting. In Proc. IEEE Computer Vision and Pattern Recognition (CVPR), Jun. 2003.
[3]
J. Demers. Depth of field a survey of techniques. In R. Fernando, editor, GPU Gems 1, chapter 23, pages 375--390. Mar. 2004.
[4]
P. F. Felzenszwalb and D. P. Huttenlocher. Distance transforms of sampled functions. Technical Report TR2004--1963, Cornell University, 2004.
[5]
T. Georgiev, C. Zheng, S. Nayar, D. Salesin, B. Curless, and C. Intwala. Spatio-angular resolution trade-offs in integral photography. Proc. of Eurographics Symposium on Rendering, 2006.
[6]
S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. B. Cohen. The lumigraph. ACM Trans. Graph., pages 43--54, 1996.
[7]
E. Hammon. Practical post-process depth of field. In H. Nguyen, editor, GPU Gems 3, chapter 28. Addison Wesley, 2007.
[8]
R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, second edition, 2004.
[9]
L.-W. He, M. F. Cohen, and D. H. Salesin. The virtual cinematographer: A paradigm for automatic real-time camera control and directing. In Proc. of SIGGRAPH 96, Computer Graphics Proceedings, Annual Conference Series, pages 217--224, Aug. 1996.
[10]
B. Heigl, R. Koch, M. Pollefeys, J. Denzler, and L. J. V. Gool. Plenoptic modeling and rendering from image sequences taken by hand-held camera. In DAGM-Symposium, pages 94--101, 1999.
[11]
D. Hoiem, A. A. Efros, and M. Hebert. Automatic photo pop-up. ACM Trans. Graph., 24(3):577--584, August 2005.
[12]
Y. Horry, K.-I. Anjyo, and K. Aral. Tour into the picture: Using a spidery mesh interface to make animation from a single image. In Proc. of SIGGRAPH 97, pages 225--232, August 1997.
[13]
A. Isaksen, L. McMillan, and S. J. Gortler. Dynamically reparameterized light fields. ACM Trans. Graph., pages 297--306, 2000.
[14]
S. B. Kang and H.-Y. Shum. A review of image-based rendering techniques. In IEEE/SPIE Visual Communications and Image Processing 2000, pages 2--13, 2002.
[15]
M. Kass, A. Lefohn, and J. D. Owens. Interactive depth of field using simulated diffusion. Technical Report 06-01, Pixar Animation Studios, Jan. 2006.
[16]
A. Klaus, M. Sormann, and K. F. Karner. Segment-based stereo matching using belief propagation and a self-adapting dissimilarity measure. In International Conference on Pattern Recognition (ICPR), pages 15--18, 2006.
[17]
A. Levin, R. Fergus, F. Durand, and W. Freeman. Image and depth from a conventional camera with a coded aperture. ACM Trans. Graph., 26(3):70:1--70:9, July 2007.
[18]
M. Levoy. Light fields and computational imaging. IEEE Computer, 39(8):46--55, 2006.
[19]
M. Levoy and P. Hanrahan. Light field rendering. ACM Trans. Graph., pages 31--42, 1996.
[20]
C.-K. Liang, T.-H. Lin, B.-Y. Wong, C. Liu, and H. Chen. Programmable aperture photography: Multiplexed light field acquisition. ACM Trans. Graph., 27(3), 2008.
[21]
F. Moreno-Noguer, P. N. Belhumeur, and S. K. Nayar. Active refocusing of images and videos. ACM Trans. Graph., 26:671--679, 2007.
[22]
R. Ng. Fourier slice photography. ACM Trans. Graph., 24(3):735--744, Aug. 2005.
[23]
R. Ng, M. Levoy, M. Bredif, G. Duval, M. Horowitz, and P. Hanrahan. Light field photography with a hand-held plenoptic camera. Technical Report CSTR 2005--02, Stanford University, 2005.
[24]
B. M. Oh, M. Chen, J. Dorsey, and F. Durand. Image-based modeling and photo editing. In Proc. of SIGGRAPH 2001, pages 433--442, August 2001.
[25]
M. Pollefeys, L. V. Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch. Visual modeling with a hand-held camera. International Journal of Computer Vision, 59(3):207--232, Sept. 2004.
[26]
K. Pulli, M. F. Cohen, T. Duchamp, H. Hoppe, L. Shapiro, and W. Stuetzle. View-based rendering: Visualizing real objects from scanned range and color data. In Eurographics Rendering Workshop 1997, pages 23--34, June 1997.
[27]
D. Scharstein and R. Szeliski. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. International Journal of Computer Vision, 47(1--3):7--42, 2002.
[28]
S. M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski. A comparison and evaluation of multi-view stereo reconstruction algorithms. In Proc. IEEE Computer Vision and Pattern Recognition (CVPR), pages 519--528, June 2006.
[29]
J. Shade, S. J. Gortler, L.-W. He, and R. Szeliski. Layered depth images. In Proc. of SIGGRAPH 98, pages 231--242, July 1998.
[30]
J. R. Shewchuk. Triangle: Engineering a 2D Quality Mesh Generator and Delaunay Triangulator. In Applied Computational Geometry: Towards Geometric Engineering, volume 1148 of Lecture Notes in Computer Science, pages 203--222. Springer-Verlag, May 1996.
[31]
N. Snavely, S. M. Seitz, and R. Szeliski. Photo tourism: Exploring photo collections in 3d. ACM Trans. Graph., 25(3):835--846, 2006.
[32]
D. Taylor. Virtual camera movement: The way of the future? American Cinematographer, 77(9):93--100, Sept. 1996.
[33]
P. Viola and M. Jones. Robust real-time object detection. International Journal of Computer Vision, 57:137--154, 2004.
[34]
K. C. Zheng, A. Colburn, A. Agarwala, M. Agrawala, B. Curless, D. Salesin, and M. Cohen. A consistent segmentation approach to image-based rendering. Technical Report CSE-09-03-02, University of Washington, 2009.
[35]
C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. High-quality video view interpolation using a layered representation. ACM Trans. Graph., 23(3):600--608, Aug. 2004.

Cited By

View all

Index Terms

  1. Parallax photography: creating 3D cinematic effects from stills

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image Guide Proceedings
    GI '09: Proceedings of Graphics Interface 2009
    May 2009
    257 pages
    ISBN:9781568814704

    Sponsors

    • The Canadian Human-Computer Communications Society / Société Canadienne du Dialogue Humaine Machine (CHCCS/SCDHM)

    Publisher

    Canadian Information Processing Society

    Canada

    Publication History

    Published: 25 May 2009

    Author Tags

    1. image-based rendering
    2. photo and image editing

    Qualifiers

    • Research-article

    Acceptance Rates

    GI '09 Paper Acceptance Rate 28 of 77 submissions, 36%;
    Overall Acceptance Rate 206 of 508 submissions, 41%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)26
    • Downloads (Last 6 weeks)9
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2019)Deferred neural renderingACM Transactions on Graphics10.1145/3306346.332303538:4(1-12)Online publication date: 12-Jul-2019
    • (2018)Deep blending for free-viewpoint image-based renderingACM Transactions on Graphics10.1145/3272127.327508437:6(1-15)Online publication date: 4-Dec-2018
    • (2017)Kinetic depth imagesThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-016-1231-233:10(1357-1369)Online publication date: 1-Oct-2017
    • (2015)An image degradation model for depth-augmented image editingProceedings of the Eurographics Symposium on Geometry Processing10.1111/cgf.12707(191-199)Online publication date: 6-Jul-2015
    • (2012)ClipletsProceedings of the 25th annual ACM symposium on User interface software and technology10.1145/2380116.2380149(251-260)Online publication date: 7-Oct-2012
    • (2012)Looking at youProceedings of the SIGCHI Conference on Human Factors in Computing Systems10.1145/2207676.2208375(2211-2220)Online publication date: 5-May-2012
    • (2010)Space-time visual effects as a post-production processProceedings of the 1st international workshop on 3D video processing10.1145/1877791.1877793(1-6)Online publication date: 29-Oct-2010

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media