article

Factored time-lapse video

Authors:

Kalyan Sunkavalli,

Wojciech Matusik,

Hanspeter Pfister,

Szymon RusinkiewiczAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 26, Issue 3

Pages 101 - es

https://doi.org/10.1145/1276377.1276504

Published: 29 July 2007 Publication History

Abstract

We describe a method for converting time-lapse photography captured with outdoor cameras into Factored Time-Lapse Video (FTLV): a video in which time appears to move faster (i.e., lapsing) and where data at each pixel has been factored into shadow, illumination, and reflectance components. The factorization allows a user to easily relight the scene, recover a portion of the scene geometry (normals), and to perform advanced image editing operations. Our method is easy to implement, robust, and provides a compact representation with good reconstruction characteristics. We show results using several publicly available time-lapse sequences.

Supplementary Material

JPG File (pps100.jpg)

Download
11.86 KB

MP4 File (pps100.mp4)

Download
49.20 MB

References

[1]

Agarwala, A., Zheng, K. C., Pal, C., Agrawala, M., Cohen, M., Curless, B., Salesin, D., and Szeliski, R. 2005. Panoramic video textures. ACM Trans. on Graph. 24, 3, 821--827.

Digital Library

[2]

Barrow, H., and Tenenbaum, J. 1978. Recovering intrinsic scene characteristics from images. Academic Press, 3--26.

[3]

Bhat, K., Seitz, S., Hodgins, J. K., and Khosla, P. 2004. Flow-based video synthesis and editing. ACM Trans. on Graph. 23, 3, 360--363.

Digital Library

[4]

Bregler, C., Covell, M., and Slaney, M. 1997. Video rewrite: driving visual speech with audio. In Proc. of ACM SIGGRAPH, ACM Press, New York, NY, USA, 353--360.

Digital Library

[5]

Chuang, Y.-Y, Agarwala, A., Curless, B., Salesin, D. H., and Szeliski, R. 2002. Video matting of complex scenes. ACM Trans. on Graph, 21, 3 (July), 243--248.

Digital Library

[6]

Chuang, Y.-Y., Goldman, D. B., Curless, B., Salesin, D. H., and Szeliski, R. 2003. Shadow matting and compositing. ACM Trans. on Graph. 22, 3, 494--500.

Digital Library

[7]

Debevec, P., Tchou, C., Gardner, A., Hawkins, T., Poullis, C., Stumpfel, J., Jones, A., Yun, N., Einarsson, P., Lundgren, T., Fajardo, M., and Martinez, P., 2004. Estimating Surface Reflectance Properties of a Complex Scene under Captured Natural Illumination. USC ICT Technical Report ICT-TR-06.2004.

[8]

Gu, J., Tu, C.-I., Ramamoorthi, R., Belhumeur, P., Matusik, W., and Nayar, S. 2006. Time-varying surface appearance: acquisition, modeling and rendering. ACM Trans. on Graph. 25, 3, 762--771.

Digital Library

[9]

Hertzmann, A., and Seitz, S. M. 2005. Example-based photometric stereo: Shape reconstruction with general, varying brdfs. IEEE Trans. on PAMI 27, 8, 1254--1264.

Digital Library

[10]

Koppal, S. J., and Narasimhan, S. G. 2006. Clustering appearance for scene analysis. In Proc. of CVPR, vol. 2, 1323--1330.

Digital Library

[11]

Lawrence, J., Ben-Artzi, A., DeCoro, C., Matusik, W., Pfister, H., Ramamoorthi, R., and Rusinkiewicz, S. 2006. Inverse shade trees for non-parametric material representation and editing. ACM Trans. on Graph. 25, 3, 735--745.

Digital Library

[12]

Li, Y., Sun J., and Shum, H.-Y. 2005. Video object cut and paste. ACM Trans. on Graph. 24, 3, 595--600.

Digital Library

[13]

Litwinowicz, P. 1997. Processing images and video for an impressionist effect. In Proc. of ACM SIGGRAPH, ACM Press, New York, NY, USA, 407--414.

Digital Library

[14]

Liu, C., Torralba, A., Freeman, W. T., Durand, F., and Adelson, E. H. 2005. Motion magnification. ACM Trans. on Graph. 24, 3, 519--526.

Digital Library

[15]

Matsushita, Y., Nishino, K., Ikeuchi, K., and Sakauchi, M. 2004. Illumination normalization with time-dependent intrinsic images for video surveillance. IEEE Trans. on PAMI 26, 10, 1336--1347.

Digital Library

[16]

Matusik, W., Loper, M., and Pfister, H. 2004. Progressively-refined reflectance functions from natural illumination. In Rendering Techniques, Eurographics Association, A. Keller and H. W. Jensen, Eds., 299--308.

[17]

McGuire, M., Matusik, W., Pfister, H., Hughes, J. F., and Durand, F. 2005. Defocus video matting. ACM Trans. on Graph. 24, 3, 567--576.

Digital Library

[18]

Nayar, S. K., Krishnan, G., Grossberg, M. D., and Raskar, R. 2006. Fast separation of direct and global components of a scene using high frequency illumination. ACM Trans. on Graph. 25, 3, 935--944.

Digital Library

[19]

Nimeroff, J. S., Simoncelli, E., and Dorsey, J. 1994. Efficient Re-rendering of Naturally Illuminated Environments.

[20]

In Fifth Eurographics Workshop on Rendering, Springer-Verlag, Darmstadt, Germany, 359--373.

[21]

Rusinkiewicz, S., Burns, M., and DeCarlo, D. 2006. Exaggerated shading for depicting shape and detail. ACM Trans, on Graph. 25, 3, 1199--1205.

Digital Library

[22]

Schödl, A., Szeliski, R., Salesin, D., and Essa, I. 2000. Video textures. In Proc. of ACM SIGGRAPH, ACM Press, New York, NY, USA, 489--498.

Digital Library

[23]

Seitz, S. M., Matsushita, Y., and Kutulakos, K. N. 2005. A theory of inverse light transport. In Proc. of ICCV, II: 1440--1447.

Digital Library

[24]

Tomasi, C., and Manduchi, R. 1998. Bilateral filtering for gray and color images. In Proc. of ICCV, 839--846.

Digital Library

[25]

Wang, J., Xu, Y., Shum, H.-Y., and Cohen, M. F. 2004. Video tooning. ACM Trans. on Graph. 23, 3, 574--583.

Digital Library

[26]

Wang, J., Bhat, P., Colburn, R. A., Agrawala, M., and Cohen, M. F. 2005. Interactive video cutout. ACM Trans. on Graph. 24, 3, 585--594.

Digital Library

[27]

Weiss, Y. 2001. Deriving intrinsic images from image sequences. In Proc. of ICCV, II: 68--75.

[28]

Winnemöller, H., Olsen, S., and Gooch, B. 2006. Realtime video abstraction. ACM Trans. on Graph. 25, 3, 1221--1226.

Digital Library

[29]

Yu, Y., and Malik, J. 1998. Recovering photometric properties of architectural scenes from photographs. In Proc. of ACM SIGGRAPH, ACM Press, New York, NY, USA, 207--217.

Digital Library

Cited By

Wang YHolynski AZhang XZhang X(2023)SunStage: Portrait Reconstruction and Relighting Using the Sun as a Light Stage2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01992(20792-20802)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01992
Yan RSun JDeng LDavis A(2022)ReCapture: AR-Guided Time-lapse PhotographyProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545641(1-14)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545641
Griffiths DRitschel TPhilip J(2022)OutCast: Outdoor Single‐image Relighting with Cast ShadowsComputer Graphics Forum10.1111/cgf.1446741:2(179-193)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14467
Show More Cited By

Index Terms

Factored time-lapse video
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Video segmentation
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Image manipulation
      1. Texturing

Recommendations

Factored time-lapse video
SIGGRAPH '07: ACM SIGGRAPH 2007 papers

We describe a method for converting time-lapse photography captured with outdoor cameras into Factored Time-Lapse Video (FTLV): a video in which time appears to move faster (i.e., lapsing) and where data at each pixel has been factored into shadow, ...
Time-Lapse Photometric Stereo and Applications

This paper presents a technique to recover geometry from time-lapse sequences of outdoor scenes. We build upon photometric stereo techniques to recover approximate shadowing, shading and normal components allowing us to alter the material and normals of ...
Axial-cones: modeling spherical catadioptric cameras for wide-angle light field rendering

Catadioptric imaging systems are commonly used for wide-angle imaging, but lead to multi-perspective images which do not allow algorithms designed for perspective cameras to be used. Efficient use of such systems requires accurate geometric ray modeling ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 26, Issue 3

July 2007

976 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1276377

Issue’s Table of Contents

Copyright © 2007 ACM.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 July 2007

Published in TOG Volume 26, Issue 3

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

51
Total Citations
View Citations
1,274
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)1

Reflects downloads up to 01 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang YHolynski AZhang XZhang X(2023)SunStage: Portrait Reconstruction and Relighting Using the Sun as a Light Stage2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.01992(20792-20802)Online publication date: Jun-2023
https://doi.org/10.1109/CVPR52729.2023.01992
Yan RSun JDeng LDavis A(2022)ReCapture: AR-Guided Time-lapse PhotographyProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology10.1145/3526113.3545641(1-14)Online publication date: 29-Oct-2022
https://dl.acm.org/doi/10.1145/3526113.3545641
Griffiths DRitschel TPhilip J(2022)OutCast: Outdoor Single‐image Relighting with Cast ShadowsComputer Graphics Forum10.1111/cgf.1446741:2(179-193)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14467
Zhang QZhou JZhu LSun WXiao CZheng W(2022)Unsupervised Intrinsic Image Decomposition Using Internal Self-Similarity CuesIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2021.312979544:12(9669-9686)Online publication date: 1-Dec-2022
https://doi.org/10.1109/TPAMI.2021.3129795
Rudnev VElgharib MSmith WLiu LGolyanik VTheobalt C(2022)NeRF for Outdoor Scene RelightingComputer Vision – ECCV 202210.1007/978-3-031-19787-1_35(615-631)Online publication date: 21-Oct-2022
https://doi.org/10.1007/978-3-031-19787-1_35
Wehrwein SBala KSnavely N(2021)Scene Summarization via Motion NormalizationIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.299319527:4(2495-2501)Online publication date: 1-Apr-2021
https://doi.org/10.1109/TVCG.2020.2993195
Kajo IKamel NRuichek YAl-Ahdal A(2021)Frequency-Aware SVD Decomposition and its Application to Color Magnification and Motion DenoisingIEEE Access10.1109/ACCESS.2021.31018239(108832-108845)Online publication date: 2021
https://doi.org/10.1109/ACCESS.2021.3101823
Philip JGharbi MZhou TEfros ADrettakis G(2019)Multi-view relighting using a geometry-aware networkACM Transactions on Graphics10.1145/3306346.332301338:4(1-14)Online publication date: 12-Jul-2019
https://dl.acm.org/doi/10.1145/3306346.3323013
Lee TWittenburg K(2019)Space-Time Slicing: Visualizing Object Detector Performance in Driving Video Sequences2019 IEEE Pacific Visualization Symposium (PacificVis)10.1109/PacificVis.2019.00045(318-322)Online publication date: Apr-2019
https://doi.org/10.1109/PacificVis.2019.00045
Nam SMa CChai MBrendel WXu NJoo Kim S(2019)End-To-End Time-Lapse Video Synthesis From a Single Outdoor Image2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2019.00150(1409-1418)Online publication date: Jun-2019
https://doi.org/10.1109/CVPR.2019.00150
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents