skip to main content
research-article

A virtual restoration stage for real-world objects

Published: 01 December 2008 Publication History

Abstract

In this paper, we introduce a system to virtually restore damaged or historically significant objects without needing to physically change the object in any way. Our work addresses both creating a restored synthetic version of the object as viewed from a camera and projecting the necessary light, using digital projectors, to give the illusion of the object being restored. The restoration algorithm uses an energy minimization method to enforce a set of criteria over the surface of the object and provides an interactive tool to the user which can compute a restoration in a few minutes. The visual compensation method develops a formulation that is particularly concerned with obtaining bright compensations under a specified maximum amount of light. The bound on the amount of light is of crucial importance when viewing and restoring old and potentially fragile objects. Finally, we demonstrate our system by restoring several deteriorated and old objects enabling the observer to view the original or restored object at will.

Supplementary Material

JPG File (a149-aliaga-mp4_hi.jpg)
MOV File (a149-aliaga-mp4_hi.mov)

References

[1]
Aliaga, D., and Xu, Y. 2008. Photogeometric Structured Light: A Self-Calibrating and Multi-Viewpoint Framework for Accurate 3D Modeling. In Proc. of IEEE Computer Vision and Pattern Recognition, 1--8.
[2]
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre, B. 2001. Recent Advances in Augmented Reality, IEEE Comp. Graphics & Applications, 21, 6, 34--47.
[3]
Barrett, W. A., and Cheney, A. S. 2002. Object-based Image Editing. ACM Trans. on Graphics, 21, 3, 777--784.
[4]
Bertalmio, M., Sapiro, G., Caselles, V., and Ballester, V. C. 2000. Image Inpainting. In Proc. of ACM SIGGRAPH 2000, 417--424.
[5]
Bertalmio, M., Vese, L., Sapiro, G., and Osher, S. 2003. Simultaneous Structure and Texture Image Inpainting. IEEE Trans. on Image Processing, 12, 8, 882--889.
[6]
Bimber, O., Frohlich, B., Schmalsteig, D., and Encarnação, L. M., 2001. The Virtual Showcase, IEEE Computer Graphics & Applications, 21, 6, 48--55.
[7]
Bornard, R., Lecan, E., Laborelli, L., and Chenot, J. H. 2002. Missing Data Correction in Still Images and Image Sequences. International Multimedia Conference, 355--361.
[8]
Chan, T. F., and Shen, J., 2001, Nontexture Inpainting by Curvature-Driven Diffusions. Journal of Visual Communication and Image Representation, 12, 4, 436--449.
[9]
Comaniciu, D., and Meer, P. 2002. Mean Shift: A Robust Approach Toward Feature Space Analysis. IEEE Trans. Pattern Analysis Machine Intelligence, 24, 5, 603--619.
[10]
Criminisi, A., Perez, P., and Toyama, K. 2003. Object Removal by Exemplar-based Inpainting. In Proc. of IEEE Computer Vision and Pattern Recognition, 2, 721--728.
[11]
Drori, I., Cohen-Or, D., and Yeshurun, H. 2003. Fragment-based Image Completion. ACM Trans. on Graphics, 22, 3, 303--312.
[12]
Durand, F., and Dorsey, J. 2002. Fast Bilateral Filtering for the Display of High-Dynamic-Range Images. In Proc. of ACM SIGGRAPH 2002, 257--266.
[13]
Efros, A. A., and Leung T. K. 1999. Texture Synthesis by Non-Parametric Sampling. In Proc. of IEEE International Conference on Computer Vision, 2, 1033--1038.
[14]
Felzenszwalb, P. F., and Huttenlocher, D. P. 2004. Efficient Graph-Based Image Segmentation. International Journal of Computer Vision, 59, 2, 167--181.
[15]
Fujii, K., Grossberg, M. D., and Nayar, S. K. 2005. A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments. In Proc. of IEEE Computer Vision and Pattern Recognition, 1, 814--821.
[16]
Grossberg, M. D., Peri, H., Nayar, S. K., and Belhumeur, P. N. 2004. Making One Object Look Like Another: Controlling Appearance using a Projector-Camera System. In Proc. of IEEE Computer Vision and Pattern Recognition, 1, 452--459.
[17]
Igehy, H., and Pereira, L. 1997. Image Replacement through Texture Synthesis. In Proc. of International Conference on Image Processing, 3, 186--189.
[18]
Jia, J., and Tang, C. K. 2003. Image Repairing: Robust Image Synthesis by Adaptive ND Tensor Voting. In Proc. of IEEE Computer Vision and Pattern Recognition, 1, 643--650.
[19]
Levin, A., Zomet, A., and Weiss, Y. 2003. Learning How to Inpaint from Global Image Statistics. In Proc. of IEEE International Conference on Computer Vision, 1, 305--312.
[20]
Mitsunaga, T., Nayar, S. K. 1999. Radiometric Self Calibration. IEEE Comp. Vision & Pattern Recognition, 1, 374--380.
[21]
Nayar, S. K., Peri, H., Grossberg, M. D., and Belhumeur, P. N. 2003. A Projection System with Radiometric Compensation for Screen Imperfections. ICCV Workshop on Projector-Camera Systems.
[22]
Perona, P., and Malik, J. 1990. Scale-Space and Edge Detection Using Anisotropic Diffusion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 12, 7, 629--639.
[23]
Raskar, R., Welch, G., Low, K. L., and Bandyopadhyay, D. 2001. Shader Lamps: Animating Real Objects With Image-Based Illumination. In Proc. of Eurographics Workshop on Rendering Techniques, 89--102.
[24]
Sen, P., Chen, B., Garg, G., Marschner, S. R., Horowitz, M., Levoy, M., and Lensch, H. P. A. 2005. Dual Photography. In Proc. of ACM SIGGRAPH 2005, 745--755.
[25]
Sun, J., Yuan, L., Jia, J., and Shum, H. Y. 2005. Image Completion with Structure Propagation. ACM Trans. on Graphics, 24, 3, 861--868.
[26]
Tomasi, C., and Manduchi, R. 1998. Bilateral Filtering for Gray and Color Images. In Proc. of IEEE International Conf. on Computer Vision, 836--846.
[27]
Wei, L. Y., and Levoy, M. 2000. Fast Texture Synthesis Using Tree-structured Vector Quantization. In Proc. of ACM SIGGRAPH 2000, 479--488.
[28]
Weiss, B. 2006. Fast Median and Bilateral Filtering. ACM Trans. of Graphics, 25, 3, 519--526.
[29]
Wetzstein, G., and Bimber, O. 2007. Radiometric Compensation through Inverse Light Transport. In Proc. of Pacific Graphics, 391--399.

Cited By

View all
  • (2023)Virtual Reconstruction with the Augmented Reality Technology of the Cultural Heritage Components that have Disappeared: The Ayazini Virgin Mary ChurchJournal on Computing and Cultural Heritage 10.1145/357936116:1(1-16)Online publication date: 1-Jun-2023
  • (2023)Shadowless Projection Mapping using Retrotransmissive OpticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324710429:5(2280-2290)Online publication date: 22-Feb-2023
  • (2020)Virtual Restoration and Visualization Changes through Light: A ReviewHeritage10.3390/heritage30400763:4(1373-1384)Online publication date: 17-Nov-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 27, Issue 5
December 2008
552 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/1409060
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 December 2008
Published in TOG Volume 27, Issue 5

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. digitization
  2. energy minimization
  3. image completion
  4. light transport
  5. radiometric calibration
  6. restoration

Qualifiers

  • Research-article

Funding Sources

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)3
  • Downloads (Last 6 weeks)1
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Virtual Reconstruction with the Augmented Reality Technology of the Cultural Heritage Components that have Disappeared: The Ayazini Virgin Mary ChurchJournal on Computing and Cultural Heritage 10.1145/357936116:1(1-16)Online publication date: 1-Jun-2023
  • (2023)Shadowless Projection Mapping using Retrotransmissive OpticsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.324710429:5(2280-2290)Online publication date: 22-Feb-2023
  • (2020)Virtual Restoration and Visualization Changes through Light: A ReviewHeritage10.3390/heritage30400763:4(1373-1384)Online publication date: 17-Nov-2020
  • (2020)Automatic Deep Inference of Procedural Cities from Global-scale Spatial DataACM Transactions on Spatial Algorithms and Systems10.1145/34234227:2(1-28)Online publication date: 27-Oct-2020
  • (2020)FibAR: Embedding Optical Fibers in 3D Printed Objects for Active Markers in Dynamic Projection MappingIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.297344426:5(2030-2040)Online publication date: May-2020
  • (2019)Fundamental Technologies of Projection Mappingプロジェクションマッピングの基盤技術The Journal of The Institute of Electrical Engineers of Japan10.1541/ieejjournal.139.34139:1(34-37)Online publication date: 1-Jan-2019
  • (2019)Augmented Environment Mapping for Appearance Editing of Glossy Surfaces2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR.2019.00-26(55-65)Online publication date: Oct-2019
  • (2018)5. Improvised Projection Mapping without Modeling5. モデリングを必要としない即興的なプロジェクションマッピングThe Journal of The Institute of Image Information and Television Engineers10.3169/itej.72.32672:5(326-331)Online publication date: 2018
  • (2018)Projector primary-based optimization for superimposed projection mappingsJournal of Electronic Imaging10.1117/1.JEI.27.1.01101127:01(1)Online publication date: 21-Feb-2018
  • (2018)Recent Advances in Projection Mapping Algorithms, Hardware and ApplicationsComputer Graphics Forum10.1111/cgf.1338737:2(653-675)Online publication date: 22-May-2018
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media