skip to main content
10.1145/1180495.1180532acmconferencesArticle/Chapter ViewAbstractPublication PagesvrstConference Proceedingsconference-collections
Article

Layered probability maps: basic framework and prototype system

Published: 01 November 2006 Publication History

Abstract

We propose an image-based rendering method called"layered prob-ability mapping." The algorithm requires modest computer power, but produces high-quality output images even from textureless or noisy input images. Thus, real-time applications such as immersive teleconferencing and live broadcasting are promising areas of use. The key idea is layered probability map (LPM) representation using a set of two-dimensional probabilistic functions. This accommodates ambiguities regarding object depth. To test the feasibility of this method, we developed a prototype system with nine synchronized cameras and a thirteen-PC cluster. By calculating the algorithm in parallel, stable 5-15fps image generation has been achieved without any off-line procedures between the image capture and display.

References

[1]
S. Baker, R. Szeliski, and P. Anandan. A layered approach to stereo reconstruction. In Proceedings of CVPR '98, pages 434--441, 1998.
[2]
C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen. Unstructured lumigraph rendering. In Proceedings of SIGGRAPH 2001, pages 425--432, 2001.
[3]
J.-X. Chai, X. Tong, S.-C. Chan, and H.-Y. Shum. Plenoptic sampling. In Proceedings of SIGGRAPH 2000, pages 307--318, 2000.
[4]
R. T. Collins. A space-sweep approach to true multi-image matching. In Proceedings of CVPR '96, pages 358--363, 1996.
[5]
P. E. Debevec, C. J. Taylor, and J. Malik. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Proceedings of SIGGRAPH '96, pages 1120, 1996.
[6]
P. E. Debevec, Y. Yu, and G. Borshukov. Efficient view-dependent image-based rendering with projective texture-mapping. In Proceedings of 9th Eurographics workshop on Rendering, pages 105--116, 1998.
[7]
B. Goldlücke, M. Magnor, and B. Wilburn. Hardware-accelerated dynamic light field rendering. Proceedings of Vision, Modeling, and Visualization (VMV-2002), pages 455--462, 2002.
[8]
S. J. Gortler, R. Grzeszczuk, R. Szeliski, and M. F. Cohen. The lumigraph. In Proceedings of SIGGRAPH '96, pages 43--54, 1996.
[9]
T. Kanade, P. J. Narayanan, and P. W. Rander. Virtualized reality: Concepts and early results. In Proceedings of IEEE Workshop on the Representation of Visual Scenes, pages 69--76, 1995.
[10]
Y. Kunita, M. Inami, T. Maeda, and S. Tachi. Real-time rendering system of moving objects. In Proceedings of the 1999 IEEE Workshop on Multi-View Modeling & Analysis of Visual Scenes (MVIEW'99), pages 81--88, 1999.
[11]
M. Levoy and P. Hanrahan. Light field rendering. In Proceedings of SIGGRAPH '96, pages 31--41, 1996.
[12]
M. Li, M. Magnor, and H.-P. Seidel. Hardware-accelerated visual hull reconstruction and rendering. In Proceedings of Graphics Interface 2003, pages 65--71, 2003.
[13]
W. Matusik, C. Buehler, R. Raskar, S. J. Gortler, and L. McMillan. Image-based visual hulls. In Proceedings of SIGGRAPH 2000, pages 369--374, 2000.
[14]
T. Naemura and H. Harashima. Ray-based approach to integrated 3d visual communication. In Proceedings of SPIE Three-Dimensional Video and Display: Devices and Systems, volume CR76, pages 282--305, 2000.
[15]
T. Naemura, J. Tago, and H. Harashima. Real-time video-based modeling and rendering of 3d scenes. IEEE Computer Graphics and Applications, 22(2):66--73, 2002.
[16]
M. Okutomi and T. Kanade. A multiple-baseline stereo. IEEE Transactions on Pattern Analysis and Machine Intelligence, 15(4):353--363, 1993.
[17]
H. Schirmacher, L. Ming, and H.-P. Seidel. On-the-fly processing of generalized lumigraphs. In Proceedings of Eurographics, pages C165--C173, 2001.
[18]
M. Segal, C. Korobkin, R. van Widenfelt, J. Foran, and P. Haeberli. Fast shadows and lighting effects using texture mapping. In Proceedings of SIGGRAPH '92, pages 249--252, 1992.
[19]
S. M. Seitz and C. R. Dyer. Photorealistic scene reconstruction by voxel coloring. In Proceedings of CVPR'97, pages 1067--1073, 1997.
[20]
J. Shade, S. Gortler, L. wei He, and R. Szeliski. Layered depth images. In Proceedings of SIGGRAPH '98, pages 231--242, 1998.
[21]
S. Suyama, S. Otsuka, H. Takada, K. Uehira, and S. Sakai. Apparent 3-d image perceived from luminance-modulated two 2-d images displayed at different depths. Visi on Research, 44:785--793, 2004.
[22]
H. Takada, S. Suyama, K. Hiruma, and K. Nakazawa. A compact depth-fused 3-d lcd. SID'03 Digest of Technical Papers, 58(2):1526--1529, 2003.
[23]
K. Takahashi, T. Naemura, and H. Harashima. Depth of field in light field rendering. In Proceedings of IEEE International Conf. Image Process (ICIP2003), pages 409--412. IEEE, 2003.
[24]
S. Vedula, S. Baker, S. Seitz, and T. Kanade. Shape and motion carving in 6d. In Proceedings of CVPR 2000, June 2000.
[25]
B. Wilburn, N. Joshi, V. Vaish, E.-V. Talvala, E. Antunez, A. Barth, A. Adams, M. Horowitz, and M. Levoy. High performance imaging using large camera arrays. In Proceedings of SIGGRAPH 2005, pages 765--776, 2005.
[26]
B. Wilburn, M. Smulski, K. Lee, and M. A. Horowitz. The light field video camera. In Proceedings of SPIE, volume 4674, pages 29--36, 2002.
[27]
J. C. Yang, M. Everett, C. Buehler, and L. McMillan. A real-time distributed light field camera. In Proceedings of Eurographics workshop on Rendering, pages 77--86, 2002.
[28]
R. Yang and M. Pollefeys. Multi-resolution real-time stereo on commodity graphics hardware. In Proceedings of CVPR 2003, pages 211--217, 2003.
[29]
R. Yang, G. Welch, and G. Bishop. Real-time consensus-based scene reconstruction using commodity graphics hardware. In Proceedings of the 10th Pacific Conference on Computer Graphics and Applications, page 225, 2002.
[30]
C. Zhang and T. Chen. A self-reconfigurable camera array. In Proceedings of Eurographics Symposium on Rendering, pages 243--254, 2004.
[31]
C. L. Zitnick, S. B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski. High-quality video view interpolation using a layered representation. In Proceedings of SIGGRAPH 2004, pages 600--608, 2004.

Cited By

View all
  • (2015)Real-time adaptive content retargeting for live multi-view capture and light field displayThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-015-1127-631:6-8(1023-1032)Online publication date: 1-Jun-2015
  • (2015)Flat3D: Browsing Stereo Images on a Conventional ScreenMultiMedia Modeling10.1007/978-3-319-14445-0_47(546-558)Online publication date: 2015
  • (2014)Probability-Based Rendering for View SynthesisIEEE Transactions on Image Processing10.1109/TIP.2013.229571623:2(870-884)Online publication date: 1-Feb-2014
  • Show More Cited By

Index Terms

  1. Layered probability maps: basic framework and prototype system

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      VRST '06: Proceedings of the ACM symposium on Virtual reality software and technology
      November 2006
      400 pages
      ISBN:1595933212
      DOI:10.1145/1180495
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 01 November 2006

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. image-based rendering
      2. real-time graphics
      3. three-dimensional video

      Qualifiers

      • Article

      Conference

      VRST06

      Acceptance Rates

      Overall Acceptance Rate 66 of 254 submissions, 26%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)1
      • Downloads (Last 6 weeks)0
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2015)Real-time adaptive content retargeting for live multi-view capture and light field displayThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-015-1127-631:6-8(1023-1032)Online publication date: 1-Jun-2015
      • (2015)Flat3D: Browsing Stereo Images on a Conventional ScreenMultiMedia Modeling10.1007/978-3-319-14445-0_47(546-558)Online publication date: 2015
      • (2014)Probability-Based Rendering for View SynthesisIEEE Transactions on Image Processing10.1109/TIP.2013.229571623:2(870-884)Online publication date: 1-Feb-2014
      • (2011)A real-time coarse-to-fine multiview capture system for all-in-focus rendering on a light-field display2011 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video (3DTV-CON)10.1109/3DTV.2011.5877176(1-4)Online publication date: May-2011
      • (2010)Image-based 3D telecopierProceedings of the 17th ACM Symposium on Virtual Reality Software and Technology10.1145/1889863.1889871(51-54)Online publication date: 22-Nov-2010
      • (2010)An image-based system for sharing a 3D object by transmitting to remote locationsProceedings of the 2010 IEEE Virtual Reality Conference10.1109/VR.2010.5444766(277-278)Online publication date: 20-Mar-2010
      • (2008)Real-time MVC viewer for free viewpoint navigation2008 IEEE International Conference on Multimedia and Expo10.1109/ICME.2008.4607715(1437-1440)Online publication date: Jun-2008
      • (2007)Multiview Imaging and 3DTVIEEE Signal Processing Magazine10.1109/MSP.2007.90587324:6(10-21)Online publication date: Nov-2007

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media