Reliability-based 3D reconstruction in real environment

ABSTRACT

We present a practical 3D reconstruction method that guarantees robust visual hull construction in real environments where segmentation errors and occlusion exist. The proposed method consists of foreground extraction and reliability-based shape-from-silhouette, and they are connected by the intra-/inter-silhouette reliabilities. In foreground extraction, all regions are classified into four categories based on their intra-reliabilities. Then the reliability-based shape-from-silhouette technique reconstructs a visual hull by carving a 3D space based on the intra-/inter-silhouette reliabilities. The proposed method provides a reliable visual hull in real environments without much increment of the system complexity compared with conventional systems.

REFERENCES

• 1: Cheung, G., Kanade, T., Bouguet, J. Y., and Holler, M. A real time system for robust 3d voxel reconstruction of human motions. Proc. CVPR (2000), 714--720.
• 2: Ahmed M. Elgammal , David Harwood , Larry S. Davis, Non-parametric Model for Background Subtraction, Proceedings of the 6th European Conference on Computer Vision-Part II, p.751-767, June 26-July 01, 2000
• 3: Grauman, K., Shakhnarovich, G., and Darrell, T. A Bayesian Approach to Image-Based Visual Hull Reconstruction. Proc. CVPR (2003), 187--194.
• 4: Markus Gross , Stephan Würmlin , Martin Naef , Edouard Lamboray , Christian Spagno , Andreas Kunz , Esther Koller-Meier , Tomas Svoboda , Luc Van Gool , Silke Lang , Kai Strehlke , Andrew Vande Moere , Oliver Staadt, blue-c: a spatially immersive display and 3D video portal for telepresence, ACM SIGGRAPH 2003 Papers, July 27-31, 2003, San Diego, California
• 5: Li Guan , Sudipta Sinha , Jean-Sebastien Franco , Marc Pollefeys, Visual Hull Construction in the Presence of Partial Occlusion, Proceedings of the Third International Symposium on 3D Data Processing, Visualization, and Transmission (3DPVT'06), p.413-420, June 14-16, 2006 [doi>10.1109/3DPVT.2006.147]
• 6: Han, B., Comaniciu, D., and Davis, L. Sequential kernel density approximation through mode propagation: applications to background modeling. Proc. ACCV (2004).
• 7: Ishikawa, T., Yamazawa, K., and Yokoya, N. Real-time generation of novel views of a dynamic scene using morphing and visual hull. Proc. ICIP (2005), 1013--1016.
• 8: Takeo Kanade , Peter Rander , P. J. Narayanan, Virtualized Reality: Constructing Virtual Worlds from Real Scenes, IEEE MultiMedia, v.4 n.1, p.34-47, January 1997 [doi>10.1109/93.580394 ]
• 9: Kim, H., Sakamoto, R., Kitahara, I., Toriyama, T., and Kogure, K. Robust Foreground Segmentation from Color Video Sequences Using Background Subtraction with Multiple Thresholds. Proc. KJPR (2006), 188--193.
• 10: Marcus A. Magnor, Video-Based Rendering, AK Peters Ltd, 2005
• 11: Matsuyama, T., Wu, X., Takai, T., and Wada, T. Real-Time Dynamic 3-D Object Shape Reconstruction and High-Fidelity Texture Mapping for 3-D Video. IEEE Trans. CSVT, 14, 3 (2004), 357--369.
• 12: Wojciech Matusik , Chris Buehler , Ramesh Raskar , Steven J. Gortler , Leonard McMillan, Image-based visual hulls, Proceedings of the 27th annual conference on Computer graphics and interactive techniques, p.369-374, July 2000 [doi>10.1145/344779.344951]
• 13: Mittal, A., and Paragios, N. Motion-based background subtraction using adaptive kernel density estimation. Proc. CVPR (2004), 302--309.
• 14: Stauffer. C., and Grimson, W. E. L. Adaptive background mixture models for real-time tracking. Proc. CVPR (1999), 246--252.
• 15: Richard Szeliski, Rapid octree construction from image sequences, CVGIP: Image Understanding, v.58 n.1, p.23-32, July 1993 [doi>10.1006/ciun.1993.1029 ]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.4 IMAGE PROCESSING AND COMPUTER VISION
      I.4.8 Scene Analysis

General Terms:
Algorithms, Reliability