Real time 3D avatar for interactive mixed reality

ABSTRACT

This paper presents real-time reconstruction of dynamic 3D avatar for interactive mixed reality. In computer graphics, one of the main goals is the combination of virtual scenes with real-world scenes. However, the views of the real world objects are often restricted to views from the cameras. True navigation through such mixed reality scenes becomes impossible unless the components from real objects can be rendered from arbitrary viewpoints. Additionally, adding a real-world object to a virtual scene requires some depth information as well, in order to handle interaction. The proposed algorithm introduces an approach to generate 3D video avatars and to augment the avatars naturally into 3D virtual environment using the calibrated camera parameters and silhouette information. As a result, we can create photo-realistic live avatars from natural scenes and the resulting 3D live avatar can guide and interact participants in VR space.

References

1. Debevec, P. E., Borshukov, G., and Yu, Y. 1998. Efficient view-dependent image-based rendering with projective texture-mapping. In 9th Eurographics Rendering Workshop, 105--116.]]
2. Markus, G., Stephan, W., Martin, N., Edouard, L., Christian, S., Andreas, K., Esther, K., Tomas, S., Luc, V. G., Silke, L., Kai, S., Andrew, V. M., and Oliver, S. 2003. blue-c: A Spatially Immersive Display and 3D Video Portal for Telepresence. In Proceedings of ACM SIGGRAPH 2003.]]
3. Matsuyama, Wu, X., Takai, T., and Nobuhara, S. 2003. Real-Time Generation and High Fidelity Visualization of 3D Video. MIRAGE 2003, 1--10.]]
4. Matusik, W., Buehler, C., Raskar, R., Gortler, S., and McMillan, L. 2000. Image-Based Visual Hulls. In Proceedings of ACM SIGGRAPH 2000, 369--374]]
5. Matusik, W., Bueler, C., and McMillan, L. 2000. Polyhedral visual hulls for real-time rendering. In Proceedings of 12th Eurographics Workshop on Rendering, 115--125.]]
6. Li, M., Magnor, M., and Seidel, H. 2003. Online Accelerated Rendering of Visual Hulls in Real Scenes. Journal of WSCG 2003, 11(2): 290--297.]]
7. Laurentini, A. 2000. The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Machine Intell., 16(2): 150--162.]]
8. Lok, B. 1995. Online Model Reconstruction for Interactive Virtual Environments. In Proceedings 2001 Symposium on Interactive 3-D Graphics 69--72.]]
9. Sawhney, H., Arpa, A., Kumar, R., Samarasekera, S., Aggarwal, M., Hsu, S., Nister, D., and Hanna, K. Video flashlights - real time rendering of multiple videos for immersive model visualization. In Proceedings of 13th Eurographics Workshop on Rendering.]]
10. Segal, M., Korobkin, C., Widenfelt, R., Foran, J., and Haeberli, P. 1992. Fast shadows and lighting effects using texture mapping. In Proceedings of ACM SIGGRAPH 1992, 249--252.]]
11. Slabaugh, G., Schafer, R., Hans, M. Image-Based Photo Hulls. In Proceedings of 3D Data Processing, Visualization, and Transmission.]]
12. Prince, S. J. D., Williamson, T., Johnson, N., Cheok, A. D., Farbiz, F., Billinghurst M., and Kato, H. 2002. 3D Live: Real Time Interaction for Mixed Reality. Computer Supported Collaborative Work.]]
13. Yang, R., Welch, R., Bishop, G. 2002. Real-Time Consensus-Based Scene Reconstruction using Commodity Graphics Hardware. In Proceedings of Pacific Graphics 2002.]]
14. Wada, T., Wu, X., Tokai, S., and Matsuyama, T. 2002. Homography Based Parallel Volume Intersection:Toward Real-Time Volume Reconstruction Using Active Cameras. In Proceedings of Computer Architectures for Machine Perception, 331--339.]]

Index Terms

• Computing methodologies

  • Computer graphics

    • Image manipulation

    • Rendering