skip to main content
research-article

Enabling multi-party 3D tele-immersive environments with ViewCast

Published: 23 March 2010 Publication History

Abstract

Three-dimensional tele-immersive (3DTI) environments have great potential to promote collaborative work among geographically distributed users. However, most existing 3DTI systems only work with two sites due to the huge demand of resources and the lack of a simple yet powerful networking model to handle connectivity, scalability, and quality-of-service (QoS) guarantees.
In this article, we explore the design space from the angle of multi-stream management to enable multi-party 3DTI communication. Multiple correlated 3D video streams are employed to provide a comprehensive representation of the physical scene in each 3DTI environment, and are rendered together to establish a common cyberspace among all participating 3DTI environments. The existence of multi-stream correlation provides the unique opportunity for new approaches in QoS provisioning. Previous work mostly concentrated on compression and adaptation techniques on the per-stream basis while ignoring the application layer semantics and the coordination required among streams. We propose an innovative and generalized ViewCast model to coordinate the multi-stream content dissemination over an overlay network. ViewCast leverages view semantics in 3D free-viewpoint video systems to fill the gap between high-level user interest and low-level stream management. In ViewCast, only the view information is specified by the user/application, while the underlying control dynamically performs stream differentiation, selection, coordination, and dissemination. We present the details of ViewCast and evaluate it through both simulation and 3DTI sessions among tele-immersive environments residing in different institutes across the Internet2. Our experimental results demonstrate the implementation feasibility and performance enhancement of ViewCast in supporting multi-party 3DTI collaboration.

References

[1]
3DAV Report. 2003. Report on 3dav exploration. International Organisation for Standardisation, ISO/IEC JTC1/SC29/WG11 N5878.
[2]
Baker, H. H., Bhatti, N., Tanguay, D., Sobel, I., Gelb, D., Goss, M. E., Culbertson, W. B., and Malzbender, T. 2005. Understanding performance in coliseum, an immersive videoconferencing system. ACM Trans. Multimedia Comput. Comm. Appl. 1, 2, 190--210.
[3]
Banerjee, S., Bhattacharjee, B., and Kommareddy, C. 2002. Scalable application layer multicast. SIGCOMM Comput. Comm. Rev. 32, 4, 205--217.
[4]
Bauer, F. and Varma, A. 1997. Aries: A rearrangeable inexpensive edge-based on-line Steiner algorithm. IEEE J. Sel. Areas Commun. 382--397.
[5]
Castro, M., Druschel, P., Kermarrec, A.-M., Nandi, A., Rowstron, A., and Singh, A. 2003. Splitstream: high-bandwidth multicast in cooperative environments. In Proceedings of the 19th ACM Symposium on Operating Systems Principles (SOSP). ACM, New York, NY, 298--313.
[6]
Chen, M. 2001. Design of a virtual auditorium. In Proceedings of the 9th ACM International Conference on Multimedia (MULTIMEDIA). ACM, New York, NY, 19--28.
[7]
Chu, Y. H., Rao, S. G., and Zhang, H. 2000. A case for end system multicast. In Proceedings of the ACM SIGMETRICS International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS). ACM, New York, NY, 1--12.
[8]
Cui, Y. and Nahrstedt, K. 2003. Layered peer-to-peer streaming. In Proceedings of the 13th International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV). ACM, New York, NY, 162--171.
[9]
Daniilidis, K., Mulligan, J., McKendall, R., Schmid, D., Kamberova, G., and Bajcsy, R. 2000. Real-time 3d-teleimmersion. In Confluence of Computer Vision and Computer Graphics, Kluwer Academic Publishers, Norwell, MA, 253--265.
[10]
Gharai, L., Perkins, C., Riley, C., and Mankin, A. 2002. Large scale video conferencing: A digital amphitheater. In Proceedings of the 8th International Conference on Distributed Multimedia Systems (DMS). 137--143.
[11]
Greenhalgh, C. and Benford, S. 1995. Massive: a collaborative virtual environment for teleconferencing. ACM Trans. Comput.-Hum. Inter. 2, 3, 239--261.
[12]
Gross, M., Würmlin, S., Naef, M., Lamboray, E., Spagno, C., Kunz, A., Koller-Meier, E., Svoboda, T., Gool, L. V., Lang, S., Strehlke, K., Moere, A. V., and Staadt, O. 2003. blue-c: a spatially immersive display and 3d video portal for telepresence. ACM Trans. Graph. 22, 3, 819--827.
[13]
Hosseini, M. and Georganas, N. D. 2003. Design of a multi-sender 3d videoconferencing application over an end system multicast protocol. In Proceedings of the 11th ACM International Conference on Multimedia (MULTIMEDIA). ACM Press, New York, NY, 480--489.
[14]
Jannotti, J., Gifford, D. K., Johnson, K. L., Kaashoek, M. F., and James W. O'Toole, J. 2000. Overcast: reliable multicasting with on overlay network. In Proceedings of the 4th Conference on Symposium on Operating System Design & Implementation (OSDI). USENIX Association, Berkeley, CA, 197--212.
[15]
Kauff, P. and Schreer, O. 2002. An immersive 3d video-conferencing system using shared virtual team user environments. In Proceedings of the 4th International Conference on Collaborative Virtual Environments (CVE). ACM, New York, NY, 105--112.
[16]
Mulligan, J., Isler, V., and Daniilidis, K. 2001. Trinocular stereo: a real-time algorithm and its evaluation. In Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV). 10--17.
[17]
Ott, D. E. and Mayer-Patel, K. 2004. Coordinated multi-streaming for 3d tele-immersion. In Proceedings of the 12th Annual ACM International Conference on Multimedia (MULTIMEDIA). ACM Press, New York, NY, 596--603.
[18]
Raskar, R., Welch, G., Cutts, M., Lake, A., Stesin, L., and Fuchs, H. 1998. The office of the future: a unified approach to image-based modeling and spatially immersive displays. In Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Technique (SIGGRAPH). ACM Press, New York, NY, 179--188.
[19]
Reynard, G., Benford, S., Greenhalgh, C., and Heath, C. 1998. Awareness driven video quality of service in collaborative virtual environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI). ACM Press/Addison-Wesley Publishing Co., New York, NY, 464--471.
[20]
Schreer, O., Brandenburg, N., Askar, S., and Trucco, E. 2001. A virtual 3d video-conferencing system providing semi-immersive telepresence: A real-time solution in hardware and software. In Proceedings of the International Conference on eWork and eBusiness. 184--190.
[21]
Shi, S., Wang, L., Calvert, K. L., and Griffioen, J. N. 2004. A multi-path routing service for immersive environments. In Proceedings of the IEEE International Symposium on Cluster Computing and the Grid (CCGRID). IEEE Computer Society, Los Alamitos, CA, 699--706.
[22]
Sripanidkulchai, K., Maggs, B., and Zhang, H. 2004. An analysis of live streaming workloads on the internet. In Proceedings of the 4th ACM SIGCOMM Conference on Internet Measurement (IMC). ACM, New York, NY, 41--54.
[23]
Sriram, R., Manimaran, G., and Murthy, C. 1999. A rearrangeable algorithm for the construction of delay-constrained dynamic multicast trees. In Proceedings of the IEEE Conference on Computer Communications (INFOCOM). New York, NY, 1073--1080.
[24]
TEEVE. 2006. Teeve project, http://cairo.cs.uiuc.edu/teleimmersion, http://tele-immersion.citris-uc.org.
[25]
Würmlin, S., Lamboray, E., and Gross, M. 2003. 3d video fragmens: dynamic point samples for real-time free-viewpoint video. Tech. rep. No. 397. Institute of Scientific Computing, ETH, Zurich.
[26]
Yang, Z., Cui, Y., Anwar, Z., Bocchino, R., Kiyanclar, N., Nahrstedt, K., Campbell, R. H., and Yurcik, W. 2006a. Real-time 3d video compression for tele-immersive environments. In Proceedings of the SPIE Multimedia Computing and Networking (MMCN). 12--23.
[27]
Yang, Z., Nahrstedt, K., Cui, Y., Yu, B., Liang, J., hack Jung, S., and Bajscy, R. 2005. Teeve: the next generation architecture for tele-immersive environments. In Proceedings of the 7th IEEE International Symposium on Multimedia (ISM). 112--119.
[28]
Yang, Z., Wu, W., Nahrstedt, K., Kurillo, G., and Bajcsy, R. 2007. Viewcast: view dissemination and management for multi-party 3d tele-immersive environments. In Proceedings of the 15th International Conference on Multimedia (MULTIMEDIA). ACM, New York, NY, 882--891.
[29]
Yang, Z., Yu, B., Nahrstedt, K., and Bajscy, R. 2006b. A multi-stream adaptation framework for bandwidth management in 3d tele-immersion. In Proceedings of the International Workshop on Network and Operating Systems Support for Digital Audio and Video (NOSSDAV). ACM, New York, NY, 1--6.

Cited By

View all
  • (2024)ImmerScope: Multi-view Video Aggregation at Edge towards Immersive Content ServicesProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699324(82-96)Online publication date: 4-Nov-2024
  • (2024)VP9 bitstream-based Tiled Multipoint Control UnitProceedings of the 15th ACM Multimedia Systems Conference10.1145/3625468.3647608(23-33)Online publication date: 15-Apr-2024
  • (2023)Volumetric video streamingImmersive Video Technologies10.1016/B978-0-32-391755-1.00021-3(425-443)Online publication date: 2023
  • Show More Cited By

Index Terms

  1. Enabling multi-party 3D tele-immersive environments with ViewCast

        Recommendations

        Comments

        Information & Contributors

        Information

        Published In

        cover image ACM Transactions on Multimedia Computing, Communications, and Applications
        ACM Transactions on Multimedia Computing, Communications, and Applications  Volume 6, Issue 2
        March 2010
        119 pages
        ISSN:1551-6857
        EISSN:1551-6865
        DOI:10.1145/1671962
        Issue’s Table of Contents
        Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

        Publisher

        Association for Computing Machinery

        New York, NY, United States

        Publication History

        Published: 23 March 2010
        Accepted: 01 February 2009
        Revised: 01 April 2008
        Received: 01 October 2007
        Published in TOMM Volume 6, Issue 2

        Permissions

        Request permissions for this article.

        Check for updates

        Author Tags

        1. 3D tele-immersion
        2. QoS adaptation
        3. application level multicast
        4. distributed multimedia system
        5. multi-stream coordination
        6. networking protocol

        Qualifiers

        • Research-article
        • Research
        • Refereed

        Funding Sources

        Contributors

        Other Metrics

        Bibliometrics & Citations

        Bibliometrics

        Article Metrics

        • Downloads (Last 12 months)11
        • Downloads (Last 6 weeks)1
        Reflects downloads up to 18 Feb 2025

        Other Metrics

        Citations

        Cited By

        View all
        • (2024)ImmerScope: Multi-view Video Aggregation at Edge towards Immersive Content ServicesProceedings of the 22nd ACM Conference on Embedded Networked Sensor Systems10.1145/3666025.3699324(82-96)Online publication date: 4-Nov-2024
        • (2024)VP9 bitstream-based Tiled Multipoint Control UnitProceedings of the 15th ACM Multimedia Systems Conference10.1145/3625468.3647608(23-33)Online publication date: 15-Apr-2024
        • (2023)Volumetric video streamingImmersive Video Technologies10.1016/B978-0-32-391755-1.00021-3(425-443)Online publication date: 2023
        • (2022)Evaluating the Impact of Tiled User-Adaptive Real-Time Point Cloud Streaming on VR Remote CommunicationProceedings of the 30th ACM International Conference on Multimedia10.1145/3503161.3548220(3094-3103)Online publication date: 10-Oct-2022
        • (2020)Group Behavior RecognitionHuman Behavior Analysis: Sensing and Understanding10.1007/978-981-15-2109-6_6(139-218)Online publication date: 1-Mar-2020
        • (2018)Multimedia SystemsMultimedia Computing10.1017/CBO9781139049351.010(95-107)Online publication date: 12-Sep-2018
        • (2017)Multicamera Summarization of Rehabilitation Sessions in Home EnvironmentProceedings of the 25th ACM international conference on Multimedia10.1145/3123266.3123387(1381-1389)Online publication date: 23-Oct-2017
        • (2016)MobiGroup: Enabling Lifecycle Support to Social Activity Organization and Suggestion With Mobile Crowd SensingIEEE Transactions on Human-Machine Systems10.1109/THMS.2015.250329046:3(390-402)Online publication date: Jun-2016
        • (2016)Security enhancement for video transmission via noise aggregation in immersive systemsMultimedia Tools and Applications10.1007/s11042-015-2936-375:9(5345-5357)Online publication date: 1-May-2016
        • (2015)Distortion score based pose selection for 3D tele-immersionProceedings of the 21st ACM Symposium on Virtual Reality Software and Technology10.1145/2821592.2821600(227-236)Online publication date: 13-Nov-2015
        • Show More Cited By

        View Options

        Login options

        Full Access

        View options

        PDF

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Figures

        Tables

        Media

        Share

        Share

        Share this Publication link

        Share on social media