skip to main content
10.1145/1180639.1180785acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
Article

Remote rendering and streaming of progressive panoramas for mobile devices

Published: 23 October 2006 Publication History

Abstract

Providing mobile devices with virtual environment walkthrough and real-time streaming movie playback capability is expected to have a profound impact to the entertainment-based applications, such as virtual guides, online gaming, and e-learning, just to name a few. However, it is well known that it is extremely difficult to render complex 3D scenes at interactive frame rates on thin mobile devices known for their lack of proper resources needed to process large volume of 3D virtual environment data. In order to provide virtual environment navigation on thin mobile clients, we propose a hybrid technique which combines both remote geometry rendering and streaming of warped images. In our approach, the server renders a partial panoramic view, which is based on the user's viewpoint and last movements. The server then warps the image's coordinates into cylindrical coordinates, and streams the images to the client device, which will progressively build the panoramic representation of the scene. Furthermore, in order to enhance streaming performance and quality of the interaction, we propose to use a rate control mechanism as well as a prediction of the user's movements within the virtual scene. In this paper we discuss our scheme for remote rendering and streaming of progressive panoramas for mobile devices, and present our experimental results we have obtained in order to validate our proposed technique. Our results indicate clearly that the proposed solution is able to achieve stable frame rates and throughput in error-prone wireless channels.

References

[1]
Balakrishnan, H., Katz, R. Explicit Loss Notification and Wireless Web Performance, IEEE Globecom Internet Mini-Conference, Nov. 1998
[2]
Biaz, S., Vaidya, N. H. Discriminating congestion loss from wireless losses using inter-arrival times at the receiver, Proc. of IEEE Symposium on Application-specific System and Software Engr. and Techn., pp. 10--17, Richardson, TX, Mar 1999
[3]
Boukerche, A., Huang, T., Pazzi, R. W. N. A real-time transport protocol for image-based rendering over heterogeneous wireless networks. In Proceedings of the 8th ACM International Symposium, MSWiM 2005
[4]
Boukerche, A., Pazzi, R. W. N. "Performance Evaluation of a Streaming Based Protocol for 3D Virtual Environment Exploration on Mobile Devices". Proceedings of the 9th ACM International Symposium MSWiM 2006.
[5]
Chang, C. and Ger, S. Enhancing 3D Graphics on Mobile Devices by Image-Based Rendering. In Proceedings of the Third IEEE Pacific Rim Conference on Multimedia. In LNCS vol. 2532. 1105--1111. 2002
[6]
Chen, M., Zakhor, A. Rate control for streaming video over wireless, IEEE Infocom 2004, Hong Kong, China, Mar. 2004.
[7]
Floyd, S., Handley, M., Padhye, J., Widmer, J. Equation-Based Congestion Control for Unicast Applications, Proc. ACM SIGCOMM 2000, Aug. 2000, pp. 43 -- 56
[8]
Handley, M., Floyd, S., Padhye, J., Widmer, J. TCP Friendly Rate Control (TFRC) Protocol Specification, RFC 3448.
[9]
Koller, D., et. al. Protected interactive 3D graphics via remote rendering. ACM Trans. Graph. 695--703. 2004
[10]
OpenGL - The Industry's Foundation for High Performance Graphics http://www.opengl.org/
[11]
OpenGL Embeded System. http://www.khronos.org/opengles/ 2006
[12]
Vicente, E., Mujica, V., Sisalem, D., Popescu-Zeletin, R., Wolisz, A. "TCP-Friendly Congestion Control over Wireless Networks". In Proc. of European Wireless 2004, Barcelona, Spain, February 2004.
[13]
QuickTime VR. http://www.virtuallyvancouver.com/index2.html.
[14]
Szeliski, R., Shum, H.-Y. Creating full view panoramic image mosaics and environment maps. In SIGGRAPH, pages 251--258. ACM Press/Addison- Wesley Publishing Co., 1997.
[15]
Thomas, G., Point, G., Bouatouch, K. A client-server approach to image-based rendering on mobile terminals. Technical Report, ISSN 0249-6399, France, January 2005.

Cited By

View all
  • (2023)Learning to Predict Head Pose in Remotely-Rendered Virtual RealityProceedings of the 14th Conference on ACM Multimedia Systems10.1145/3587819.3590972(27-38)Online publication date: 7-Jun-2023
  • (2023)RenderFusion: Balancing Local and Remote Rendering for Interactive 3D Scenes2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00046(312-321)Online publication date: 16-Oct-2023
  • (2021)Rendering and Transmission Method based on Model Resolution Expression and User Action Prediction2021 3rd International Conference on Artificial Intelligence and Advanced Manufacture (AIAM)10.1109/AIAM54119.2021.00071(318-322)Online publication date: Oct-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
MM '06: Proceedings of the 14th ACM international conference on Multimedia
October 2006
1072 pages
ISBN:1595934472
DOI:10.1145/1180639
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 October 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. image-based rendering
  2. remote virtual walkthrough
  3. wireless multimedia streaming

Qualifiers

  • Article

Conference

MM06
MM06: The 14th ACM International Conference on Multimedia 2006
October 23 - 27, 2006
CA, Santa Barbara, USA

Acceptance Rates

Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Learning to Predict Head Pose in Remotely-Rendered Virtual RealityProceedings of the 14th Conference on ACM Multimedia Systems10.1145/3587819.3590972(27-38)Online publication date: 7-Jun-2023
  • (2023)RenderFusion: Balancing Local and Remote Rendering for Interactive 3D Scenes2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00046(312-321)Online publication date: 16-Oct-2023
  • (2021)Rendering and Transmission Method based on Model Resolution Expression and User Action Prediction2021 3rd International Conference on Artificial Intelligence and Advanced Manufacture (AIAM)10.1109/AIAM54119.2021.00071(318-322)Online publication date: Oct-2021
  • (2021)CMCSF: a collaborative service framework for mobile web augmented reality base on mobile edge computingComputing10.1007/s00607-021-00952-8Online publication date: 29-Apr-2021
  • (2019)A hybrid remote rendering method for mobile applicationsMultimedia Tools and Applications10.1007/s11042-019-7306-079:5-6(3333-3358)Online publication date: 1-Mar-2019
  • (2018)A framework for cost-effective communication system for 3D data streaming and real-time 3D reconstructionProceedings of the 3rd International Workshop on Interactive and Spatial Computing10.1145/3191801.3191804(88-96)Online publication date: 12-Apr-2018
  • (2016)Proxy-guided Image-based Rendering for Mobile DevicesComputer Graphics Forum10.5555/3151666.315170235:7(353-362)Online publication date: 1-Oct-2016
  • (2016)Proxy‐guided Image‐based Rendering for Mobile DevicesComputer Graphics Forum10.1111/cgf.1303235:7(353-362)Online publication date: 27-Oct-2016
  • (2016)Remote rendering solutions using web technologiesMultimedia Tools and Applications10.1007/s11042-015-2481-075:8(4383-4410)Online publication date: 1-Apr-2016
  • (2015)A Survey of Interactive Remote Rendering SystemsACM Computing Surveys10.1145/271992147:4(1-29)Online publication date: 26-May-2015
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media