Low Bit-rate Mobile Cloud Gaming

ABSTRACT

In mobile cloud gaming, high-quality, high-frame-rate game images of immense data-size need to be delivered to the clients under stringent delay requirement. For good gaming experience, reducing the transmission bit-rate of the game images is necessary. In this work, we investigate a framework to achieve low bit-rate game image transmission. In particular, we extend our previously proposed layered coding to reduce transmission bandwidth and latency. In layered coding, we leverage the rendering capability of modern mobile devices to locally render low-quality game images, or the base layer. Instead of sending high quality game images directly, cloud servers can send enhancement layer information, which clients can utilize to improve the quality of the base layer. We investigate the design of a complexity-scalable base layer (BL) rendering pipeline that can be executed on a range of power-constrained mobile devices for layered coding. We focus on the popular Blinn-Phong lighting and formulate the design of BL Blinn-Phong lighting as an information-complexity optimization problem. Furthermore, we investigate a solution to remove the initialization latency of layered coding. Experiment results show that the information rate of the enhancement layer could be much less than that of the high quality game image; while the base layer can be generated with only a very small amount of computation. In particular, up to 84 percent reduction in bandwidth usage can be achieved by the proposed layered coding scheme.

References

1. 4G LTE showdown: How fast is your carrier? http://www.cnet.com/news/4g-lte-showdown-how-fast-is-your-carrier/.
2. E. Angel. Interactive Computer Graphics: A Top-Down Approach Using OpenGL. Addison-Wesley Publishing Company, USA, 5th edition, 2008.
3. S. P. Chuah, N. M. Cheung, and C. Yuen. Information-theoretic analysis of blinn-phong lighting with application to mobile cloud gaming. In Image Processing (ICIP), 2015 IEEE International Conference on, pages 382--386, Sept 2015.
4. S.-P. Chuah, N.-M. Cheung, and C. Yuen. Layered coding for mobile cloud gaming using scalable blinn-phong lighting. IEEE Transactions on Image Processing, 2016.
5. D. Cohen-Or, Y. Mann, and S. Fleishman. Deep compression for streaming texture intensive animations. In Proc. of, SIGGRAPH '99, pages 261--267, New York, NY, USA, 1999. ACM Press/Addison-Wesley Publishing Co.
6. P. Eisert and P. Fechteler. Low delay streaming of computer graphics. In Image Processing, 2008. ICIP 2008. 15th IEEE International Conference on, pages 2704--2707, 2008.
7. L. Fang, N. M. Cheung, D. Tian, A. Vetro, H. Sun, and O. C. Au. An analytical model for synthesis distortion estimation in 3d video. IEEE Transactions on Image Processing, 23(1):185--199, Jan 2014.
8. C.-Y. Huang, C.-H. Hsu, Y.-C. Chang, and K.-T. Chen. Gaminganywhere: an open cloud gaming system. In Proc. of, MMSys '13, pages 36--47, New York, NY, USA, 2013. ACM.
9. A. Jurgelionis, P. Fechteler, P. Eisert, F. Bellotti, H. David, J. Laulajainen, R. Carmichael, V. Poulopoulos, A. Laikari, P. Perala, A. D. Gloria, and C. Bouras. Platform for distributed 3d gaming. International Journal of Computer Games Technology, 2009(1):1--15, January 2009.
10. M. Levoy. Polygon-assisted JPEG and MPEG compression of synthetic images. In Proc. of SIGGRAPH, pages 21--28, New York, NY, USA, 1995. ACM.
11. X. Nan, X. Guo, Y. Lu, Y. He, L. Guan, S. Li, and B. Guo. A novel cloud gaming framework using joint video and graphics streaming. In Multimedia and Expo (ICME), 2014 IEEE International Conference on, pages 1--6, July 2014.
12. J. Pool, A. Lastra, and M. Singh. An energy model for graphics processing units. In Computer Design (ICCD), 2010 IEEE International Conference on, pages 409--416, 2010.
13. S. Shi, C.-H. Hsu, K. Nahrstedt, and R. Campbell. Using graphics rendering contexts to enhance the real-time video coding for mobile cloud gaming. In Proc. of, MM '11, pages 103--112, New York, NY, USA, 2011. ACM.
14. Unity3D. unity3d.com.
15. S. Wang and S. Dey. Adaptive mobile cloud computing to enable rich mobile multimedia applications. Multimedia, IEEE Transactions on, 15(4):870--883, 2013.
16. L. A. Wolsey. Integer Programming. Wiley, 1998.
17. J. Wu, C. Yuen, N. M. Cheung, J. Chen, and C. W. Chen. Enabling adaptive high-frame-rate video streaming in mobile cloud gaming applications. IEEE Transactions on Circuits and Systems for Video Technology, 25(12):1988--2001, Dec 2015.

Index Terms

• Information systems

  • Information systems applications

    • Multimedia information systems

      • Multimedia streaming