Measuring the Reliability of Mobile Broadband Networks

ABSTRACT

Mobile broadband networks play an increasingly important role in society, and there is a strong need for independent assessments of their robustness and performance. A promising source of such information is active end-to-end measurements. It is, however, a challenging task to go from individual measurements to an assessment of network reliability, which is a complex notion encompassing many stability and performance related metrics. This paper presents a framework for measuring the user-experienced reliability in mobile broadband networks. We argue that reliability must be assessed at several levels, from the availability of the network connection to the stability of application performance. Based on the proposed framework, we conduct a large-scale measurement study of reliability in 5 mobile broadband networks. The study builds on active measurements from hundreds of measurement nodes over a period of 10 months. The results show that the reliability of mobile broadband networks is lower than one could hope: more than 20% of connections from stationary nodes are unavailable more than 10 minutes per day. There is, however, a significant potential for improving robustness if a device can connect simultaneously to several networks. We find that in most cases, our devices can achieve 99.999% ("five nines") connection availability by combining two operators. We further show how both radio conditions and network configuration play important roles in determining reliability, and how external measurements can reveal weaknesses and incidents that are not always captured by the operators' existing monitoring tools.

References

1. ITU-T recommendation G.711. Pulse code modulation (PCM) of voice frequencies, 1988.
2. Mobiperf. http://www.mobiperf.com, 2014.
3. M. Bagnulo, P. Eardley, T. Burbridge, B. Trammell, and R. Winter. Standardizing Large-scale Measurement Platforms. SIGCOMM Comput. Commun. Rev., 43, 2013.
4. Cisco Systems, Inc. Cisco visual networking index: Global mobile data traffic forecast update, 2012--2017, February 2013.
5. Communications Regulatory Authority of the Republic of Lithuania. Annual report of the Communications Regulatory Authority (RTT) of the Republic of Lithuania, 2012.
6. Digium. Asterisk. http://www.asterisk.org/.
7. FCC. 2013 Measuring Broadband America February Report. Technical report, FCC's Office of Engineering and Technology and Consumer and Governmental Affairs Bureau, 2013.
8. O. Goga and R. Teixeira. Speed Measurements of Residential Internet Access. In Proc. of PAM, 2012.
9. E. Halepovic, J. Pang, and O. Spatscheck. Can you GET me now?: Estimating the time-to-first-byte of HTTP transactions with passive measurements. In Proc. of IMC, 2012.
10. H. Holma and A. Toskala. WCDMA for UMTS: HSPA Evolution and LTE. John Wiley & Sons Ltd., 4th edition, 2007.
11. J. Huang, F. Qian, Y. Guo, Y. Zhou, Q. Xu, Z. M. Mao, S. Sen, and O. Spatscheck. An In-depth Study of LTE: Effect of Network Protocol and Application Behavior on Performance. In Proc. of SIGCOMM, 2013.
12. IEEE. P802.16.3 Project: Mobile Broadband Network Performance Measurements. http://www.ieee802.org/16/mbnpm/index.html.
13. H. Jiang, Y. Wang, K. Lee, and I. Rhee. Tackling bufferbloat in 3g/4g networks. In Proc. of IMC, 2012.
14. Z. Koradia, G. Mannava, A. Raman, G. Aggarwal, V. Ribeiro, A. Seth, S. Ardon, A. Mahanti, and S. Triukose. First Impressions on the State of Cellular Data Connectivity in India. In Proceedings of the 4th Annual Symposium on Computing for Development, ACM DEV-4 '13, 2013.
15. A. Kvalbein, D. Baltrunas, J. Xiang, K. R. Evensen, A. Elmokash, and S. Ferlin-Oliveira. The Nornet Edge platform for mobile broadband measurements. Elsevier Computer Networks special issue on Future Internet Testbeds, 2014.
16. A. Nikravesh, D. R. Chones, E. Katz-Bassett, Z. M. Mao, and M. Welsh. Mobile Network Performance from User Devices: A Longitudinal, Multidimensional Analysis. In Procs. of PAM, 2014.
17. F. Qian, Z. Wang, A. Gerber, Z. M. Mao, S. Sen, and O. Spatscheck. Characterizing Radio Resource Allocation for 3G Networks. In Proc. of IMC, 2010.
18. S. Sen, J. Yoon, J. Hare, J. Ormont, and S.Banerjee. Can they hear me now?: A case for a client-assisted approach to monitoring wide-area wireless networks. In Proc. of IMC, 2011.
19. M. Z. Shafiq, L. Ji, A. X. Liu, J. Pang, S. Venkataraman, and J. Wang. A first look at cellular network performance during crowded events. In Proc. of SIGMETRICS, 2013.
20. M. Z. Shafiq, L. Ji, A. X. Liu, and J. Wang. Characterizing and Modeling Internet Traffic Dynamics of Cellular Devices. In Proc. of SIGMETRICS, 2011.
21. J. Sommers and P. Barford. Cell vs. WiFi: On the Performance of Metro Area Mobile Connections. In Proc. of IMC, 2012.
22. M. Stiemerling. IP Performance Metrics charter-ietf-ippm-05. http: //datatracker.ietf.org/doc/charter-ietf-ippm/, 2013.
23. S. Sundaresan, W. de Donato, N. Feamster, R. Teixeira, S. Crawford, and A. Pescapè. Broadband Internet performance: A view from the gateway. SIGCOMM Comput. Commun. Rev., 41:134--145, 2011.
24. Viber-Media. Viber. http://www.viber.com/.

Index Terms

• Computer systems organization

  • Dependable and fault-tolerant systems and networks

    • Availability

    • Maintainability and maintenance

    • Reliability

• General and reference

  • Cross-computing tools and techniques

    • Measurement

    • Metrics

    • Reliability