skip to main content
research-article
Free access

Taking a long look at QUIC: an approach for rigorous evaluation of rapidly evolving transport protocols

Published: 24 June 2019 Publication History

Abstract

Google's Quick UDP Internet Connections (QUIC) protocol, which implements TCP-like properties at the application layer atop a UDP transport, is now used by the vast majority of Chrome clients accessing Google properties but has no formal state machine specification, limited analysis, and ad-hoc evaluations based on snapshots of the protocol implementation in a small number of environ-merits. Further frustrating attempts to evaluate QUIC is the fact that the protocol is under rapid development, with extensive rewriting of the protocol occurring over the scale of months, making individual studies of the protocol obsolete before publication.
Given this unique scenario, there is a need for alternative techniques for understanding and evaluating QUIC when compared with previous transport-layer protocols. First, we develop an approach that allows us to conduct analysis across multiple versions of QUIC to understand how code changes impact protocol effectiveness. Next, we instrument the source code to infer QUIC's state machine from execution traces. With this model, we run QUIC in a large number of environments that include desktop and mobile, wired and wireless environments and use the state machine to understand differences in transport-and application-layer performance across multiple versions of QUIC and in different environments. QUIC generally outperforms TCP, but we also identified performance issues related to window sizes, re-ordered packets, and multiplexing large number of small objects; further, we identify that QUIC's performance diminishes on mobile devices and over cellular networks.

References

[1]
Chrome debugging protocol. https://developer.chrome.com/devtools/docs/debugger-protocol.
[2]
https://github.com/quicwg/basedrafts/wiki/implementations.
[3]
I. grigorik. Deciphering the critical rendering path. https://calendar.perfplanet.com/2012/deciphering-the-critical-rendering-path/.
[4]
QUIC at 10,000 feet. https://docs.google.com/document/d/1gY9-YNDNAB1eip-RTPbqphgySwSNSDHLq9D5Bty4FSU.
[5]
TCP probe. https://wiki.linuxfoundation.org/networking/tcpprobe.
[6]
Welch's t-test. https://en.wikipedia.org/wiki/Welch%27s_t-test.
[7]
Beschastnikh, I., Brun, Y., Schneider, S., Sloan, M., Ernst, M.D. Leveraging existing instrumentation to automatically infer invariant-constrained models. In Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering (2011).
[8]
Carlucci, G., De Cicco, L., Mascolo, S. HTTP over UDP: An experimental investigation of QUIC. In Proc. of SAC (2015).
[9]
Chromium Blog. A QUIC update on Google's experimental transport, April 2015. http://blog.chromium.org/2015/04/a-quic-update-on-googles-experimental.html.
[10]
Cimpanu, C. Google creates new algorithm for handling TCP traffic congestion control, September 2016. http://news.softpedia.com/news/google-creates-new-algorithm-for-handling-tcp-traffic-congshtml.
[11]
Das, S.R. Evaluation of QUIC on Web page performance. Master's thesis, Massachusetts Institute of Technology (2014).
[12]
Dukkipati, N., Cardwell, N., Cheng, Y., Mathis, M. Tail loss probe (TLP): An algorithm for fast recovery of tail losses, February 2013. https://tools.ietf.org/html/draft-dukkipati-tcpm-tcp-loss-probe-01.
[13]
Ha, S., Rhee, I. Taming the elephants: New TCP slow start. Comput. Netw., 2011.
[14]
Kakhki, A.M., Jero, S., Choffnes, D., Nita-Rotaru, C., Mislove, A. Taking a long look at QUIC: An approach for rigorous evaluation of rapidly evolving transport protocols. In Proceedings of the 2017 Internet Measurement Conference (2017).
[15]
Langley, A., Riddoch, A., Wilk, A., Vicente, A., Krasic, C., Zhang, D., Yang, F., Kouranov, F., Swett, I., Iyengar, J., Bailey, J., Dorfman, J., Kulik, J., Roskind, J., Westin, P., Tenneti, R., Shade, R., Hamilton, R., Vasiliev, V., Chang, W.-T., Shi, Z. The QUIC transport protocol: Design and Internet-scale deployment. In Proc. of ACM SIGCOMM (2017).
[16]
Mathis, M., Dukkipati, N., Cheng, Y. Proportional rate reduction for TCP, May 2013. https://tools.ietf.org/html/rfc6937.
[17]
Megyesi, P., Krämer, Z., Molnár, S. How quick is QUIC? In Proc. of ICC (May 2016).
[18]
Nikravesh, A., Yao, H., Xu, S., Choffnes, D.R., Mao, Z.M. Mobilyzer: An open platform for controllable mobile network measurements. In Proc. of MobiSys (2015).
[19]
Odvarko, J., Jain, A., Davies, A. HTTP Archive (HAR) format, August 2012. https://dvcs.w3.org/hg/webperf/raw-file/tip/specs/HAR/Overview.html.
[20]
Swett, I. QUIC congestion control and loss recovery. https://docs.google.com/presentation/d/1T9GtMz1CvPpZtmF8g-W7j9XHZBOCp9cu1fW0sMsmpoo.
[21]
Swett, I. QUIC deployment experience "Google, 2016. https://www.ietf.org/proceedings/96/slides/slides-96-quic-3.pdf.
[22]
Swett, I. QUIC FEC v1, February 2016. https://docs.google.com/document/d/1Hg1SaLEl6T4rEU9j-isovCo8VEjjnuCPTcLNJewj7Nk/edit.
[23]
Wang, X.S., Balasubramanian, A., Krishnamurthy, A., Wetherall, D. How speedy is SPDY? In Proc. of USENIX NSDI (2014).
[24]
Zhang, M., Karp, B., Floyd, S., Peterson, L. RR-TCP: A reordering-robust TCP with DSACK. In Proceedings of the 11th IEEE International Conference on Network Protocols, ICNP '03 (Washington, DC, USA, 2003). IEEE Computer Society.

Cited By

View all
  • (2024)Use of QUIC for Mobile-Oriented Future Internet (Q-MOFI)Electronics10.3390/electronics1302043113:2(431)Online publication date: 19-Jan-2024
  • (2023)Security and Performance Characteristics of QUIC and HTTP/3Proceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623757(124-126)Online publication date: 9-Oct-2023
  • (2022)Event Model Algorithm with Microservice Architecture Implementation2022 IEEE 9th International Conference on Problems of Infocommunications, Science and Technology (PIC S&T)10.1109/PICST57299.2022.10238498(561-566)Online publication date: 10-Oct-2022
  • Show More Cited By

Index Terms

  1. Taking a long look at QUIC: an approach for rigorous evaluation of rapidly evolving transport protocols

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image Communications of the ACM
      Communications of the ACM  Volume 62, Issue 7
      July 2019
      87 pages
      ISSN:0001-0782
      EISSN:1557-7317
      DOI:10.1145/3342113
      Issue’s Table of Contents
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 24 June 2019
      Published in CACM Volume 62, Issue 7

      Permissions

      Request permissions for this article.

      Check for updates

      Qualifiers

      • Research-article
      • Research
      • Refereed

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)899
      • Downloads (Last 6 weeks)152
      Reflects downloads up to 15 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Use of QUIC for Mobile-Oriented Future Internet (Q-MOFI)Electronics10.3390/electronics1302043113:2(431)Online publication date: 19-Jan-2024
      • (2023)Security and Performance Characteristics of QUIC and HTTP/3Proceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623757(124-126)Online publication date: 9-Oct-2023
      • (2022)Event Model Algorithm with Microservice Architecture Implementation2022 IEEE 9th International Conference on Problems of Infocommunications, Science and Technology (PIC S&T)10.1109/PICST57299.2022.10238498(561-566)Online publication date: 10-Oct-2022
      • (2021)It's over 9000Proceedings of the 21st ACM Internet Measurement Conference10.1145/3487552.3487826(261-275)Online publication date: 2-Nov-2021
      • (2021)An Early Benchmark of Quality of Experience Between HTTP/2 and HTTP/3 using LighthouseICC 2021 - IEEE International Conference on Communications10.1109/ICC42927.2021.9500258(1-6)Online publication date: Jun-2021
      • (2021)When QUIC's Connection Migration Meets Middleboxes A case study on mobile Wi-Fi hotspot2021 IEEE Global Communications Conference (GLOBECOM)10.1109/GLOBECOM46510.2021.9685048(1-6)Online publication date: Dec-2021
      • (2020)Analysis of Influence of UDP Parameters on QoS in IP Network2020 IEEE International Conference on Problems of Infocommunications. Science and Technology (PIC S&T)10.1109/PICST51311.2020.9467892(551-556)Online publication date: 6-Oct-2020
      • (2020)Does QUIC Suit Well With Modern Adaptive Bitrate Streaming Techniques?IEEE Networking Letters10.1109/LNET.2020.29918672:2(85-89)Online publication date: Jun-2020
      • (2020)Concept of a Regulatory Compliant Blockchain Based Instant Payment System. Limitations and Compromises2020 61st International Scientific Conference on Information Technology and Management Science of Riga Technical University (ITMS)10.1109/ITMS51158.2020.9259310(1-7)Online publication date: 2020

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Digital Edition

      View this article in digital edition.

      Digital Edition

      Magazine Site

      View this article on the magazine site (external)

      Magazine Site

      Login options

      Full Access

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media