Improving TCP Congestion Control with Machine Intelligence

ABSTRACT

In a TCP/IP network, a key to ensure efficient and fair sharing of network resources among its users is the TCP congestion control (CC) scheme. Previously, the design of TCP CC schemes is based on hard-wiring of predefined actions to specific feedback signals from the network. However, as networks become more complex and dynamic, it becomes harder to design the optimal feedback-action mapping. Recently, learning-based TCP CC schemes have attracted much attention due to their strong capabilities to learn the actions from interacting with the network. In this paper, we design two learning-based TCP CC schemes for wired networks with under-buffered bottleneck links, a loss predictor (LP) based TCP CC (LP-TCP), and a reinforcement learning (RL) based TCP CC (RL-TCP). We implement both LP-TCP and RL-TCP in NS2. Compared to the existing NewReno and Q-learning based TCP, LP-TCP and RL-TCP both achieve a better tradeoff between throughput and delay, under various simulated network scenarios.

References

1. Guido Appenzeller. 2005. Sizing router buffers. Ph.D. Dissertation. Stanford University, Palo Alto, CA.
2. Saad Biaz and Nitin H Vaidya. 1998. Distinguishing congestion losses from wireless transmission losses: A negative result. In Proc. 7th International Conference on Computer Communications and Networks. IEEE, Lafayette, LA, 722--731.
3. Lawrence S. Brakmo and Larry L. Peterson. 1995. TCP Vegas: End to end congestion avoidance on a global Internet. IEEE Journal on selected Areas in communications 13, 8 (1995), 1465--1480.
4. Mo Dong, Qingxi Li, and Doron Zarchy. 2015. PCC: Re-architecting Congestion Control for Consistent High Performance. In Proc. 12th USENIX Symposium on Networked Systems Design and Implementation (NSDI'15). ACM, Oakland, CA, 395--408.
5. Sally Floyd and Tom Henderson. 1999. The NewReno modification to TCP's fast recovery algorithm. RFC 2582 (1999).
6. Sangtae Ha, Injong Rhee, and Lisong Xu. 2008. CUBIC: a new TCP-friendly high-speed TCP variant. ACM SIGOPS Operating Systems Review 42, 5 (2008), 64--74.
7. Tin Kam Ho. 1995. Random decision forests. In Proc. the 3rd international conference on Document analysis and recognition. IEEE, Montreal, Que., Canada, 278--282.
8. Tin Kam Ho. 1998. The random subspace method for constructing decision forests. IEEE transactions on pattern analysis and machine intelligence 20, 8 (1998), 832--844.
9. Van Jacobson. 1988. Congestion avoidance and control. In Proc. ACM SIGCOMM. ACM, Stanford, CA, 314--329.
10. Van Jacobson. 1990. Modified TCP congestion avoidance algorithm. note sent to end2end-interest mailing list (1990).
11. Raj Jain. 1989. A delay-based approach for congestion avoidance in interconnected heterogeneous computer networks. ACM SIGCOMM Computer Communication Review 19, 5 (1989), 56--71.
12. Raj Jain, Dah-Ming Chiu, and William R Hawe. 1984. A quantitative measure of fairness and discrimination for resource allocation in shared computer system. Vol. 38. Eastern Research Laboratory, Digital Equipment Corporation Hudson, MA.
13. A Jayaraj, T Venkatesh, and C Siva Ram Murthy. 2008. Loss classification in optical burst switching networks using machine learning techniques: Improving the performance of TCP. IEEE Journal on Selected Areas in Communications 26, 6 (2008).
14. Junchen Jiang, Shijie Sun, Vyas Sekar, and Hui Zhang. 2017. Pytheas: Enabling Data-Driven Quality of Experience Optimization Using Group-Based Exploration-Exploitation. In Proc. 14th USENIX Symposium on Networked Systems Design and Implementation (NSDI'17). ACM, BOSTON, MA, 393--406.
15. Dina Katabi, Mark Handley, and Charlie Rohrs. 2002. Congestion control for high bandwidth-delay product networks. ACM SIGCOMM computer communication review 32, 4 (2002), 89--102.
16. Wei Li, Fan Zhou, Waleed Meleis, and Kaushik Chowdhury. 2016. Learning-based and Data-driven TCP Design for Memory-constrained IoT. In Proc. International Conference on Distributed Computing in Sensor Systems (DCOSS). IEEE, Washington, DC, 199--205.
17. Dapeng Liu, Youjian Zhao, Kaixin Sui, Lei Zou, Dan Pei, Qingqian Tao, Xiyang Chen, and Dai Tan. 2016. FOCUS: Shedding light on the high search response time in the wild. In Proc. The 35th Annual IEEE International Conference on Computer Communications (INFOCOM). IEEE, San Francisco, CA, 1--9.
18. Dapeng Liu, Youjian Zhao, Haowen Xu, Yongqian Sun, Dan Pei, Jiao Luo, Xiaowei Jing, and Mei Feng. 2015. Opprentice: Towards practical and automatic anomaly detection through machine learning. In Proc. the Internet Measurement Conference. ACM, Tokyo, Japan, 211--224.
19. Hongzi Mao, Mohammad Alizadeh, Ishai Menache, and Srikanth Kandula. 2016. Resource management with deep reinforcement learning. In Proc. the 15th ACM Workshop on Hot Topics in Networks. ACM, Atlanta, GA, 50--56.
20. Hongzi Mao, Ravi Netravali, and Mohammad Alizadeh. 2017. Neural adaptive video streaming with Pensieve. In Proc. ACM SIGCOMM. ACM, Los Angeles, CA, 197--210.
21. Mariyam Mirza, Joel Sommers, Paul Barford, and Xiaojin Zhu. 2010. A machine learning approach to TCP throughput prediction. IEEE/ACM Transactions on Networking 18, 4 (2010), 1026--1039.
22. Bruno Astuto Arouche Nunes, Kerry Veenstra, William Ballenthin, Stephanie Lukin, and Katia Obraczka. 2014. A machine learning framework for TCP round-trip time estimation. EURASIP Journal on Wireless Communications and Networking 1 (2014), 47.
23. Lopamudra Roychoudhuri and Ehab S Al-Shaer. 2005. Real-time packet loss prediction based on end-to-end delay variation. IEEE transactions on Network and Service Management 2, 1 (2005), 29--38.
24. Anirudh Sivaraman, Keith Winstein, Pratiksha Thaker, and Hari Balakrishnan. 2014. An experimental study of the learnability of congestion control. In Proc. ACM SIGCOMM. ACM, Chicago, IL, 479--490.
25. Yi Sun, Xiaoqi Yin, Junchen Jiang, Vyas Sekar, Fuyuan Lin, Nanshu Wang, Tao Liu, and Bruno Sinopoli. 2016. CS2P: Improving video bitrate selection and adaptation with data-driven throughput prediction. In Proc. ACM SIGCOMM. ACM, Florianópolis, Brazil, 272--285.
26. Richard S Sutton and Andrew G Barto. 1998. Reinforcement learning: An introduction. Vol. 1. Cambridge: MIT press.
27. C-H Tai, Jiang Zhu, and Nandita Dukkipati. 2008. Making large scale deployment of RCP practical for real networks. In Proc. The 27th Conference on Computer Communications. IEEE, Phoenix, AZ, 2180--2188.
28. Kun Tan, Jingmin Song, Qian Zhang, and Murad Sridharan. 2006. A compound TCP approach for high-speed and long distance networks. In Proc. IEEE INFOCOM. IEEE, Barcelona, Spain, 1--12.
29. Zheng Wang and Jon Crowcroft. 1991. A new congestion control scheme: Slow start and search (Tri-S). ACM SIGCOMM Computer Communication Review 21, 1 (1991), 32--43.
30. Christopher John Cornish Hellaby Watkins. 1989. Learning from delayed rewards. Ph.D. Dissertation. King's College, Cambridge, Cambridge, UK.
31. Keith Winstein and Hari Balakrishnan. 2013. TCP ex machina: Computer-generated congestion control. In Proc. ACM SIGCOMM. ACM, Hong Kong, China, 123--134.
32. Yasir Zaki, Thomas Pötsch, Jay Chen, Lakshminarayanan Subramanian, and Carmelita Görg. 2015. Adaptive congestion control for unpredictable cellular networks. In Proc. ACM SIGCOMM. ACM, London, United Kingdom, 509--522.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Reinforcement learning

      • Supervised learning

    • Machine learning approaches

      • Classification and regression trees

• Networks

  • Network protocols

    • Transport protocols