Michio Honda
Felipe Huici
Abstract
Recent studies show that more than 86% of Internet paths allow well-designed TCP extensions, meaning that it is still possible to deploy transport layer improvements despite the existence of middleboxes in the network. Hence, the blame for the slow evolution of protocols (with extensions taking many years to nbecome widely used) should be placed on end systems.
In this paper, we revisit the case for moving protocols stacks up into user space in order to ease the deployment of new protocols, extensions, or performance optimizations. We present MultiStack, operating system support for user-level protocol stacks. MultiStack runs within commodity operating systems, can concurrently host a large number of isolated stacks, has a fall-back path to the legacy host stack, and is able to process packets at rates of 10Gb/s.
We validate our design by showing that our mux/demux layer can validate and switch packets at line rate (up to 14.88 Mpps) on a 10 Gbit port using 1-2 cores, and that a proof-of-concept HTTP server running over a basic userspace TCP outperforms by 18-90% both the same server and nginx running over the kernel's stack.
- M. Allman. Comments on buffer bloat. ACM CCR, 43:31--37, 2013. Google Scholar
Digital Library
- A. Bittau, M. Hamburg, M. Handley, D. Mazieres, and D. Boneh. The case for ubiquitous transport-level encryption. In Proc. USENIX Security Symposium, Aug 2010. Google ScholarDigital Library
- K. Cho, K. Mitsuya, and A. Kato. Traffic data repository at the WIDE project. USENIX ATC, 2000. Google ScholarDigital Library
- P. Druschel and G. Banga. Lazy receiver processing (lrp): A network subsystem architecture for server systems. In Proc. USENIX OSDI, Oct. 1996. Google ScholarDigital Library
- T. Eicken, A. Basu, V. Buch, and W. Vogels. U-Net: A User-Level Network Interface for Parallel and Distributed Computing. In Proc. ACM SOSP, pages 40--53, 1995. Google ScholarDigital Library
- F. Fusco and L. Deri. High speed network traffic analysis with commodity multi-core systems. In Proc. ACM IMC, pages 218--224, 2010. Google ScholarDigital Library
- G. Ganger, D. Engler, M. Kaashoek, H. Briceno, R. Hunt, and T. Pinckney. Fast and flexible application-level networking on exokernel systems. ACM ToCS, 20(1):49--83, 2002. Google ScholarDigital Library
- GitHub. Modern HTTP benchmarking tool. https://github.com/wg/wrk, July 2013.Google Scholar
- S. Han, S. Marshall, B. Chun, and S. Ratnasamy. Megapipe: A new programming interface for scalable network i/o. In Proc. USENIX OSDI, Oct. 2012. Google ScholarDigital Library
- M. Honda, Y. Nishida, C. Raiciu, A. Greenhalgh, M. Handley, and H. Tokuda. Is it Still Possible to Extend TCP? In Proc. ACM IMC, pages 181--192, 2011. Google ScholarDigital Library
- Intel Open Source Technology Center. Intel DPDK vSwitch. https://01.org/packet-processing/intelo-ovdk, 2014.Google Scholar
- V. Jacobson, R. Braden, and D. Borman. TCP Extensions for High Performance. RFC 1323, May. 1992. Google ScholarDigital Library
- A. Kantee. Environmental Independence: BSD Kernel TCP/IP in Userspace. AsiaBSDCon, 2009.Google Scholar
- C. Maeda and B. Bershad. Networking performance for microkernels. In Proc. IEEE WOS, pages 154--159, 1992.Google ScholarCross Ref
- C. Maeda and B. Bershad. Protocol service decomposition for high-performance networking. In Proc. ACM SOSP, pages244--255, 1993. Google ScholarDigital Library
- M. Mathis, J. Mahdavi, S. Floyd, and A. Romanow. TCP Selective Acknowledgment Options. RFC 2018, Oct. 1996. Google ScholarDigital Library
- B. Penoff, A. Wagner, M. Tuxen, and I. Rungeler. Portable and Performant Userspace SCTP Stack. In Proc. IEEE ICCCN, pages 1--9, 2012.Google ScholarCross Ref
- D. Porter, S. Boyd-Wickizer, J. Howell, R. Olinsky, and G. Hunt. Rethinking the library os from the top down. In Proc. ACM ASPLOS, Mar. 2011. Google ScholarDigital Library
- S. Radhakrishnan, Y. Cheng, J. Chu, A. Jain, and B. Raghavan. Tcp fast open. In Proc. ACM CoNEXT, December 2011. Google ScholarDigital Library
- C. Raiciu, C. Paasch, S. Barre, A. Ford, M. Honda, F. Duchene, O. Bonaventure, and M. Handley. How Hard Can It Be? Designing and Implementing a Deployable Multipath TCP. In Proc. USENIX NSDI, 2012. Google ScholarDigital Library
- I. Rhee and L. Xu. Cubic: A new tcp-friendly high-speed tcp variant. Proc. PFLDNeT, 2005.Google Scholar
- L. Rizzo. netmap: a novel framework for fast packet I/O. In Proc. USENIX ATC, 2012. Google ScholarDigital Library
- L. Rizzo and G. Lettieri. Vale: a switched ethernet for virtual machines. In Proc. ACM CoNEXT, December 2012. Google ScholarDigital Library
- SolarFlare. OpenOnLoad. http://www.openonload.org, 2013.Google Scholar
- K. Tan, J. Song, Q. Zhang, and M. Sridharan. A compound tcp approach for high-speed and long distance networks. In Proc. IEEE INFOCOM, pages 1--12, 2006.Google ScholarCross Ref
- C. Thekkath, T. Nguyen, E. Moy, and E. Lazowska. Implementing network protocols at user level. IEEE/ACM ToN, 1(5):554--565, 1993. Google ScholarDigital Library
- Wikipedia. QUIC. http://en.wikipedia.org/wiki/QUIC, 2014.Google Scholar
- D. Wischik, C. Raiciu, A. Greenhalgh, and M. Handley. Design, implementation and evaluation of congestion control for multipath TCP. In Proc. USENIX NSDI, 2011. Google ScholarDigital Library
Index Terms
- Rekindling network protocol innovation with user-level stacks
Recommendations
Non-Renegable Selective Acknowledgments (NR-SACKs) for SCTP
ICNP '08: Proceedings of the 2008 IEEE International Conference on Network ProtocolsIn both TCP and SCTP, selectively acked (SACKed) out-of-order data is implicitly renegable; that is, the receiver can later discard SACKed data. The possibility of reneging forces the transport sender to maintain copies of SACKed data in the send buffer ...
Protocol design for scalable and adaptive multicast for group communications
ICNP '08: Proceedings of the 2008 IEEE International Conference on Network ProtocolsCurrently, IP multicast and Explicit Multi-Unicast (Xcast) are two approaches for multicast communications. IP multicast is designed for large groups but is not scalable in terms of the group number because every router in a multicast tree needs to ...
TCP Westwood(+) protocol implementation in ns-3
SimuTools '13: Proceedings of the 6th International ICST Conference on Simulation Tools and TechniquesThe poor performance of conventional TCP protocols in error prone channels is a well studied issue. Numerous optimizations to TCP to address this problem have been proposed. TCP Westwood is one such protocol engineered for use in wireless networks and ...
Comments