skip to main content
research-article

Ananta: cloud scale load balancing

Published: 27 August 2013 Publication History

Abstract

Layer-4 load balancing is fundamental to creating scale-out web services. We designed and implemented Ananta, a scale-out layer-4 load balancer that runs on commodity hardware and meets the performance, reliability and operational requirements of multi-tenant cloud computing environments. Ananta combines existing techniques in routing and distributed systems in a unique way and splits the components of a load balancer into a consensus-based reliable control plane and a decentralized scale-out data plane. A key component of Ananta is an agent in every host that can take over the packet modification function from the load balancer, thereby enabling the load balancer to naturally scale with the size of the data center. Due to its distributed architecture, Ananta provides direct server return (DSR) and network address translation (NAT) capabilities across layer-2 boundaries. Multiple instances of Ananta have been deployed in the Windows Azure public cloud with combined bandwidth capacity exceeding 1Tbps. It is serving traffic needs of a diverse set of tenants, including the blob, table and relational storage services. With its scale-out data plane we can easily achieve more than 100Gbps throughput for a single public IP address. In this paper, we describe the requirements of a cloud-scale load balancer, the design of Ananta and lessons learnt from its implementation and operation in the Windows Azure public cloud.

References

[1]
A10 Networks AX Series. http://www.a10networks.com.
[2]
Aryaka WAN Optimization. http://www.aryaka.com.
[3]
Amazon Web Services. http://aws.amazon.com.
[4]
Microsoft Windows Azure. http://www.windowsazure.com.
[5]
T. Benson, A. Akella, A. Shaikh, and S. Sahu. CloudNaaS: A Cloud Networking Platform for Enterprise Applications. In Symposium on Cloud Computing, 2011.
[6]
M. Dobrescu, N. Egi, K. Argyraki, B.-G. Chun, K. Fall, G. Iannaccone, A. Knies, M. Manesh, and S. Ratnasamy. RouteBricks: exploiting parallelism to scale software routers. In SOSP, 2009.
[7]
N. Egi, A. Greenhalgh, M. Handley, M. Hoerdt, F. Huici, and L. Mathy. Towards high performance virtual routers on commodity hardware. In CoNEXT, 2008.
[8]
Embrane. http://www.embrane.com.
[9]
F5 BIG-IP. http://www.f5.com.
[10]
Google Cloud Platform. http://cloud.google.com.
[11]
A. Greenberg et al. VL2: A scalable and flexible data center network. In SIGCOMM, 2009.
[12]
HA Proxy Load Balancer. http://haproxy.1wt.eu.
[13]
A. Heffernan. RFC 2385: Protection of BGP Sessions via the TCP MD5 Signature Option, 1998.
[14]
L. Lamport. The Part-Time Parliament. ACM TOCS, 16(2):133--169, May 1998.
[15]
LoadBalancer.org Virtual Appliance. http://www.load-balancer.org.
[16]
N. Mckeown, T. Anderson, H. Balakrishnan, G. M. Parulkar, L. L. Peterson, J. Rexford, S. Shenker, and J. S. Turner. OpenFlow: Enabling Innovation in Campus Networks. In SIGCOMM, 2008.
[17]
NetScalar VPX Virtual Appliance. http://www.citrix.com.
[18]
C. Perkins. RFC 2003: IP Encapsulation within IP, 1996.
[19]
Rackspace. http://www.rackspace.com.
[20]
Y. Rekhter, T. Li, and S. Hares. RFC 4271: A Border Gateway Protocol 4 (BGP-4), 2006.
[21]
Riverbed Virtual Steelhead. http://www.riverbed.com.
[22]
Receive Side Scaling. http://msdn.microsoft.com.
[23]
V. Sekar, S. Ratnasamy, M. K. Reiter, N. Egi, and G. Shi. The Middlebox Manifesto: Enabling Innovation in Middlebox Deployment. In HotNets, 2011.
[24]
J. Sherry, S. Hasan, C. Scott, A. Krishnamurthy, S. Ratnasamy, and V. Sekar. Making Middleboxes Someone Else's Problem: Network Processing as a Cloud Service. In SIGCOMM, 2012.
[25]
D. Thaler and C. Hopps. RFC 2991: Multipath Issues in Unicast and Multicast Next-Hop Selection, 2000.
[26]
H. Uppal, V. Brajkovic, D. Brandon, T. Anderson, and A. Krishnamurthy. ETTM: A Scalable Fault Tolerant Network Manager. In NSDI, 2011.
[27]
Vyatta Software Middlebox. http://www.vyatta.com.
[28]
R. Wang, D. Butnariu, and J. Rexford. OpenFlow-Based Server Load Balancing GoneWild. In Hot-ICE, 2011.
[29]
M. Welsh, D. Culler, and E. Brewer. SEDA: An Architecture for Well-Conditioned, Scalable Internet Services. In SOSP, 2001.
[30]
Windows Filtering Platform. http://msdn.microsoft.com.
[31]
ZScalar Cloud Security. http://www.zscalar.com.

Cited By

View all
  • (2025)Verifying Network-level Properties for Large-scale Networks with Header Transformations in RealtimeJournal of Information Processing10.2197/ipsjjip.33.4133(41-54)Online publication date: 2025
  • (2025)Rethinking Cost-Efficient VM Scheduling on Public Edge Platforms: A Service Provider’s PerspectiveIEEE Transactions on Mobile Computing10.1109/TMC.2024.348808224:3(1846-1858)Online publication date: Mar-2025
  • (2024)Diagnosing application-network anomalies for millions of IPs in production cloudsProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692046(885-899)Online publication date: 10-Jul-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGCOMM Computer Communication Review
ACM SIGCOMM Computer Communication Review  Volume 43, Issue 4
October 2013
595 pages
ISSN:0146-4833
DOI:10.1145/2534169
Issue’s Table of Contents
  • cover image ACM Conferences
    SIGCOMM '13: Proceedings of the ACM SIGCOMM 2013 conference on SIGCOMM
    August 2013
    580 pages
    ISBN:9781450320566
    DOI:10.1145/2486001
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 August 2013
Published in SIGCOMM-CCR Volume 43, Issue 4

Check for updates

Author Tags

  1. distributed systems
  2. server load balancing
  3. software defined networking

Qualifiers

  • Research-article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)340
  • Downloads (Last 6 weeks)42
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Verifying Network-level Properties for Large-scale Networks with Header Transformations in RealtimeJournal of Information Processing10.2197/ipsjjip.33.4133(41-54)Online publication date: 2025
  • (2025)Rethinking Cost-Efficient VM Scheduling on Public Edge Platforms: A Service Provider’s PerspectiveIEEE Transactions on Mobile Computing10.1109/TMC.2024.348808224:3(1846-1858)Online publication date: Mar-2025
  • (2024)Diagnosing application-network anomalies for millions of IPs in production cloudsProceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference10.5555/3691992.3692046(885-899)Online publication date: 10-Jul-2024
  • (2024)BeaverProceedings of the 18th USENIX Conference on Operating Systems Design and Implementation10.5555/3691938.3691951(233-249)Online publication date: 10-Jul-2024
  • (2024)POSEIDONProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691885(1083-1099)Online publication date: 16-Apr-2024
  • (2024)LuoShenProceedings of the 21st USENIX Symposium on Networked Systems Design and Implementation10.5555/3691825.3691874(877-892)Online publication date: 16-Apr-2024
  • (2024)Unraveling Physical Space Limits for LEO Network ScalabilityProceedings of the 23rd ACM Workshop on Hot Topics in Networks10.1145/3696348.3696885(43-51)Online publication date: 18-Nov-2024
  • (2024) Tango : Harmonious Optimization for Mixed Services in Kubernetes-Based Edge Clouds IEEE Transactions on Services Computing10.1109/TSC.2024.3479926(1-14)Online publication date: 2024
  • (2024)SFCache: Hybrid NF Synthesization in Runtime With Rule-Caching in Programmable SwitchesIEEE Transactions on Network and Service Management10.1109/TNSM.2024.339014021:4(4613-4624)Online publication date: 1-Aug-2024
  • (2024)Proactive Telemetry in Large-Scale Multi-Tenant Cloud Overlay NetworksIEEE/ACM Transactions on Networking10.1109/TNET.2024.338178632:4(3002-3017)Online publication date: Aug-2024
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media