A measurement-based deployment proposal for IP anycast

ABSTRACT

Despite its growing use in critical infrastructure services, the performance of IP(v4) Anycast and its interaction with IP routing practices is not well understood. In this paper, we present the results of a detailed measurement study of IP Anycast. Our study uses a two-pronged approach. First, using a variant of known latency estimation techniques, we measure the performance of current commercially operational IP Anycast deployments from a large number (>20,000) of vantage points. Second, we deploy our own small-scale anycast service that allows us to perform controlled tests under different deployment and failure scenarios. To the best of our knowledge, our study represents the first large-scale evaluation of existing anycast services and the first evaluation of the behavior of IP Anycast under failure.We find that: (1) IP Anycast, if deployed in an ad-hoc manner, does not offer good latency-based proximity, (2) IP Anycast, if deployed in an ad-hoc manner, does not provide fast failover to clients, (3) IP Anycast typically offers good affinity to all clients with the exception of those that explicitly load balance traffic across multiple providers, (4) IP Anycast, by itself, is not effective in balancing client load across multiple sites. We thus propose and evaluate practical means by which anycast deployments can achieve good proximity, fast failover and control over the distribution of client load. Overall, our results suggest that an IP Anycast service, if deployed carefully, can offer good proximity, load balance, and failover behavior.

REFERENCES

• 1: Abley, J. Hierarchical Anycast for Global Service Distribution. ISC Technical Note ISC-TN-2003-1 www.isc.org/tn/isc-tn-2003-1.html.
• 2: Abley, J. A Software Approach to Distributing Requests for DNS Service Using GNU Zebra, ISC BIND 9, and FreeBSD. In Proc. of USENIX Annual Technical Conference (2004).
• 3: Ballani, H., Ermolinskiy, A., Ratnasamy, S., and Francis, P. An Experiment in Deploying Next Generation Network Protocols. Tech. rep., Cornell University Technical Report, 2006.
• 4: Hitesh Ballani , Paul Francis, Towards a global IP anycast service, Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications, August 22-26, 2005, Philadelphia, Pennsylvania, USA
• 5: Ballani, H., and Francis, P. Understanding IP Anycast. Tech. rep., Cornell University Technical Report, 2006. http://pias.gforge.cis.cornell.edu/unpub/any-measure.pdf.
• 6: Barber, P., Larson, M., Kosters, M., and Toscano, P. Life and Times of J-Root. NANOG 32 meeting, October 2004. http://www.nanog.org/mtg-0410/kosters.html.
• 7: Basturk, E., Haas, R., Engel, R., Kandlur, D., Peris, V., and Saha, D. Using IP Anycast For Load Distribution And Server Location. In Proc. of IEEE Globecom Global Internet Mini Conference (1998).
• 8: Boothe, P., and Bush, R. Anycast Measurements Used to Highlight Routing Instabilities. NANOG 34 meeting, May 2005. http://www.nanog.org/mtg-0505/boothe.html.
• 9: Chang, R., and Lo, M. Inbound traffic engineering for multi-homed AS's using AS path prepending. IEEE Network (2005), 18--25.
• 10: E. Chen , T. Bates, An Application of the BGP Community Attribute in Multi-home Routing, RFC Editor, 1996
• 11: Brent Chun , David Culler , Timothy Roscoe , Andy Bavier , Larry Peterson , Mike Wawrzoniak , Mic Bowman, PlanetLab: an overlay testbed for broad-coverage services, ACM SIGCOMM Computer Communication Review, v.33 n.3, July 2003 [doi>10.1145/956993.956995]
• 12: Colitti, L. Effect of anycast on K-root. DNS-OARC Workshop, July 2005. http://www.ripe.net/info/ncc/presentations/anycast-kroot.pdf.
• 13: Fei, Z., Bhattacharjee, S., Zegura, E. W., and Ammar, M. H. A Novel Server Selection Technique for Improving the Response Time of a Replicated Service. In Proc. of INFOCOM (1998).
• 14: Anja Feldmann , Olaf Maennel , Z. Morley Mao , Arthur Berger , Bruce Maggs, Locating internet routing instabilities, Proceedings of the 2004 conference on Applications, technologies, architectures, and protocols for computer communications, August 30-September 03, 2004, Portland, Oregon, USA
• 15: Freedman, M. J., Lakshminarayanan, K., and Maziéres, D. OASIS: Anycast for Any Service. In Proc. of 3rd USENIX/ACM Symposium on Networked Systems Design and Implementation (2006).
• 16: Greene, B., and McPherson, D. ISP Security: Deploying and Using Sinkholes. NANOG meeting, June 2003. www.nanog.org/mtg-0306/sink.html.
• 17: Krishna P. Gummadi , Stefan Saroiu , Steven D. Gribble, King: estimating latency between arbitrary internet end hosts, Proceedings of the 2nd ACM SIGCOMM Workshop on Internet measurment, November 06-08, 2002, Marseille, France [doi>10.1145/637201.637203]
• 18: Hardy, T. RFC 3258 - Distributing Authoritative Name Servers via Shared Unicast Addresses, April 2002.
• 19: C. Huitema, An Anycast Prefix for 6to4 Relay Routers, RFC Editor, 2001
• 20: Dina Katabi , John Wroclawski, A framework for scalable global IP-anycast (GIA), Proceedings of the conference on Applications, Technologies, Architectures, and Protocols for Computer Communication, p.3-15, August 28-September 01, 2000, Stockholm, Sweden
• 21: D. Kim , D. Meyer , H. Kilmer , D. Farinacci, Anycast Rendevous Point (RP) mechanism using Protocol Independent Multicast (PIM) and Multicast Source Discovery Protocol (MSDP), RFC Editor, 2003
• 22: Balachander Krishnamurthy , Craig Wills , Yin Zhang, On the use and performance of content distribution networks, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, November 01-02, 2001, San Francisco, California, USA [doi>10.1145/505202.505224]
• 23: Craig Labovitz , Abha Ahuja , Abhijit Bose , Farnam Jahanian, Delayed Internet routing convergence, IEEE/ACM Transactions on Networking (TON), v.9 n.3, p.293-306, June 2001 [doi>10.1109/90.929852]
• 24: Lo, S. S. M., and Chang, R. K. C. Active Measurement of the AS Path Prepending Method. In Proc. of ICNP (Poster) (2005).
• 25: Z. Morley Mao , Randy Bush , Timothy G. Griffin , Matthew Roughan, BGP beacons, Proceedings of the 3rd ACM SIGCOMM conference on Internet measurement, October 27-29, 2003, Miami Beach, FL, USA [doi>10.1145/948205.948207]
• 26: Matsunaga, S., Ata, S., Kitamura, H., and Murata, M. Applications of IPv6 Anycasting. draft-ata-ipv6-anycast-app-00, February 2005.
• 27: Miller, K. Deploying IP Anycast. NANOG 29 meeting, 2003. http://www.net.cmu.edu/pres/anycast/.
• 28: C. Partridge , T. Mendez , W. Milliken, Host Anycasting Service, RFC Editor, 1993
• 29: Bruno Quoitin , Cristel Pelsser , Olivier Bonaventure , Steve Uhlig, A performance evaluation of BGP-based traffic engineering, International Journal of Network Management, v.15 n.3, p.177-191, May 2005 [doi>10.1002/nem.559]
• 30: Sylvia Ratnasamy , Scott Shenker , Steven McCanne, Towards an evolvable internet architecture, Proceedings of the 2005 conference on Applications, technologies, architectures, and protocols for computer communications, August 22-26, 2005, Philadelphia, Pennsylvania, USA
• 31: Sarat, S., Pappas, V., and Terzis, A. On the use of Anycast in DNS. Tech. rep., HiNRG, Johns Hopkins University Technical Report, 2004.
• 32: Sandeep Sarat , Vasileios Pappas , Andreas Terzis, On the use of anycast in DNS, Proceedings of the 2005 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, June 06-10, 2005, Banff, Alberta, Canada
• 33: Shaikh, A., Tewari, R., and Agrawal, M. On the Effectiveness of DNS-based Server Selection. In Proc. of IEEE INFOCOM 2001 (2001).
• 34: Neil Spring , Ratul Mahajan , Thomas Anderson, The causes of path inflation, Proceedings of the 2003 conference on Applications, technologies, architectures, and protocols for computer communications, August 25-29, 2003, Karlsruhe, Germany [doi>10.1145/863955.863970]
• 35: Ion Stoica , Daniel Adkins , Shelley Zhuang , Scott Shenker , Sonesh Surana, Internet indirection infrastructure, Proceedings of the 2002 conference on Applications, technologies, architectures, and protocols for computer communications, August 19-23, 2002, Pittsburgh, Pennsylvania, USA
• 36: Bernard Wong , Emin Gün Sirer, ClosestNode.com: an open access, scalable, shared geocast service for distributed systems, ACM SIGOPS Operating Systems Review, v.40 n.1, January 2006 [doi>10.1145/1113361.1113373]
• 37: Ellen W. Zegura , Mostafa H. Ammar , Zongming Fei , Samrat Bhattacharjee, Application-layer anycasting: a server selection architecture and use in a replicated Web service, IEEE/ACM Transactions on Networking (TON), v.8 n.4, p.455-466, Aug. 2000 [doi>10.1109/90.865074]
• 38: Akamai, May 2006. http://www.akamai.com/.
• 39: AS112 Project Home Page, May 2006. www.as112.net.
• 40: CacheFly, May 2006. http://www.cachefly.com.
• 41: CIDR Report, May 2006. http://www.cidr-report.org/.
• 42: Global Server Load Balancing, May 2006. http://www.tenereillo.com/GSLBPageOfShame.htm.
• 43: History of DNS Root Anycast controversy, May 2006. http://www.av8.net/IETF-watch/DNSRootAnycast/History.html.
• 44: Internal Anycast Service deployment, May 2006. http://pias.gforge.cis.cornell.edu/deployment.php.
• 45: ISC F Root-Server, May 2006. http://www.isc.org/index.pl?/ops/f-root/.
• 46: National Lambda Rail, April 2006. www.nlr.net.
• 47: Report on DNS Amplification attacks, May 2006. http://www.circleid.com/posts/report_on_dns_amplification_attacks/.
• 48: Root-Server Technical Operations, May 2006. http://www.root-servers.org/.
• 49: Route Views Project Page, May 2006. www.route-views.org.
• 50: SprintLink's BGP Policy, May 2006. http://www.sprintlink.net/policy/bgp.html.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Measurement techniques

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Performance attributes

General Terms:
Measurement, Performance

Keywords:
BGP, IP anycast, affinity, failover, load distribution, proximity