DI-GRUBER: A Distributed Approach to Grid Resource Brokering

ABSTRACT

Managing usage service level agreements (USLAs) within environments that integrate participants and resources spanning multiple physical institutions is a challenging problem. Maintaining a single unified USLA management decision point over hundreds to thousands of jobs and sites can become a bottleneck in terms of reliability as well as performance. DIGRUBER, an extension to our GRUBER brokering framework, was developed as a distributed grid USLAbased resource broker that allows multiple decision points to coexist and cooperate in real-time. DIGRUBER addresses issues regarding how USLAs can be stored, retrieved, and disseminated efficiently in a large distributed environment. The key question this paper addresses is the scalability and performance of DI-GRUBER in large Grid environments. We conclude that as little as three to five decision points can be sufficient in an environment with 300 sites and 60 VOs, an environment ten times larger than today's Open Science Grid.

REFERENCES

• 1: Catalin Dumitrescu , Ian Foster, Usage Policy-Based CPU Sharing in Virtual Organizations, Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing, p.53-60, November 08-08, 2004 [doi>10.1109/GRID.2004.62]
• 2: 2. Condor Project, Condor-G, www.cs.wisc.edu/condor/, 2002.
• 3: 3. Altair Grid Technologies, LLC, A Batching Queuing System, Software Project, Software Project, 2003.
• 4: 4. Platform Computing Corporation, Administrator's Guide, Version 4.1. February 2001.
• 5: 5. Cluster Resources, Inc., Maui Scheduler, Software Project, 2001-2005.
• 6: 6. Foster, I., et al., "End-to-End Quality of Service for High-end Applications ", Computer Communications, 2004. 27 (14): p. 1375-1388.
• 7: 7. S. Tuecke, et al., "Grid Service Specification".
• 8: Asit Dan , Catalin Dumitrescu , Matei Ripeanu, Connecting client objectives with resource capabilities: an essential component for grid service managent infrastructures, Proceedings of the 2nd international conference on Service oriented computing, November 15-19, 2004, New York, NY, USA [doi>10.1145/1035167.1035177]
• 9: Heiko Ludwig , Asit Dan , Robert Kearney, Cremona: an architecture and library for creation and monitoring of WS-agreents, Proceedings of the 2nd international conference on Service oriented computing, November 15-19, 2004, New York, NY, USA [doi>10.1145/1035167.1035178]
• 10: L. Pearlman , V. Welch , I. Foster , C. Kesselman , S. Tuecke, A Community Authorization Service for Group Collaboration, Proceedings of the 3rd International Workshop on Policies for Distributed Systems and Networks (POLICY'02), p.50, June 05-07, 2002
• 11: 11. Avery, P. and I. Foster, "The GriPhyN Project: Towards Petascale Virtual Data Grids", 2001.
• 12: 12. Chervenak, A., et al., "The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Data Sets", J. Network and Computer Applications, 2001(23): p. 187-200.
• 13: Catalin Dumitrescu , Ioan Raicu , Matei Ripeanu , Ian Foster, DiPerF: An Automated DIstributed PERformance Testing Framework, Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing, p.289-296, November 08-08, 2004 [doi>10.1109/GRID.2004.21]
• 14: Matei Ripeanu , Ian Foster, A Decentralized, Adaptive Replica Location Mechanism, Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing, p.24, July 24-26, 2002
• 15: 15. Gimpel, H., et al., "PANDA: Specifying Policies for Automated Negotiations of Service Contracts ", in the 1st International Conference on Service Oriented Computing. 2003. Trento, Italy.
• 16: 16. Ranganathan, K. and I. Foster, "Simulation Studies of Computation and Data Scheduling Algorithms for Data Grids", Journal of Grid Computing, 2003, 1(1).
• 17: Kavitha Ranganathan , Ian Foster, Decoupling Computation and Data Scheduling in Distributed Data-Intensive Applications, Proceedings of the 11th IEEE International Symposium on High Performance Distributed Computing, p.352, July 24-26, 2002
• 18: I. Foster , J. Gieraltowski , S. Gose , N. Maltsev , E. May , A. Rodriguez , D. Sulakhe , A. Vaniachine , J. Shank , S. Youssef , D. Adams , R. Baker , W. Deng , J. Smith , D. Yu , I. Legrand , S. Singh , C. Steenberg , Y. Xia , A. Afaq , E. Berman , J. Annis , L. A. T. Bauerdick , M. Ernst , I. Fisk , L. Giacchetti , G. Graham , A. Heavey , J. Kaiser , N. Kuropatkin , R. Pordes , V. Sekhri , J. Weigand , Y. Wu , K. Baker , L. Sorrillo , J. Huth , M. Allen , L. Grundhoefer , J. Hicks , F. Luehring , S. Peck , R. Quick , S. Simms , G. Fekete , J. vandenBerg , K. Cho , K. Kwon , D. Son , H. Park , S. Canon , K. Jackson , D. E. Konerding , J. Lee , D. Olson , I. Sakrejda , B. Tierney , M. Green , R. Miller , J. Letts , T. Martin , D. Bury , C. Dumitrescu , D. Engh , R. Gardner , M. Mambelli , Y. Smirnov , J. Voeckler , M. Wilde , Y. Zhao , X. Zhao , P. Avery , R. Cavanaugh , B. Kim , C. Prescott , J. Rodriguez , A. Zahn , S. McKee , C. Jordan , J. Prewett , T. Thomas , H. Severini , B. Clifford , E. Deelman , L. Flon , C. Kesselman , G. Mehta , N. Olomu , K. Vahi , K. De , P. McGuigan , M. Sosebee , D. Bradley , P. Couvares , A. De Smet , C. Kireyev , E. Paulson , A. Roy , S. Koranda , B. Moe , B. Brown , P. Sheldon, The Grid2003 Production Grid: Principles and Practice, Proceedings of the 13th IEEE International Symposium on High Performance Distributed Computing, p.236-245, June 04-06, 2004 [doi>10.1109/HPDC.2004.36]
• 19: 19. Legrand, I. C., et al., "MonALISA: A Distributed Monitoring Service Architecture", in Computing in High Energy Physics. 2003. La Jolla, CA.
• 20: 20. Kay, J. and P. Lauder, "A Fair Share Scheduler", University of Sydney, AT&T Bell Labs, 1998.
• 21: 21. Henry, G. J., "A Fair Share Scheduler", AT&T Bell Laboratory Technical Journal, October 1984, 3(8).
• 22: 22. I., In, J., P. Avery, R. Cavanaugh, and S. Ranka, "Policy Based Scheduling for Simple Quality of Service in Grid Computing", in International Parallel & Distributed Processing Symposium (IPDPS), April'04, New Mexico.
• 23: 23. Buyya, R., "GridBus: A Economy-based Grid Resource Broker", The University of Melbourne: Melbourne, 2004.
• 24: C. L. Dumitrescu , I. Foster, GangSim: a simulator for grid scheduling studies, Proceedings of the Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2, p.1151-1158, May 09-12, 2005
• 25: 25. Dumitrescu, C., Foster, I., "GRUBER: A Grid Resource SLA Broker", in Euro-Par, Portugal, September 2005.
• 26: Andy Bavier , Mic Bowman , Brent Chun , David Culler , Scott Karlin , Steve Muir , Larry Peterson , Timothy Roscoe , Tammo Spalink , Mike Wawrzoniak, Operating system support for planetary-scale network services, Proceedings of the 1st conference on Symposium on Networked Systems Design and Implementation, p.19-19, March 29-31, 2004, San Francisco, California
• 27: Brent Chun , David Culler , Timothy Roscoe , Andy Bavier , Larry Peterson , Mike Wawrzoniak , Mic Bowman, PlanetLab: an overlay testbed for broad-coverage services, ACM SIGCOMM Computer Communication Review, v.33 n.3, July 2003 [doi>10.1145/956993.956995]
• 28: 28. Voeckler, J., "Euryale: Yet Another Concrete Planner", in Virtual Data Workshop, May 18th, 2004.
• 29: 29. M. Humphrey, G. Wasson, K. Jackson, J. Boverhof, M. Rodriguez, Joe Bester, J. Gawor, S. Lang, I. Foster, S. Meder, S. Pickles, and M. McKeown, "State and Events for Web Services: A Comparison of Five WS-Resource Framework and WS-Notification Implementations", 4th IEEE International Symposium on High Performance Distributed Computing (HPDC-14), Research Triangle Park, NC, 24-27 July 2005.

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.4 System Management

Additional Classification:
  C. Computer Systems Organization
  C.2 COMPUTER-COMMUNICATION NETWORKS
      C.2.4 Distributed Systems

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.2 Installation Management
          Subjects: Pricing and resource allocation

General Terms:
Design, Experimentation, Measurement, Performance