A case study in top-down performance estimation for a large-scale parallel application

ABSTRACT

This work presents a general methodology for estimating the performance of an HPC workload when running on a future hardware architecture. Further, it demonstrates the methodology by estimating the performance of a significant scientific application -- the Gyrokinetic Toroidal Code (GTC) -- when executing on Sun's proposed next-generation petascale computer architecture.For GTC, we identify the important phases of the iteration and perform low-level analysis that includes instruction tracing and component simulations of processor and memory systems. Low-level analysis is complemented with scalability estimates based on modeling MPI, OpenMP and I/O activity in the code. The work's approach permits accurate end-to-end performance projections from the microarchitecture level to the petascale.

REFERENCES

• 1: T.F. Chan and T. Mathew. Domain decomposition algorithms. In Acta Numerica, pages 61--143. Cambridge University Press, 1994.
• 2: Robit Chandra , Leonardo Dagum , Dave Kohr , Dror Maydan , Jeff McDonald , Ramesh Menon, Parallel programming in OpenMP, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2001
• 3: F. Chen. Introduction to Plasma Physics and Controlled Fusion, volume 1. Plenum Publishing Corporation, 2nd edition, January 1984.
• 4: Mee-Chow Chiang , Gurindar S. Sohi, Experience with mean value analysis model for evaluating shared bus, throughput-oriented multiprocessors, Proceedings of the 1991 ACM SIGMETRICS conference on Measurement and modeling of computer systems, p.90-100, May 21-24, 1991, San Diego, California, United States
• 5: Yuan Chou , Brian Fahs , Santosh Abraham, Microarchitecture Optimizations for Exploiting Memory-Level Parallelism, Proceedings of the 31st annual international symposium on Computer architecture, p.76, June 19-23, 2004, München, Germany
• 6: S. Ethier. Performance study of the 3D particle-in-cell code GTC on the Cray X1. Presentation at CUG04, May 2004.
• 7: Rajat P. Garg , Illya Sharapov, Techniques for Optimizing Applications: High Performance Computing, Prentice Hall PTR, Upper Saddle River, NJ, 2001
• 8: Ilya Gluhovsky , Brian O'Krafka, Comprehensive multiprocessor cache miss rate generation using multivariate models, ACM Transactions on Computer Systems (TOCS), v.23 n.2, p.111-145, May 2005 [doi>10.1145/1062247.1062248]
• 9: I. Gluhovsky and D. Vengerov. Nonnegative monotone convex-concave multivariate extrapolation models with application to computer cache rates. to appear in Technometrics.
• 10: William Gropp , Ewing Lusk , Anthony Skjellum, Using MPI (2nd ed.): portable parallel programming with the message-passing interface, MIT Press, Cambridge, MA, 1999
• 11: Donald Gross , Carl M. Harris, Fundamentals of queueing theory (2nd ed.)., John Wiley & Sons, Inc., New York, NY, 1985
• 12: John L. Gustafson, Reevaluating Amdahl's law, Communications of the ACM, v.31 n.5, p.532-533, May 1988 [doi>10.1145/42411.42415]
• 13: Adolfy Hoisie. Performance Prediction: Where the Rubber Meets the Road. Presentation at the Salishan Conference on High Speed Computing, April 2002.
• 14: Adolfy Hoisie. Performance Prediction: Where the Rubber Meets the Road. Presentation at the Salishan Conference on High Speed Computing, April 2002.
• 15: S. Kunkel, R. Eickemeyer, M. Lipasti, T. Mullins, B. O'Krafka, H. Rosenberg, S. Vanderweil, P. Vitale, and L. Whitely. A performance methodology for commercial servers. IBM Journal of Research and Development, 44(6):851--873, 2000.
• 16: Edward D. Lazowska , John Zahorjan , G. Scott Graham , Kenneth C. Sevcik, Quantitative system performance: computer system analysis using queueing network models, Prentice-Hall, Inc., Upper Saddle River, NJ, 1984
• 17: Z. Lin, T. S. Hahm, W. W. Lee, W. M. Tang, and R. B. White. Turbulent transport reduction by zonal flows: Massively parallel simulations. Science, (281):1835, 1998.
• 18: P. Luszczek, J.J. Dongarra, D. Koester, R. Rabenseifner, B. Lucas, J. Kepner, J. McCalpin, D. Bailey, and D. Takahashi. Introduction to the hpc challenge benchmark suite. Technical report, April 2005. Lawrence Berkeley National Laboratory. Paper LBNL-57493.
• 19: R. E. Matick, Comparison of analytic performance models using closed mean-value analysis versus open-queuing theory for estimating cycles per instruction of memory hierarchies, IBM Journal of Research and Development, v.47 n.4, p.495-517, July 2003
• 20: Leonid Oliker , Andrew Canning , Jonathan Carter , John Shalf , Stephane Ethier, Scientific Computations on Modern Parallel Vector Systems, Proceedings of the 2004 ACM/IEEE conference on Supercomputing, p.10, November 06-12, 2004 [doi>10.1109/SC.2004.54]
• 21: Daniel J. Sorin , Vijay S. Pai , Sarita V. Adve , Mary K. Vernon , David A. Wood, Analytic evaluation of shared-memory systems with ILP processors, Proceedings of the 25th annual international symposium on Computer architecture, p.380-391, June 27-July 02, 1998, Barcelona, Spain
• 22: W.M. Tang. Introduction to gyrokinetic theory with applications in magnetic confinement research in plasma physics. Technical report, Princeton Plasma Physics Laboratory, 2005.
• 23: R.B. White and A.H. Boozer. Rapid guiding center calculations. Physics of Plasmas, 2(8):2915--2919, 1995.
• 24: R.W. Wolff. Stochastic Modeling and the Theory of Queues. Prentice-Hall, 1989.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Measurement techniques

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Modeling techniques

  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

General Terms:
Algorithms, Performance, Theory

Keywords:
mean value analysis, particle-in-cell codes, performance analysis, petascale