Performance characteristics of an adaptive mesh refinement calculation on scalar and vector platforms

ABSTRACT

Adaptive mesh refinement (AMR) is a powerful technique that reduces the resources necessary to solve otherwise intractable problems in computational science. The AMR strategy solves the problem on a relatively coarse grid, and dynamically refines it in regions requiring higher resolution. However, AMR codes tend to be far more complicated than their uniform grid counterparts due to the software infrastructure necessary to dynamically manage the hierarchical grid framework. Despite this complexity, it is generally believed that future multi-scale applications will increasingly rely on adaptive methods to study problems at unprecedented scale and resolution. Recently, a new generation of parallel-vector architectures have become available that promise to achieve extremely high sustained performance for a wide range of applications, and are the foundation of many leadership-class computing systems worldwide. It is therefore imperative to understand the tradeoffs between conventional scalar and parallel-vector platforms for solving AMR-based calculations. In this paper, we examine the LibraryHyperCLaw AMR framework to compare and contrast performance on the Cray X1E, IBM Power3 and Power5, and SGI Altix. To the best of our knowledge, this is the first work that investigates and characterizes the performance of an AMR calculation on modern parallel-vector systems.

REFERENCES

• 1: J. B. Bell , M. S. Day , C. A. Rendleman , S. E. Woosley , M. A. Zingale, Adaptive low Mach number simulations of nuclear flame microphysics, Journal of Computational Physics, v.195 n.2, p.677-694, 10 April 2004 [doi>10.1016/j.jcp.2003.10.035]
• 2: M. J. Berger , P. Colella, Local adaptive mesh refinement for shock hydrodynamics, Journal of Computational Physics, v.82 n.1, p.64-84, May 1989 [doi>10.1016/0021-9991(89)90035-1]
• 3: M. J. Berger and J. Oliger. Adaptive mesh refinement for hyperbolic partial differential equations. Journal of Computational Physics, 53:484--512, 1984.
• 4: Center for Computational~Sciences and Engineering, Lawrence Berkeley National Laboratory. http://seesar.lbl.gov/CCSE.
• 5: Chombo software package for AMR applications design document, September 2003. http://seesar.lbl.gov/anag/chombo/ChomboDesign-1.4.pdf.
• 6: P. Colella. A direct Eulerian MUSCL scheme for gas dynamics. SIAM Journal on Scientific and Statistical Computing, 6:104--117, 1985.
• 7: P. Colella and W. Y. Crutchfield. A parallel adaptive mesh refinement algorithm on the C-90. In Proceedings of the Energy Research Power Users Symposium, July 1994.
• 8: W. Y. Crutchfield. Load balancing irregular algorithms. Technical Report UCRL-JC-107679, Lawrence Livermore National Laboratory, July 1991.
• 9: B. Fryxell, K. Olson, P. Ricker, F. X. Timmes, M. A. Zingale, D. Q. Lamb, P. MacNeice, R. Rosner, J.~W. Truran, and H. Tufo. FLASH: An adaptive mesh hydrodynamics code for modeling astrophysical thermonuclear flashes. The Astrophysical Journal Supplement Series, 131:273--334, 2000.
• 10: J.-F. Haas and B. Sturtevant. Interaction of weak shock waves with cylindrical and spherical gas inhomogeneities. Journal of Fluid Mechanics, 181:41--76, 1987.
• 11: HPC Challenge benchmark. http://icl.cs.utk.edu/hpcc/index.html.
• 12: IPM Homepage. http://www.nersc.gov/projects/ipm.
• 13: ORNL Cray X1 evaluation. http://www.csm.ornl.gov/~dunigan/cray.
• 14: M. Parashar and J. C. Browne. http://www.caip.rutgers.edu/~parashar/DAGH.
• 15: C. A. Rendleman, V. E. Beckner, M. J. Lijewski, W. Y. Crutchfield, and J. B. Bell. Parallelization of structured, hierarchical adaptive mesh refinement algorithms. Computing and Visualization in Science, 3(3):147--157, 2000.
• 16: SAMRAI home page. http://www.llnl.gov/CASC/SAMRAI.
• 17: STREAM home page. http://www.cs.virginia.edu/stream.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Performance attributes

Additional Classification:
  B. Hardware
  B.8 Performance and Reliability
      B.8.2 Performance Analysis and Design Aids

  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

General Terms:
Experimentation, Measurement, Performance

Keywords:
IBM power3 and power4, SGI, altix, cray X1E, high end computing, hyperCLaw framework, integrated performance monitoring