Performance characteristics of the SPEC OMP2001 benchmarks

ABSTRACT

Parallel computing is becoming mainstream with the advent of general purpose cost effective Shared-memory Multiprocessor (SMP) systems. At the same time, new developments in parallel programming environments allow more rapid and efficient programming of these systems. To this end, OpenMP has emerged as a flexible and fairly comprehensive set of compiler directives, library routines, and environment variables to facilitate parallel programming of SMP systems in Fortran and C/C++. The Standard Performance Evaluation Corporation (SPEC) has created a benchmark suite of eleven applications, named SPEC OMP2001, to be used for the performance evaluation and comparison of moderate size SMP systems. Each of the benchmarks in SPEC OMP2001 is either automatically or manually parallelized using OpenMP directives. In this paper, we present basic static and runtime characteristics of these benchmarks. We present data gathered using high resolution timers and the hardware counters available on our SMP system. We explain some of the benchmark performance characteristics with measured data and with a quantitative model.

REFERENCES

• 1: Vishal Aslot. Performance Characterization of the SPEC OMP2001 Benchmarks. Master's thesis, Purdue University, 2001.
• 2: Vishal Aslot , Max J. Domeika , Rudolf Eigenmann , Greg Gaertner , Wesley B. Jones , Bodo Parady, SPEComp: A New Benchmark Suite for Measuring Parallel Computer Performance, Proceedings of the International Workshop on OpenMP Applications and Tools: OpenMP Shared Memory Parallel Programming, p.1-10, July 30-31, 2001
• 3: M. Berry and et. al. The Perfect Club Benchmarks: Effective Performance Evaluation of Supercomputers. Int'l. Journal of Supercomputer Applications, 3(3):5-40, Fall 1989.
• 4: David E. Culler , Anoop Gupta , Jaswinder Pal Singh, Parallel Computer Architecture: A Hardware/Software Approach, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1997
• 5: Roger Hockney , Michael Berry, Public international benchmarks for parallel computers: PARKBENCH committee: Report-1, Scientific Programming, v.3 n.2, p.100-146, Summer 1994
• 6: Seon Wook Kim and Rudolf Eigenmann. Where Does the Speedup Go: Quantitative Modeling of Performance Losses in Shared-Memory Programs. Parallel Processing Letters, 10(2 and 3):227-238, 2000.
• 7: S. Mukherjee, S. Reinhardt, B. Falsafi, M. Litzkow, S Huss-Lederman, M. Hill, J. Larus, and D. Wood. Wisconsin Wind Tunnel II: A Fast and Portable Parallel Architecture Simulator. Workshop on Performance Analysis and Its Impact on Design (PAID), July 1997.
• 8: OpenMP Forum, http://www.openmp.org. OpenMP: A Proposed Industry Standard API for Shared Memory Programming, October 1997.
• 9: Vijay S. Pai, Parthasarathy Ranganathan, and Sarita V. Adve. RSIM Reference Manual, Version 1.0. Technical Report 9705, Electrical and Computer Engineering Department, Rice University, July 1997.
• 10: Sun Microsystems Inc. UltraSPARC User's Manual, 1997.
• 11: Steven Cameron Woo , Moriyoshi Ohara , Evan Torrie , Jaswinder Pal Singh , Anoop Gupta, The SPLASH-2 programs: characterization and methodological considerations, Proceedings of the 22nd annual international symposium on Computer architecture, p.24-36, June 22-24, 1995, S. Margherita Ligure, Italy

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.4 Parallel Architectures

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS
      Subjects: Performance attributes

  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming

General Terms:
Experimentation, Languages, Performance