Evaluating the correspondence between training and reference workloads in SPEC CPU2006

Abstract

Profile feedback (sometimes called Feedback Directed Optimisation FDO) is a useful technique for providing the compiler with additional information about runtime program flow. The compiler is able to use this information to make optimisation decisions that improve the way the code is laid out in memory or determine which routines are inlined, and hence improve the performance of the application.

The use of profile feedback requires the code to be compiled twice. The first time the compiler generates an instrumented version of the application. This instrumented version is then run on one or more 'representative' training workloads to gather profile data. This profile data contains information such as how many times each routine is executed and how frequently each branch is taken. The second pass through the compiler uses this information to make more enlighted optimisation decisions.

The quality of the training data impacts the ability of the compiler to do the best job that it can. This paper discusses a method of assessing the similarity of the training workload to the reference workload, and applies this methodology to evaluate the training workloads in the SPEC CPU2006 benchmark suite.

References

1. "SPEC CPU2006 Benchmark Suite," http://www.spec.org/cpu2006/.
2. D. Gove and L. Spracklen, "Evaluating Whether the Training Data Provided for Profile Feedback is a Realistic Control Flow for the Real Workload," in Proc. SPEC Benchmark Workshop http://www.spec.org/workshops/2006/.
3. D. Gove, "Selecting Representative Training Workloads for Profile Feedback Through Coverage and Branch Analysis," http://developers.sun.com/sunstudio/artides/coverage.html.
4. "Cool Tools -- Binary Improvement Tool (BIT)," http://cooltools.sunsource.net/bit/.
5. G. Langdale and T. Gross, "Evaluating the Relationship Between the Usefulness and Accuracy of Profiles," in Proc. Workshop on Duplicating, Deconstructing, and Debunking, 2003.
6. S. McFarling, "Reality-based optimization," in Proc. Intl. Symp. on Code generation and optimization, 2003, pp. 59--68.
7. J. A. Fisher and S. M. Freudenberger, "Predicting conditional branch directions from previous runs of a program," in Proc. Intl. Conf. on Architectural support for programming languages and operating systems, 1992, pp. 85--95.
8. A. Phansalkar, A. Joshi, L. Eeckhout, and L. K. John, "Measuring Program Similarity: Experiments with SPEC CPU Benchmark Suites," in Proc. Intl. Symp. Performance Analysis of Systems and Software, 2005.
9. W. Hsu, H. Chen, P. Yew, and D. Chen, "On the Predictability of Program Behavior Using Different Input Data Sets," in Proc. Workshop of Interaction between Compilers and Computer Architectures, 2002.
10. L. Eeckhout, H. Vandierendonck, and K. De Bosschere, "Workload Design: Selecting Representative Program-Input Pairs," in Proc. Intl. Conf. on Parallel Architectures and Compilation Techniques, 2002, pp. 83--94.

Index Terms

• Computer systems organization

  • Architectures

    • Parallel architectures

  • Dependable and fault-tolerant systems and networks

• General and reference

  • Cross-computing tools and techniques

    • Evaluation

    • Metrics

    • Performance

• Networks

  • Network performance evaluation