Jozo Dujmović
ABSTRACT
In this tutorial, we describe techniques for automatic generation of benchmark and test workloads. Generated programs have adjustable parameters that are used to select the program size and structure, as well as the relative frequencies of basic operations (or program modules) that characterize the workload.
- Asanovic, K. et al., A View of the Parallel Computing Landscape. CACM, Vol. 52, No. 10, pp. 56--67, 2009. Google Scholar
Digital Library
- Dujmovic, J.J., Evaluation and Design of Benchmark Suites. Chapter 12 in Advanced Computer Performance Modeling and Simulation, Edited by K. Bagchi, J. Walrand, and G.W. Zobrist, Gordon and Breach, 1998, pp. 278--323.Google Scholar
- Dujmovic, J.J., Universal Benchmark Suites -- A Quantitative Approach to Benchmark Design. Performance Evaluation and Benchmarking with Realistic Applications, Edited by Rudolf Eigenmann, MIT Press, pp. 257--287, 2000. Google ScholarDigital Library
- Dujmovic, J.J and I. Dujmović, Evolution and Evaluation of SPEC Benchmarks. Performance Evaluation Review, Vol. 26, No. 3, pp. 2--9, 1998. Google ScholarDigital Library
- Dujmovic, J.J., E. Horvath, and H. Lew, Benchmark Program Generator for Compiler Performance Analysis. CMG99 Proceedings, Vol. 2, pp.838--847, 1999.Google Scholar
- Dujmovic, J.J. and Howard Lew, A Method for Generating Benchmark Programs. CMG 2000 Proceedings, Vol. 1, pp. 379--388, 2000.Google Scholar
- Dujmovic, J.J. and Murat Cengiz, A Kernel Library for Benchmark Program Generators. CMG 2003 Proceedings, Vol. 2 pp. 609--618, 2003.Google Scholar
- Gray, J. What Next? A Dozen Information-Technology Research Goals. Microsoft Research Technical Report MS-TR-99-50, 1999.Google Scholar
- Herder, C. and J.J. Dujmovic, Workload Characterization Using Metrics Based on Instruction Grouping. International Journal of Computer and Information Sciences, Vol. 5, No. 1, March 2004.Google Scholar
- Jain, R., The Art of Computer Systems Performance Analysis. John Wiley, 1991.Google Scholar
- Kubiatowicz, J., Introduction to Parallel Architectures. http://parlab.eecs.berkeley.edu/bootcampagenda, 2009Google Scholar
- Lew, H. and J.J. Dujmovic, Performance Evaluation and Comparison of C++ Compilers. CMG 2000 Proceedings, Vol. 1, pp. 241--252, 2000.Google Scholar
- Mirghafori, N., M. Jacoby, and D. Patterson, Truth in SPEC Benchmarks. Computer Architecture News, Vol. 23, No. 5, pp. 34--42, December 1995. Google ScholarDigital Library
- Nelder, J.A. and Mead, R. (1965), "A simplex method for function minimization", Comput. J., 7, pp. 308--313. (See http://www.scholarpedia.org/article/Nelder-Mead_algorithm)Google Scholar
- SPEC, The Current Benchmarks. http://www.spec.org/osg/, 2009.Google Scholar
- Wikipedia, Moore's law. http://en.wikipedia.org/wiki/Moore's_lawGoogle Scholar
Index Terms
- Automatic generation of benchmark and test workloads
Recommendations
A Benchmark Characterization of the EEMBC Benchmark Suite
Benchmark consumers expect benchmark suites to be complete, accurate, and consistent, and benchmark scores serve as relative measures of performance. However, it is important to understand how benchmarks stress the processors that they aim to test. This ...
The LDBC Social Network Benchmark: Interactive Workload
SIGMOD '15: Proceedings of the 2015 ACM SIGMOD International Conference on Management of DataThe Linked Data Benchmark Council (LDBC) is now two years underway and has gathered strong industrial participation for its mission to establish benchmarks, and benchmarking practices for evaluating graph data management systems. The LDBC introduced a ...
Comments