A framework for reducing the cost of instrumented code

ABSTRACT

Instrumenting code to collect profiling information can cause substantial execution overhead. This overhead makes instrumentation difficult to perform at runtime, often preventing many known offline feedback-directed optimizations from being used in online systems. This paper presents a general framework for performing instrumentation sampling to reduce the overhead of previously expensive instrumentation. The framework is simple and effective, using code-duplication and counter-based sampling to allow switching between instrumented and non-instrumented code.

Our framework does not rely on any hardware or operating system support, yet provides a high frequency sample rate that is tunable, allowing the tradeoff between overhead and accuracy to be adjusted easily at runtime. Experimental results are presented to validate that our technique can collect accurate profiles (93-98% overlap with a perfect profile) with low overhead (averaging 6% total overhead with a naive implementation). A Jalape~ no-specific optimization is also presented that reduces overhead further, resulting in an average total overhead of 3%.

References

1. 1.A.-R. Adl-Tabatabai, M. Cierniak, C.-Y. Lueh, V. M. Parikh, and J. M.Stichnoth. Fast, effective code generation in a Just-in- Time Java compiler. In Proceedings of the ACM SIGPLAN'98 Conference onProgramming Language Design and Implementation (PLDI), pages 280-290, Montreal, Canada, 17-19 June 1998. SIGPLAN Notices 33(5), May 1998.]]
2. 2.B. Alpern, C. R. Attanasio, J. J. Barton, M. G. Burke, P. Cheng, J.-D. Choi, A. Cocchi, S. J. Fink, D. Grove, M. Hind, S. F. Hummel, D. Lieber, V. Litvinov, M. F. Mergen, T. Ngo, J. R. Russell, V. Sarkar, M. J. Serrano, J. C. Shepherd, S. E. Smith, V. C. Sreedhar, H. Srinivasan, and J. Whaley. The Jalapeno virtual machine. IBM Systems Journal, 39(1), 2000.]]
3. 3.G. Ammons, T. Ball, and J. Larus. Exploiting hardware performance counters with ow and context sensitive profiling. In SIGPLAN '97 Conf. on Programming Language Design and Implementation, 1997.]]
4. 4.J. M. Andersen, L. M. Berc, J. Dean, S. Ghemawat, M. R. Henzinger, S.-T. A. Leung, R. L. Sites, M. T. Vandevoorde, C. A. Waldspurger, and W. E. Weihl. Continuous profiling: Where have all the cycles gone? Technical Note 1997-016a, Digital Systems Research Center, www.research.digital.com/SRC, Sept. 1997.]]
5. 5.M. Arnold, S. Fink, D. Grove, M. Hind, and P. Sweeney. Adaptive optimization in the Jalape~ no JVM. In ACM Conference on Object-Oriented Programming Systems, Languages, and Applications, Oct. 2000.]]
6. 6.M. Arnold, S. Fink, V. Sarkar, and P. Sweeney. A comparative study of static and dynamic heuristics for inlining. In ACM SIGPLAN Workshop on Dynamic and Adaptive Compilation and Optimization, 2000.]]
7. 7.M. Arnold, M. Hind, and B. G. Ryder. An empirical study of selective optimization. In 13th International Workshop on Languages and Compilers for Parallel Computing, Aug. 2000.]]
8. 8.M. Arnold and P. F.Sweeney. Approximating the calling context tree via sampling. Technical Report RC 21789, IBM T.J. Watson Research Center, July 2000.]]
9. 9.V. Bala, E. Duesterwald, and S. Banerjia. Dynamo: A transparent dynamic optimization system. In SIGPLAN 2000 Conference onProgramming Language Design and Implementation, June 2000.]]
10. 10.T. Ball and J. R. Larus. Optimally profiling and tracing programs. ACM Transactions on Programming Languages and Systems, 16(4):1319-1360, July 1994.]]
11. 11.T. Ball and J. R. Larus. Efficient path profiling. In Proceedings of the 29th Annual International Symposium on Microarchitecture, pages 46-57. ACM Press, 1996.]]
12. 12.S. Baylor, M. Devarakonda, S. Fink, E. Gluzberg, M. Kalantar, P. Muttineni, E. Barsness, R. Arora, R. Dimpsey, and S. Munroe. Java server benchmarks. IBM Systems Journal, 39(1), 2000.]]
13. 13.M. G. Burke, J.-D. Choi, S. Fink, D. Grove, M. Hind, V. Sarkar, M. J. Serrano, V. C. Sreedhar, H. Srinivasan, and J. Whaley. The Jalapeno dynamic optimizing compiler for Java. In ACM 1999 Java Grande Conference, pages 129-141, June 1999.]]
14. 14.M. Burrows, U. Erlingson, S.-T. A. Leung, M. T. Vandevoorde, C. A. Waldspurger, K. Walker, and W. E. Weihl. Efficient and exible value sampling. In Ninth International Conference on Architectural Support for Programming Languages and Operating Systems, Nov. 2000.]]
15. 15.B. Calder, P. Feller, and A. Eustace. Value profiling. In the 30th International Symposium on Microarchitecture, pages 259-269, Dec. 1997.]]
16. 16.B. Calder, P. Feller, and A. Eustace. Value profiling and optimization. Journal of Instruction Level Parallelism, Vol 1, Mar. 1999.]]
17. 17.B. Calder, C. Krintz, S. John, and T. Austin. Cache-conscious data placement. In Proceedings of the Eighth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 139-149, San Jose, California, Oct. 1998. ACM SIGARCH, SIGOPS, SIGPLAN, and the IEEE Computer Society.]]
18. 18.C. Chambers and D. Ungar. Making pure object-oriented languages practical. In ACM Conference on Object-Oriented Programming Systems, Languages, and Applications, pages 1-15, Nov. 1991. SIGPLAN Notices 26(11).]]
19. 19.P. P. Chang, S. A. Mahlke, W. Y. Chen, and W. mei W. Hwu. Profile-guided automatic inline expansion for C programs. Software - Practice and Experience, 22(5):349-369, May 1992.]]
20. 20.T. M. Chilimbi, B. Davidson, and J. R. Larus. Cache-conscious structure definition. In Proceedings of SIGPLAN'99 Conference on Programming Languages Design and Implementation, ACM SIGPLAN Notices, pages 13-24, Atlanta, May 1999. ACM Press.]]
21. 21.T. M. Chilimbi and J. R. Larus. Using generational garbage collection to implement cache-conscious data placement. In Proceedings of the International Symposium on Memory Management (ISMM-98), volume 34, 3 ofACM SIGPLAN Notices, pages 37-48, New York, Oct. 17-19 1999. ACM Press.]]
22. 22.M. Cierniak, G.-Y. Lueh, and J. M. Stichnoth. Practicing JUDO: Java Under Dynamic Optimizations. In SIGPLAN 2000 Conference onProgramming Language Design and Implementation, June 2000.]]
23. 23.T. S. P. E. Corporation. SPEC JVM98 Benchmarks. http://www.spec.org/osg/jvm98, 1998.]]
24. 24.L. P. Deutsch and A. M. Schiffman. Efficient implementation of the Smalltalk-80 system. In 11th Annual ACM Symposium on the Principles of Programming Languages, pages 297-302, Jan. 1984.]]
25. 25.E. Duesterwald and V. Bala. Software profiling for hot path prediction: Less is more. In Ninth International Conference on Architectural Support for Programming Languages and Operating Systems, Nov. 2000.]]
26. 26.P. T. Feller. Value profiling for instructions and memory locations. Masters Thesis CS98-581, University of California, San Diego, Apr. 1998.]]
27. 27.D. Grove, J. Dean, C. Garrett, and C. Chambers. Profileguided receiver class prediction. In ACM Conference on Object- Oriented Programming Systems, Languages, and Applications, pages 108-123, Oct. 1995.]]
28. 28.J. K. Hollingsworth, B. P. Miller, M. J. R. Goncalves, O. Naim, Z. Xu, and L. Zheng. MDL: A language and compiler for dynamic program instrumentation. In Proceedings of the 1997 International Conference on Parallel Architectures and Compilation Techniques (PACT '97), pages 201-212, San Francisco, California, Nov. 10-14, 1997. IEEE Computer Society Press.]]
29. 29.U. Holzle, C. Chambers, and D. Ungar. Debugging optimized code with dynamic deoptimization. In Proceedings of the ACM SIGPLAN'92 Conference on Programming Language Design and Implementation (PLDI), pages 32-43, San Francisco, California, 17-19 June 1992. SIGPLAN Notices 27(7), July 1992.]]
30. 30.U. Holzle and D. Ungar. Reconciling responsiveness with performance in pure object-oriented languages. ACM Transactions on Programming Languages and Systems, 18(4):355-400, July 1996.]]
31. 31.The Java Hotspot performance engine architecture. White paper available at http://java.sun.com/products/hotspot/whitepaper.html, Apr. 1999.]]
32. 32.T. P. Kistler. Continuous Program Optimization. PhD thesis, University of California, Irvine, 1999.]]
33. 33.A. Krall. Efficient JavaVM Just-in-Time compilation. In J.-L. Gaudiot, editor, International Conference on Parallel Architectures and Compilation Techniques, pages 205-212, Oct. 1998.]]
34. 34.S. Sastry, R. Bodik, and J. E. Smith. Rapid profiling via stratified sampling. In To appear in 28th International Symposium on Computer Architecture, 2001.]]
35. 35.T. Suganama, T. Ogasawara, M. Takeuchi, T. Yasue, M. Kawahito, K. Ishizaki, H. Komatsu, and T. Nakatani. Overview of the IBM Java Just-in-Time compiler. IBM Systems Journal, 39(1), 2000.]]
36. 36.O. Traub, S. Schechter, and M. D. Smith. Ephemeral instrumentation for lightweight program profiling. Technical report, Harvard University, 1999.]]
37. 37.D. Viswanathan and S. Liang. Java Virtual Machine Profiler Interface. IBM Systems Journal, 39(1):82-95, 2000.]]
38. 38.VolanoMark 2.1. http://www.volano.com/benchmarks.html.]]
39. 39.J. Whaley. A portable sampling-based profiler for Java virtual machines. In ACM 2000 Java Grande Conference, June 2000.]]
40. 40.B.-S. Yang, S.-M. Moon, S. Park, J. Lee, S. Lee, J. Park, Y. C. Chung, S. Kim, K. Ebcioglu, and E. Altman. LaTTe: A Java VM Just-in-Time compiler with fast and efficient register allocation. In International Conference onParallel Architectures and Compilation Techniques, Oct. 1999.]]

Index Terms

• Software and its engineering

  • Software notations and tools

    • Compilers

      • Runtime environments

      • Source code generation

    • General programming languages

      • Language features

        • Frameworks