Vertical profiling: understanding the behavior of object-priented applications

ABSTRACT

Object-oriented programming languages provide a rich set of features that provide significant software engineering benefits. The increased productivity provided by these features comes at a justifiable cost in a more sophisticated runtime system whose responsibility is to implement these features efficiently. However, the virtualization introduced by this sophistication provides a significant challenge to understanding complete system performance, not found in traditionally compiled languages, such as C or C++. Thus, understanding system performance of such a system requires profiling that spans all levels of the execution stack, such as the hardware, operating system, virtual machine, and application.

In this work, we suggest an approach, called <i>vertical profiling</i>, that enables this level of understanding. We illustrate the efficacy of this approach by providing deep understandings of performance problems of Java applications run on a VM with vertical profiling support. By incorporating vertical profiling into a programming environment, the programmer will be able to understand how their program interacts with the underlying abstraction levels, such as application server, VM, operating system, and hardware.

REFERENCES

• 1: Dong H. Ahn , Jeffrey S. Vetter, Scalable analysis techniques for microprocessor performance counter metrics, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-16, November 16, 2002, Baltimore, Maryland
• 2: Bowen Alpern , C. R. Attanasio , Anthony Cocchi , Derek Lieber , Stephen Smith , Ton Ngo , John J. Barton , Susan Flynn Hummel , Janice C. Sheperd , Mark Mergen, Implementing jalapeño in Java, ACM SIGPLAN Notices, v.34 n.10, p.314-324, Oct. 1999
• 3: Jennifer M. Anderson , Lance M. Berc , Jeffrey Dean , Sanjay Ghemawat , Monika R. Henzinger , Shun-Tak A. Leung , Richard L. Sites , Mark T. Vandevoorde , Carl A. Waldspurger , William E. Weihl, Continuous profiling: where have all the cycles gone?, ACM Transactions on Computer Systems (TOCS), v.15 n.4, p.357-390, Nov. 1997 [doi>10.1145/265924.265925]
• 4: Matthew Arnold , Stephen Fink , David Grove , Michael Hind , Peter F. Sweeney, Adaptive optimization in the Jalapeño JVM, ACM SIGPLAN Notices, v.35 n.10, p.47-65, Oct. 2000
• 5: Rudolf Berrendorf, Heinz Ziegler, and Bernd Mohr. PCL - the performance counter library. http://www.fz-juelich.de/zam/PCL.
• 6: Stephen M. Blackburn , Perry Cheng , Kathryn S. McKinley, Oil and Water? High Performance Garbage Collection in Java with MMTk, Proceedings of the 26th International Conference on Software Engineering, p.137-146, May 23-28, 2004
• 7: S. Browne , J. Dongarra , N. Garner , K. London , P. Mucci, A scalable cross-platform infrastructure for application performance tuning using hardware counters, Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM), p.42-es, November 04-10, 2000, Dallas, Texas, United States
• 8: Morris Marden, An Architectural Evaluation of Java TPC-W, Proceedings of the 7th International Symposium on High-Performance Computer Architecture, p.229, January 20-24, 2001
• 9: Standard Performance Evaluation Corporation. SPECjbb2000 (Java Business Benchmark). http://www.spec.org/jbb2000.
• 10: Standard Performance Evaluation Corporation. Specjvm98 benchmarks. http://www.spec.org/jvm98.
• 11: Luiz A. de Rose , Daniel A. Reed, SvPablo: A Multi-Language Architecture-Independent Performance Analysis System, Proceedings of the 1999 International Conference on Parallel Processing, p.311, September 21-24, 1999
• 12: Luiz De Rose, The Hardware Performance Monitor Toolkit, Proceedings of the 7th International Euro-Par Conference Manchester on Parallel Processing, p.122-131, August 28-31, 2001
• 13: Sylvia Dieckmann , Urs Hölzle, A Study of the Allocation Behavior of the SPECjvm98 Java Benchmark, Proceedings of the 13th European Conference on Object-Oriented Programming, p.92-115, June 14-18, 1999
• 14: Bruno Dufour , Karel Driesen , Laurie Hendren , Clark Verbrugge, Dynamic metrics for java, Proceedings of the 18th annual ACM SIGPLAN conference on Object-oriented programing, systems, languages, and applications, October 26-30, 2003, Anaheim, California, USA
• 15: Lieven Eeckhout , Andy Georges , Koen De Bosschere, How java programs interact with virtual machines at the microarchitectural level, Proceedings of the 18th annual ACM SIGPLAN conference on Object-oriented programing, systems, languages, and applications, October 26-30, 2003, Anaheim, California, USA
• 16: Stephen J. Fink , Feng Qian, Design, implementation and evaluation of adaptive recompilation with on-stack replacement, Proceedings of the international symposium on Code generation and optimization: feedback-directed and runtime optimization, March 23-26, 2003, San Francisco, California
• 17: Urs Hölzle , Craig Chambers , David Ungar, Debugging optimized code with dynamic deoptimization, Proceedings of the ACM SIGPLAN 1992 conference on Programming language design and implementation, p.32-43, June 15-19, 1992, San Francisco, California, United States
• 18: IBM. Power4 system microarchitecture. http://www-1.ibm.com/servers/eserver/pseries/hardware/ -whitepapers/ -power4%.html, 2001.
• 19: Jikes Research Virtual Machine (RVM). http://www.ibm.com/developerworks/oss/jikesrvm.
• 20: Martin Karlsson , Kevin E. Moore , Erik Hagersten , David A. Wood, Memory System Behavior of Java-Based Middleware, Proceedings of the 9th International Symposium on High-Performance Computer Architecture, p.217, February 08-12, 2003
• 21: Doug Kimelman , Bryan Rosenburg , Tova Roth, Strata-various: multi-layer visualization of dynamics in software system behavior, Proceedings of the conference on Visualization '94, October 17-21, 1994, Washinton, D.C.
• 22: Yue Luo and Lizy Kurian John. Workload characterization of multithreaded Java servers. In Proceedings of the 2001 IEEE International Symposium on Performance Analysis of Systems and Software, pages 128--136, Tucson, AZ, November 2001.
• 23: Cathy May, Ed Silha, Rick Simpson, and Hank Warren. The PowerPC Architecture. Morgan Kaufmann Publishers, Inc., San Francisco, California, 1994.
• 24: John Mellor-Crummey, Robert Fowler, and Gabriel Marin. HPCView: A tool for top-down analysis of node performance. In Proceedings of the Los Alamos Computer Science Institute Second Annual Symposium, Santa Fe, NM, October 2001.
• 25: Microsoft. Windows management instrumentation. http://msdn.microsoft.com/library/en-us/wmisdk/wmi/wmi_start_page.asp, 2004.
• 26: Barton P. Miller , Mark D. Callaghan , Jonathan M. Cargille , Jeffrey K. Hollingsworth , R. Bruce Irvin , Karen L. Karavanic , Krishna Kunchithapadam , Tia Newhall, The Paradyn Parallel Performance Measurement Tool, Computer, v.28 n.11, p.37-46, November 1995 [doi>10.1109/2.471178 ]
• 27: Oprofile. http://oprofile.sourceforge.net, 2003.
• 28: HSQL Database Engine (HSQLDB) Project. Hsql database engine. http://hsqldb.sourceforge.net.
• 29: Daniel A. Reed, Ruth. A. Aydt, Roger J. Noe, Philip C. Roth, Keith A. Shields, Bradley Schwartz, and Luis F. Tavera. Scalable performance analysis: The P ablo performance analysis environment. In Proceedings of the Scalable Parallel Libraries Conference, October 1993.
• 30: Ashish Sen and Muni Srivastava. Regression Analysis: Theory, Methods and Applications. Springer-Verlag, 1997.
• 31: Pattabi Seshadri, Lizy John, and Alex Mericas. Workload characterization of Java server applications on two PowerPC processors. In Proceedings of the Third Annual Austin Center for Advanced Studies Conference, Austin, TX, February 2002.
• 32: Yefim Shuf , Mauricio J. Serrano , Manish Gupta , Jaswinder Pal Singh, Characterizing the memory behavior of Java workloads: a structured view and opportunities for optimizations, Proceedings of the 2001 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.194-205, June 2001, Cambridge, Massachusetts, United States
• 33: Peter F. Sweeney, Matthias Hauswirth, Brendon Cahoon, Perry Cheng, Amer Diwan, David Grove, and Michael Hind. Using hardware performance monitors to understand the behavior of Java applications. In Proceedings of the 3rd Virtual Machine Research and Technology Symposium (VM'04), May 2004.
• 34: Bob Urquhart, Enio Pineda, Scott Jones, Frank Levine, Ron Cadima, Jimmy DeWitt, Theresa Halloran, and Aakash Parekh.Performance inspector. http://www-124.ibm.com/developerworks/oss/pi, 2004.
• 35: D. Viswanathan , S. Liang, Java virtual machine profiler interface, IBM Systems Journal, v.39 n.1, p.82-95, January 2000
• 36: Intel VTune performance analyzers. http://www.intel.com/software/products/vtune.
• 37: C. Eric Wu , Anthony Bolmarcich , Marc Snir , David Wootton , Farid Parpia , Anthony Chan , Ewing Lusk , William Gropp, From trace generation to visualization: a performance framework for distributed parallel systems, Proceedings of the 2000 ACM/IEEE conference on Supercomputing (CDROM), p.50-es, November 04-10, 2000, Dallas, Texas, United States
• 38: Zhichen Xu , Barton P. Miller , Oscar Naim, Dynamic instrumentation of threaded applications, Proceedings of the seventh ACM SIGPLAN symposium on Principles and practice of parallel programming, p.49-59, May 04-06, 1999, Atlanta, Georgia, United States
• 39: Marco Zagha , Brond Larson , Steve Turner , Marty Itzkowitz, Performance analysis using the MIPS R10000 performance counters, Proceedings of the 1996 ACM/IEEE conference on Supercomputing (CDROM), p.16-es, January 01-01, 1996, Pittsburgh, Pennsylvania, United States [doi>10.1145/369028.369059]
• 40: Omer Zaki , Ewing Lusk , William Gropp , Deborah Swider, Toward Scalable Performance Visualization with Jumpshot, International Journal of High Performance Computing Applications, v.13 n.3, p.277-288, August 1999 [doi>10.1177/109434209901300310]

INDEX TERMS

Primary Classification:
  B. Hardware
  B.8 Performance and Reliability
      B.8.2 Performance Analysis and Design Aids

Additional Classification:
  C. Computer Systems Organization
  C.4 PERFORMANCE OF SYSTEMS

General Terms:
Experimentation, Measurement, Performance

Keywords:
hardware performance monitors, perturbation, software performance monitors, vertical profiling, whole-system analysis