Using complete system simulation to characterize SPECjvm98 benchmarks

ABSTRACT

Complete system simulation to understand the influence of architecture and operating systems on application execution has been identified to be crucial for systems design. While there have been previous attempts at understanding the architectural impact of Java programs, there has been no prior work investigating the operating system (kernel) activity during their executions. This problem is particularly interesting in the context of Java since it is not only the application that can invoke kernel services, but so does the underlying Java Virtual Machine (JVM) implementation which runs these programs. Further, the JVM style (JIT compiler or interpreter) and the manner in which the different JVM components (such as the garbage collector and class loader) are exercised, can have a significant impact on the kernel activities.To investigate these issues, this research uses complete system simulation of the SPECjvm98 benchmarks on the SimOS simulation platform. The execution of these benchmarks on both JIT compilers and interpreters is profiled in detail, to identify and quantify where time is spent in each component. The kernel activity of SPECjvm98 applications constitutes up to 17% of the execution time in the large dataset and up to 31% in the small dataset. The average kernel activity in the large dataset is approximately 10%, in comparison to around 2% in four SPECInt benchmarks studied. Of the kernel services, TLB miss handling is the most dominant in all applications. The TLB miss rates in the JIT compiler, dynamic class loader and garbage collector portions of the JVM are individually analyzed. In addition to such execution profiles, the ILP in the user and kernel mode are also quantified. The Java code is seen to limit exploitable parallelism and aggressive instruction issue is seen to be less efficient for SPECjvm98 benchmarks in comparison to SPEC95 programs. Also, the kernel mode of execution does not exhibit as much ILP as the user mode.

REFERENCES

• 1: C.-H. A. Hsieh , M. T. Conte , T. L. Johnson , J. C. Gyllenhaal , W.-M. W. Hwu, A study of the cache and branch performance issues with running Java on current hardware platforms, Proceedings of the 42nd IEEE International Computer Conference, p.211, February 23-26, 1997
• 2: R. Radhakrishnan, N. Vijaykrishnan, L. K. John and A. Sivasubramaniam, Architectural Issue in Java Runtime Systems, In Proceedings of the 6th International Conference on High Performance Computer Architecture, pages 387-398, 2000.
• 3: Narayanan Vijaykrishnan , N. Ranganathan , Ravi Gadekarla, Object-Oriented Architectural Support for a Java Processor, Proceedings of the 12th European Conference on Object-Oriented Programming, p.330-354, July 20-24, 1998
• 4: N. Vijaykrishnan and N. Ranganathan, Tuning Branch Predictors to Support Virtual Method Invocation in Java, In Proceedings of the 5th USENIX Conference of Object-Oriented Technologies and Systems, pages 217-228, 1999.
• 5: A. Barisone, F. Bellotti, R. Berta and A. D. Gloria, Ultrasparc Instruction Level Characterization of Java Virtual Machine Workload, from the 2nd Annual Workshop on Workload Characterization, Workload Characterization for Computer System Design, Kluwer Academic Publishers, pages 1-24, 1999.
• 6: J.-S. Kim and Y. Hsu, Analyzing Memory Reference Traces of Java Programs, from the 2nd Annual Workshop on Workload Characterization, Workload Characterization for Computer System Design, Kluwer Academic Publishers, pages 25-48, 1999.
• 7: J. Michael O'Connor , Marc Tremblay, picoJava-I: The Java Virtual Machine in Hardware, IEEE Micro, v.17 n.2, p.45-53, March 1997 [doi>10.1109/40.592314 ]
• 8: Characterization of Java Applications at Bytecode and Ultra-SPARC Machine Code Levels, Proceedings of the 1999 IEEE International Conference on Computer Design, p.281, October 10-13, 1999
• 9: Luiz André Barroso , Kourosh Gharachorloo , Edouard Bugnion, Memory system characterization of commercial workloads, Proceedings of the 25th annual international symposium on Computer architecture, p.3-14, June 27-July 02, 1998, Barcelona, Spain
• 10: Stephen Alan Herrod , Mendel Rosenblum, Using complete machine simulation to understand computer system behavior, 1998
• 11: Mendel Rosenblum , Stephen A. Herrod , Emmett Witchel , Anoop Gupta, Complete Computer System Simulation: The SimOS Approach, IEEE Parallel & Distributed Technology: Systems & Technology, v.3 n.4, p.34-43, December 1995 [doi>10.1109/88.473612 ]
• 12: M. Rosenblum , E. Bugnion , S. A. Herrod , E. Witchel , A. Gupta, The impact of architectural trends on operating system performance, Proceedings of the fifteenth ACM symposium on Operating systems principles, p.285-298, December 03-06, 1995, Copper Mountain, Colorado, United States
• 13: Cheng-Hsueh A. Hsieh , John C. Gyllenhaal , Wen-mei W. Hwu, Java bytecode to native code translation: the caffeine prototype and preliminary results, Proceedings of the 29th annual ACM/IEEE international symposium on Microarchitecture, p.90-99, December 02-04, 1996, Paris, France
• 14: Timothy Cramer , Richard Friedman , Terrence Miller , David Seberger , Robert Wilson , Mario Wolczko, Compiling Java Just in Time, IEEE Micro, v.17 n.3, p.36-43, May 1997 [doi>10.1109/40.591653 ]
• 15: A. Krall, Efficient JavaVM Just-in-Time Compilation, Proceedings of the 1998 International Conference on Parallel Architectures and Compilation Techniques, p.205, October 12-18, 1998
• 16: Ali-Reza Adl-Tabatabai , Michał Cierniak , Guei-Yuan Lueh , Vishesh M. Parikh , James M. Stichnoth, Fast, effective code generation in a just-in-time Java compiler, Proceedings of the ACM SIGPLAN 1998 conference on Programming language design and implementation, p.280-290, June 17-19, 1998, Montreal, Quebec, Canada
• 17: Harlan McGhan , Mike O'Connor, PicoJava: A Direct Execution Engine For Java Bytecode, Computer, v.31 n.10, p.22-30, October 1998 [doi>10.1109/2.722273 ]
• 18: N. Vijaykrishnan, Issues in the Design of a Java Processor Architecture. PhD Thesis, College of Engineering, University of South Florida, July 1998.
• 19: Matthew C. Merten , Andrew R. Trick , Christopher N. George , John C. Gyllenhaal , Wen-mei W. Hwu, A hardware-driven profiling scheme for identifying program hot spots to support runtime optimization, Proceedings of the 26th annual international symposium on Computer architecture, p.136-147, May 01-04, 1999, Atlanta, Georgia, United States
• 20: SPEC Jvm98 Benchmarks, http://www.spec.org/osg/jvm98/
• 21: SPEC CPU95 Benchmarks, hnp://www.spec.org/osg/cpu95/
• 22: S. A. Herrod, M. Rosenblum, E. Bugnion, S. Devine, R. Bosch, J. Chapin, K. Govil, D. Teodosiu, E. Witchel, and B. Verghese, The SimOS User Guide, http ://simos.stanford.edu/userguide/
• 23: Overview of Java Platform Product Family, http://www.javasoft, com/products/O V jdkProduct, html
• 24: Emmett Witchel , Mendel Rosenblum, Embra: fast and flexible machine simulation, Proceedings of the 1996 ACM SIGMETRICS international conference on Measurement and modeling of computer systems, p.68-79, May 23-26, 1996, Philadelphia, Pennsylvania, United States
• 25: James E. Bennett , Michael J. Flynn, Performance Factors for Superscalar Processors, Stanford University, Stanford, CA, 1995
• 26: MIPS Technologies, Incorporated, R10000 Microprocessor Product Overview, MIPS Open RISC Technology, Oct. 1994.
• 27: Kenneth C. Yeager, The MIPS R10000 Superscalar Microprocessor, IEEE Micro, v.16 n.2, p.28-40, April 1996 [doi>10.1109/40.491460 ]
• 28: K. I. Farkas , N. P. Jouppi, Complexity/performance tradeoffs with non-blocking loads, Proceedings of the 21ST annual international symposium on Computer architecture, p.211-222, April 18-21, 1994, Chicago, Illinois, United States
• 29: Kunle Olukotun , Basem A. Nayfeh , Lance Hammond , Ken Wilson , Kunyung Chang, The case for a single-chip multiprocessor, Proceedings of the seventh international conference on Architectural support for programming languages and operating systems, p.2-11, October 01-04, 1996, Cambridge, Massachusetts, United States
• 30: Basem A. Nayfeh , Lance Hammond , Kunle Olukotun, Evaluation of design alternatives for a multiprocessor microprocessor, Proceedings of the 23rd annual international symposium on Computer architecture, p.67-77, May 22-24, 1996, Philadelphia, Pennsylvania, United States
• 31: Tao Li, Using Complete System Simulation to Characterize the Execution Behaviors of SPECjvm98 Benchmarks, http://www.ece.utexas.edu/tli3/tao-jvm98.ps
• 32: Todd M. Austin , Gurindar S. Sohi, Dynamic dependency analysis of ordinary programs, Proceedings of the 19th annual international symposium on Computer architecture, p.342-351, May 19-21, 1992, Queensland, Australia
• 33: R. Sathe , M. Franklin, Available Parallelism with Data Value Prediction, Proceedings of the Fifth International Conference on High Performance Computing, p.194, December 17-20, 1998
• 34: J. Sabarinathan, A Study of Instruction Level Parallelism in Contemporary Computer Applications, Master Report, University of Texas at Austin, Dec. 1999.
• 35: Sylvia Dieckmann , Urs Hölzle, A Study of the Allocation Behavior of the SPECjvm98 Java Benchmark, Proceedings of the 13th European Conference on Object-Oriented Programming, p.92-115, June 14-18, 1999
• 36: Kenneth M. Wilson , Kunle Olukotun , Mendel Rosenblum, Increasing cache port efficiency for dynamic superscalar microprocessors, Proceedings of the 23rd annual international symposium on Computer architecture, p.147-157, May 22-24, 1996, Philadelphia, Pennsylvania, United States

INDEX TERMS

Primary Classification:
  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.2 Installation Management
          Subjects: Benchmarks

Additional Classification:
  C. Computer Systems Organization

  D. Software
  D.3 PROGRAMMING LANGUAGES
      D.3.2 Language Classifications

          Nouns: Java
      D.3.4 Processors
          Subjects: Compilers

  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.2 Modes of Computation
          Subjects: Parallelism and concurrency

General Terms:
Design, Experimentation, Languages, Measurement, Performance, Standardization, Theory