Exploring single and multilevel JIT compilation policy for modern machines ¹

Abstract

Dynamic or Just-in-Time (JIT) compilation is essential to achieve high-performance emulation for programs written in managed languages, such as Java and C#. It has been observed that a conservative JIT compilation policy is most effective to obtain good runtime performance without impeding application progress on single-core machines. At the same time, it is often suggested that a more aggressive dynamic compilation strategy may perform best on modern machines that provide abundant computing resources, especially with virtual machines (VMs) that are also capable of spawning multiple concurrent compiler threads. However, comprehensive research on the best JIT compilation policy for such modern processors and VMs is currently lacking. The goal of this work is to explore the properties of single-tier and multitier JIT compilation policies that can enable existing and future VMs to realize the best program performance on modern machines.

In this work, we design novel experiments and implement new VM configurations to effectively control the compiler aggressiveness and optimization levels (if and when methods are compiled) in the industry-standard Oracle HotSpot Java VM to achieve this goal. We find that the best JIT compilation policy is determined by the nature of the application and the speed and effectiveness of the dynamic compilers. We extend earlier results showing the suitability of conservative JIT compilation on single-core machines for VMs with multiple concurrent compiler threads. We show that employing the free compilation resources (compiler threads and hardware cores) to aggressively compile more program methods quickly reaches a point of diminishing returns. At the same time, we also find that using the free resources to reduce compiler queue backup (compile selected hot methods early) significantly benefits program performance, especially for slower (highly optimizing) JIT compilers. For such compilers, we observe that accurately prioritizing JIT method compiles is crucial to realize the most performance benefit with the smallest hardware budget. Finally, we show that a tiered compilation policy, although complex to implement, greatly alleviates the impact of more and early JIT compilation of programs on modern machines.

References

1. Arnold, M., Fink, S., Grove, D., Hind, M., and Sweeney, P. F. 2000a. Adaptive optimization in the Jalapeno JVM. In Proceedings of the 15th ACM SIGPLAN Conference on Object-Oriented Programming, Systems, Languages, and Applications. 47--65.
2. Arnold, M., Fink, S., Grove, D., Hind, M., and Sweeney, P. F. 2000b. Adaptive optimization in the Jalapeo JVM: The controller’s analytical model. In Proceedings of the 3rd ACM Workshop on Feedback Directed and Dynamic Optimization (FDDO’00).
3. Arnold, M., Fink, S., Grove, D., Hind, M., and Sweeney, P. F. 2005. A survey of adaptive optimization in virtual machines. Proc. IEEE 92, 2, 449--466.
4. Arnold, M., Hind, M., and Ryder, B. G. 2002. Online feedback-directed optimization of Java. SIGPLAN Not. 37, 11, 111--129.
5. Blackburn, S. M., Garner, R., Hoffmann, C., Khang, A. M., McKinley, K. S., Bentzur, R., Diwan, A., Feinberg, D., Frampton, D., Guyer, S. Z., Hirzel, M., Hosking, A., Jump, M., Lee, H., Moss, J. E. B., Moss, B., Phansalkar, A., Stefanović, D., VanDrunen, T., von Dincklage, D., and Wiedermann, B. 2006. The DaCapo benchmarks: Java benchmarking development and analysis. In Proceedings of the 21st annual ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA’06). 169--190.
6. Böhm, I., von Koch, T. J. K. E., Kyle, S. C., Franke, B., and Topham, N. 2011. Generalized just-in-time trace compilation using a parallel task farm in a dynamic binary translator. In Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI’11). 74--85.
7. Bruening, D. and Duesterwald, E. 2000. Exploring optimal compilation unit shapes for an embedded just-in-time compiler. In Proceedings of the 3rd ACM Workshop on Feedback-Directed and Dynamic Optimization. 13--20.
8. Cavazos, J. and O’Boyle, M. F. P. 2006. Method-specific dynamic compilation using logistic regression. In Proceedings of the 21st Annual ACM SIGPLAN Conference on Object-Oriented Programming Systems, Languages, and Applications (OOPSLA’06). ACM, New York, 229--240.
9. Chang, P. P., Mahlke, S. A., and Hwu, W. m W. 1991. Using profile information to assist classic code optimizations. Software Prac. Experience 21, 1301--1321.
10. Deutsch, L. P. and Schiffman, A. M. 1984. Efficient implementation of the smalltalk-80 system. In Proceedings of the 11th ACM SIGACT-SIGPLAN Symposium on Principles of Programming Languages (POPL’84). ACM, New York, 297--302.
11. Esmaeilzadeh, H., Cao, T., Xi, Y., Blackburn, S. M., and McKinley, K. S. 2011. Looking back on the language and hardware revolutions: Measured power, performance, and scaling. In Proceedings of the 16th International Conference on Architectural Support for Programming Languages and Operating Systems (ASPLOS XVI). ACM, New York, 319--332.
12. Georges, A., Buytaert, D., and Eeckhout, L. 2007. Statistically rigorous java performance evaluation. In Proceedings of the Conference on Object-Oriented Programming Systems and Applications (OOPSLA’07). 57--76.
13. Gosling, J., Joy, B., Steele, G., and Bracha, G. 2005. The Java(TM) Language Specification 3rd Ed. http://dl.acm.org/citation.cfm?id=1036643.
14. Graham, S. L., Kessler, P. B., and Mckusick, M. K. 1982. Gprof: A call graph execution profiler. SIGPLAN Not. 17, 6, 120--126.
15. Grcevski, N., Kielstra, A., Stoodley, K., Stoodley, M., and Sundaresan, V. 2004. Java just-in-time compiler and virtual machine improvements for server and middleware applications. In Proceedings of the Conference on Virtual Machine Research and Technology Symposium. 12.
16. Gu, D. and Verbrugge, C. 2008. Phase-based adaptive recompilation in a JVM. In Proceedings of the 6th IEEE/ACM Symposium on Code Generation and Optimization (CGO’08). 24--34.
17. Haneda, M., Knijnenburg, P. M. W., and Wijshoff, H. A. G. 2005. Generating new general compiler optimization settings. In Proceedings of the 19th Annual International Conference on Supercomputing (ICS’05). 161--168.
18. Hansen, G. J. 1974. Adaptive Systems for the Dynamic Run-time Optimization of Programs. Ph.D. Dissertation. Carnegie-Mellon Univ., Pittsburgh, PA.
19. Harris, T. 1998. Controlling run-time compilation. In Proceedings of the IEEE Workshop on Programming Languages for Real-Time Industrial Applications. 75--84.
20. Hazelwood, K. and Grove, D. 2003. Adaptive online context-sensitive inlining. In Proceedings of the International Symposium on Code Generation and Optimization (CGO’03). IEEE Computer Society, Washington, DC, 253--264.
21. Hölzle, U. and Ungar, D. 1996. Reconciling responsiveness with performance in pure object-oriented languages. ACM Trans. Program. Lang. Syst. 18, 4, 355--400.
22. Jantz, M. R. and Kulkarni, P. A. 2013. Performance potential of optimization phase selection during dynamic JIT compilation. In Proceedings of the 9th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEEE’13).
23. Knuth, D. E. 1971. An empirical study of FORTRAN programs. Software: Pract. Experience 1, 2, 105--133.
24. Kotzmann, T., Wimmer, C., Mössenböck, H., Rodriguez, T., Russell, K., and Cox, D. 2008. Design of the Java HotSpot#8482; client compiler for Java 6. ACM Trans. Archit. Code Optim. 5, 1, 1--32.
25. Krintz, C. 2003. Coupling on-line and off-line profile information to improve program performance. In Proceedings of the International Symposium on Code Generation and Optimization (CGO’03). Washington, DC, 69--78.
26. Krintz, C. and Calder, B. 2001. Using annotations to reduce dynamic optimization time. In Proceedings of the ACM SIGPLAN 2001 Conference on Programming Language Design and Implementation. 156--167.
27. Krintz, C., Grove, D., Sarkar, V., and Calder, B. 2000. Reducing the overhead of dynamic compilation. Software: Pract. Experience 31, 8, 717--738.
28. Kulkarni, P., Arnold, M., and Hind, M. 2007. Dynamic compilation: The benefits of early investing. In VEE ’07: Proceedings of the 3rd International Conference on Virtual Execution Environments. 94--104.
29. Kulkarni, P. A. 2011. JIT compilation policy for modern machines. In Proceedings of the 2011 ACM International Conference on Object-Oriented Programming Systems Languages and Applications (OOPSLA’11). 773--788.
30. Kulkarni, P. A. and Fuller, J. 2011. JIT compilation policy on single-core and multi-core machines. In Proceedings of the 15th Workshop on Interaction between Compilers and Computer Architectures (INTERACT ’11). 54--62.
31. Microsoft. 2001. Microsoft C# Language Specifications 1st Ed. Microsoft Press.
32. Namjoshi, M. A. and Kulkarni, P. A. 2010. Novel online profiling for virtual machines. In Proceedings of the 6th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE’10). 133--144.
33. Paleczny, M., Vick, C., and Click, C. 2001. The Java hotspot TM server compiler. In Proceedings of the 2001 Symposium on JavaTM Virtual Machine Research and Technology Symposium (JVM’01). USENIX Association, Berkeley, CA, 1--12.
34. Sanchez, R. N., Amaral, J. N., Szafron, D., Pirvu, M., and Stoodley, M. 2011. Using machines to learn method-specific compilation strategies. In Proceedings of the 9th Annual IEEE/ACM International Symposium on Code Generation and Optimization (CGO’11). 257--266.
35. Smith, J. and Nair, R. 2005. Virtual Machines: Versatile Platforms for Systems and Processes (The Morgan Kaufmann Series in Computer Architecture and Design). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
36. SPEC2008. 2008. SPECjvm2008 Benchmarks. http://www.spec.org/jvm2008/.
37. SPEC98. 1998. SPECjvm98 Benchmarks. http://www.spec.org/jvm98/.
38. Sundaresan, V., Maier, D., Ramarao, P., and Stoodley, M. 2006. Experiences with multi-threading and dynamic class Loading in a Java Just-In-Time compiler. In Proceedings of the International Symposium on Code Generation and Optimization(CGO’06). 87--97.
39. Whitfield, D. L. and Soffa, M. L. 1997. An approach for exploring code improving transformations. ACM Trans. Program. Lang. Syst. 19, 6, 1053--1084.

Index Terms

• Software and its engineering

  • Software notations and tools

    • Compilers

      • Runtime environments