Modelling multi-tier enterprise applications behaviour with design of experiments technique

ABSTRACT

Queueing network models are commonly used for performance modelling. However, through application development stage analytical models might not be able to continuously reflect performance, for example due to performance bugs or minor changes in the application code that cannot be readily reflected in the queueing model. To cope with this problem, a measurement-based approach adopting Design of Experiments (DoE) technique is proposed. The applicability of the proposed method is demonstrated on a complex 3-tier e-commerce application that is difficult to model with queueing networks.

References

1. NIST/SEMATECH e-Handbook of Statistical Methods http://www.itl.nist.gov/div898/handbook/
2. Java Modelling Tools. http://jmt.sourceforge.net/
3. Lazowska E et. al. ‘Quantitative system performance’. Available online from: http://homes.cs.washington.edu/~lazowska/qsp/
4. Z. Li, L. O'Brien, H. Zhang, and R. Cai. A factor framework for experimental design for performance evaluation of commercial cloud services. In Cloud Computing Technology and Science, 2012 IEEE 4th International Conference on, pages 169, 176.
5. MDload load generation simulator. https://github.com/imperialmodaclouds?query=modaclouds-mdload
6. OFBiz web-based 3 tier e-commerce application. http://ofbiz.apache.org/
7. J. Rai. Art of Computer Systems Performance Analysis Techniques For Experimental Design Measurements Simulation And Modeling. Wiley Computer Publishing, John Wiley & Sons, Inc. ISBN: 0471503363 Pub Date: 05/01/91
8. Software Engineering Institute - Blog https://blog.sei.cmu.edu/post.cfm/continuous-integration-in-devops
9. Spinner S., Casale G., Zhu X., and Kounev S. LibReDE: A Library for Resource Demand Estimation (Demonstration Paper). In Proceedings of the 5th ACM/SPEC International Conference on Performance Engineering (ICPE 2014), Dublin, Ireland, March 22- 26, 2014. ACM. March 2014
10. D. Westermann, R. Krebs, and J. Happe. Efficient experiment selection in automated software performance evaluations. In Computer Performance Engineering, pages 325-339. Springer, 2011.

Index Terms

• Computer systems organization

  • Architectures

    • Distributed architectures

  • Dependable and fault-tolerant systems and networks

• General and reference

  • Cross-computing tools and techniques

    • Metrics

    • Performance

• Networks

  • Network performance evaluation

• Software and its engineering

  • Software organization and properties

    • Extra-functional properties

      • Software performance

    • Software system structures

      • Distributed systems organizing principles