Article

Traffic-aware stress testing of distributed systems based on UML models

Authors:

Lionel C. Briand,

Yvan LabicheAuthors Info & Claims

ICSE '06: Proceedings of the 28th international conference on Software engineering

Pages 391 - 400

https://doi.org/10.1145/1134285.1134340

Published: 28 May 2006 Publication History

Abstract

A stress test methodology aimed at increasing chances of discovering faults related to network traffic in distributed systems is presented. The technique uses the UML 2.0 model of the distributed system under test, augmented with timing information, and is based on an analysis of the control flow in sequence diagrams. It yields stress test requirements that are made of specific control flow paths along with time values indicating when to trigger them. Different variants of our stress testing technique already exist (they stress different aspects of a distributed system) and we focus here on one variant that is designed to identify and to stress test the system at the instant when data traffic on a network is maximal. Using a real-world distributed system specification, we design and implement a prototype distributed system and describe, for that particular system, how the stress test cases are derived and executed using our methodology. The stress test results indicate that the technique is significantly more effective at detecting network traffic-related faults when compared to test cases based on an operational profile.

References

[1]

A. Avritzer and E. J. Weyuker, "The Automatic Generation of Load Test Suites and the Assessment of the Resulting Software," IEEE Trans. on Software Eng, vol. 21, no. 9, pp. 705--716, 1995.

Digital Library

[2]

Borland Corporation, "Borland Delphi," www.borland.com/delphi, 2005.

[3]

L. Briand and Y. Labiche, "A UML-based Approach to System Testing," Journal of Software and Systems Modeling, vol. 1, no. 1, pp. 10--42, 2002.

[4]

L. C. Briand, Y. Labiche, and M. Shousha, "Automating Stress Testing for Real-Time Systems Using Genetic Algorithms," Genetic and Evolutionary Computation Conference (GECCO), pp. 1021--1028, 2005.

Digital Library

[5]

T. Brown, A. Pasetti, W. Pree, T. A. Henzinger, and C. M. Kirsch, "A Reusable and Platform-independent Framework for Distributed Control Systems," Proc. of Digital Avionics Systems Conference, pp. 1--11, 2001.

[6]

R. J. A. Buhr, "Use Case Maps as Architectural Entities for Complex Systems," IEEE Transactions on Software Engineering, vol. 24, no. 12, 1998.

Digital Library

[7]

BWI Co., "ElipseSCADA," http://www.bwi.com/proot/2775, 2004.

[8]

D. Coleman, P. Arnold, S. Bodoff, C. Dollin, H. GilChrist, F. Hayes, and P. Jeremaes, Object-Oriented Development - The Fusion Method: Prentice Hall, 1994.

Digital Library

[9]

Z. Constantinescu, P. Petrovic, A. Pedersen, D. Federici, and J. Campos, "QADPZ (Quite Advanced Distributed Parallel Zystem)," in http://qadpz.sourceforge.net, 2003.

[10]

A. Daneels and W. Salter, "What is SCADA?," Proc. of Int. Conf. on Accelerator and Large Experimental Physics Control Systems, pp. 339--343, 1999.

[11]

European Information Society Technologies, "Component Based Open Source Architecture for Distributed Telecom Applications," in http://coach.objectweb.org, 2003.

[12]

R. Fletcher, Practical Methods of Optimization, 2nd ed: John Wiley & Sons, 2000.

Digital Library

[13]

V. Garousi, L. Briand, and Y. Labiche, "Control Flow Analysis of UML 2.0 Sequence Diagrams," Proceedings of the European Conference on Model Driven Architecture-Foundations and Applications, LNCS 3748, pp. 160--174, 2005.

Digital Library

[14]

V. Garousi, L. Briand, and Y. Labiche, "Traffic-aware Model-based Stress Testing of Distributed Systems," Technical Report SCE-05-13, Carleton University, http://www.sce.carleton.ca/squall/pubs/tech_report/TR_SCE-05-13.pdf, 2005.

[15]

H. Gomaa, Designing Concurrent, Distributed, and Real-Time Applications with UML: Addison-Wesley, 2000.

Digital Library

[16]

R. Kuhn, "Sources of Failure in the Public Switched Telephone Network," IEEE Computer, vol. 30, no. 4, pp. 31--36, 1997.

Digital Library

[17]

S. Mackay, E. Wright, and J. Park, Practical Data Communications for Instrumentation and Control: Newnes, June, 2003.

[18]

S. Muchnick, Advanced Compiler Design and Implementation, First ed: Morgan Kaufmann, 1997.

Digital Library

[19]

J. D. Musa, "The Operational Profile in Software Reliability Engineering: An Overview," Proc. of Int. Symp. on Software Reliability Engineering, 1992.

[20]

C. Nebut, F. Fleurey, Y. L. Traon, and J.-M. Jézéquel, "Requirements by Contracts allow Automated System Testing," th International Symposium on Software Reliability Engineering (ISSRE), Denver, Colorado, 2003.

Digital Library

[21]

Object Management Group (OMG), "OCL 2.0 Specification," 2005.

[22]

Object Management Group (OMG), "UML 2.0 Superstructure Specification," 2005.

[23]

Object Management Group (OMG), "UML 2.0 Testing Profile Specification," 2003.

[24]

Object Management Group (OMG), "UML Profile for Schedulability, Performance, and Time (v1.0)," 2003.

[25]

T. Pender, UML Bible: Wiley, Sept. 2003.

Digital Library

[26]

B. Stojkovic and I. Vujosevic, "A compact SCADA system for a smaller size electric power system control-a fast, object-oriented and cost-effective approach," IEEE Power Engineering Society Winter Meeting, pp. 695--700, 2002.

[27]

N. Toshida, M. Uesugi, Y. Nakata, M. Nomoto, and T. Uchida, "Open Distributed EMS/SCADA System," Hitachi Review, vol. 47, no. 5, pp. 208--213, 1998.

[28]

J. J. P. Tsai, Y. Bi, S. J. H. Yang, and R. A. W. Smith, Distributed real-time systems: monitoring, visualization, debugging, and analysis: John Wiley & Sons, 1996.

Digital Library

[29]

R. Wakizono, T. Kawamura, T. Tsuchiya, T. Hatanaka, and T. Tanaka, "Object-oriented database management system for process control systems-development and evaluation," Proc. of Symp. on Applied Computing, pp. 204--209, 1999.

Digital Library

[30]

E. Weyuker and F. I. Vokolos, "Experience with Performance testing of Software Systems: Issues, an Approach and Case Study," IEEE Transactions of Software Engineering, vol. 26, no. 12, pp. 1147--1156, 2000.

Digital Library

[31]

C. S. D. Yang, "Identifying Potentially Load Sensitive Code Regions for Stress Testing," Proc. of Mid-Atlantic Student Workshop on Programming Languages and Systems, 1996.

[32]

J. Zhang and S. C. Cheung, "Automated Test Case Generation for the Stress Testing of Multimedia Systems," Software Practice & Experience, vol. 32, no. 15, pp. 1411--1435, 2002.

Digital Library

Cited By

Tuli SBojarczuk KGucevska NHarman MWang XWright G(2023)Simulation-Driven Automated End-to-End Test and Oracle Inference2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)10.1109/ICSE-SEIP58684.2023.00016(122-133)Online publication date: May-2023
https://doi.org/10.1109/ICSE-SEIP58684.2023.00016
Ajami AArtail H(2021)Analyzing The Throughput of Online Home Users During the COVID-19 Pandemic Using the Neyman-Scott Cluster Process2021 17th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob)10.1109/WiMob52687.2021.9606408(54-59)Online publication date: 11-Oct-2021
https://doi.org/10.1109/WiMob52687.2021.9606408
Krichen MMaâlej ALahami MJmaiel M(2019)A Resource-Aware Model-Based Framework for Load Testing of WS-BPEL CompositionsEnterprise Information Systems10.1007/978-3-030-26169-6_7(130-157)Online publication date: 28-Jul-2019
https://doi.org/10.1007/978-3-030-26169-6_7
Show More Cited By

Index Terms

Traffic-aware stress testing of distributed systems based on UML models
1. Computer systems organization
  1. Architectures
    1. Distributed architectures
2. Software and its engineering
  1. Software creation and management
    1. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
  2. Software organization and properties
    1. Software system structures
      1. Distributed systems organizing principles

Recommendations

Fault-driven stress testing of distributed real-time software based on UML models

In a previous article, a stress testing methodology was reported to detect network traffic-related Real-Time (RT) faults in distributed RT systems based on the design UML model of a System Under Test (SUT). The stress methodology, referred to as Test ...
Traffic-aware stress testing of distributed real-time systems based on UML models using genetic algorithms

This paper presents a model-driven, stress test methodology aimed at increasing chances of discovering faults related to network traffic in distributed real-time systems (DRTS). The technique uses the UML 2.0 model of the distributed system under test, ...
Traffic-aware Stress Testing of Distributed Real-Time Systems Based on UML Models in the Presence of Time Uncertainty
ICST '08: Proceedings of the 2008 International Conference on Software Testing, Verification, and Validation

In a previous work, we reported and experimented with a stress testing methodology to detect network traffic-related Real-Time (RT) faults in Distributed Real-Time Systems (DRTSs) based on the design UML models. The stress methodology, referred to as ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICSE '06: Proceedings of the 28th international conference on Software engineering

May 2006

1110 pages

ISBN:1595933751

DOI:10.1145/1134285

General Chair:
Leon J. Osterweil
University of Massachusetts, USA
,
Program Chairs:
Dieter Rombach
TU Kaiserslautern, Germany
,
Mary Lou Soffa
University of Virginia, USA

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 May 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ICSE06

Sponsor:

ICSE06: International Conference on Software Engineering

May 20 - 28, 2006

Shanghai, China

Acceptance Rates

Overall Acceptance Rate 276 of 1,856 submissions, 15%

Upcoming Conference

ICSE 2025

2025 IEEE/ACM 46th International Conference on Software Engineering

April 26 - May 3, 2025

Ottawa , ON , Canada

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
1,322
Total Downloads

Downloads (Last 12 months)14
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Tuli SBojarczuk KGucevska NHarman MWang XWright G(2023)Simulation-Driven Automated End-to-End Test and Oracle Inference2023 IEEE/ACM 45th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP)10.1109/ICSE-SEIP58684.2023.00016(122-133)Online publication date: May-2023
https://doi.org/10.1109/ICSE-SEIP58684.2023.00016
Ajami AArtail H(2021)Analyzing The Throughput of Online Home Users During the COVID-19 Pandemic Using the Neyman-Scott Cluster Process2021 17th International Conference on Wireless and Mobile Computing, Networking and Communications (WiMob)10.1109/WiMob52687.2021.9606408(54-59)Online publication date: 11-Oct-2021
https://doi.org/10.1109/WiMob52687.2021.9606408
Krichen MMaâlej ALahami MJmaiel M(2019)A Resource-Aware Model-Based Framework for Load Testing of WS-BPEL CompositionsEnterprise Information Systems10.1007/978-3-030-26169-6_7(130-157)Online publication date: 28-Jul-2019
https://doi.org/10.1007/978-3-030-26169-6_7
(2018)A model-based approach to combine conformance and load testsInternational Journal of Critical Computer-Based Systems10.5555/3302642.33026458:3-4(282-310)Online publication date: 1-Jan-2018
https://dl.acm.org/doi/10.5555/3302642.3302645
Paz SBernardino J(2018)Web Platform Assessment Tools: An Experimental EvaluationWeb Information Systems and Technologies10.1007/978-3-319-93527-0_3(45-63)Online publication date: 19-Jun-2018
https://doi.org/10.1007/978-3-319-93527-0_3
Almasri NTahat LKorel B(2017)Toward automatically quantifying the impact of a change in systemsSoftware Quality Journal10.1007/s11219-016-9316-825:3(601-640)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1007/s11219-016-9316-8
Maâlej AKrichen MJmaïel M(2017)WSCLim: A Tool for Model-Based Testing of WS-BPEL Compositions Under Load ConditionsTests and Proofs10.1007/978-3-319-61467-0_9(139-151)Online publication date: 18-Jun-2017
https://doi.org/10.1007/978-3-319-61467-0_9
Portillo‐Dominguez APerry PMagoni DMurphy J(2017)PHOEBE: an automation framework for the effective usage of diagnosis tools in the performance testing of clustered systemsSoftware: Practice and Experience10.1002/spe.250047:11(1837-1874)Online publication date: 11-Apr-2017
https://doi.org/10.1002/spe.2500
Zhen Ming Jiang Hassan A(2015)A Survey on Load Testing of Large-Scale Software SystemsIEEE Transactions on Software Engineering10.1109/TSE.2015.244534041:11(1091-1118)Online publication date: 10-Nov-2015
https://dl.acm.org/doi/10.1109/TSE.2015.2445340
Wang YLi ZZhao R(2015)Dependence Based Model-HealingProceedings of the 2015 IEEE 39th Annual Computer Software and Applications Conference - Volume 0310.1109/COMPSAC.2015.274(556-561)Online publication date: 1-Jul-2015
https://dl.acm.org/doi/10.1109/COMPSAC.2015.274
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents