Article

Empirical estimates of software availability of deployed systems

Author:

Audris MockusAuthors Info & Claims

ISESE '06: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering

Pages 222 - 231

https://doi.org/10.1145/1159733.1159767

Published: 21 September 2006 Publication History

Abstract

We consider empirical evaluation of the availability of the deployed software. Evaluation of real systems is more realistic, more accurate, and provides higher level of confidence than simulations, testing, or models. We process and model information gathered from a variety of operational and service support systems to obtain estimates of software reliability and availability. The three principal quantities are the total runtime, the number of outages, and the duration of outages. We consider methods to assess the quality of information in customer support systems, discuss advantages and disadvantages of various sources, consider methods to deal with missing data, and ways to construct bounds on measures that are not directly available. We propose a method to assess empirically software availability and reliability based on information from operational customer support and inventory systems and use a case study of a large communications system to investigate factors affecting software reliability. We find large variations among platforms and releases and find the failure rate to vary over time.

References

[1]

D. Coit and W. Turkowski. Practical reliability data and analysis. Reliability Engineering, 14(1):1--17, 1986.

[2]

D. W. Coit and K. A. Dey. Analysis of grouped data from field-failure reporting systems. Reliability Engineering and System Safety, 65(2):95--101, 1999.

[3]

J. Fahy. Estimating warranty and service costs from mtbf estimates. In Electro/95 International. Professional Program Proceedings, pages 35--47, 21--23 June 1995.

[4]

T. H. Fleming and D. Harrington. Nonparametric estimation of the survival distribution in censored data. Comm. in Statistics, 13:2469--86, 1984.

[5]

K. Kanoun, M. Kaaniche, and J. Laprie. Experience in software reliability: From data collection to quantitative evaluation. In Proceedings, Fourth International Symposium on Software Reliability Engineering, pages 234--245, 3--6 Nov 1993.

[6]

E. Kaplan and P. Meyer. Non-paramentric estimation from incomplete observations. J Am Stat Assoc, pages 457--481, 1958.

[7]

C. Kjaergaard. Field failure data collection and analysis of repairable systems. In Reliability Data Collection and Use in Risk and Availability Assessment. Proceedings of the 6th EuReDatA Conference, pages 848--858, 1989.

[8]

K. Liu. Refrigerator failure early prediction based on warranty data. In Proceedings of the Annual Reliability and Maintainability Symposium, pages 195--199, 2002.

[9]

M. W. Lu. Automotive reliability prediction based on early field failure warranty data. Quality and Reliability Engineering International, 14(2):103--108, March-April 1998.

[10]

H. Malec. Communications reliability: a historical perspective. IEEE Transactions on Reliability, 47(3):333--345, Sept. 1998.

[11]

A. Mockus, P. Zhang, and P. Li. Drivers for customer perceived software quality. In ICSE 2005, St Louis, Missouri, May 2005. ACM Press.

Digital Library

[12]

H. Mueller and J. Wang. Hazard rates estimation under random censoring with varying kernels and bandwidths. Biometrics, 50:61--76, March 1994.

[13]

J. D. Musa, A. Iannino, and K. Okumoto. Software Reliability. McGraw-Hill Publishing Co., 1990.

[14]

S. original by Kenneth Hess and R. port by R. Gentleman. muhaz: Hazard Function Estimation in Survival Analysis. R package version 1.2.2.

[15]

S. original by Terry Therneau and ported by Thomas Lumley. survival: Survival analysis, including penalised likelihood. R package version 2.20.

[16]

R Development Core Team. R: A Language and Environment for Statistical Computing. R Foundation for Statistical Computing, Vienna, Austria, 2005. ISBN3-900051-07-0.

[17]

B. Rai and N. Singh. Hazard rate estimation from incomplete and unclean warranty data. Reliability Engineering and System Safety, 81(1):79--92, July 2003.

[18]

N. Sarawgi. A simple method for predicting the cumulative failures of consumer products during the warranty period. In Annual Reliability and Maintainability Symposium 1995 Proceedings, pages 384--390, 1995.

[19]

X. Zhang and H. Pham. Predicting operational software availability and its applications to telecommunication systems. International Journal of Systems Science, 33(11):923--930, Sep 15 2002.

Cited By

Afanaseva T(2019)Search of Software Artefacts Based on the Project Quantitative CharacteristicsProceedings of the 2019 2nd International Conference on Geoinformatics and Data Analysis10.1145/3318236.3318252(26-30)Online publication date: 15-Mar-2019
https://dl.acm.org/doi/10.1145/3318236.3318252
Scanniello G(2014)An Investigation of Object-Oriented and Code-Size Metrics as Dead Code PredictorsProceedings of the 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications10.1109/SEAA.2014.67(392-397)Online publication date: 27-Aug-2014
https://dl.acm.org/doi/10.1109/SEAA.2014.67
Scanniello G(2011)Source code survival with the Kaplan MeierProceedings of the 2011 27th IEEE International Conference on Software Maintenance10.1109/ICSM.2011.6080823(524-527)Online publication date: 25-Sep-2011
https://dl.acm.org/doi/10.1109/ICSM.2011.6080823
Show More Cited By

Index Terms

Empirical estimates of software availability of deployed systems
1. Social and professional topics
  1. Professional topics
    1. Management of computing and information systems
      1. System management
        Quality assurance
2. Software and its engineering
  1. Software creation and management
    1. Software development techniques
      1. Reusability
        Software product lines

Recommendations

An empirical study of high availability in stream processing systems
Middleware '09: Proceedings of the 10th ACM/IFIP/USENIX International Conference on Middleware

High availability (HA) is critical for many stream processing applications such as financial data analysis and disaster response. Existing HA schemes use either active standby or passive standby to guard the system against unexpected failures such as ...
Maximizing the Availability of Replicated Services in Widely Distributed Systems Considering Network Availability
SERE '13: Proceedings of the 2013 IEEE 7th International Conference on Software Security and Reliability

Availability is an important issue in distributed environments that promise to provide quality of services and thus the mechanisms to improve data or service availability is critical to ensure such quality of service. Replication schemes have been ...
Effects of replication on availability in distributed database systems

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISESE '06: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering

September 2006

388 pages

ISBN:1595932186

DOI:10.1145/1159733

General Chair:
Guilherme Horta Travassos
COPPE/UFRJ, Brazil
,
Program Chairs:
José Carlos Maldonado
ICMC-USP, Brazil
,
Claes Wohlin
Blekinge Institute of Technology, Sweden

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 September 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Article

Conference

ISESE06

Sponsor:

ISESE06: ACM-IEEE International Symposium on Empirical Software Engineering 2006

September 21 - 22, 2006

Rio de Janeiro, Brazil

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
476
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Afanaseva T(2019)Search of Software Artefacts Based on the Project Quantitative CharacteristicsProceedings of the 2019 2nd International Conference on Geoinformatics and Data Analysis10.1145/3318236.3318252(26-30)Online publication date: 15-Mar-2019
https://dl.acm.org/doi/10.1145/3318236.3318252
Scanniello G(2014)An Investigation of Object-Oriented and Code-Size Metrics as Dead Code PredictorsProceedings of the 2014 40th EUROMICRO Conference on Software Engineering and Advanced Applications10.1109/SEAA.2014.67(392-397)Online publication date: 27-Aug-2014
https://dl.acm.org/doi/10.1109/SEAA.2014.67
Scanniello G(2011)Source code survival with the Kaplan MeierProceedings of the 2011 27th IEEE International Conference on Software Maintenance10.1109/ICSM.2011.6080823(524-527)Online publication date: 25-Sep-2011
https://dl.acm.org/doi/10.1109/ICSM.2011.6080823
Khomh FChan BZou YSinha ADietz D(2011)Predicting post-release defects using pre-release field testing resultsProceedings of the 2011 27th IEEE International Conference on Software Maintenance10.1109/ICSM.2011.6080792(253-262)Online publication date: 25-Sep-2011
https://dl.acm.org/doi/10.1109/ICSM.2011.6080792
Jiang ZAvritzer AShihab EHassan AFlora P(2010)An Industrial Case Study on Speeding Up User Acceptance Testing by Mining Execution LogsProceedings of the 2010 Fourth International Conference on Secure Software Integration and Reliability Improvement10.1109/SSIRI.2010.15(131-140)Online publication date: 9-Jun-2010
https://dl.acm.org/doi/10.1109/SSIRI.2010.15
Mockus A(2006)Software support tools and experimental workProceedings of the 2006 international conference on Empirical software engineering issues: critical assessment and future directions10.5555/1767399.1767435(91-99)Online publication date: 26-Jun-2006
https://dl.acm.org/doi/10.5555/1767399.1767435

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten