Predicting component failures at design time

ABSTRACT

How do design decisions impact the quality of the resulting software? In an empirical study of 52 ECLIPSE plug-ins, we found that the software design as well as past failure history, can be used to build models which accurately predict failure-prone components in new programs. Our prediction only requires usage relationships between components, which are typically defined in the design phase; thus, designers can easily explore and assess design alternatives in terms of predicted quality. In the ECLIPSE study, 90% of the 5% most failure-prone components, as predicted by our model from design data, turned out to actually produce failures later; a random guess would have predicted only 33%.

REFERENCES

• 1: John Anvik , Lyndon Hiew , Gail C. Murphy, Coping with an open bug repository, Proceedings of the 2005 OOPSLA workshop on Eclipse technology eXchange, p.35-39, October 16-17, 2005, San Diego, California [doi>10.1145/1117696.1117704]
• 2: Victor R. Basili , Lionel C. Briand , Walcélio L. Melo, A Validation of Object-Oriented Design Metrics as Quality Indicators, IEEE Transactions on Software Engineering, v.22 n.10, p.751-761, October 1996 [doi>10.1109/32.544352 ]
• 3: Aaron B. Binkley , Stephen R. Schach, Validation of the coupling dependency metric as a predictor of run-time failures and maintenance measures, Proceedings of the 20th international conference on Software engineering, p.452-455, April 19-25, 1998, Kyoto, Japan
• 4: Davor Cubranic , Gail Murphy, Project history as a group memory: learning from the past, University of British Columbia, Vancouver, BC, Canada, 2005
• 5: Davor Cubranic , Gail C. Murphy , Janice Singer , Kellogg S. Booth, Hipikat: A Project Memory for Software Development, IEEE Transactions on Software Engineering, v.31 n.6, p.446-465, June 2005 [doi>10.1109/TSE.2005.71]
• 6: Giovanni Denaro , Sandro Morasca , Mauro Pezzè, Deriving models of software fault-proneness, Proceedings of the 14th international conference on Software engineering and knowledge engineering, July 15-19, 2002, Ischia, Italy [doi>10.1145/568760.568824]
• 7: Giovanni Denaro , Mauro Pezzè, An empirical evaluation of fault-proneness models, Proceedings of the 24th International Conference on Software Engineering, May 19-25, 2002, Orlando, Florida [doi>10.1145/581339.581371]
• 8: Norman E. Fenton , Niclas Ohlsson, Quantitative Analysis of Faults and Failures in a Complex Software System, IEEE Transactions on Software Engineering, v.26 n.8, p.797-814, August 2000 [doi>10.1109/32.879815 ]
• 9: Michael Fischer , Martin Pinzger , Harald Gall, Analyzing and Relating Bug Report Data for Feature Tracking, Proceedings of the 10th Working Conference on Reverse Engineering, p.90, November 13-17, 2003
• 10: Michael Fischer , Martin Pinzger , Harald Gall, Populating a Release History Database from Version Control and Bug Tracking Systems, Proceedings of the International Conference on Software Maintenance, p.23, September 22-26, 2003
• 11: Todd L. Graves , Alan F. Karr , J. S. Marron , Harvey Siy, Predicting Fault Incidence Using Software Change History, IEEE Transactions on Software Engineering, v.26 n.7, p.653-661, July 2000 [doi>10.1109/32.859533 ]
• 12: John P. Hudepohl , Stephen J. Aud , Taghi M. Khoshgoftaar , Edward B. Allen , Jean Mayrand, Emerald: Software Metrics and Models on the Desktop, IEEE Software, v.13 n.5, p.56-60, September 1996 [doi>10.1109/52.536459 ]
• 13: T. M. Khoshgoftaar , E. B. Allen , N. Goel , A. Nandi , J. McMullan, Detection of software modules with high debug code churn in a very large legacy system, Proceedings of the The Seventh International Symposium on Software Reliability Engineering (ISSRE '96), p.364, October 30-November 02, 1996
• 14: Audris Mockus , Lawrence G. Votta, Identifying Reasons for Software Changes Using Historic Databases, Proceedings of the International Conference on Software Maintenance (ICSM'00), p.120, October 11-14, 2000
• 15: Audris Mockus , Ping Zhang , Paul Luo Li, Predictors of customer perceived software quality, Proceedings of the 27th international conference on Software engineering, p.225-233, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062506]
• 16: K.-H. Moller and D. Paulish. An empirical investigation of software fault distribution. In Proc. IEEE First International Software Metrics Symposium, pages 82--90, May 1993.
• 17: Nachiappan Nagappan , Thomas Ball, Use of relative code churn measures to predict system defect density, Proceedings of the 27th international conference on Software engineering, p.284-292, May 15-21, 2005, St. Louis, MO, USA [doi>10.1145/1062455.1062514]
• 18: Nachiappan Nagappan , Thomas Ball , Andreas Zeller, Mining metrics to predict component failures, Proceeding of the 28th international conference on Software engineering, May 20-28, 2006, Shanghai, China [doi>10.1145/1134285.1134349]
• 19: Niclas Ohlsson , Hans Alberg, Predicting Fault-Prone Software Modules in Telephone Switches, IEEE Transactions on Software Engineering, v.22 n.12, p.886-894, December 1996 [doi>10.1109/32.553637 ]
• 20: Thomas J. Ostrand , Elaine J. Weyuker , Robert M. Bell, Predicting the Location and Number of Faults in Large Software Systems, IEEE Transactions on Software Engineering, v.31 n.4, p.340-355, April 2005 [doi>10.1109/TSE.2005.49]
• 21: J. Rivières. How to use the Eclipse API, May 2001. http://eclipse.org/articles/Article-API%20use/eclipse-apiusage-rules.html.
• 22: Jacek Śliwerski , Thomas Zimmermann , Andreas Zeller, When do changes induce fixes?, Proceedings of the 2005 international workshop on Mining software repositories, p.1-5, May 17-17, 2005, St. Louis, Missouri
• 23: Ramanath Subramanyam , M. S. Krishnan, Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects, IEEE Transactions on Software Engineering, v.29 n.4, p.297-310, April 2003 [doi>10.1109/TSE.2003.1191795]
• 24: The Bugzilla Team. The Bugzilla Guide - 2.18 Release, Jan. 2005. http://www.bugzilla.org/docs/2.18/html/.
• 25: T. Zimmermann and P. Weißgerber. Preprocessing CVS data for fine-grained analysis. In Proceedings of International Workshop on Mining Software Repositories (MSR 2004), pages 2--6, Edinburgh, Scotland, UK, May 2004.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.7 Distribution, Maintenance, and Enhancement
          Subjects: Version control

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics
          Subjects: Product metrics; Process metrics; Complexity measures
      D.2.9 Management
          Subjects: Software quality assurance (SQA)

General Terms:
Management, Measurement, Reliability