An experimental study of fault detection in user requirements documents

ABSTRACT

This paper describes a software engineering experiment designed to confirm results from an earlier project which measured fault detection rates in user requirements documents (URD). The experiment described in this paper involves the creation of a standardized URD with a known number of injected faults of specific type. Nine independent inspection teams were given this URD with instructions to locate as many faults as possible using the N-fold requirements inspection technique developed by the authors. Results obtained from this experiment confirm earlier conclusions about the low rate of fault detection in requirements documents using formal inspections and the advantages to be gained using the N-fold inspection method. The experiment also provides new results concerning variability in inspection team performance and the relative difficulty of locating different classes of URD faults.

REFERENCES

• 1: ~AvIzmMs, A. The N-version approach to fault tolerant software. IEEE Trans. Softw. Eng. ~11, 12 (Dec. 1985).
• 2: ~BO~M, B. Industrial software metrics top ten list IEEE Softw. (Sept 1987).
• 3: ~COLL~U*ELLO J. S. The software technical review process. Curriculum Module SEI-CM-3-1.5, ~Software Engineering Institute, Carnegie Mellon Univ., Pittsburgh, Pa., 1988.
• 4: ~FACAN, M.E. Design and code inspections to reduce errors in program developments. IBM ~Syst J., 3 (1976)
• 5: ~FAmLEY, R. Software Engineering Concepts. McGraw-Hfil, New York, 1985.
• 6: C B Jones, Systematic software development using VDM, Prentice Hall International (UK) Ltd., Hertfordshire, UK, 1986
• 7: Johnny Martin , W. T. Tsai, N-Fold inspection: a requirements analysis technique, Communications of the ACM, v.33 n.2, p.225-232, Feb. 1990 [doi>10.1145/75577.75587]
• 8: ~MOILER, T., AND SCHNEIDER, G M. Methodology and experimental research m software ~engineering. Int. J Man-Mach. Stud. 16 (1982).
• 9: ~RAMAMOOR?HY, R. Application of a methodology for the development and validation of ~process control software. IEEE Trans. Softw. Eng. SE-7, 6 (Nov. 1981).

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications
          Subjects: Methodologies (e.g., object-oriented, structured)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications
          Subjects: Tools
      D.2.9 Management
          Subjects: Programming teams

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management
          Subjects: Software development

General Terms:
Design, Experimentation, Management, Performance, Verification

Keywords:
fault detection, inspections, user requirements