Determining the cost-quality trade-off for automated software traceability

ABSTRACT

Major software development standards mandate the establishment of trace links among software artifacts such as requirements, architectural elements, or source code without explicitly stating the required level of detail of these links. However, the level of detail vastly affects the cost and quality of trace link generation and important applications of trace analysis such as conflict analysis, consistency checking, or change impact analysis. In this paper, we explore these cost-quality trade-offs with three case study systems from different contexts - the open-source ArgoUML modeling tool, an industrial route-planning system, and a movie player. We report the cost-quality trade-off of automated trace generation with the Trace Analyzer approach and discuss its expected impact onto several applications that consume its trace information. In the study we explore simple techniques to predict and manipulate the cost-benefit trade-off with threshold-based filtering. We found that (a) 80% of the benefit comes from only 20% of the cost and (b) weak trace links are predominantly false trace links and can be efficiently eliminated through thresholds.

REFERENCES

• 1: G. Antoniol , B. Caprile , A. Potrich , P. Tonella, Design-code traceability recovery: selecting the basic linkage properties, Science of Computer Programming, v.40 n.2-3, p.213-234, July 2001 [doi>10.1016/S0167-6423(01)00016-8 ]
• 2: Stefan Biffl , Aybüke Aurum , Barry Boehm , Hakan Erdogmus , Paul Grünbacher, Value-Based Software Engineering, Springer-Verlag New York, Inc., Secaucus, NJ, 2005
• 3: Barry Boehm, Value-based software engineering: reinventing, ACM SIGSOFT Software Engineering Notes, v.28 n.2, March 2003 [doi>10.1145/638750.638775]
• 4: Alexander Egyed , Paul Grünbacher, Automating Requirements Traceability: Beyond the Record & Replay Paradigm, Proceedings of the 17th IEEE international conference on Automated software engineering, p.163, September 23-27, 2002
• 5: Alexander Egyed, A Scenario-Driven Approach to Trace Dependency Analysis, IEEE Transactions on Software Engineering, v.29 n.2, p.116-132, February 2003 [doi>10.1109/TSE.2003.1178051 ]
• 6: Alexander Egyed , Paul Grunbacher, Identifying Requirements Conflicts and Cooperation: How Quality Attributes and Automated Traceability Can Help, IEEE Software, v.21 n.6, p.50-58, November 2004 [doi>10.1109/MS.2004.40]
• 7: Egyed, A., Tailoring Software Traceability to Value-based Needs, In: Biffl, S., Aurum, A., Boehm, B.W., Erdogmus, H., and Grünbacher, P. (eds.), Value-Based Software Engineering, Sept. 2005, Springer Verlag
• 8: Egyed, A. and Grünbacher, P., Supporting Software Understanding with Automated Traceability. in: International Journal of Software Engineering and Knowledge Engineering (IJSEKE) (in press), 2005.
• 9: Gotel, O. and Finkelstein, A., An Analysis of the Requirements Traceability Problem. Proc. 1st International Conference on Rqts. Eng., pp. 94--101, 1994.
• 10: Matthias Heindl , Stefan Biffl, A case study on value-based requirements tracing, Proceedings of the 10th European software engineering conference held jointly with 13th ACM SIGSOFT international symposium on Foundations of software engineering, September 05-09, 2005, Lisbon, Portugal [doi>10.1145/1081706.1081717]
• 11: Ramesh, B., Stubbs, L., and Edwards, M., "Lessons Learned from Implementing Requirements Traceability." Crosstalk, Journal of Defense Software Engineering 8, 4 (April 1995): 11-15. Online at: http://www.stsc.hill.af.mil/crosstalk/1995/apr/Lessons.asp.
• 12: Julian Richardson , Jeff Green, Automating Traceability for Generated Software Artifacts, Proceedings of the 19th IEEE international conference on Automated software engineering, p.24-33, September 20-24, 2004 [doi>10.1109/ASE.2004.20]
• 13: Spanoudakis, G., Zisman, A., Perez-Minana, E., and Krause, P. Rule-based generation of requirements traceability relations. J. Systems and Software, 72(2):105--127, 2004.

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.1 Requirements/Specifications

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.8 Metrics

General Terms:
Design, Documentation, Economics, Experimentation, Management

Keywords:
automated trace analysis, cost-quality trade-off, experience report, value-based software engineering