Specialization and extrapolation of software cost models

ABSTRACT

Despite the widespread availability of software effort estimation models (e.g. COCOMO [2], Price-S [12], SEER-SEM [13], SLIM [14]), most managers still estimate new projects by extrapolating from old projects [3, 5, 7]. In this delta method, the cost of the next project is the cost of the last project multiplied by some factors modeling the difference between old and new projects [2].Delta estimation is simple, fast, and best of all, can take full advantage of local costing information. However delta estimation fails when the experience base (the old projects) can not be extrapolated to the new projects. Previously [10], we have shown that for a set of NASA projects, delta estimation would usually fail since most of the features and coefficients of the learned model vary wildly across sub-samples of the training data. In that prior work, no solution was offered for this problem.Here, we offer a solution and report the results of experiment with feature subset selection (FSS) and extrapolation. FSS methods are usually assessed via the mean change in model performance. However, as shown below, FSS can significantly reduce the variance as well. Hence, FSS should be routinely used in cost estimation.Our results should stop the trend in the effort modeling community of continually adding to the number of features in a model in order to improve estimation performance. Here we show that there are benefits in intelligently subtracting model features.

REFERENCES

• 1: Barry W. Boehm, Software Engineering Economics, Prentice Hall PTR, Upper Saddle River, NJ, 1981
• 2: Barry Boehm, Safe and Simple Software Cost Analysis, IEEE Software, v.17 n.5, p.14-17, September 2000 [doi>10.1109/52.877854 ]
• 3: B. Boehm. Personnel communication, 2003.
• 4: Barry W. Boehm , Clark , Horowitz , Brown , Reifer , Chulani , Ray Madachy , Bert Steece, Software Cost Estimation with Cocomo II with Cdrom, Prentice Hall PTR, Upper Saddle River, NJ, 2000
• 5: A. Griesel, J. Hihn, K. Bruno, and R. Tausworthe. Software Forecasting: As it is Really Done: A Study of JPL Software Engineers. In Proceedings of the Eighteenth Annual Software Engineering Workshop, Goddard Space Flight Center, Decemeber 1993.
• 6: Mark A. Hall , Geoffrey Holmes, Benchmarking Attribute Selection Techniques for Discrete Class Data Mining, IEEE Transactions on Knowledge and Data Engineering, v.15 n.6, p.1437-1447, November 2003 [doi>10.1109/TKDE.2003.1245283]
• 7: Jairus Hihn , Hamid Habib-agahi, Cost estimation of software intensive projects: a survey of current practices, Proceedings of the 13th international conference on Software engineering, p.276-287, May 13-17, 1991, Austin, Texas, United States
• 8: C. Kirsopp and M. Shepperd. Case and feature subset selection in case-based software project effort prediction. In Proc. of 22nd SGAI International Conference on Knowledge-Based Systems and Applied Artificial Intelligence, Cambridge, UK, 2002.
• 9: Ron Kohavi , George H. John, Wrappers for feature subset selection, Artificial Intelligence, v.97 n.1-2, p.273-324, Dec. 1997 [doi>10.1016/S0004-3702(97)00043-X ]
• 10: Tim Menzies , Dan Port , Zhihao Chen , Jairus Hihn, Simple software cost analysis: safe or unsafe?, Proceedings of the 2005 workshop on Predictor models in software engineering, p.1-6, May 15-15, 2005, St. Louis, Missouri
• 11: A. Miller. Subset Selection in Regression (second edition). Chapman & Hall, 2002.
• 12: P. S. L. M. L. NJ. Your guide to price-s: Estimating cost and schedule of software development and support, 1998.
• 13: D. of USA. Parametric cost estimating handbook, second edition, 1999.
• 14: L. H. Putnam. Software Cost Estimating and Life-Cycle Control: Getting the Software Numbers, New York. The Institute of Electrical and Electronics Engineers, Inc., 1980.
• 15: J. R. Quinlan. Learning with Continuous Classes. In 5th Australian Joint Conference on Artificial Intelligence, pages 343-348, 1992. Available from http://citeseer.nj.nec.com/quinlan92learning.html.
• 16: Ian H. Witten , Eibe Frank, Data mining: practical machine learning tools and techniques with Java implementations, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2000

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.6 SIMULATION AND MODELING

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.9 Management

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management
          Subjects: Software process