skip to main content
10.1145/1159733.1159766acmconferencesArticle/Chapter ViewAbstractPublication PagesesemConference Proceedingsconference-collections
Article

A family of empirical studies to compare informal and optimization-based planning of software releases

Published: 21 September 2006 Publication History

Abstract

Replication of experiments, or performing a series of related studies, aims at attaining a higher level of validity of results. This paper reports on a series of empirical studies devoted to comparing informal release planning with two variants of optimization-based release planning.Two research questions were studied: How does optimization-based release planning compare with informal planning in terms of (i) time to generate release plans, and the feasibility and quality of those plans, and (ii) understanding and confidence of generated solutions and trust in the release planning process.For the family of empirical studies, the paper presents two types of results related to (i) the two research questions to compare the release planning techniques, and (ii) the evolution and lessons learned while conducting the studies.

References

[1]
Bagnall, A. J., Rayward-Smith, V. J., and Whittley, I. M. The Next Release Problem. Information and Software Technology, Vol. 43, No. 14, 2001, 883--890.
[2]
Basili, V. The Role of Experimentation in Software Engineering: Past, Present, and Future. Proceedings of the International Conference on Software Engineering. Berlin, Germany, 1996, 442--449.
[3]
Basili, V., Green, S., Laitenberger, O., Lanubile, F., Shull, F., Sørumgård, S., and Zelkowitz, M. The Empirical Investigation of Perspective-Based Reading. International Journal of Empirical Software Engineering, Vol. 1, No. 2, 1996, 133--164.
[4]
Basilli, V., and Lanubile, F. Building Knowledge through Families of Experiments. IEEE Transactions on Software Engineering, Vol. 25, No. 4, 1999, 456--473.
[5]
Basili, V., Selby, R., and Hutchens, D. Experimentation in Software Engineering. IEEE Transactions on Software Engineering, Vol. 12, No. 7, 1986, 733--743.
[6]
Bhawnani, P., Ruhe, G., Kudorfer, F., and Meyer, L. Intelligent Decision Support for Road Mapping - A Technology Transfer Case Study with Siemens Corporate Technology. Workshop on Technology Transfer in Software Engineering, Shanghai, China, 2006, 35--40.
[7]
Brooks, A., Daly, J., Miller, J., Roper, M., and Wood M. Replication of Experimental Results in Software Engineering. Technical Report ISERN-96-10, Department of Computer Science, University of Strathclyde, Glasgow, Scotland, 1996.
[8]
Canfora, G., Garcia, F., Piattini, M., Ruiz, F., and Visaggio, C. A Family of Experiments to Validate Metrics for Software Process Models. Journal of Systems and Software, Vol. 77, No. 2, 2005, 113--129.
[9]
Chi-square tests, www.lv.psu.edu/jxm57/irp/chisquar.html, February 2006.
[10]
Carlsson, C., and Turban, E. Decision Support Systems: Directions for the Next Decade. Decision Support Systems Vol. 33, 2002, 105--110.
[11]
Ciolkowski, M., Shull, F., and Biffl, S. A Family of Experiments to Investigate the Influence of Context on the Effect of Inspection Techniques. Proceedings of the 6th International Conference on Empirical Assessment in Software Engineering, Keele, UK, 2002, 48--60.
[12]
Denne, M., and Cleland-Huang, J. The Incremental Funding Method: Data Driven Software Development. IEEE Software, Vol. 21, No. 3, 2004, 39--47.
[13]
Du, G., Richter, M. M., and Ruhe, G. An Explanation Oriented Dialogue Approach and its Application to Wicked Planning Problems. To appear: Journal of Computing and Informatics, Vol. 25, No.3, 2006.
[14]
Garey, M.R., and Johnson, D.S. Computers and Intractability: A Guide to the Theory of NP-Completeness. W.H. Freeman and Co., 1979.
[15]
Greer, D. Decision Support for Planning Software Evolution with Risk Management. Proceedings of 16th International Conference on Software Engineering and Knowledge Engineering, Banff, Canada, 2004, 503--508.
[16]
ISERN (International Software Engineering Research Network) Annual Meeting, Noosa Heads, Australia, 2005.
[17]
Jung, H W. Optimizing Value and Cost in Requirements Analysis. IEEE Software, Vol. 15, No. 4, 1998, 74--78.
[18]
Karlsson, J., and Ryan, K. A Cost-Value Approach for Prioritizing Requirements. IEEE Software, Vol. 14, No. 5, 1997, 67--74.
[19]
Laitenberger, O., Atkinson, C., Schlich, M., and Emam, K. An Experimental Comparison of Reading Techniques for Defect Detection in UML Design Documents. Journal of Systems and Software, Vol. 53, No. 2, 2000, 183--204.
[20]
Lanubile, F. Empirical Evaluation of Software Maintenance Technologies. International Journal of Empirical Software Engineering, Vol. 2, No. 2, 1997, 95--106.
[21]
Lethola, L., Kauppinen, M., and Kujala, S. Requirements Prioritization Challenges in Practice. Proceedings of 4th International Conference on Product Focused Software Process Improvement, Lecture Notes on Computer Science, Vol. 3009, 2004, 497--508.
[22]
Lott, C. and Rombach, H. Repeatable Software Engineering Experiments for Comparing Defect-Detection Techniques. International Journal of Empirical Software Engineering, Vol. 1, No. 3, 1996, 241--277.
[23]
Pawlak, Z. Rough Sets: Theoretical Aspects of Reasoning about Data. Kluwer Academic, Dordrecht, 1991.
[24]
Penny, D. A. An Estimation-Based Management Framework for Enhancive Maintenance in Commercial Software Products. Proceedings of International Conference on Software Maintenance, Montreal, Canada, 2002, 122--130.
[25]
Pfahl, D., Laitenberger, O., Dorsch, J. and Ruhe, G. An Externally Replicated Experiment for Evaluating the Learning Effectiveness of Using Simulations in Software Project Management Education. Journal of Empirical Software Engineering, Vol. 8, No. 4, 2003, 367--395.
[26]
Porter, A., and Votta, L. Comparing Detection Methods for Software Requirements Inspections: A Replication using Professional Subjects. IEEE Transactions on Software Engineering, Vol. 21, No. 6, 1995, 563--575.
[27]
ReleasePlanner, www.releaseplanner.com
[28]
ROSE2, Institute of Computing Science, Poznan University of Technology, http://idss.cs.put.poznan.pl/site/rose.html, January 2006.
[29]
Ruhe, G., and Ngo-The, A. Hybrid Intelligence in Software Release Planning. International Journal of Hybrid Intelligent Systems, Vol. 1, No. 2, 2004, 99--110.
[30]
Ruhe, G., and Saliu, O. The Art and Science of Software Release Planning. IEEE Software, Vol. 22, No. 6, 2005, 47--53.
[31]
Saliu, O., and Ruhe, G. Supporting Software Release Planning Decisions for Evolving Systems. Proceedings of 29th IEEE/NASA Software Engineering Workshop, Greenbelt, MD, USA, 2005, 14--24.
[32]
Shull, F., Basili, V., Carver, J., Maldonado, J., Travassos, G., Mendonça, M., and Fabbri, S. Replicating Software Engineering Experiments: Addressing the Tacit Knowledge Problem. Proceedings of the International Symposium on Empirical Software Engineering, Nara, Japan, 2002, 7--16.
[33]
Wiegers, K. Software Requirements. Microsoft Press, Redmont, 1999.

Cited By

View all
  • (2019)Two machine-learning techniques for mining solutions of the ReleasePlannerTM decision support systemInformation Sciences: an International Journal10.1016/j.ins.2009.12.017259(474-489)Online publication date: 6-Jan-2019
  • (2017)Towards continuous software release planning2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER.2017.7884642(402-406)Online publication date: Feb-2017
  • (2014)Industrial evaluation of the impact of quality-driven release planningProceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2652524.2652579(1-8)Online publication date: 18-Sep-2014
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ISESE '06: Proceedings of the 2006 ACM/IEEE international symposium on Empirical software engineering
September 2006
388 pages
ISBN:1595932186
DOI:10.1145/1159733
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 September 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. decision support
  2. family of empirical studies
  3. release planning

Qualifiers

  • Article

Conference

ISESE06
Sponsor:

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 15 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2019)Two machine-learning techniques for mining solutions of the ReleasePlannerTM decision support systemInformation Sciences: an International Journal10.1016/j.ins.2009.12.017259(474-489)Online publication date: 6-Jan-2019
  • (2017)Towards continuous software release planning2017 IEEE 24th International Conference on Software Analysis, Evolution and Reengineering (SANER)10.1109/SANER.2017.7884642(402-406)Online publication date: Feb-2017
  • (2014)Industrial evaluation of the impact of quality-driven release planningProceedings of the 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/2652524.2652579(1-8)Online publication date: 18-Sep-2014
  • (2010)ReferencesProduct Release Planning10.1201/EBK0849326202-b(305-318)Online publication date: 8-Jul-2010
  • (2009)Towards automation of iteration planningProceedings of the 24th ACM SIGPLAN conference companion on Object oriented programming systems languages and applications10.1145/1639950.1640065(965-972)Online publication date: 25-Oct-2009
  • (2009)Optimized Resource Allocation for Software Release PlanningIEEE Transactions on Software Engineering10.1109/TSE.2008.8035:1(109-123)Online publication date: 1-Jan-2009
  • (2009)Does explanation improve the acceptance of decision support for product release planning?Proceedings of the 2009 3rd International Symposium on Empirical Software Engineering and Measurement10.1109/ESEM.2009.5316049(56-68)Online publication date: 15-Oct-2009
  • (2009)Evaluating the Applicability of a Use Case Precedence Diagram Based Approach in Software Development Projects through a Controlled ExperimentAdvances in Security Technology10.1007/978-3-642-10240-0_10(122-137)Online publication date: 2009
  • (2008)An Approach to Determine Software Requirement Construction Sequences Based on Use CasesProceedings of the 2008 Advanced Software Engineering and Its Applications10.1109/ASEA.2008.33(17-22)Online publication date: 13-Dec-2008
  • (2007)An experiment with a release planning method for web application developmentProceedings of the 14th European conference on Software Process Improvement10.5555/2396167.2396181(106-117)Online publication date: 26-Sep-2007
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media