ABSTRACT
Agile development methods are increasing in popularity, yet there are limited studies on the reasons and use of metrics in industrial agile development. This paper presents preliminary results from a systematic literature review. Based on our study, metrics and their use are focused to the following areas: Iteration planning, Iteration tracking, Motivating and improving, Identifying process problems, Pre-release quality, Post-release quality and Changes in processes or tools. The findings are mapped against agile principles and it seems that the use of metrics supports the principles with some deviations. Surprisingly, we find little evidence of the use of code metrics. Also, we note that there is a lot of evidence on the use of planning and tracking metrics. Finally, the use of metrics to motivate and enforce process improvements as well as applicable quality metrics can be interesting future research topics.
- K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, B. Marick, R. C. Martin, S. Mellor, K. Schwaber, J. Sutherland, and D. Thomas. Manifesto for agile software development.Google Scholar
- C. Catal and B. Diri. A systematic review of software fault prediction studies. Expert Systems with Applications, 36(4):7346–7354, 2009. Google ScholarDigital Library
- T.-H. Cheng, S. Jansen, and M. Remmers. Controlling and monitoring agile software development in three dutch product software companies. In Proceedings of the 2009 ICSE Workshop on Software Development Governance, SDG 2009, pages 29–35, Vancouver, BC, 2009. Google ScholarDigital Library
- D. Cruzes and T. Dyba. Recommended steps for thematic synthesis in software engineering. In Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, pages 275–284, 2011. Google ScholarDigital Library
- P. S. M. dos Santos, A. Varella, C. R. Dantas, and D. B. ao Borges. Visualizing and managing technical debt in agile development: An experience report. In Agile Processes in Software Engineering and Extreme Programming, volume 149 of Lecture Notes in Business Information Processing, pages 121–134, 2013.Google Scholar
- Y. Dubinsky, D. Talby, O. Hazzan, and A. Keren. Agile metrics at the israeli air force. In Proceedings - AGILE Confernce 2005, volume 2005, pages 12–19, Denver, CO, 2005. Google ScholarDigital Library
- T. Dyb˚ a and T. Dingsøyr. Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9):833–859, Aug. 2008. Google ScholarDigital Library
- A. Elssamadisy and G. Schalliol. Recognizing and responding to ”bad smells” in extreme programming. In Proceedings - International Conference on Software Engineering, pages 617–622, Orlando, FL, 2002. Google ScholarDigital Library
- P. Green. Measuring the impact of scrum on product development at adobe systems. In Proceedings of the Annual Hawaii International Conference on System Sciences, Koloa, Kauai, HI, 2011. Google ScholarDigital Library
- D. Greening. Enterprise scrum: Scaling scrum to the executive level. In Proceedings of the Annual Hawaii International Conference on System Sciences, Koloa, Kauai, HI, 2010. Google ScholarDigital Library
- D. Hartmann and R. Dymond. Appropriate agile measurement: using metrics and diagnostics to deliver business value. In Agile Conference, 2006, pages 6 pp.–134, 2006. Google ScholarDigital Library
- N. Haugen. An empirical study of using planning poker for user story estimation. In Proceedings - AGILE Conference, 2006, volume 2006, pages 23–31, Minneapolis, MN, 2006. Google ScholarDigital Library
- P. Hodgetts. Refactoring the development process: Experiences with the incremental adoption of agile practices. In Proceedings of the Agile Development Conference, ADC 2004, pages 106–113, Salt Lake City, UT, 2004. Google ScholarDigital Library
- P. Hodgkins and L. Hohmann. Agile program management: Lessons learned from the verisign managed security services team. In Proceedings - AGILE 2007, pages 194–199, Washington, DC, 2007. Google ScholarDigital Library
- N. Hong, J. Yoo, and S. Cha. Customization of scrum methodology for outsourced e-commerce projects. In Proceedings - Asia-Pacific Software Engineering Conference, APSEC, pages 310–315, Sydney, NSW, 2010. Google ScholarDigital Library
- ISO/IEC. ISO/IEC 25010 - Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. Technical report, 2010.Google Scholar
- C. Jakobsen and T. Poppendieck. Lean as a scrum troubleshooter. In Proceedings - 2011 Agile Conference, Agile 2011, pages 168–174, Salt Lake City, UT, 2011. Google ScholarDigital Library
- A. Janus, R. Dumke, A. Schmietendorf, and J. Jager. The 3c approach for agile quality assurance. In Emerging Trends in Software Metrics (WETSoM), 2012 3rd International Workshop on, pages 9–13, 2012.Google ScholarCross Ref
- B. Kitchenham. Procedures for performing systematic reviews. Keele, UK, Keele University, 33:2004, 2004.Google Scholar
- B. Kitchenham. What’s up with software metrics? - a preliminary mapping study. Journal of Systems and Software, 83(1):37–51, Jan. 2010. Google ScholarDigital Library
- B. Kitchenham and P. Brereton. A systematic review of systematic review process research in software engineering. Information and Software Technology, 55(12):2049–2075, 2013. Google ScholarDigital Library
- K. Korhonen. Migrating defect management from waterfall to agile software development in a large-scale multi-site organization: A case study. In P. Abrahamsson, M. Marchesi, and F. Maurer, editors, Agile Processes in Software Engineering and Extreme Programming, volume 31 of Lecture Notes in Business Information Processing, pages 73–82. Springer Berlin Heidelberg, 2009.Google Scholar
- J. Landis and G. Koch. The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174, 1977.Google ScholarCross Ref
- V. Mahnic and N. Zabkar. Measuring progress of scrum-based software projects. Electronics and Electrical Engineering, 18(8):73–76, 2012.Google Scholar
- P. Middleton, P. Taylor, A. Flaxel, and A. Cookson. Lean principles and techniques for improving the quality and productivity of software development projects: A case study. International Journal of Productivity and Quality Management, 2(4):387–403, 2007.Google ScholarCross Ref
- S. Mujtaba, R. Feldt, and K. Petersen. Waste and lead time reduction in a software product customization process with value stream maps. In Proceedings of the Australian Software Engineering Conference, ASWEC, pages 139–148, Auckland, 2010. Google ScholarDigital Library
- C. Patel, M. Lycett, R. Macredie, and S. de Cesare. Perceptions of agility and collaboration in software development practice. In System Sciences, 2006. HICSS ’06. Proceedings of the 39th Annual Hawaii International Conference on, volume 1, pages 10c–10c, 2006. Google ScholarDigital Library
- K. Petersen. A palette of lean indicators to detect waste in software maintenance: A case study. Lecture Notes in Business Information Processing, 111 LNBIP:108–122, 2012.Google Scholar
- K. Petersen and C. Wohlin. The effect of moving from a plan-driven to an incremental software development approach with agile practices: An industrial case study. Empirical Software Engineering, 15(6):654–693, 2010. Google ScholarDigital Library
- K. Petersen and C. Wohlin. Software process improvement through the lean measurement (spi-leam) method. Journal of Systems and Software, 83(7):1275–1287, 2010. cited By (since 1996)6. Google ScholarDigital Library
- K. Petersen and C. Wohlin. Measuring the flow in lean software development. Software - Practice and Experience, 41(9):975–996, 2011. Google ScholarDigital Library
- R. Polk. Agile & kanban in coordination. In Proceedings - 2011 Agile Conference, Agile 2011, pages 263––268, Salt Lake City, UT, 2011. Google ScholarDigital Library
- S. Purao and V. Vaishnavi. Product metrics for object-oriented systems. ACM Computing Surveys (CSUR), 35(2):191–221, 2003. Google ScholarDigital Library
- G. Seber. The Estimation of Animal Abundance and Related Parameters. Blackburn Press, 2002.Google Scholar
- M. Seikola, H.-M. Loisa, and A. Jagos. Kanban implementation in a telecom product maintenance. In Proceedings - 37th EUROMICRO Conference on Software Engineering and Advanced Applications, SEAA 2011, pages 321–329, Oulu, 2011. Google ScholarDigital Library
- B. Shen and D. Ju. On the measurement of agility in software process. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 4470 LNCS:25–36, 2007. Google ScholarDigital Library
- M. Staron and W. Meding. Monitoring bottlenecks in agile and lean software development projects - a method and its industrial use. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6759 LNCS:3–16, 2011. Google ScholarDigital Library
- M. Staron, W. Meding, and B. Söderqvist. A method for forecasting defect backlog in large streamline software development projects and its industrial evaluation. Information and Software Technology, 52(10):1069–1079, 2010. Google ScholarDigital Library
- D. Talby and Y. Dubinsky. Governance of an agile software project. In Proceedings of the 2009 ICSE Workshop on Software Development Governance, SDG 2009, pages 40–45, Vancouver, BC, 2009. Google ScholarDigital Library
- D. Talby, O. Hazzan, Y. Dubinsky, and A. Keren. Reflections on reflection in agile software development. In Proceedings - AGILE Conference, 2006, volume 2006, pages 100–110, Minneapolis, MN, 2006. Google ScholarDigital Library
- V. Trapa and S. Rao. T3 - tool for monitoring agile development. In Proceedings - AGILE Conference, 2006, volume 2006, pages 243–248, Minneapolis, MN, 2006. Google ScholarDigital Library
- J. Trimble and C. Webster. From traditional, to lean, to agile development: Finding the optimal software engineering cycle. In Proceedings of the Annual Hawaii International Conference on System Sciences, pages 4826–4833, Wailea, Maui, HI, 2013. Google ScholarDigital Library
- D. Tudor and G. Walter. Using an agile approach in a large, traditional organization. In Proceedings - AGILE Conference, 2006, volume 2006, pages 367–373, Minneapolis, MN, 2006. Google ScholarDigital Library
Index Terms
Why are industrial agile teams using metrics and how do they use them?
Recommendations
Using metrics in Agile and Lean Software Development - A systematic literature review of industrial studies
ContextSoftware industry has widely adopted Agile software development methods. Agile literature proposes a few key metrics but little is known of the actual metrics use in Agile teams. ObjectiveThe objective of this paper is to increase knowledge of ...
How Agile Organizations Use Metrics: A Systematic Literature Mapping
SBQS '22: Proceedings of the XXI Brazilian Symposium on Software QualityAgile Software Development Methods (ASDMs) have addressed the complexity of managing software development processes through small increments, continuous and frequent delivery, openness to change, and work visibility. These practices have been ...
Group development and group maturity when building agile teams
Overlap between what is meant by an agile team and a mature group in psychology.Group developmental issues are dealt with in practice in agile groups.Quantitative survey data support the connections.Focusing on group development is a key success factor ...
Comments