skip to main content
10.1145/2593868.2593873acmconferencesArticle/Chapter ViewAbstractPublication PagesicseConference Proceedingsconference-collections
Article

Why are industrial agile teams using metrics and how do they use them?

Published:03 June 2014Publication History

ABSTRACT

Agile development methods are increasing in popularity, yet there are limited studies on the reasons and use of metrics in industrial agile development. This paper presents preliminary results from a systematic literature review. Based on our study, metrics and their use are focused to the following areas: Iteration planning, Iteration tracking, Motivating and improving, Identifying process problems, Pre-release quality, Post-release quality and Changes in processes or tools. The findings are mapped against agile principles and it seems that the use of metrics supports the principles with some deviations. Surprisingly, we find little evidence of the use of code metrics. Also, we note that there is a lot of evidence on the use of planning and tracking metrics. Finally, the use of metrics to motivate and enforce process improvements as well as applicable quality metrics can be interesting future research topics.

References

  1. K. Beck, M. Beedle, A. van Bennekum, A. Cockburn, W. Cunningham, M. Fowler, J. Grenning, J. Highsmith, A. Hunt, R. Jeffries, J. Kern, B. Marick, R. C. Martin, S. Mellor, K. Schwaber, J. Sutherland, and D. Thomas. Manifesto for agile software development.Google ScholarGoogle Scholar
  2. C. Catal and B. Diri. A systematic review of software fault prediction studies. Expert Systems with Applications, 36(4):7346–7354, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. T.-H. Cheng, S. Jansen, and M. Remmers. Controlling and monitoring agile software development in three dutch product software companies. In Proceedings of the 2009 ICSE Workshop on Software Development Governance, SDG 2009, pages 29–35, Vancouver, BC, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  4. D. Cruzes and T. Dyba. Recommended steps for thematic synthesis in software engineering. In Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, pages 275–284, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. P. S. M. dos Santos, A. Varella, C. R. Dantas, and D. B. ao Borges. Visualizing and managing technical debt in agile development: An experience report. In Agile Processes in Software Engineering and Extreme Programming, volume 149 of Lecture Notes in Business Information Processing, pages 121–134, 2013.Google ScholarGoogle Scholar
  6. Y. Dubinsky, D. Talby, O. Hazzan, and A. Keren. Agile metrics at the israeli air force. In Proceedings - AGILE Confernce 2005, volume 2005, pages 12–19, Denver, CO, 2005. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. T. Dyb˚ a and T. Dingsøyr. Empirical studies of agile software development: A systematic review. Information and Software Technology, 50(9):833–859, Aug. 2008. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. A. Elssamadisy and G. Schalliol. Recognizing and responding to ”bad smells” in extreme programming. In Proceedings - International Conference on Software Engineering, pages 617–622, Orlando, FL, 2002. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. P. Green. Measuring the impact of scrum on product development at adobe systems. In Proceedings of the Annual Hawaii International Conference on System Sciences, Koloa, Kauai, HI, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. D. Greening. Enterprise scrum: Scaling scrum to the executive level. In Proceedings of the Annual Hawaii International Conference on System Sciences, Koloa, Kauai, HI, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. D. Hartmann and R. Dymond. Appropriate agile measurement: using metrics and diagnostics to deliver business value. In Agile Conference, 2006, pages 6 pp.–134, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. N. Haugen. An empirical study of using planning poker for user story estimation. In Proceedings - AGILE Conference, 2006, volume 2006, pages 23–31, Minneapolis, MN, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. P. Hodgetts. Refactoring the development process: Experiences with the incremental adoption of agile practices. In Proceedings of the Agile Development Conference, ADC 2004, pages 106–113, Salt Lake City, UT, 2004. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. P. Hodgkins and L. Hohmann. Agile program management: Lessons learned from the verisign managed security services team. In Proceedings - AGILE 2007, pages 194–199, Washington, DC, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. N. Hong, J. Yoo, and S. Cha. Customization of scrum methodology for outsourced e-commerce projects. In Proceedings - Asia-Pacific Software Engineering Conference, APSEC, pages 310–315, Sydney, NSW, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. ISO/IEC. ISO/IEC 25010 - Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQuaRE) - System and software quality models. Technical report, 2010.Google ScholarGoogle Scholar
  17. C. Jakobsen and T. Poppendieck. Lean as a scrum troubleshooter. In Proceedings - 2011 Agile Conference, Agile 2011, pages 168–174, Salt Lake City, UT, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. A. Janus, R. Dumke, A. Schmietendorf, and J. Jager. The 3c approach for agile quality assurance. In Emerging Trends in Software Metrics (WETSoM), 2012 3rd International Workshop on, pages 9–13, 2012.Google ScholarGoogle ScholarCross RefCross Ref
  19. B. Kitchenham. Procedures for performing systematic reviews. Keele, UK, Keele University, 33:2004, 2004.Google ScholarGoogle Scholar
  20. B. Kitchenham. What’s up with software metrics? - a preliminary mapping study. Journal of Systems and Software, 83(1):37–51, Jan. 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. B. Kitchenham and P. Brereton. A systematic review of systematic review process research in software engineering. Information and Software Technology, 55(12):2049–2075, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. K. Korhonen. Migrating defect management from waterfall to agile software development in a large-scale multi-site organization: A case study. In P. Abrahamsson, M. Marchesi, and F. Maurer, editors, Agile Processes in Software Engineering and Extreme Programming, volume 31 of Lecture Notes in Business Information Processing, pages 73–82. Springer Berlin Heidelberg, 2009.Google ScholarGoogle Scholar
  23. J. Landis and G. Koch. The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174, 1977.Google ScholarGoogle ScholarCross RefCross Ref
  24. V. Mahnic and N. Zabkar. Measuring progress of scrum-based software projects. Electronics and Electrical Engineering, 18(8):73–76, 2012.Google ScholarGoogle Scholar
  25. P. Middleton, P. Taylor, A. Flaxel, and A. Cookson. Lean principles and techniques for improving the quality and productivity of software development projects: A case study. International Journal of Productivity and Quality Management, 2(4):387–403, 2007.Google ScholarGoogle ScholarCross RefCross Ref
  26. S. Mujtaba, R. Feldt, and K. Petersen. Waste and lead time reduction in a software product customization process with value stream maps. In Proceedings of the Australian Software Engineering Conference, ASWEC, pages 139–148, Auckland, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. C. Patel, M. Lycett, R. Macredie, and S. de Cesare. Perceptions of agility and collaboration in software development practice. In System Sciences, 2006. HICSS ’06. Proceedings of the 39th Annual Hawaii International Conference on, volume 1, pages 10c–10c, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. K. Petersen. A palette of lean indicators to detect waste in software maintenance: A case study. Lecture Notes in Business Information Processing, 111 LNBIP:108–122, 2012.Google ScholarGoogle Scholar
  29. K. Petersen and C. Wohlin. The effect of moving from a plan-driven to an incremental software development approach with agile practices: An industrial case study. Empirical Software Engineering, 15(6):654–693, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. K. Petersen and C. Wohlin. Software process improvement through the lean measurement (spi-leam) method. Journal of Systems and Software, 83(7):1275–1287, 2010. cited By (since 1996)6. Google ScholarGoogle ScholarDigital LibraryDigital Library
  31. K. Petersen and C. Wohlin. Measuring the flow in lean software development. Software - Practice and Experience, 41(9):975–996, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  32. R. Polk. Agile & kanban in coordination. In Proceedings - 2011 Agile Conference, Agile 2011, pages 263––268, Salt Lake City, UT, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. S. Purao and V. Vaishnavi. Product metrics for object-oriented systems. ACM Computing Surveys (CSUR), 35(2):191–221, 2003. Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. G. Seber. The Estimation of Animal Abundance and Related Parameters. Blackburn Press, 2002.Google ScholarGoogle Scholar
  35. M. Seikola, H.-M. Loisa, and A. Jagos. Kanban implementation in a telecom product maintenance. In Proceedings - 37th EUROMICRO Conference on Software Engineering and Advanced Applications, SEAA 2011, pages 321–329, Oulu, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  36. B. Shen and D. Ju. On the measurement of agility in software process. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 4470 LNCS:25–36, 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. M. Staron and W. Meding. Monitoring bottlenecks in agile and lean software development projects - a method and its industrial use. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 6759 LNCS:3–16, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  38. M. Staron, W. Meding, and B. Söderqvist. A method for forecasting defect backlog in large streamline software development projects and its industrial evaluation. Information and Software Technology, 52(10):1069–1079, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. D. Talby and Y. Dubinsky. Governance of an agile software project. In Proceedings of the 2009 ICSE Workshop on Software Development Governance, SDG 2009, pages 40–45, Vancouver, BC, 2009. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. D. Talby, O. Hazzan, Y. Dubinsky, and A. Keren. Reflections on reflection in agile software development. In Proceedings - AGILE Conference, 2006, volume 2006, pages 100–110, Minneapolis, MN, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. V. Trapa and S. Rao. T3 - tool for monitoring agile development. In Proceedings - AGILE Conference, 2006, volume 2006, pages 243–248, Minneapolis, MN, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. J. Trimble and C. Webster. From traditional, to lean, to agile development: Finding the optimal software engineering cycle. In Proceedings of the Annual Hawaii International Conference on System Sciences, pages 4826–4833, Wailea, Maui, HI, 2013. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. D. Tudor and G. Walter. Using an agile approach in a large, traditional organization. In Proceedings - AGILE Conference, 2006, volume 2006, pages 367–373, Minneapolis, MN, 2006. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Why are industrial agile teams using metrics and how do they use them?

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      WETSoM 2014: Proceedings of the 5th International Workshop on Emerging Trends in Software Metrics
      June 2014
      72 pages
      ISBN:9781450328548
      DOI:10.1145/2593868

      Copyright © 2014 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 3 June 2014

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • Article

      Upcoming Conference

      ICSE 2024

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader