Does adding manpower also affect quality?: an empirical, longitudinal analysis

ABSTRACT

With each new developer to a software development team comes a greater challenge to manage the communication, coordination, and knowledge transfer amongst teammates. Fred Brooks discusses this challenge in The Mythical Man-Month by arguing that rapid team expansion can lead to a complex team organization structure. While Brooks focuses on productivity loss as the negative outcome, poor product quality is also a substantial concern. But if team expansion is unavoidable, can any quality impacts be mitigated? Our objective is to guide software engineering managers by empirically analyzing the effects of team size, expansion, and structure on product quality. We performed an empirical, longitudinal case study of a large Cisco networking product over a five year history. Over that time, the team underwent periods of no expansion, steady expansion, and accelerated expansion. Using team-level metrics, we quantified characteristics of team expansion, including team size, expansion rate, expansion acceleration, and modularity with respect to department designations. We examined statistical correlations between our monthly team-level metrics and monthly product-level metrics. Our results indicate that increased team size and linear growth are correlated with later periods of better product quality. However, periods of accelerated team expansion are correlated with later periods of reduced software quality. Furthermore, our linear regression prediction model based on team metrics was able to predict the product's post-release failure rate within a 95% prediction interval for 38 out of 40 months. Our analysis provides insight for project managers into how the expansion of development teams can impact product quality.

References

1. A. Begel, Y. P. Khoo, and T. Zimmermann, "Codebook: discovering and exploiting relationships in software repositories," in Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1, pp. 125--134, 2010.
2. C. Bird, D. Pattison, R. D'Souza, V. Filkov, and P. Devanbu, "Latent social structure in open source projects," in Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, pp. 24--35, 2008.
3. U. Brandes and T. Erlebach, Network Analysis: Methodological Foundations, 1st ed. Springer, 2005.
4. F. P. Brooks, The Mythical Man-Month: Essays on Software Engineering, Anniversary Edition, Anniversary. Addison-Wesley Professional, 1995.
5. T. G. Cummings, "Self-Regulating Work Groups: A Socio-Technical Synthesis," The Academy of Management Review, vol. 3, no. 3, pp. 625--634, Jul. 1978.
6. ISO/IEC 24765, "Software and Systems Engineering Vocabulary," 2006.
7. A. Meneely and L. Williams, "Secure Open Source Collaboration: an Empirical Study of Linus' Law," in Computer and Communications Security, pp. 453--462, 2009.
8. A. Meneely and L. Williams, "Strengthening the empirical analysis of the relationship between Linus' Law and software security," in Proceedings of the 2010 ACM-IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 1--10, 2010.
9. A. Meneely and L. Williams, "Socio-Technical Developer Networks: Should We Trust Our Measurements?," presented at the International Conference on Software Engineering, Waikiki, Hawaii, USA, p. to appear, 2011.
10. A. Meneely, L. Williams, W. Snipes, and J. Osborne, "Predicting failures with developer networks and social network analysis," in Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, pp. 13--23, 2008.
11. A. Meneely, L. Williams, W. Snipes, and J. Osborne, "Predicting failures with developer networks and social network analysis," in Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, pp. 13--23, 2008.
12. M. Pinzger, N. Nagappan, and B. Murphy, "Can developer-module networks predict failures?," in Proceedings of the 16th ACM SIGSOFT International Symposium on Foundations of software engineering, pp. 2--12, 2008.
13. A. Sarma, L. Maccherone, P. Wagstrom, and J. Herbsleb, "Tesseract: Interactive visual exploration of socio-technical relationships in software development," in Proceedings of the 31st International Conference on Software Engineering, pp. 23--33, 2009.
14. Y. Shin, A. Meneely, L. Williams, and J. Osborne, "Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities," IEEE Trans. Softw. Eng., 2011.
15. C. Treude and M. Storey, "Awareness 2.0: staying aware of projects, developers and tasks using dashboards and feeds," in Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1, pp. 365--374, 2010.
16. E. L. Trist and K. W. Bamforth, "Some social and psychological consequences of the longwall method of coal-getting," Technology, Organizations and Innovation: The early debates, p. 79, 2000.
17. E. J. Weyuker, T. J. Ostrand, and R. M. Bell, "Do too many cooks spoil the broth? Using the number of developers to enhance defect prediction models," Empirical Softw. Engg., vol. 13, no. 5, pp. 539--559, 2008.
18. R. Williams, J. Walker, and A. Dorofee, "Putting risk management into practice," Software, IEEE, vol. 14, no. 3, pp. 75--82, 1997.
19. T. Wolf, A. Schroter, D. Damian, and T. Nguyen, "Predicting build failures using social network analysis on developer communication," in Proceedings of the 31st International Conference on Software Engineering, pp. 1--11, 2009.

Index Terms

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Process validation