An empirical investigation into the role of API-level refactorings during software evolution

ABSTRACT

It is widely believed that refactoring improves software quality and programmer productivity by making it easier to maintain and understand software systems. However, the role of refactorings has not been systematically investigated using fine-grained evolution history. We quantitatively and qualitatively studied API-level refactorings and bug fixes in three large open source projects, totaling 26523 revisions of evolution.

The study found several surprising results: One, there is an increase in the number of bug fixes after API-level refactorings. Two, the time taken to fix bugs is shorter after API-level refactorings than before. Three, a large number of refactoring revisions include bug fixes at the same time or are related to later bug fix revisions. Four, API-level refactorings occur more frequently before than after major software releases. These results call for re-thinking refactoring's true benefits. Furthermore, frequent floss refactoring mistakes observed in this study call for new software engineering tools to support safe application of refactoring and behavior modifying edits together.

References

1. J. Aranda and G. Venolia. The secret life of bugs: Going past the errors and omissions in software repositories. In ICSE '09: Proceedings of the 31st International Conference on Software Engineering, pages 298--308, Washington, DC, USA, 2009. IEEE Computer Society.
2. C. Bird, A. Bachmann, E. Aune, J. Duffy, A. Bernstein, V. Filkov, and P. Devanbu. Fair and balanced?: bias in bug-fix datasets. In ESEC/FSE '09: Proceedings of the the 7th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, pages 121--130, New York, NY, USA, 2009. ACM.
3. J. Carriere, R. Kazman, and I. Ozkaya. A cost-benefit framework for making architectural decisions in a business context. In ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, pages 149--157, New York, NY, USA, 2010. ACM.
4. W. Cunningham. The wycash portfolio management system. In OOPSLA '92: Addendum to the proceedings on Object-oriented programming systems, languages, and applications (Addendum), pages 29--30, New York, NY, USA, 1992. ACM.
5. P. Dalgaard. Introductory statistics with R. Springer, New York, 2. ed. edition, 2008.
6. B. Daniel, D. Dig, K. Garcia, and D. Marinov. Automated testing of refactoring engines. In ESEC-FSE '07: Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering, pages 185--194, New York, NY, USA, 2007. ACM.
7. D. Dig and R. Johnson. The role of refactorings in api evolution. In ICSM '05: Proceedings of the 21st IEEE International Conference on Software Maintenance, pages 389--398, Washington, DC, USA, 2005. IEEE Computer Society.
8. D. Dig and R. Johnson. Automated detection of refactorings in evolving components. In ECOOP '06: Proceedings of European Conference on Object-Oriented Programming, pages 404--428. Springer, 2006.
9. M. Fischer, M. Pinzger, and H. Gall. Populating a release history database from version control and bug tracking systems. In ICSM '03: Proceedings of the International Conference on Software Maintenance, page 23, Washington, DC, USA, 2003. IEEE Computer Society.
10. B. Fluri, M. Würsch, M. Pinzger, and H. C. Gall. Change distilling: Tree differencing for fine-grained source code change extraction. IEEE Transactions on Software Engineering, 33(11):725--743, August 2007.
11. M. Fowler. Refactoring: Improving the Design of Existing Code. Addison-Wesley Professional, 2000.
12. C. Görg and P. Weißgerber. Error detection by refactoring reconstruction. In MSR '05: Proceedings of the 2005 international workshop on Mining software repositories, pages 1--5, New York, NY, USA, 2005. ACM.
13. W. G. Griswold. Program Restructuring as an Aid to Software Maintenance. PhD thesis, University of Washington, WA, USA, 1991.
14. A. Hindle, D. M. German, and R. Holt. What do large commits tell us?: a taxonomical study of large commits. In MSR '08: Proceedings of the 2008 international working conference on Mining software repositories, pages 99--108, New York, NY, USA, 2008. ACM.
15. M. Kim and D. Notkin. Discovering and representing systematic code changes. In ICSE '09: Proceedings of the 2009 IEEE 31st International Conference on Software Engineering, pages 309--319, Washington, DC, USA, 2009. IEEE Computer Society.
16. M. Kim, D. Notkin, and D. Grossman. Automatic inference of structural changes for matching across program versions. In ICSE '07: Proceedings of the 29th International Conference on Software Engineering, pages 333--343, Washington, DC, USA, 2007. IEEE Computer Society.
17. S. Kim, T. Zimmermann, E. J. Whitehead, Jr., and A. Zeller. Predicting faults from cached history. In ICSE '07: Proceedings of the 29th international conference on Software Engineering, pages 489--498, Washington, DC, USA, 2007. IEEE Computer Society.
18. M. M. Lehman. On understanding laws, evolution, and conservation in the large-program life cycle. Journal of Systems and Software, 1:213--221, 1980.
19. T. Mens and T. Tourwe. A survey of software refactoring. IEEE Transactions on Software Engineering, 30(2):126--139, February 2004.
20. E. Murphy-hill and A. P. Black. Why don't people use refactoring tools. In ECOOP '07: Proceedings of the 1st Workshop on Refactoring Tools, TU Berlin, Germany, 2007.
21. E. Murphy-Hill, C. Parnin, and A. P. Black. How we refactor, and how we know it. In ICSE '09: Proceedings of the 31st International Conference on Software Engineering, pages 287--297, Washington, DC, USA, 2009. IEEE Computer Society.
22. N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In ICSE '05: Proceedings of the 27th International Conference on Software Engineering, pages 284--292, New York, NY, USA, 2005. ACM.
23. H. A. Nguyen, T. T. Nguyen, G. Wilson, Jr., A. T. Nguyen, M. Kim, and T. N. Nguyen. A graph-based approach to api usage adaptation. In Proceedings of the ACM international conference on Object oriented programming systems languages and applications, OOPSLA '10, pages 302--321, New York, NY, USA, 2010. ACM.
24. K. Prete, N. Rachatasumrit, N. Sudan, and M. Kim. Template-based reconstruction of complex refactorings. In Software Maintenance (ICSM), 2010 IEEE International Conference on, pages 1--10, 2010.
25. J. Ratzinger, T. Sigmund, and H. C. Gall. On the relation of refactorings and software defect prediction. In MSR '08: Proceedings of the 2008 international working conference on Mining software repositories, pages 35--38, New York, NY, USA, 2008. ACM.
26. C. Reichenbach, D. Coughlin, and A. Diwan. Program metamorphosis. In Genoa: Proceedings of the 23rd European Conference on ECOOP 2009 Object-Oriented Programming, pages 394--418, Berlin, Heidelberg, 2009. Springer-Verlag.
27. D. Roberts, J. Brant, and R. Johnson. A refactoring tool for smalltalk. Theory and Practice of Object Systems, 3(4):253--263, 1997.
28. J. Śliwerski, T. Zimmermann, and A. Zeller. When do changes induce fixes? In MSR '05: Proceedings of the 2005 international workshop on Mining software repositories, pages 1--5, New York, NY, USA, 2005. ACM.
29. D. Čubranić and G. C. Murphy. Hipikat: recommending pertinent software development artifacts. In ICSE '03: Proceedings of the 25th International Conference on Software Engineering, pages 408--418, Washington, DC, USA, 2003. IEEE Computer Society.
30. P. Weißgerber and S. Diehl. Are refactorings less error-prone than other changes? In MSR '06: Proceedings of the 2006 international workshop on Mining software repositories, pages 112--118, New York, NY, USA, 2006. ACM.
31. W. Wu, Y.-G. Guéhéneuc, G. Antoniol, and M. Kim. Aura: a hybrid approach to identify framework evolution. In ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering, pages 325--334, New York, NY, USA, 2010. ACM.
32. Z. Xing and E. Stroulia. Umldiff: an algorithm for object-oriented design differencing. In ASE '05: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering, pages 54--65, New York, NY, USA, 2005. ACM.
33. Z. Xing and E. Stroulia. Refactoring practice: How it is and how it should be supported - an eclipse case study. In ICSM '06: Proceedings of the 22nd IEEE International Conference on Software Maintenance, pages 458--468, Washington, DC, USA, 2006. IEEE Computer Society.

Index Terms

• Software and its engineering

  • Software creation and management

    • Software post-development issues

      • Software reverse engineering