An exploratory study of the pull-based software development model

ABSTRACT

The advent of distributed version control systems has led to the development of a new paradigm for distributed software development; instead of pushing changes to a central repository, developers pull them from other repositories and merge them locally. Various code hosting sites, notably Github, have tapped on the opportunity to facilitate pull-based development by offering workflow support tools, such as code reviewing systems and integrated issue trackers. In this work, we explore how pull-based software development works, first on the GHTorrent corpus and then on a carefully selected sample of 291 projects. We find that the pull request model offers fast turnaround, increased opportunities for community engagement and decreased time to incorporate contributions. We show that a relatively small number of factors affect both the decision to merge a pull request and the time to process it. We also examine the reasons for pull request rejection and find that technical ones are only a small minority.

References

1. J. Anvik, L. Hiew, and G. C. Murphy. Who should fix this bug? In Proceedings of ICSE ’06, pages 361–370. ACM, 2006.
2. E. T. Barr, C. Bird, P. C. Rigby, A. Hindle, D. M. German, and P. Devanbu. Cohesive and isolated development with branches. In Proceedings of FASE ’12. Springer, 2012.
3. O. Baysal, R. Holmes, and M. W. Godfrey. Mining usage data and development artifacts. In Proceedings of MSR ’09, pages 98–107. IEEE, 2012.
4. C. Bird, A. Gourley, and P. Devanbu. Detecting patch submission and acceptance in oss projects. In Proceedings of MSR ’07, page 26. IEEE Computer Society, 2007.
5. C. Bird, A. Gourley, P. Devanbu, A. Swaminathan, and G. Hsu. Open borders? Immigration in open source projects. In Proceedings of MSR ’07, page 6. IEEE Computer Society, 2007.
6. C. Bird, P. C. Rigby, E. T. Barr, D. J. Hamilton, D. M. German, and P. Devanbu. The promises and perils of mining Git. In Proceedings of MSR ’09, pages 1–10, 2009.
7. C. Bird and T. Zimmermann. Assessing the value of branches with what-if analysis. In Proceedings of FSE ’12, pages 45:1–45:11. ACM, 2012.
8. S. Chacon. Pro Git. Expert’s Voice in Software Development. Apress, 1rst edition, Aug 2009.
9. L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb. Social coding in Github: transparency and collaboration in an open software repository. In Proceedings of CSCW ’12, pages 1277–1286. ACM, 2012.
10. L. Dabbish, C. Stuart, J. Tsay, and J. Herbsleb. Leveraging transparency. IEEE Software, 30(1):37–43, 2013.
11. B. Fluri, M. Wursch, M. PInzger, and H. Gall. Change distilling: Tree differencing for fine-grained source code change extraction. IEEE Trans. Soft. Eng., 33(11):725–743, 2007.
12. R. Genuer, J.-M. Poggi, and C. Tuleau-Malot. Variable selection using random forests. Pattern Recognition Letters, 31(14):2225 – 2236, 2010.
13. E. Giger, M. D’Ambros, M. Pinzger, and H. C. Gall. Method-level bug prediction. In In Proceedings of ESEM ’12, pages 171–180. ACM, 2012.
14. E. Giger, M. Pinzger, and H. Gall. Predicting the fix time of bugs. In In Proceedings of RSSE ’10, pages 52–56. ACM, 2010.
15. T. Girba, S. Ducasse, and M. Lanza. Yesterday’s weather: guiding early reverse engineering efforts by summarizing the evolution of changes. In Proceedings of ICSM ’04, pages 40 – 49, sept. 2004.
16. G. Gousios. The GHTorrent dataset and tool suite. In Proceedings of MSR ’13, May 2013.
17. N. V. Ivankova, J. W. Creswell, and S. L. Stick. Using mixed-methods sequential explanatory design: From theory to practice. Field Methods, 18(1):3–20, 2006.
18. C. Jensen and W. Scacchi. Role migration and advancement processes in OSSD projects: A comparative case study. In Proceedings of ICSE ’07, pages 364–374. IEEE Computer Society, 2007.
19. G. Jeong, S. Kim, T. Zimmermann, and K. Yi. Improving code review by predicting reviewers and acceptance of patches. Research on Software Analysis for Error-free Computing Center Tech-Memo (ROSAEC MEMO), 2009.
20. Y. Jiang, B. Adams, and D. M. German. Will my patch make it? and how fast?: case study on the Linux kernel. In Proceedings of MSR ’13, pages 101–110. IEEE Press, 2013.
21. S. Lessmann, B. Baesens, C. Mues, and S. Pietsch. Benchmarking classification models for software defect prediction: A proposed framework and novel findings. IEEE Trans. Softw. Eng., 34(4):485–496, July 2008.
22. N. McDonald and S. Goggins. Performance and participation in open source software on github. In CHI ’13 Extended Abstracts on Human Factors in Computing Systems, CHI EA ’13, pages 139–144. ACM, 2013.
23. A. Mockus, R. T. Fielding, and J. D. Herbsleb. Two case studies of open source software development: Apache and Mozilla. ACM Trans. Softw. Eng. Methodol., 11(3):309–346, 2002.
24. N. Nagappan and T. Ball. Use of relative code churn measures to predict system defect density. In Proceedings of ICSE ’05, pages 284–292. ACM, 2005.
25. K. Peterson. The github open source development process. Technical report, Mayo Clinic, May 2013.
26. R. Pham, L. Singer, O. Liskin, F. Figueira Filho, and K. Schneider. Creating a shared understanding of testing culture on a social coding site. In Proceedings of ICSE ’13, pages 112–121. IEEE Press, 2013.
27. J. Ratzinger, M. Pinzger, and H. Gall. EQ-mine: predicting short-term defects for software evolution. In Proceedings of FASE ’07, pages 12–26. Springer-Verlag, 2007.
28. P. C. Rigby and C. Bird. Convergent software peer review practices. In Proceedings of FSE ’13, 2013.
29. P. C. Rigby and D. M. German. A preliminary examination of code review processes in open source projects. University of Victoria, Canada, Tech. Rep. DCS-305-IR, 2006.
30. P. C. Rigby, D. M. German, and M.-A. Storey. Open source software peer review practices: a case study of the Apache server. In Proceedings of ICSE ’08, pages 541–550. ACM, 2008.
31. E. Shihab, C. Bird, and T. Zimmermann. The effect of branching strategies on software quality. In In Proceedings of ESEM ’12, pages 301–310. ACM, 2012.
32. P. Weißgerber, D. Neu, and S. Diehl. Small patches get in! In Proceedings of MSR ’08, pages 67–76. ACM, 2008.

Index Terms

• Social and professional topics

  • Professional topics

    • Management of computing and information systems

      • Project and people management

      • Software management

        • Software maintenance

• Software and its engineering

  • Software creation and management

    • Software development process management

    • Software post-development issues

  • Software notations and tools

    • Software configuration management and version control systems