Almost there: a study on quasi-contributors in open source software projects

ABSTRACT

Recent studies suggest that well-known OSS projects struggle to find the needed workforce to continue evolving---in part because external developers fail to overcome their first contribution barriers. In this paper, we investigate how and why quasi-contributors (external developers who did not succeed in getting their contributions accepted to an OSS project) fail. To achieve our goal, we collected data from 21 popular, non-trivial GitHub projects, identified quasi-contributors, and analyzed their pull-requests. In addition, we conducted surveys with quasi-contributors, and projects' integrators, to understand their perceptions about nonacceptance. We found 10,099 quasi-contributors --- about 70% of the total actual contributors --- that submitted 12,367 nonaccepted pull-requests. In five projects, we found more quasi-contributors than actual contributors. About one-third of the developers who took our survey disagreed with the nonacceptance, and around 30% declared the nonacceptance demotivated or prevented them from placing another pull-request. The main reasons for pull-request nonacceptance from the quasi-contributors' perspective were "superseded/duplicated pull-request" and "mismatch between developer's and team's vision/opinion." A manual analysis of a representative sample of 263 pull-requests corroborated with this finding. We also found reasons related to the relationship with the community and lack of experience or commitment from the quasi-contributors. This empirical study is particularly relevant to those interested in fostering developers' participation and retention in OSS communities.

References

1. Guilherme Avelino, Leonardo Teixeira Passos, André C. Hora, and Marco Tulio Valente. 2016. A novel approach for estimating Truck Factors. In 24th IEEE International Conference on Program Comprehension, ICPC 2016, Austin, TX, USA, May 16--17, 2016. 1--10.
2. H. Borges, A. Hora, and M. T. Valente. 2016. Understanding the Factors That Impact the Popularity of GitHub Repositories. In 2016 IEEE International Conference on Software Maintenance and Evolution (ICSME). 334--344.
3. Jailton Coelho and Marco Tulio Valente. 2017. Why Modern Open Source Projects Fail. In 25th International Symposium on the Foundations of Software Engineering (FSE). 186--196.
4. Luiz Felipe Dias, Igor Steinmacher, Gustavo Pinto, Daniel Alencar da Costa, and Marco Aurélio Gerosa. 2016. How Does the Shift to GitHub Impact Project Collaboration?. In 2016 IEEE International Conference on Software Maintenance and Evolution, ICSME 2016, Raleigh, NC, USA, October 2--7, 2016. 473--477.
5. Nicolas Ducheneaut. 2005. Socialization in an Open Source Software Community: A Socio-Technical Analysis. CSCW 14, 4 (Aug. 2005), 323--368.
6. Susan Elliott Sim and Richard C. Holt. 1998. The Ramp-up Problem in Software Projects: A Case Study of How Software Immigrants Naturalize. In 20th International Conference on Software Engineering (ICSE '98). 361--370.
7. Fabian Fagerholm, Alejandro S. Guinea, JÃijrgen MÃijnch, and Jay Borenstein. 2014. The Role of Mentoring and Project Characteristics for Onboarding in Open Source Software Projects. In 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM '14). ACM, New York, NY, USA, Article 55, 10 pages.
8. Karl Fogel. 2013. Producing Open Source Software: How to Run a Successful Free Software Project (first ed.). O'Reilly Media. http://www.producingoss.com/.Accessedon01-15-2015
9. Georgios Gousios and Alberto Bacchelli. 2016. Work Practices and Challenges in Pull-based Development: The Contributor's Perspective. In ICSE. 358--368.
10. Georgios Gousios, Martin Pinzger, and Arie van Deursen. 2014. An Exploratory Study of the Pull-based Software Development Model. In 36th International Conference on Software Engineering (ICSE 2014). ACM, New York, NY, USA, 345--355.
11. Georgios Gousios, Martin Pinzger, and Arie van Deursen. 2014. An Exploratory Study of the Pull-based Software Development Model. In 36th International Conference on Software Engineering (ICSE 2014). 345--355.
12. Georgios Gousios, Andy Zaidman, Margaret-Anne D. Storey, and Arie van Deursen. 2015. Work Practices and Challenges in Pull-Based Development: The Integrator's Perspective. In ICSE. 358--368.
13. Gordon Haff. 2017. Node.js: A project for casual contributors. https://opensource.com/article/17/3/nodejs-community-casual-contributors. (2017). {Online; accessed Jan 24, 2018}.
14. Vincent J. Hellendoorn, Premkumar T. Devanbu, and Alberto Bacchelli. 2015. Will They Like This?: Evaluating Code Contributions with Language Models. In 12th Working Conference on Mining Software Repositories (MSR '15). IEEE Press, Piscataway, NJ, USA, 157--167. http://dl.acm.org/citation.cfm?id=2820518.2820539
15. Guido Hertel, Sven Niedner, and Stefanie Herrmann. 2003. Motivation of software developers in Open Source projects: an Internet-based survey of contributors to the Linux kernel. Research Policy 32, 7 (2003), 1159 -- 1177. Open Source Software Development.
16. Yujuan Jiang, Bram Adams, and Daniel M. Germán. 2013. Will my patch make it? and how fast?: case study on the Linux kernel. In 10th Working Conference on Mining Software Repositories, MSR '13, San Francisco, CA, USA, May 18--19, 2013. 101--110.
17. B.A. Kitchenham, S.L. Pfleeger, L.M. Pickard, P.W. Jones, D.C. Hoaglin, K. El Emam, and J. Rosenberg. 2002. Preliminary guidelines for empirical research in software engineering. Software Engineering, IEEE Transactions on 28, 8 (Aug 2002), 721--734.
18. Nora McDonald and Sean Goggins. 2013. Performance and Participation in Open Source Software on GitHub. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHIEA '13). ACM, New York, NY, USA, 139--144.
19. Mozilla. {n. d.}. Mozilla Community Wiki: Casual Contributors. https://wiki.mozilla.org/Community#Casual_Contributor. ({n. d.}). {Online; accessed Jan 24, 2018}.
20. Meiyappan Nagappan, Thomas Zimmermann, and Christian Bird. 2013. Diversity in Software Engineering Research. In 2013 9th Joint Meeting on Foundations of Software Engineering (ESEC/FSE 2013). 466--476.
21. Rohan Padhye, Senthil Mani, and Vibha Singhal Sinha. 2014. A Study of External Community Contribution to Open-source Projects on GitHub. In 11th Working Conference on Mining Software Repositories (MSR 2014). ACM, New York, NY, USA, 332--335.
22. G. Pinto, I. Steinmacher, and M. A. Gerosa. 2016. More Common Than You Think: An In-depth Study of Casual Contributors. In 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER), Vol. 1. 112--123.
23. Mohammad Masudur Rahman, Chanchal K. Roy, and Jason A. Collins. 2016. CoRReCT: Code Reviewer Recommendation in GitHub Based on Cross-project and Technology Experience. In 38th International Conference on Software Engineering Companion (ICSE '16). ACM, New York, NY, USA, 222--231.
24. Baishakhi Ray, Daryl Posnett, Vladimir Filkov, and Premkumar Devanbu. 2014. A Large Scale Study of Programming Languages and Code Quality in Github. In 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering (FSE 2014). 155--165.
25. P. C. Rigby and M. A. Storey. 2011. Understanding broadcast based peer review on open source software projects. In 2011 33rd International Conference on Software Engineering (ICSE). 541--550.
26. J. Romano, J.D. Kromrey, J. Coraggio, and J. Skowronek. 2006. Appropriate statistics for ordinal level data: Should we really be using t-test and Cohen'sd for evaluating group differences on the NSSE and other surveys?. In annual meeting of the Florida Association of Institutional Research. 1--3.
27. E. Smith, R. Loftin, E. Murphy-Hill, C. Bird, and T. Zimmermann. 2013. Improving developer participation rates in surveys. In CHASE. 89--92.
28. Daricélio Moreira Soares, Manoel Limeira de Lima Júnior, Leonardo Murta, and Alexandre Plastino. 2015. Acceptance Factors of Pull Requests in Open-source Projects. In 30th Annual ACM Symposium on Applied Computing (SAC '15). ACM, New York, NY, USA, 1541--1546.
29. Igor Steinmacher, Ana Paula Chaves, Tayana Conte, and Marco AurÃl'lio Gerosa. 2014. Preliminary empirical identification of barriers faced by newcomers to Open Source Software projects.. In 28th Brazilian Symposium on Software Engineering (SBES '14). IEEE Computer Society, 1--10.
30. Igor Steinmacher, Tayana Conte, and Marco AurÃl'lio Gerosa. 2015. Understanding and Supporting the Choice of an Appropriate Task to Start With In Open Source Software Communities. In 48th Hawaiian International Conference in Software Systems (HICSS '15). 1--10.
31. Igor Steinmacher, Tayana Conte, Marco AurÃl'lio Gerosa, and David F. Redmiles. 2015. Social Barriers Faced by Newcomers Placing Their First Contribution in Open Source Software Projects. In 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, New York, NY, USA, 1--13.
32. Anselm Strauss and Juliet M. Corbin. 2007. Basics of Qualitative Research : Techniques and Procedures for Developing Grounded Theory (3rd ed.). SAGE Publications.
33. Y. Tao, D. Han, and S. Kim. 2014. Writing Acceptable Patches: An Empirical Study of Open Source Project Patches. In 2014 IEEE International Conference on Software Maintenance and Evolution. 271--280.
34. Jason Tsay, Laura Dabbish, and James Herbsleb. 2014. Influence of social and technical factors for evaluating contribution in GitHub. In ICSE. 356--366.
35. Georg von Krogh and Eric von Hippel. 2003. Editorial: Special issue on open source software development. Research Policy 32, 7 (July 2003), 1149--1157.
36. Peter Weißgerber, Daniel Neu, and Stephan Diehl. 2008. Small patches get in!. In 2008 International Working Conference on Mining Software Repositories, MSR 2008 (Co-located with ICSE), Leipzig, Germany, May 10--11, 2008, Proceedings. 67--76.
37. D.S. Wilks. 2011. Statistical Methods in the Atmospheric Sciences. Academic Press. https://books.google.com.br/books?id=IJuCVtQ0ySIC
38. Laurie Williams and Robert Kessler. 2002. Pair Programming Illuminated. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.
39. Yue Yu, Huaimin Wang, Vladimir Filkov, Premkumar Devanbu, and Bogdan Vasilescu. 2015. Wait for It: Determinants of Pull Request Evaluation Latency on GitHub. In 12th Working Conference on Mining Software Repositories (MSR '15). IEEE Press, Piscataway, NJ, USA, 367--371. http://dl.acm.org/citation.cfm?id=2820518.2820564
40. Yue Yu, Huaimin Wang, Gang Yin, and Tao Wang. 2016. Reviewer Recommendation for Pull-requests in GitHub. Inf. Softw. Technol. 74, C (June 2016), 204--218.
41. Minghui Zhou and Audris Mockus. 2015. Who Will Stay in the FLOSS Community? Modelling Participant's Initial Behaviour. IEEE Transactions on Software Engineering 41, 1 (2015), 82--99.

Index Terms

• Human-centered computing

  • Collaborative and social computing

• Software and its engineering

  • Software creation and management

    • Collaboration in software development

      • Open source model