research-article

A quantitative study of Java software buildability

Authors:

Matúš Sulír,

Jaroslav PorubänAuthors Info & Claims

PLATEAU 2016: Proceedings of the 7th International Workshop on Evaluation and Usability of Programming Languages and Tools

Pages 17 - 25

https://doi.org/10.1145/3001878.3001882

Published: 01 November 2016 Publication History

Abstract

Researchers, students and practitioners often encounter a situation when the build process of a third-party software system fails. In this paper, we aim to confirm this observation present mainly as anecdotal evidence so far. Using a virtual environment simulating a programmer's one, we try to fully automatically build target archives from the source code of over 7,200 open source Java projects. We found that more than 38% of builds ended in failure. Build log analysis reveals the largest portion of errors are dependency-related. We also conduct an association study of factors affecting build success.

References

[1]

M. Beller, G. Gousios, and A. Zaidman. Oops, my tests broke the build: An analysis of Travis CI builds with GitHub. PeerJ Preprints 4:e1984v1, 2016.

[2]

S. I. Feldman. Make – a program for maintaining computer programs. Software: Practice and Experience, 9(4):255–265, 1979. ISSN 1097-024X.

[3]

L. Hochstein and Y. Jiao. The cost of the build tax in scientific software. In Empirical Software Engineering and Measurement (ESEM), 2011 International Symposium on, pages 384–387, Sept 2011.

Digital Library

[4]

E. Kalliamvakou, G. Gousios, K. Blincoe, L. Singer, D. M. German, and D. Damian. The promises and perils of mining GitHub. In Proceedings of the 11th Working Conference on Mining Software Repositories, MSR 2014, pages 92–101, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2863-0.

Digital Library

[5]

N. Kerzazi, F. Khomh, and B. Adams. Why do automated builds break? An empirical study. In Software Maintenance and Evolution (ICSME), 2014 IEEE International Conference on, pages 41–50, Sept 2014.

Digital Library

[6]

G. Marceau, K. Fisler, and S. Krishnamurthi. Mind your language: On novices’ interactions with error messages. In Proceedings of the 10th SIGPLAN Symposium on New Ideas, New Paradigms, and Reflections on Programming and Software, Onward! 2011, pages 3–18, New York, NY, USA, 2011. ACM. ISBN 978-1-4503-0941-7.

Digital Library

[7]

S. McIntosh, B. Adams, and A. Hassan. The evolution of ANT build systems. In Mining Software Repositories (MSR), 2010 7th IEEE Working Conference on, pages 42–51, May 2010.

[8]

[9]

S. McIntosh, M. Nagappan, B. Adams, A. Mockus, and A. E. Hassan. A large-scale empirical study of the relationship between build technology and build maintenance. Empirical Software Engineering, 20(6):1587–1633, 2015. ISSN 1382- 3256.

Digital Library

[10]

A. Neitsch, K. Wong, and M. Godfrey. Build system issues in multilanguage software. In Software Maintenance (ICSM), 2012 28th IEEE International Conference on, pages 140–149, Sept 2012.

Digital Library

[11]

H. Seo, C. Sadowski, S. Elbaum, E. Aftandilian, and R. Bowdidge. Programmers’ build errors: A case study (at Google). In Proceedings of the 36th International Conference on Software Engineering, ICSE 2014, pages 724–734, New York, NY, USA, 2014. ACM. ISBN 978-1-4503-2756-5.

Digital Library

[12]

P. Smith. Software Build Systems: Principles and Experience. Addison-Wesley Professional, 2011.

Digital Library

[13]

J. Spolsky. Joel on Software, chapter The Joel Test: 12 Steps to Better Code, pages 17–30. Apress, Berkeley, CA, 2004. ISBN 978-1-4302-0753-5.

[14]

M. Sulír and M. Nosál’. Sharing developers’ mental models through source code annotations. In Computer Science and Information Systems (FedCSIS), 2015 Federated Conference on, pages 997–1006, Sept 2015.

[15]

M. Sulír and J. Porubän. Locating user interface concepts in source code. In M. Mernik, J. P. Leal, and H. G. Oliveira, editors, 5th Symposium on Languages, Applications and Technologies (SLATE’16), volume 51 of OpenAccess Series in Informatics (OASIcs), pages 1–9, Dagstuhl, Germany, 2016.

[16]

Schloss Dagstuhl–Leibniz-Zentrum fuer Informatik. ISBN 978-3-95977-006-4.

[17]

B. Vasilescu, S. v. Schuylenburg, J. Wulms, A. Serebrenik, and M. G. J. v. d. Brand. Continuous integration in a social-coding world: Empirical evidence from GitHub. In Proceedings of the 2014 IEEE International Conference on Software Maintenance and Evolution, ICSME ’14, pages 401–405, Washington, DC, USA, 2014. IEEE Computer Society. ISBN 978-1-4799-6146- 7.

Digital Library

Cited By

Huang SWang XFilkov VRay BZhou M(2024)Build Issue Resolution from the Perspective of Non-ContributorsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695304(2304-2308)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695304
Misu MAchar RLopes C(2024)SourcererJBF: A Java Build Framework For Large-Scale CompilationACM Transactions on Software Engineering and Methodology10.1145/363571033:3(1-35)Online publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1145/3635710
Schott SFischer WPonta SKlauke JBodden E(2024)Compilation of Commit Changes Within Java Source Code Repositories2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00038(325-336)Online publication date: 6-Oct-2024
https://doi.org/10.1109/ICSME58944.2024.00038
Show More Cited By

Index Terms

A quantitative study of Java software buildability
1. Software and its engineering
  1. Software notations and tools
    1. Compilers

Recommendations

The evolution of Java build systems

Build systems are responsible for transforming static source code artifacts into executable software. While build systems play such a crucial role in software development and maintenance, they have been largely ignored by software evolution researchers. ...
Software Builders

The tools and processes we use to transform our system's source code into an application that we can deploy or ship have always been important, but nowadays they can mean the difference between success and failure. Software building's golden rule is ...
JaMaBuild: Mass Building of Java Projects
SPLASH 2023: Companion Proceedings of the 2023 ACM SIGPLAN International Conference on Systems, Programming, Languages, and Applications: Software for Humanity

Many large-scale Java empirical studies require not only source code but also resulting binaries such as JAR files. Pre-compiled datasets quickly become obsolete, and the creation of a custom corpus for every study is tedious. We present a prototype ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

PLATEAU 2016: Proceedings of the 7th International Workshop on Evaluation and Usability of Programming Languages and Tools

November 2016

33 pages

ISBN:9781450346382

DOI:10.1145/3001878

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGPLAN: ACM Special Interest Group on Programming Languages

In-Cooperation

SIGAda: ACM Special Interest Group on Ada Programming Language

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Kultúrna a Edukaná Grantová Agentúra MŠVVaŠ SR

Conference

SPLASH '16

Sponsor:

SIGPLAN

SPLASH '16: Conference on Systems, Programming, Languages, and Applications: Software for Humanity

November 1, 2016

Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 5 of 8 submissions, 63%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

26
Total Citations
View Citations
320
Total Downloads

Downloads (Last 12 months)19
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Huang SWang XFilkov VRay BZhou M(2024)Build Issue Resolution from the Perspective of Non-ContributorsProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695304(2304-2308)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695304
Misu MAchar RLopes C(2024)SourcererJBF: A Java Build Framework For Large-Scale CompilationACM Transactions on Software Engineering and Methodology10.1145/363571033:3(1-35)Online publication date: 15-Mar-2024
https://dl.acm.org/doi/10.1145/3635710
Schott SFischer WPonta SKlauke JBodden E(2024)Compilation of Commit Changes Within Java Source Code Repositories2024 IEEE International Conference on Software Maintenance and Evolution (ICSME)10.1109/ICSME58944.2024.00038(325-336)Online publication date: 6-Oct-2024
https://doi.org/10.1109/ICSME58944.2024.00038
Maes-Bermejo MGallego MGortázar FRobles GGonzalez-Barahona J(2024)Testing the past: can we still run tests in past snapshots for Java projects?Empirical Software Engineering10.1007/s10664-024-10530-z29:5Online publication date: 30-Jul-2024
https://dl.acm.org/doi/10.1007/s10664-024-10530-z
Khatami AZaidman A(2024)State‐of‐the‐practice in quality assurance in Java‐based open source software developmentSoftware: Practice and Experience10.1002/spe.332154:8(1408-1446)Online publication date: 4-Mar-2024
https://doi.org/10.1002/spe.3321
Hassan FMeng NWang X(2023)UniLoc: Unified Fault Localization of Continuous Integration FailuresACM Transactions on Software Engineering and Methodology10.1145/359379932:6(1-31)Online publication date: 8-May-2023
https://dl.acm.org/doi/10.1145/3593799
Zhu HRubio-González C(2023)On the Reproducibility of Software Defect Datasets2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00195(2324-2335)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00195
Nejati MAlfadel MMcIntosh S(2023)Code Review of Build System Specifications: Prevalence, Purposes, Patterns, and Perceptions2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE)10.1109/ICSE48619.2023.00108(1213-1224)Online publication date: May-2023
https://doi.org/10.1109/ICSE48619.2023.00108
Chen AChen TChen J(2022)How Useful is Code Change Information for Fault Localization in Continuous Integration?Proceedings of the 37th IEEE/ACM International Conference on Automated Software Engineering10.1145/3551349.3556931(1-12)Online publication date: 10-Oct-2022
https://dl.acm.org/doi/10.1145/3551349.3556931
Rzig DHassan FBansal CNagappan N(2022)Characterizing the Usage of CI Tools in ML ProjectsProceedings of the 16th ACM / IEEE International Symposium on Empirical Software Engineering and Measurement10.1145/3544902.3546237(69-79)Online publication date: 19-Sep-2022
https://dl.acm.org/doi/10.1145/3544902.3546237
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten