Kui Liu
Jacques Klein
Skip Abstract Section
Abstract
Fix pattern-based patch generation is a promising direction in automated program repair (APR). Notably, it has been demonstrated to produce more acceptable and correct patches than the patches obtained with mutation operators through genetic programming. The performance of pattern-based APR systems, however, depends on the fix ingredients mined from fix changes in development histories. Unfortunately, collecting a reliable set of bug fixes in repositories can be challenging. In this article, we propose investigating the possibility in an APR scenario of leveraging fix patterns inferred from code changes that address violations detected by static analysis tools. To that end, we build a fix pattern-based APR tool, Avatar, which exploits fix patterns of static analysis violations as ingredients for the patch generation of repairing semantic bugs. Evaluated on four benchmarks (i.e., Defects4J, Bugs.jar, BEARS, and QuixBugs), Avatar presents the potential feasibility of fixing semantic bugs with the fix patterns inferred from the patches for fixing static analysis violations and can correctly fix 26 semantic bugs when Avatar is implemented with the normal program repair pipeline. We also find that Avatar achieves performance metrics that are comparable to that of the closely related approaches in the literature. Compared with CoCoNut, Avatar can fix 18 new bugs in Defects4J and 3 new bugs in QuixBugs. When compared with HDRepair, JAID, and SketchFix, Avatar can newly fix 14 Defects4J bugs. In terms of the number of correctly fixed bugs, Avatar is also comparable to the program repair tools with the normal fault localization setting and presents better performance than most program repair tools. These results imply that Avatar is complementary to current program repair approaches. We further uncover that Avatar can present different bug-fixing performances when it is configured with different fault localization tools, and the stack trace information from the failed executions of test cases can be exploited to improve the bug-fixing performance of Avatar by fixing more bugs with fewer generated patch candidates. Overall, our study highlights the relevance of static bug-finding tools as indirect contributors of fix ingredients for addressing code defects identified with functional test cases (i.e., dynamic information).
- [1] . 2007. On the accuracy of spectrum-based fault localization. In Proceedings of the Testing: Academic and Industrial Conference—Practice and Research Techniques-MUTATION.
IEEE , 89–98.Google Scholar
Digital Library
- [2] . 2015. Tracking static analysis violations over time to capture developer characteristics. In Proceedings of the 37th International Conference on Software Engineering.
ACM , 437–447.Google ScholarCross Ref
- [3] . 2012. GZoltar: An eclipse plug-in for testing and debugging. In Proceedings of the 27th ACM International Conference on Automated Software Engineering.
ACM , 378–381.Google ScholarDigital Library - [4] . 2017. Contract-based program repair without the contracts. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering.
IEEE , 637–647.Google ScholarDigital Library - [5] . 2019. Empirical review of Java program repair tools: A large-scale experiment on 2,141 bugs and 23,551 repair attempts. In Proceedings of the 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering. ACM, 302–313.Google ScholarDigital Library
- [6] . 2016. DynaMoth: Dynamic code synthesis for automatic program repair. In Proceedings of the 11th International Workshop in Automation of Software Test.
ACM , 85–91.Google ScholarDigital Library - [7] . 2014. Fine-grained and accurate source code differencing. In Proceedings of the 29th ACM/IEEE International Conference on Automated Software Engineering.
ACM , 313–324.Google ScholarDigital Library - [8] . 2011. An experience report on using code smells detection tools. In Proceedings of the 4th International Conference on Software Testing, Verification and Validation Workshops.
IEEE , 450–457.Google ScholarDigital Library - [9] . 2019. Practical program repair via bytecode mutation. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis.
ACM , 19–30.Google ScholarDigital Library - [10] . 2018. How many of all bugs do we find? A study of static bug detectors. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering.
ACM , 317–328.Google ScholarDigital Library - [11] . 2008. On establishing a benchmark for evaluating static analysis alert prioritization and classification techniques. In Proceedings of the 2nd ACM-IEEE International Symposium on Empir. Softw. Eng. Measur.
ACM , 41–50.Google ScholarDigital Library - [12] . 2013. The impact of tangled code changes. In Proceedings of the 10th Working Conference on Mining Software Repositories.
ACM , 121–130.Google ScholarCross Ref - [13] . 2004. Finding bugs is easy. ACM SIGPLAN Not. 39, 12 (2004), 92–106.Google ScholarDigital Library
- [14] . 2018. Towards practical program repair with on-demand candidate generation. In Proceedings of the 40th International Conference on Software Engineering.
ACM , 12–23.Google ScholarDigital Library - [15] . 2019. Inferring program transformations from singular examples via big code. In Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering. IEEE, 255–266.Google ScholarDigital Library
- [16] . 2018. Shaping program repair space with existing patches and similar code. In Proceedings of the 27th ACM SIGSOFT International Symposium on Software Testing and Analysis.
ACM , 298–309.Google ScholarDigital Library - [17] . 2014. Defects4J: A database of existing faults to enable controlled testing studies for Java programs. In Proceedings of the 23rd International Symposium on Software Testing and Analysis.
ACM , 437–440.Google ScholarDigital Library - [18] . 2013. Automatic patch generation learned from human-written patches. In Proceedings of the 35th International Conference on Software Engineering.
IEEE , 802–811.Google ScholarCross Ref - [19] . 2019. Automatic patch generation with context-based change application. Empir. Softw. Eng. 24, 6 (2019), 4071–4106.Google ScholarCross Ref
- [20] . 2017. Impact of tool support in patch construction. In Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, 237–248.Google ScholarDigital Library
- [21] . 2020. FixMiner: Mining relevant fix patterns for automated program repair. Empir. Softw. Eng. 25, 3 (2020), 1980–2024.Google ScholarDigital Library
- [22] . 2019. On reliability of patch correctness assessment. In Proceedings of the 41st International Conference on Software Engineering.
IEEE , 524–535.Google ScholarDigital Library - [23] . 2016. Enhancing automated program repair with deductive verification. In Proceedings of the International Conference on Software Maintenance and Evolution. IEEE, 428–432.Google ScholarCross Ref
- [24] . 2016. History driven program repair. In Proceedings of the 23rd IEEE International Conference on Software Analysis, Evolution, and Reengineering.
IEEE , 213–224.Google ScholarCross Ref - [25] . 2012. GenProg: A generic method for automatic software repair. IEEE Trans. Softw. Eng. 38, 1 (2012), 54–72.Google ScholarDigital Library
- [26] . 2020. DLFix: Context-based code transformation learning for automated program repair. In Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering. ACM, 602–614.Google ScholarDigital Library
- [27] . 2018. Mining fix patterns for findbugs violations. IEEE Trans. Softw. Eng. 47, 1 (2018).Google ScholarDigital Library
- [28] . 2018. A closer look at real-world patches. In Proceedings of the 34th International Conference on Software Maintenance and Evolution.
IEEE , 275–286.Google ScholarCross Ref - [29] . 2019. You cannot fix what you cannot find! An investigation of fault localization bias in benchmarking automated program repair systems. In Proceedings of the 12th IEEE International Conference on Software Testing, Verification and Validation.
IEEE , 102–113.Google ScholarCross Ref - [30] . 2019. AVATAR: Fixing semantic bugs with fix patterns of static analysis violations. In Proceedings of the 26th IEEE International Conference on Software Analysis, Evolution and Reengineering.
IEEE , 456–467.Google ScholarCross Ref - [31] . 2019. TBar: Revisiting template-based automated program repair. In Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis.
ACM , 31–42.Google ScholarDigital Library - [32] . 2018. LSRepair: Live search of fix ingredients for automated program repair. In Proceedings of the 25th Asia-Pacific Software Engineering Conference ERA Track.
IEEE , 658–662.Google ScholarCross Ref - [33] . 2020. A critical review on the evaluation of automated program repair systems. J. Syst. Softw. 171 (2020).Google Scholar
- [34] . 2020. On the efficiency of test suite based program repair: A systematic assessment of 16 automated repair systems for Java programs. In Proceedings of the 42nd International Conference on Software Engineering.
ACM , 625–627.Google ScholarDigital Library - [35] . 2017. Automatic inference of code transforms for patch generation. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering.
ACM , 727–739.Google ScholarDigital Library - [36] . 2015. Staged program repair with condition synthesis. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering.
ACM , 166–178.Google ScholarDigital Library - [37] . 2016. An analysis of the search spaces for generate and validate patch generation systems. In Proceedings of the 38th International Conference on Software Engineering.
IEEE , 702–713.Google ScholarDigital Library - [38] . 2016. Automatic patch generation by learning correct code. In Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages.
ACM , 298–312.Google ScholarDigital Library - [39] . 2020. CoCoNuT: Combining context-aware neural translation models using ensemble for program repair. In Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis. ACM, 101–114.Google ScholarDigital Library
- [40] . 2019. BEARS: An extensible Java bug benchmark for automatic program repair studies. In Proceedings of the 26th International Conference on Software Analysis, Evolution and Reengineering.
IEEE , 468–478.Google ScholarCross Ref - [41] . 2017. Automatic repair of real bugs in Java: A large-scale experiment on the Defects4J dataset. Empir. Softw. Eng. 22, 4 (2017), 1936–1964.Google ScholarDigital Library
- [42] . 2016. ASTOR: A program repair library for Java (demo). In Proceedings of the 25th International Symposium on Software Testing and Analysis.
ACM , 441–444.Google ScholarDigital Library - [43] . 2018. Ultra-large repair search space with automatically mined templates: The Cardumen mode of Astor. In Proceedings of the 10th International Symposium on Search Based Software Engineering.
Springer , 65–86.Google ScholarCross Ref - [44] . 2014. Do the fix ingredients already exist? An empirical inquiry into the redundancy assumptions of program repair approaches. In Proceedings of the 36th International Conference on Software Engineering.
ACM , 492–495.Google Scholar - [45] . 2003. Subject independent facial expression recognition with robust face detection using a convolutional neural network. Neural Netw. 16, 5-6 (2003), 555–559.Google ScholarDigital Library
- [46] . 2018. Semantic program repair using a reference implementation. In Proceedings of the 40th International Conference on Software Engineering. ACM, 298–309.Google ScholarDigital Library
- [47] . 2010. DECOR: A method for the specification and detection of code and design smells. IEEE Trans. Softw. Eng. 36, 1 (2010), 20–36.Google ScholarDigital Library
- [48] . 2014. A critical review of automatic patch generation learned from human-written patches: Essay on the problem statement and the evaluation of automatic software repair. In Proceedings of the 36th International Conference on Software Engineering.
ACM , 234–242.Google ScholarDigital Library - [49] . 2018. Automatic software repair: A bibliography. Comput. Surv. 51, 1 (2018), 17:1–17:24.Google Scholar
- [50] . 2013. SemFix: Program repair via semantic analysis. In Proceedings of the 35th International Conference on Software Engineering.
IEEE , 772–781.Google ScholarCross Ref - [51] . 2009. Toward an understanding of bug fix patterns. Empir. Softw. Eng. 14, 3 (2009), 286–315.Google ScholarDigital Library
- [52] . 2017. Evaluating and improving fault localization. In Proceedings of the 39th International Conference on Software Engineering. ACM, 609–620.Google ScholarDigital Library
- [53] . 2015. An analysis of patch plausibility and correctness for generate-and-validate patch generation systems. In Proceedings of the 24th International Symposium on Software Testing and Analysis.
ACM , 24–36.Google ScholarDigital Library - [54] . 2018. Learning quick fixes from code repositories. arXiv preprint arXiv:1803.03806 (2018).Google Scholar
- [55] . 2018. Bugs.jar: A large-scale, diverse dataset of real-world Java bugs. In Proceedings of the 15th IEEE/ACM International Conference on Mining Software Repositories.
ACM , 10–13.Google ScholarDigital Library - [56] . 2017. ELIXIR: Effective object-oriented program repair. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering.
IEEE , 648–659.Google ScholarDigital Library - [57] . 2019. Harnessing evolution for multi-hunk program repair. In Proceedings of the 41st International Conference on Software Engineering.
IEEE , 13–24.Google ScholarDigital Library - [58] . 2015. Is the cure worse than the disease? Overfitting in automated program repair. In Proceedings of the 10th Joint Meeting on Foundations of Software Engineering.
ACM , 532–543.Google ScholarDigital Library - [59] . 2013. Threats to the validity and value of empirical assessments of the accuracy of coverage-based fault locators. In Proceedings of the International Symposium on Software Testing and Analysis.
ACM , 314–324.Google ScholarDigital Library - [60] . 2016. Anti-patterns in search-based program repair. In Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering.
ACM , 727–738.Google ScholarDigital Library - [61] . 2018. An empirical investigation into learning bug-fixing patches in the wild via neural machine translation. In Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering.
ACM , 832–837.Google ScholarDigital Library - [62] . 2009. Automatically finding patches using genetic programming. In Proceedings of the 31st International Conference on Software Engineering.
IEEE , 364–374.Google ScholarDigital Library - [63] . 2018. Context-aware patch generation for better automated program repair. In Proceedings of the 40th International Conference on Software Engineering.
ACM , 1–11.Google ScholarDigital Library - [64] . 2019. Sorting and transforming program repair ingredients via deep learning code similarities. In Proceedings of the IEEE 26th International Conference on Software Analysis, Evolution and Reengineering.
IEEE , 479–490.Google ScholarCross Ref - [65] . 2013. A theoretical analysis of the risk evaluation formulas for spectrum-based fault localization. ACM Trans. Softw. Eng. Methodol 22, 4 (2013), 31.Google ScholarDigital Library
- [66] . 2017. Leveraging syntax-related code for automated program repair. In Proceedings of the 32nd IEEE/ACM International Conference on Automated Software Engineering.
IEEE , 660–670.Google ScholarDigital Library - [67] . 2018. Identifying patch correctness in test-based program repair. In Proceedings of the 40th International Conference on Software Engineering.
ACM , 789–799.Google ScholarDigital Library - [68] . 2017. Precise condition synthesis for program repair. In Proceedings of the 39th IEEE/ACM International Conference on Software Engineering.
IEEE , 416–426.Google ScholarDigital Library - [69] . 2019. VFix: Value-flow-guided precise program repair for null pointer dereferences. In Proceedings of the 41st International Conference on Software Engineering.
IEEE , 512–523.Google ScholarDigital Library - [70] . 2017. Nopol: Automatic repair of conditional statement bugs in java programs. IEEE Trans. Softw. Eng. 43, 1 (2017), 34–55.Google ScholarDigital Library
- [71] . 2014. Learning to combine multiple ranking metrics for fault localization. In Proceedings of the IEEE International Conference on Software Maintenance and Evolution.
IEEE , 191–200.Google ScholarDigital Library - [72] . 2014. Test case purification for improving fault localization. In Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering.
ACM , 52–63.Google ScholarDigital Library - [73] . 2013. Do developers care about code smells? An exploratory survey. In Proceedings of the 20th Working Conference on Reverse Engineering.
IEEE , 242–251.Google ScholarCross Ref - [74] . 2017. Better test cases for better automated program repair. In Proceedings of the 11th Joint Meeting on Foundations of Software Engineering.
ACM , 831–841.Google ScholarDigital Library - [75] . 2019. A comprehensive study of automatic program repair on the QuixBugs benchmark. In Proceedings of the 1st International Workshop on Intelligent Bug Fixing.
IEEE , 1–10.Google ScholarCross Ref - [76] . 2018. ARJA: Automated repair of Java programs via multi-objective genetic programming. IEEE Trans. Softw. Eng. 46, 10 (2018).Google ScholarCross Ref
Index Terms
- Reliable Fix Patterns Inferred from Static Checkers for Automated Program Repair
Recommendations
TBar: revisiting template-based automated program repair
ISSTA 2019: Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and AnalysisWe revisit the performance of template-based APR to build comprehensive knowledge about the effectiveness of fix patterns, and to highlight the importance of complementary steps such as fault localization or donor code retrieval. To that end, we first ...
Mining Python fix patterns via analyzing fine-grained source code changes
AbstractMany code changes are inherently repetitive, and researchers employ repetitiveness of the code changes to generate bug fix patterns. Automatic Program Repair (APR) can automatically detect and fix bugs, thus helping developers to improve the ...
Identifying static analysis techniques for finding non-fix hunks in fix revisions
DSMM '09: Proceedings of the ACM first international workshop on Data-intensive software management and miningMining software repositories for bug detection requires accurate techniques of identifying bug-fix revisions. There have been many researches to find exact bug-fix revisions. However there are still noises, we call these noises non-fix hunks, even in ...
Comments