Sarah Smith Heckman
Abstract
Static analysis tools are useful for finding common programming mistakes that often lead to field failures. However, static analysis tools regularly generate a high number of false positive alerts, requiring manual inspection by the developer to determine if an alert is an indication of a fault. The adaptive ranking model presented in this paper utilizes feedback from developers about inspected alerts in order to rank the remaining alerts by the likelihood that an alert is an indication of a fault. Alerts are ranked based on the homogeneity of populations of generated alerts, historical developer feedback in the form of suppressing false positives and fixing true positive alerts, and historical, application-specific data about the alert ranking factors. The ordering of alerts generated by the adaptive ranking model is compared to a baseline of randomly-, optimally-, and static analysis tool-ordered alerts in a small role-based health care application. The adaptive ranking model provides developers with 81% of true positive alerts after investigating only 20% of the alerts whereas an average of 50 random orderings of the same alerts found only 22% of true positive alerts after investigating 20% of the generated alerts.
- 1 Y. Brun and M. D. Ernst, "Finding Latent Code Errors via Machine Learning Over Program Executions," in 26th International Conference on Software Engineering, Edinburgh, Scotland, 2004, pp. 480-490. Google Scholar
Digital Library
- 2 B. Chess and G. McGraw, "Static Analysis for Security," in IEEE Security & Privacy. vol. 2, no. 6, 2004, pp. 76-79. Google ScholarDigital Library
- 3 D. Engler, B. Chelf, A. Chou, and S. Hallem, "Checking System Rules Using System-Specific, Programmer-Written Compiler Extensions," in Operating Systems Design and Implementation no. San Diego, CA, 2000. Google ScholarDigital Library
- 4 S. Heckman and L. Williams, "Automated Adaptive Ranking and Filtering of Static Analysis Alerts," in 17th International Symposium on Software Reliability Engineering, Fast Abstract, Raleigh, NC, USA, 2006.Google Scholar
- 5 D. Hovemeyer and W. Pugh, "Finding Bugs is Easy," in 19th ACM Conference on Object-Oriented Programming, Systems, Languages, and Applications, Vancouver, British Columbia, Canada, 2004, pp. 132-136. Google ScholarDigital Library
- 6 T. Kremenek, K. Ashcraft, J. Yang, and D. Engler, "Correlation Exploitation in Error Ranking," in 12th ACM SIGSOFT International Symposium on Foundations of Software Engineering, Newport Beach, CA, USA, 2004, pp. 83-93. Google ScholarDigital Library
- 7 T. Kremenek and D. Engler, "Z-Ranking: Using Statistical Analysis to Counter the Impact of Static Analysis Approximations," in 10th International Static Analysis Symposium, San Diego, California, 2002. Google ScholarDigital Library
- 8 N. Nagappan, T. Ball, and A. Zeller, "Mining Metrics to Predict Component Failures," in 28th International Conference on Software Engineering, Shanghai, China, 2006, pp. 452-461. Google ScholarDigital Library
- 9 G. Rothermel, R. H. Untch, C. Chu, and M. J. Harrold, "Prioritizing Test Cases For Regression Testing," IEEE Transactions on Software Engineering, vol. 27, no. 10, pp. 929-948, October 2001. Google ScholarDigital Library
- 10 N. Rutar, C. B. Almazan, and J. S. Foster, "A Comparison of Bug Finding Tools for Java," in 15th IEEE International Symposium on Software Reliability Engineering, Saint-Malo, Bretagne, France, 2004, pp. 245-256. Google ScholarDigital Library
- 11 S. E. Smith, L. Williams, and J. Xu, "Expediting Programmer AWAREness of Anomalous Code," in 16th IEEE International Symposium on Software Reliability Engineering, Fast Abstract, Chicago, IL, USA, 2005.Google Scholar
Index Terms
- Adaptively ranking alerts generated from automated static analysis
Recommendations
Finding patterns in static analysis alerts: improving actionable alert ranking
MSR 2014: Proceedings of the 11th Working Conference on Mining Software RepositoriesStatic analysis (SA) tools that find bugs by inferring programmer beliefs (e.g., FindBugs) are commonplace in today's software industry. While they find a large number of actual defects, they are often plagued by high rates of alerts that a developer ...
An efficient approach to reduce alerts generated by multiple IDS products
Intrusion detection systems IDSs often trigger a huge number of unnecessary alerts. Managing the overwhelming number of alerts, especially from multiple IDS products, is a concern to every security analyst. Analyzing and evaluating these alerts is a ...
Comments