ABSTRACT
In object-oriented software development, a plethora of studies have been carried out to present the application of machine learning algorithms for fault prediction. Furthermore, it has been empirically validated that an ensemble method can improve classification performance as compared to a single classifier. But, due to the inherent differences among machine learning and data mining approaches, the classification performance of ensemble methods will be varied. In this study, we investigated and evaluated the performance of different ensemble methods with itself and base-level classifiers, in predicting the faults proneness classes. Subsequently, we used three ensemble methods AdaboostM1, Vote and StackingC with five base-level classifiers namely Naivebayes, Logistic, J48, VotedPerceptron and SMO in Weka tool. In order to evaluate the performance of ensemble methods, we retrieved twelve datasets of open source projects from PROMISE repository. In this experiment, we used k-fold (k=10) cross-validation and ROC analysis for validation. Besides, we used recall, precision, accuracy, F-value measures to evaluate the performance of ensemble methods and base-level Classifiers. Finally, we observed significant performance improvement of applying ensemble methods as compared to its base-level classifier, and among ensemble methods we observed StackingC outperformed other selected ensemble methods for software fault prediction.
- S. R. Chidamber and C. F. Kemerer, A Metrics Suite for OO Design, IEEE Transaction on SE, Vol 20, No 6, Jun 1994. Google ScholarDigital Library
- F. B. Abreu and R. Carapuc, Object-Oriented software engineering: measuring and controlling the development Process, Proceedings of the 4th international conference on software quality, vol 186, 1994.Google Scholar
- J. Bansiya. and C. G. Davis. A hierarchical model for OO design quality assessment, IEEE Transaction on SE, 2002 Google ScholarDigital Library
- L. C. Briand, W. L. Melo, J. Wu, "Assessing the Applicability of Fault Proneness Models Across OO Software Projects", IEEE Transaction on SE, 28 (7), pp. 706--720, 2002. Google ScholarDigital Library
- D. Radjenovic et al, Software Fault Prediction Metrics : A systematic literature review, Journal of Information and Software Technology, 2013 Google ScholarDigital Library
- S. Kanmani et al, Object-oriented software fault prediction using neural networks, Jounal of Information and Software Technology, 9(5): p. 483--492, 2007. Google ScholarDigital Library
- R. Subramanyam and M. S. Kishnan, Empirical Analysis of CK Metrics for OO Design Complexity: Implication for Software Defects, IEEE Transaction on SE, Vol. 29, 2003 Google ScholarDigital Library
- Yogesh Singh, Arvinder Kaur and Ruchika Malhotra, Software Fault Proneness Prediction Using Support Vector Machines, Proceedings of the World Congress on Engineering (WCE), Volume I. 2009.Google Scholar
- Zenko B, Todorovski L. and Dzeroski S., A comparison of stacking with MDTs to bagging, boosting and other Stacking methods, In Proceedings of the First IEEE International Conference on Data Mining, pp 669--670, 2001. Google ScholarDigital Library
- M. Jureczko and L. Madeyski, Towards identifying software project clusters with regard to defect prediction. Proceedings of the 6th International Conference on Predictive Models in Software Engineering (PROMISE '10), USA, 2010. Google ScholarDigital Library
- V. Basili, L. Briand, and W. Melo, A Validation of Object-OO Metrics as Quality Indicators, IEEE Transaction on SE, vol. 22, no. 10, pp. 751--761, Oct. 1996. Google ScholarDigital Library
- G. J. Pai and B. Dugan, Empirical Analysis of Software Fault Content and Fault Proneness Using Bayesian Methods, IEEE Transaction on SE, vol. 33, no. 10, Oct 2007. Google ScholarDigital Library
- E. Alpaydm, Techniques for Combining Multiple Learners, Proceddings of Engineering of Intelligent Systems'98 Conference, Vol 2, 6--12, 1998.Google Scholar
- T. M. Khoshgoftaar, P. Rebours and N. Seliya, Software Quality analysis by Combining Multiple Projects and Learners, Journal of Software Quality Control, vol 55, pp. 119--139, 1997. Google ScholarDigital Library
- M. C. K Vinaykumar and V. Ravi, Software Cost Estimation Using Soft Computing Approaches, Headbook of Research on Machine Learning Applications and Trends: Algorithms, Methods and Techniques, pp. 4999--518, 2010.Google Scholar
- E. Kocaguneli, T. Menzies and J. W. Keung, On the Value of Ensemble Effort Estimation, IEEE Transaction on Software Engineering, vol. 38, no. 6, Nov. 2012. Google ScholarDigital Library
- L. C. Briand and J. Wust, Empirical studies of quality models in object-oriented systems. Elsevier, Vol 56:98--167, 2002Google Scholar
- C. Catal and B. Diri, A systematic review of software fault prediction studies, Journal of Expert Systems with Applications, Vol 36 Issue 4, pp. 7346--7354 May 2009. Google ScholarDigital Library
- S. S. Rathore and A. Gupta, Investigating object-oriented design metrics to predict fault-proneness of software modules, Proceeding os of 6th CSI international conference on software engineering, pp 1--10, 2012.Google Scholar
- R. Malhotra and A. Jain, Fault prediction using statistical and machine learning methods for improving software quality, Journal of Information Processing Systems, Vol.8, No.2, June 2012.Google Scholar
- Y. Singh, A. Kaur and R. Malhotra, Application of logistic regression and artificial neural network for predicting software quality models. Proceeding of PROFES, 2008.Google Scholar
- Y. Singh A. Kaur A and R. Malhotra R, Empirical validation of object-oriented metrics for predicting fault proneness models. Journal of Software Quality, 2010. Google ScholarDigital Library
- R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval. Reading, MA, USA: Addison-Wesley, 1999. Google ScholarDigital Library
- P. Singh and S. Verma, Empirical investigation of fault prediction capability of object oriented metrics of open source software. Proceeding of the international joint conference on CS and SE, IEEE, pp 323--327, 2012Google Scholar
- K. M. Ali and M. J. Pazzani, Error Reduction through learning Multiple Descriptiond, TR 95-39, Deaprtment of ICS University of Ca, Irvine, 1995.Google Scholar
- L. K. Hansen and P. Salamon, Neural Network Ensembles, IEEE Transaction on Pattern Analysis and Machine Intelligence, Vol 12, pp. 993--1001, 1990 Google ScholarDigital Library
- S. Dezeroski and and B. Zenko. Is Combining Classifiers with Stacking Better than Selecting the Best One, Kluwer Academics Publishers, Manufactured in The Netherland, 54, 255--273, 2004. Google ScholarDigital Library
- D. Opitz and R. Maclin, Popular Ensemble Methods: An Empirical Study, Journal of Artificial Research, 1 1: 169--198, 1999.Google Scholar
- A. J. C. Sharkey, Types of multinet system, in: Proc. Int. Workshop on Multiple Classifier Systems (LNCS 2364), Springer, Calgiari, Italy, 2002, pp. 108--117. Google ScholarDigital Library
- T. K. Ho, Data complexity analysis for classifier combination, in: Proc. Int. Workshop on Multiple Classifier Systems (LNCS 2096), Springer, Cambridge, UK, 2001, pp. 53--67 Google ScholarDigital Library
- m L. I. Kuncheva, Combining Pattern Classifiers, Wiley Press 2005.Google Scholar
- G. Valentini and F. Masulli, Ensembles of learning machines. In R. Tagliaferri and M. Marinaro, editors, Neural Nets, WIRN, Vol. 2486 of Lecture Notes in ComputerScience, Springer, pp. 3--19, 2002. Google ScholarDigital Library
Index Terms
- Performance Evaluation of Ensemble Methods For Software Fault Prediction: An Experiment
Recommendations
Empirical investigation of fault predictors in context of class membership probability estimation
SAC '16: Proceedings of the 31st Annual ACM Symposium on Applied ComputingIn the domain of software fault prediction, class membership probability of a selected classifier and the factors related to its estimation can be considered as necessary information for tester to take informed decisions about software quality issues. ...
An Experimental Evaluation of Ensemble Methods for Pattern Classification
CICSYN '11: Proceedings of the 2011 Third International Conference on Computational Intelligence, Communication Systems and NetworksEnsemble methods are used in many pattern recognition problems to improve the classification accuracy. Thus, in this paper, the key goal is to evaluate the performance of three popular ensemble methods bagging, boosting, and random forest for pattern ...
Evaluation of ensemble methods for diagnosing of valvular heart disease
In this work, we investigate the use of ensemble learning for improving classifiers which is one of the important directions in the current research of machine learning, in which bagging, boosting and random subspace are three powerful and popular ...
Comments