Optimizing floating centroids method neural network classifier using dynamic multilayer particle swarm optimization

ABSTRACT

The floating centroids method (FCM) effectively enhances the performance of neural network classifiers. However, the problem of optimizing the neural network continues to restrict the further improvement of FCM. Traditional particle swarm optimization algorithm (PSO) sometimes converges to a local optimal solution in multimodal landscape, particularly for optimizing neural networks. Therefore, the dynamic multilayer PSO (DMLPSO) is proposed to optimize the neural network for improving the performance of FCM. DMLPSO adopts the basic concepts of multi-layer PSO to introduce a dynamic reorganizing strategy, which achieves that valuable information dynamically interacts among different subswarms. This strategy increases population diversity to promote the performance of DMLPSO when optimizing multimodal functions. Experimental results indicate that the proposed DMLPSO enables FCM to obtain improved solutions in many data sets.

References

1. Zahra Beheshti and Siti Mariyam Hj. Shamsuddin. 2014. CAP-SO: Centripetal accelerated particle swarm optimization. Information Sciences 258 (2014), 54 -- 79.
2. M. Bonner, T. S. Hassan, and T. Li. 2017. Fite-Hille-Wintner-type oscillation criteria for second-order half-linear dynamic equations with deviating arguments. Indagationes Mathematicae (2017), in press.
3. John S. Bridle. 1990. Probabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition. In Neurocomputing, Françoise Fogelman Soulié and Jeanny Hérault (Eds.). Springer Berlin Heidelberg, 227--236.
4. Hajo Broersma, Faustino Gomez, Julian Miller, Mike Petty, and Gunnar Tufte. 2012. Nascence project: Nanoscale engineering for novel computation using evolution. International Journal of Unconventional Computing 8, 4 (2012), 313--317.
5. R. Burbidge, M. Trotter, B. Buxton, and S. Holden. 2001. Drug design by machine learning: support vector machines for pharmaceutical data analysis. Computers & Chemistry 26, 1 (2001), 5 -- 14.
6. L. J. Cao and F. E. H. Tay. 2003. Support vector machine with adaptive parameters in financial time series forecasting. IEEE Transactions on Neural Networks 14, 6 (2003), 1506--1518.
7. C. L. P. Chen and Z. Liu. 2018. Broad learning system: An effective and efficient incremental learning system without the need for deep architecture. IEEE Transactions on Neural Networks and Learning Systems 29, 1 (2018), 10--24.
8. Ireneusz Czarnowski and Piotr Jedrzejowicz. 2012. Agent-Based Approach to RBF Network Training with Floating Centroids. In Computational Collective Intelligence. Technologies and Applications, Ngoc-Thanh Nguyen, Kiem Hoang, and Piotr Jedrzejowicz (Eds.). Springer Berlin Heidelberg, 453--462.
9. Thomas G. Dietterich and Ghulum Bakiri. 1995. Solving multi-class learning problems via error-correcting output codes. Journal of Artificial Intelligence Research 2, 1 (1995), 263--286.
10. M.G. Epitropakis, V.P. Plagianakos, and M.N. Vrahatis. 2012. Evolving cognitive and social experience in Particle Swarm Optimization through Differential Evolution: A hybrid approach. Information Sciences 216 (2012), 50 -- 92.
11. Shu-Kai S. Fan and Erwie Zahara. 2007. A hybrid simplex search and particle swarm optimization for unconstrained optimization. European Journal of Operational Research 181, 2 (2007), 527 -- 548.
12. Wenlong Fu, Mark Johnston, and Mengjie Zhang. 2011. Hybrid Particle Swarm Optimisation Based on History Information Sharing. In Proceedings of the 13th Annual Conference on Genetic and Evolutionary Computation (GECCO '11). ACM, 77--84.
13. D. Geebelen, J. A. K. Suykens, and J. Vandewalle. 2012. Reducing the Number of Support Vectors of SVM Classifiers Using the Smoothed Separable Case Approximation. IEEE Transactions on Neural Networks and Learning Systems 23, 4 (2012), 682--688.
14. B. Gu and V. S. Sheng. 2017. A Robust Regularization Path Algorithm for nu -Support Vector Classification. IEEE Transactions on Neural Networks and Learning Systems 28, 5 (2017), 1241--1248.
15. Shi-Yuan Han, Yue-Hui Chen, and Gong-You Tang. 2017. Fault diagnosis and fault-tolerant tracking control for discrete-time systems with faults and delays in actuator and measurement. Journal of the Franklin Institute 354, 12 (2017), 4719 -- 4738.
16. Robert Hecht-nielsen. 1992. Theory of the Backpropagation Neural Network. In Neural Networks for Perception, Harry Wechsler (Ed.). Academic Press, 65 -- 93.
17. Fengjun Hu. 2017. Sliding Mode Control of Discrete Chaotic System Based on Multimodal Function Series Coupling. Mathematical Problems in Engineering 2017 (2017).
18. S. A. Kashchenko. 2014. Dynamics of a second-order nonlinear equation with a large coefficient of delay control. Doklady Mathematics 90, 1 (2014), 503--506.
19. X. Li. 2010. Niching Without Niching Parameters: Particle Swarm Optimization Using a Ring Topology. IEEE Transactions on Evolutionary Computation 14, 1 (2010), 150--169.
20. Shuangrong Liu, Bo Yang, Lin Wang, Xiuyang Zhao, Jin Zhou, and Jifeng Guo. 2016. Prediction of share price trend using FCM neural network classifier. In 2016 3rd International Conference on Informative and Cybernetics for Computational Social Systems (ICCSS). 81--86.
21. R. Mendes, J. Kennedy, and J. Neves. 2004. The fully informed particle swarm: simpler, maybe better. IEEE Transactions on Evolutionary Computation 8, 3 (2004), 204--210.
22. Silja Meyer-Nieberg. 2017. Coordinating a Team of Searchers: Of Ants, Swarms, and Slime Molds. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO '17). ACM, 27--34.
23. D. Niu, J. G. Dy, and M. I. Jordan. 2014. Iterative Discovery of Multiple Alternative Clustering Views. IEEE Transactions on Pattern Analysis and Machine Intelligence 36, 7 (2014), 1340--1353.
24. E. J. Palomo and E. Lpez-Rubio. 2017. The Growing Hierarchical Neural Gas Self-Organizing Neural Network. IEEE Transactions on Neural Networks and Learning Systems 28, 9 (2017), 2000--2009.
25. K. E. Parsopoulos and M. N. Vrahatis. 2004. UPSO: A unified particle swarm optimization scheme. Lecture Semes on Computer and Computational Science 1 (2004), 868--873.
26. Angel Arturo Rojas-García and Arturo Hernández-Aguirre. 2016. Using Mutual Information to Build Dynamic Neighbourhoods for Particle Swarm Optimisation. In Proceedings of the Genetic and Evolutionary Computation Conference 2016 (GECCO '16). ACM, 45--52.
27. Yang Shi, Hongcheng Liu, Liang Gao, and Guohui Zhang. 2011. Cellular particle swarm optimization. Information Sciences 181, 20 (2011), 4460 -- 4493.
28. Radha Thangaraj, Millie Pant, Ajith Abraham, and Pascal Bouvry. 2011. Particle swarm optimization: Hybridization perspectives and experimental illustrations. Appl. Math. Comput. 217, 12 (2011), 5208 -- 5226.
29. Chengyan Wang, Bing Xue, and Lin Shang. 2017. PSO-based Parameters Selection for the Bilateral Filter in Image Denoising. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO '17). ACM, 51--58.
30. Hui Wang, Hui Sun, Changhe Li, Shahryar Rahnamayan, and Jeng shyang Pan. 2013. Diversity enhanced particle swarm optimization with neighborhood search. Information Sciences 223 (2013), 119 -- 135.
31. Lin Wang and Jeff Orchard. 2017. Investigating the Evolution of a Neuroplasticity Network for Learning. IEEE Transactions on Systems, Man and Cybernetics: Systems (2017), in press
32. Lin Wang, Bo Yang, and Yuehui Chen. 2014. Improving particle swarm optimization using multi-layer searching strategy. Information Sciences 274 (2014), 70 -- 94.
33. Lin Wang, Bo Yang, Yuehui Chen, Ajith Abraham, Hongwei Sun, Zhenxiang Chen, and Haiyang Wang. 2012. Improvement of neural network classifier using floating centroids. Knowledge and Information Systems 31, 3 (2012), 433--454.
34. Lin Wang, Bo Yang, Yuehui Chen, and Jeff Orchard. 2017. Improving Neural-Network Classifiers Using Nearest Neighbor Partitioning. IEEE Transactions on Neural Networks and Learning Systems 28, 10 (2017), 2255--2267.
35. Lin Wang, Bo Yang, and Jeff Orchard. 2016. Particle swarm optimization using dynamic tournament topology. Applied Soft Computing 48 (2016), 584--596.
36. Lin Wang, Bo Yang, Shoude Wang, and Zhifeng Liang. 2015. Building Image Feature Kinetics for Cement Hydration Using Gene Expression Programming With Similarity Weight Tournament Selection. IEEE Transactions on Evolutionary Computation 19, 5 (2015), 679--693.
37. Yitian Xu, Zhiji Yang, and Xianli Pan. 2017. A Novel Twin Support-Vector Machine With Pinball Loss. IEEE Transactions on Neural Networks and Learning Systems 28, 2 (2017), 359--370.
38. Bing Xue, Mengjie Zhang, Yan Dai, and Will N. Browne. 2013. PSO for Feature Construction and Binary Classification. In Proceedings of the 15th Annual Conference on Genetic and Evolutionary Computation (GECCO '13). ACM, 137--144.
39. Ziqiang Yu, Yang Liu, Xiaohui Yu, and Ken Q. Pu. 2015. Scalable Distributed Processing of K Nearest Neighbor Queries over Moving Objects. IEEE Transactions on Knowledge and Data Engineering 27, 5 (2015), 1383--1396.
40. Saul Zapotecas-Martinez, Alberto Moraglio, Hernan E. Aguirre, and Kiyoshi Tanaka. 2016. Geometric Particle Swarm Optimization for Multi-objective Optimization Using Decomposition. In Proceedings of the Genetic and Evolutionary Computation Conference 2016 (GECCO '16). ACM, 69--76.
41. Lei Zhang, Lin Wang, Xujiewen Wang, Keke Liu, and Ajith Abraham. 2012. Research of Neural Network Classifier Based on FCM and PSO for Breast Cancer Classification. In Hybrid Artificial Intelligent Systems, Emilio Corchado, Václav Snášel, Ajith Abraham, Michał Woźniak, Manuel Graña, and Sung-Bae Cho (Eds.). Springer Berlin Heidelberg, 647--654.
42. Wan-Lei Zhao, Cheng-Hao Deng, and Chong-Wah Ngo. 2018. k-means: A revisit. Neurocomputing 291 (2018), 195 -- 206.
43. Jin Zhou, Long Chen, C.L. Philip Chen, Yuan Zhang, and Han-Xiong Li. 2016. Fuzzy clustering with the entropy of attribute weights. Neurocomputing 198 (2016), 125 -- 134.

Index Terms

• Computer systems organization

  • Architectures

    • Other architectures

      • Neural networks