An Adaptive Particle Swarm Algorithm for Unconstrained Global Optimization of Multimodal Functions

ABSTRACT

Conventional Particle Swarm Optimization (PSO) algorithms often suffer premature convergences. Hybrid algorithms, for instance, the Simulated Annealing-based PSO, present low convergence speeds. In this paper, we develop an Adaptive Particle Swarm Optimization (APSO) algorithm to solve unconstrained global optimization problems with highly multimodal functions, in which two adaptive strategies (including an adaptive inertia weight strategy with hybrid time-varying dynamics and an adaptive random mutation strategy) are merged into the basic PSO algorithm to guarantee the algorithm performance. The proposed algorithm is numerically verified by fifteen classical multimodal functions which include Ackley, Bukin f6, Cross-in-Tray, Drop-Wave, Eggholder, Griewank, Holder Table, Langermann, Levy, Levy f13, Rastrigin, Schaffer f2, Schaffer f4, Schwefel, and Shubert. Numerical experiments demonstrate that the proposed algorithm has a potential to achieve better solutions with acceptable computational time, especially for high-dimensional optimization problems.

References

1. Bremermann, H. J. 1958. The evolution of intelligence. The Nervous System as a Model of its Environment. Technical report, no 1, contract no 477(17), Department of Mathematics, University of Washington, Seattle.
2. Storn, R. and Price, K. 1995. Differential Evolution -- A Simple and Efficient Adaptive Scheme for Global Optimization over Continuous Spaces. International Computer Science Institute, Berkeley. CA, Technical Report TR-95-012, ICSI Available.
3. Kennedy, J. and Eberhart, R. C. 1995. Particle swarm optimization. In Proceedings of IEEE International Conference on Neural Networks (Perth, Australia), Piscataway, NJ, 1942--1948.
4. Yuan, Q. and Yin, G. 2015. Analyzing Convergence and Rates of Convergence of Particle Swarm Optimization Algorithms Using Stochastic Approximation Methods. IEEE T. Automat. Contr. 60, 7, 1760--1773.
5. Hassan, R., Cohanim, B., De Weck, O., and Venter, G. 2005. A comparison of particle swarm optimization and the genetic algorithm. In Proceedings of the 1st AIAA multidisciplinary design optimization specialist conference. No. AIAA-2005-1897, Austin, TX, 18--21.
6. Banks, A., Vincent, J., and Anyakoha, C. 2007. A review of particle swarm optimization. Part I: background and development. Nat. Comp. 6, 4, 467--484.
7. Lin, S. W., Ying, K. C., Chen, S. C., and Lee, Z. J. 2008. Particle swarm optimization for parameter determination and feature selection of support vector machines. Expert. Syst. Appl. 35, 4, 1817--1824.
8. Jiang, P., Chen, C., and Liu, X. 2016. Time series prediction for evolutions of complex systems: A deep learning approach. In Proceedings of IEEE International Conference on Control and Robotics Engineering (Singapore), 2016. pp. 1--6.
9. Jiang, P., Liu, X., Zhang, J., and Yuan, X. 2016. A framework based on hidden Markov model with adaptive weighting for microcystin forecasting and early-warning. Decis. Support. Syst. 84, 89--103.
10. Kuremoto, T., Kimura, S., Kobayashi, K., and Obayashi, M. 2014. Time series forecasting using a deep belief network with restricted Boltzmann machines. Neurocomputing, 137, 47--56.
11. Shi, Y. and Eberhart, R. 1998. A modified particle swarm optimizer. In Proceedings of the IEEE congress on evolutionary computation, Piscataway, 69--73.
12. Eberhart, R. C. and Shi, Y. 2000. Comparing Inertia Weights and Constriction Factors in Particle Swarm Optimization. In Proceedings of IEEE congress on evolutionary computation, San Diego, CA, 84--88.
13. Clerc, M. 1999. The swarm and the queen: towards a deterministic and adaptive particle swarm optimization. In Proceedings of IEEE Congress on Evolutionary Computation, vol. 3, pp. 1957.
14. Clerc, M. and Kennedy, J. 2002. The particle swarm: explosion, stability and convergence in a multi-dimensional complex space. IEEE Trans. Evol. Comput. 6, 58--73.
15. Banks, A., Vincent, J., and Anyakoha, C. 2008. A review of particle swarm optimization. Part II: hybridisation, combinatorial, multicriteria and constrained optimization, and indicative applications. Nat. Comp. 7, 1, 109--124.
16. Locatelli, M. 1996. Convergence properties of simulated annealing for continuous global optimization. J. Appl. Probab. 33, 4 (Dec., 1996), 1127--1140.
17. Shieh, H. L., Kuo, C. C., and Chiang, C. M. 2011. Modified particle swarm optimization algorithm with simulated annealing behavior and its numerical verification. Appl. Math. Comput. 218, 8, 4365--4383.
18. Hu, M., Wu, T., and Weir, J. D. 2012. An intelligent augmentation of particle swarm optimization with multiple adaptive methods. Inform. Sciences. 213, 68--83.
19. Liang, X., Li, W., Zhang, Y., and Zhou, M. 2015. An adaptive particle swarm optimization method based on clustering. Soft. Comput. 19, 2, 431--448.
20. Shi, Y. and Eberhart, R. C. 1999. Empirical study of particle swarm optimization. In Proceedings of the IEEE Congress on Evolutionary Computation, Vol. 3, 1945--1950.
21. Ratnaweera, A., Halgamuge, S. K., and Watson, H. C. 2004. Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans. Evol. Comput. 8, 3, 240--255.
22. Chatterjee, A. and Siarry, P. 2006. Nonlinear inertia weight variation for dynamic adaptation in particle swarm optimization. Comput. Oper. Res. 33, 3, 859--871.
23. Liu, B., Wang, L., Jin, Y. H., Tang, F., and Huang, D. X. 2005. Improved particle swarm optimization combined with chaos. Chaos. Soliton. Fract. 25, 5, 1261--1271.
24. Zhan, Z. H., Zhang, J., Li, Y., and Chung, H. S. H. 2009. Adaptive particle swarm optimization. IEEE T. Syst. Man. CY. B. 39, 6, 1362--1381.
25. Xu, G. 2013. An adaptive parameter tuning of particle swarm optimization algorithm. Appl. Math. Comput. 219, 9, 4560--4569.
26. Hu, M., Wu, T., and Weir, J. D. 2013. An adaptive particle swarm optimization with multiple adaptive methods. IEEE Trans. Evol. Comput. 17, 5, 705--720.