Jakob Bossek
ABSTRACT
Assessing the performance of stochastic optimization algorithms in the field of multi-objective optimization is of utmost importance. Besides the visual comparison of the obtained approximation sets, more sophisticated methods have been proposed in the last decade, e. g., a variety of quantitative performance indicators or statistical tests. In this paper, we present tools implemented in the R package ecr, which assist in performing comprehensive and sound comparison and evaluation of multi-objective evolutionary algorithms following recommendations from the literature.
- Thomas Bartz-Beielstein, Christian Lasarczyk, and Mike Preuss. 2005. Sequential Parameter Optimization. In Proceedings Congress on Evolutionary Computation 2005 (CEC'05). Edinburgh, Scotland, 1553. http://www.spotseven.de/wp-content/papercite-data/pdf/blp05.pdfGoogle Scholar
Cross Ref
- Nicola Beume, Boris Naujoks, and Michael Emmerich. 2007. SMS-EMOA: Multi-objective selection based on dominated hypervolume. European Journal of Operational Research 181, 3 (2007), 1653--1669.Google ScholarCross Ref
- Mickael Binois and Victor Picheny. 2018. GPareto: Gaussian Processes for Pareto Front Estimation and Optimization. https://CRAN.R-project.org/package=GPareto R package version 1.1.1.Google Scholar
- Bernd Bischl, Jakob Richter, Jakob Bossek, Daniel Horn, Janek Thomas, and Michel Lang. 2017. mlrMBO: A Modular Framework for Model-Based Optimization of Expensive Black-Box Functions. http://arxiv.org/abs/1703.03373Google Scholar
- S Bleuler, M Laumanns, L Thiele, and E Zitzler. 2003. PISA - a platform and programming language independent interface for search algorithms. In Proceedings of International Conference on Evolutionary Multi-Criterion Optimization (EMO), C M Fonseca, P J Fleming, E Zitzler, K Deb, and L Thiele (Eds.). Springer, Berlin, Germany, 494--508. Google ScholarDigital Library
- Jakob Bossek. 2017. ecr 2.0: A Modular Framework for Evolutionary Computation in R. In Genetic and Evolutionary Computation Conference. Berlin, Germany. Google ScholarDigital Library
- Jakob Bossek and Christian Grimme. 2017. A Pareto-Beneficial Sub-Tree Mutation for the Multi-Criteria Minimum Spanning Tree Problem. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, Honolulu, HI, USA, 3280--3287.Google ScholarCross Ref
- Jakob Bossek and Christian Grimme. 2017. An Extended Mutation-Based Priority-Rule Integration Concept for Multi-Objective Machine Scheduling. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, Honolulu, HI, USA, 3288--3295.Google Scholar
- Jakob Bossek and Christian Grimme. 2018. Solving Scalarized Subproblems within Evolutionary Algorithms for Multi-Criteria Shortest Path Problems. In Proceedings of the 12th International Conference on Learning and Intelligent Optimization (LION 2018). Springer International Publishing, Kalamata, Greece, accepted.Google Scholar
- Jakob Bossek and Heike Trautmann. 2016. Understanding characteristics of evolved instances for state-of-the-art inexact TSP solvers with maximum performance difference. In AI*IA 2016 Advances in Artificial Intelligence, G. Adorni, S. Cagnoni, M. Gori, and M. Maratea (Eds.), Vol. 10037 LNAI. Springer International Publishing, Genova, Italy, 3--12. Google ScholarDigital Library
- Carlos A Coello Coello, Gary B Lamont, and David A Van Veldhuizen. 2006. Evolutionary Algorithms for Solving Multi-Objective Problems (Genetic and Evolutionary Computation). Springer-Verlag New York, Inc., Secaucus, NJ, USA. Google ScholarDigital Library
- Kalyanmoy Deb, Samir Agrawal, Amrit Pratap, and T Meyarivan. 2000. A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimization: NSGA-II. In Parallel Problem Solving from Nature PPSN VI, Marc Schoenauer, Kalyanmoy Deb, Günther Rudolph, Xin Yao, Evelyne Lutton, Juan Julian Merelo, and Hans-Paul Schwefel (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 849--858. Google ScholarDigital Library
- Matt Dowle and Arun Srinivasan. 2017. data.table: Extension of 'data.frame'. https://CRAN.R-project.org/package=data.table R package version 1.10.4-3.Google Scholar
- N. Hansen and A. Jaszkiewicz. 2006. Evaluating the quality of approximations to the non-dominated set. Technical Report. Technical University of Denmark.Google Scholar
- Joshua Knowles and David Corne. 2002. On Metrics for Comparing Non-Dominated Sets. In Proceedings of the 2002 Congress on Evolutionary Computation Conference (CEC02). Institute of Electrical and Electronics Engineers, Honolulu, HI, USA, 711--716.Google Scholar
- J Knowles, L Thiele, and E Zitzler. 2006. A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. Technical Report. Computer Engineering and Networks Laboratory (TIK), ETH Zurich, Switzerland, https://sop.tik.ee.ethz.ch/KTZ2005a.pdfGoogle Scholar
- Asep Maulana, Marios Kefalas, and Michael T. M. Emmerich. 2017. Immunization of Networks Using Genetic Algorithms and Multiobjective Metaheuristics. In 2017 IEEE Symposium Series on Computational Intelligence (SSCI). IEEE, Honolulu, HI, USA, 2953--2960.Google Scholar
- Olaf Mersmann. 2012. emoa: Evolutionary Multiobjective Optimization Algorithms. https://CRAN.R-project.org/package=emoa R package version 0.5-0.Google Scholar
- Olaf Mersmann. 2014. mco: Multiple Criteria Optimization Algorithms and Related Functions. https://CRAN.R-project.org/package=mco R package version 1.0-15.1.Google Scholar
- Adriano Passos. 2017. moko: Multi-Objective Kriging Optimization. https://CRAN.R-project.org/package=moko R package version 1.0.1.Google Scholar
- Serpil Saym. 2000. Measuring the quality of discrete representations of efficient sets in multiple objective mathematical programming. Mathematical Programming 87, 3 (01 May 2000), 543--560.Google Scholar
- V. T'kindt and J.-C. Billaut. 2006. Multicriteria Scheduling: Theory, Models and Algorithms (2nd ed.). Berlin Heidelberg. Google ScholarDigital Library
- David Allen Van Veldhuizen. 1999. Multiobjective Evolutionary Algorithms: Classifications, Analyses, and New Innovations. Ph.D. Dissertation. Wright Patterson AFB, OH, USA. Advisor(s) Lamont, Gary B. AAI9928483.Google Scholar
- Hadley Wickham. 2009. ggplot2: Elegant Graphics for Data Analysis. Springer New York, http://ggplot2.org Google ScholarDigital Library
- Hadley Wickham, Romain Francois, Lionel Henry, and Kirill MÃijller. 2017. dplyr: A Grammar of Data Manipulation. https://CRAN.R-project.org/package=dplyr R package version 0.7.4.Google Scholar
- E. Zitzler and L. Thiele. 1999. Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach. IEEE Transactions on Evolutionary Computation 3, 4 (1999), 257--271. Google ScholarDigital Library
- E. Zitzler, L. Thiele, M. Laumanns, C. M. Fonseca, and V. G. Da Fonseca. 2003. Performance assessment of multiobjective optimizers: An analysis and review. IEEE Transactions of Evolutionary Computation 7, 2 (2003), 117--132. Google ScholarDigital Library
Index Terms
- Performance assessment of multi-objective evolutionary algorithms with the R package ecr
Recommendations
A robust evolutionary framework for multi-objective optimization
GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computationEvolutionary multi-objective optimization (EMO) methodologies, suggested in the beginning of Nineties, focussed on the task of finding a set of well-converged and well-distributed set of solutions using evolutionary optimization principles. Of the EMO ...
An archived-based stochastic ranking evolutionary algorithm (asrea) for multi-objective optimization
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computationIn this paper, we propose a new multi-objective optimization algorithm called Archived-based Stochastic Ranking Evolutionary Algorithm (ASREA) that ranks the population by comparing individuals with members of an archive. The stochastic comparison ...
A reference points-based evolutionary algorithm for many-objective optimization
GECCO Comp '14: Proceedings of the Companion Publication of the 2014 Annual Conference on Genetic and Evolutionary ComputationMany-objective optimization problems are common in real-world applications, few evolutionary optimization methods, however, are suitable for them up to date due to their difficulties. We proposed a reference points-based evolutionary algorithm (RPEA) to ...
Comments