skip to main content
10.1145/3205651.3208297acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Benchmarking the PSA-CMA-ES on the BBOB noiseless testbed

Published: 06 July 2018 Publication History

Abstract

We evaluate the CMA-ES with population size adaptation mechanism (PSA-CMA-ES) on the BBOB noiseless testbed. On one hand, the PSA-CMA-ES with a simple restart strategy shows performance competitive with the best 2009 portfolio on most well-structured multimodal functions. On the other hand, it is not effective on weakly-structured multimodal functions. Moreover, on most uni-modal functions, the scale-up of performance measure w.r.t. the dimension tends to be worse than the default CMA-ES, implying that the population size is adapted greater than needed on the unimodal functions. To improve performance on unimodal functions and weakly-structured multimodal functions, we additionally propose a restart strategy for the PSA-CMA-ES. The proposed strategy consists of three search regimes. The resulted restart strategy shows improved performance on unimodal functions and weakly-structured multimodal functions with a little compromise in the performance on well-structured multimodal functions. The overall performance is competitive to the BIPOP-CMA-ES.

References

[1]
Y. Akimoto, A. Auger, and N. Hansen. 2017. Quality Gain Analysis of the Weighted Recombination Evolution Strategy on General Convex Quadratic Functions. (2017). arXiv:1608.04813v5
[2]
S. Finck, N. Hansen, R. Ros, and A. Auger. 2009. Real-Parameter Black-Box Optimization Benchmarking 2009: Presentation of the Noiseless Functions. Technical Report 2009/20. Research Center PPE. http://coco.lri.fr/downloads/download15.03/bbobdocfunctions.pdf Updated February 2010.
[3]
N. Hansen. 2009. Benchmarking a BI-population CMA-ES on the BBOB-2009 function testbed. In Workshop Proceedings of the GECCO Genetic and Evolutionary Computation Conference. ACM Press, New York, New York, USA, 2389--2395.
[4]
N. Hansen. 2016. The CMA Evolution Strategy: A Tutorial. ArXiv e-prints (April 2016). arXiv:cs.LG/1604.00772
[5]
N. Hansen, A Auger, D. Brockhoff, D. Tušar, and T. Tušar. 2016. COCO: Performance Assessment. ArXiv e-prints arXiv:1605.03560 (2016).
[6]
N. Hansen, A. Auger, S. Finck, and R. Ros. 2012. Real-Parameter Black-Box Optimization Benchmarking 2012: Experimental Setup. Technical Report. INRIA. http://coco.gforge.inria.fr/bbob2012-downloads
[7]
N. Hansen, A. Auger, O. Mersmann, T. Tušar, and D. Brockhoff. 2016. COCO: A Platform for Comparing Continuous Optimizers in a Black-Box Setting. ArXiv e-prints arXiv:1603.08785 (2016).
[8]
N. Hansen, S. Finck, R. Ros, and A. Auger. 2009. Real-Parameter Black-Box Optimization Benchmarking 2009: Noiseless Functions Definitions. Technical Report RR-6829. INRIA. http://coco.lri.fr/downloads/download15.03/bbobdocfunctions.pdf Updated February 2010.
[9]
N. Hansen and S. Kern. 2004. Evaluating the CMA Evolution Strategy on Multi-modal Test Functions. In Parallel Problem Solving from Nature-PPSN VIII. Springer, 282--291.
[10]
N. Hansen, S. D. Muller, and P. Koumoutsakos. 2003. Reducing the time complexity of the derandomized evolution strategy with covariance matrix adaptation (CMA-ES). Evolutionary Computation 11, 1 (2003), 1--18.
[11]
N. Hansen and A. Ostermeier. 2001. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation 9, 2 (2001), 159--195.
[12]
N. Hansen, T. Tušar, O. Mersmann, A. Auger, and D. Brockhoff. 2016. COCO: The Experimental Procedure. ArXiv e-prints arXiv:1603.08776 (2016).
[13]
K. Nishida and Y. Akimoto. 2018. PSA-CMA-ES: CMA-ES with Population Size Adaptation. In Genetic and Evolutionary Computation Conference, GECCO 2018, Kyoto, Japan, July 15--19, 2018. ACM, To appear.
[14]
K Price. 1997. Differential evolution vs. the functions of the second ICEO. In Proceedings of the IEEE International Congress on Evolutionary Computation. IEEE, Piscataway, NJ, USA, 153--157.

Cited By

View all
  • (2024)Adapting the population size in CMA-ES using nearest-better clustering method for multimodal optimizationApplied Soft Computing10.1016/j.asoc.2024.112361167(112361)Online publication date: Dec-2024
  • (2024)Avoiding Redundant Restarts in Multimodal Global OptimizationParallel Problem Solving from Nature – PPSN XVIII10.1007/978-3-031-70068-2_17(268-283)Online publication date: 7-Sep-2024
  • (2022)Improving LSHADE by means of a pre-screening mechanismProceedings of the Genetic and Evolutionary Computation Conference10.1145/3512290.3528805(884-892)Online publication date: 8-Jul-2022
  • Show More Cited By

Index Terms

  1. Benchmarking the PSA-CMA-ES on the BBOB noiseless testbed

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO '18: Proceedings of the Genetic and Evolutionary Computation Conference Companion
    July 2018
    1968 pages
    ISBN:9781450357647
    DOI:10.1145/3205651
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 06 July 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. benchmarking
    2. black-box optimization
    3. covariance matrix adaptation
    4. population size adaptation

    Qualifiers

    • Research-article

    Conference

    GECCO '18
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)8
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 02 Mar 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Adapting the population size in CMA-ES using nearest-better clustering method for multimodal optimizationApplied Soft Computing10.1016/j.asoc.2024.112361167(112361)Online publication date: Dec-2024
    • (2024)Avoiding Redundant Restarts in Multimodal Global OptimizationParallel Problem Solving from Nature – PPSN XVIII10.1007/978-3-031-70068-2_17(268-283)Online publication date: 7-Sep-2024
    • (2022)Improving LSHADE by means of a pre-screening mechanismProceedings of the Genetic and Evolutionary Computation Conference10.1145/3512290.3528805(884-892)Online publication date: 8-Jul-2022
    • (2022)Surrogate-Assisted LSHADE Algorithm Utilizing Recursive Least Squares FilterParallel Problem Solving from Nature – PPSN XVII10.1007/978-3-031-14714-2_11(146-159)Online publication date: 14-Aug-2022
    • (2019)GECCO black-box optimization competitionsProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3319619.3321996(275-276)Online publication date: 13-Jul-2019

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media