skip to main content
10.1145/1389095.1389365acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
research-article

Accelerating neuroevolutionary methods using a Kalman filter

Published: 12 July 2008 Publication History

Abstract

In recent years, neuroevolutionary methods have shown great promise in solving learning tasks, especially in domains that are stochastic, partially observable, and noisy. In this paper, we show how the Kalman filter can be exploited (1) to efficiently find an optimal solution (i. e. reducing the number of evaluations needed to find the solution), (2) to find solutions that are robust against noise, and (3) to recover or reconstruct missing state variables, traditionally known as state estimation in control engineering community. Our algorithm has been tested on the double pole balancing without velocities benchmark, and has achieved significantly better results on this benchmark than the published results of other algorithms to date.

References

[1]
Y. Bar-Shalom, X. Li, and T. Kirubarajan. Estimation with Applications to Tracking and Navigation. John Wiley & Sons, New York, USA, 2001.
[2]
P. Dürr, C. Mattiussi, and D. Floreano. Neuroevolution with analog genetic encoding. In Proceedings of the 9th Conference on Parallel Problem Solving from Nature (PPSN IX), pages 671--680, 2006.
[3]
F. J. Gomez and R. Miikkulainen. Incremental evolution of complex general behavior. Adaptive Behavior, 5:317--342, 1997.
[4]
F. J. Gomez, J. Schmidhuber, and R. Miikkulainen. Efficient non-linear control through neuroevolution. In Proceedings of the European Conference on Machine Learning (ECML 2006), pages 654--662, 2006.
[5]
F. Gruau. Neural Network Synthesis Using Cellular Encoding and the Genetic Algorithm. PhD thesis, Ecole Normale Superieure de Lyon, Laboratoire de l'Informatique du Parallelisme, France, January 1994.
[6]
F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming: Proceedings of the First Annual Conference, pages 81--89, Standford University, CA, USA, 1996. MIT Press.
[7]
N. Hansen and A. Ostermeier. Completely derandomized self-adaptation in evolution strategies. Evolutionary Computation, 9(2):159--195, 2001.
[8]
C. Igel. Neuroevolution for reinforcement learning using evolution strategies. In R. Sarker, R. Reynolds, H. Abbass, K. C. Tan, B. McKay, D. Essam, and T. Gedeon, editors, Congress on Evolutionary Computation (CEC2003), volume 4, pages 2588--2595. IEEE Press, 2003.
[9]
L. P. Kaelbling, M. L. Littman, and A. P. Moore. Reinforcement learning: A survey. Journal of Artificial Intelligence Research, 4:237--285, 1996.
[10]
P. R. Kalata. Alpha-beta target tracking systems: A survey. In American Control Conference, pages 832--836, 1992.
[11]
R. E. Kalman. A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering, Series D:35--45, 1960.
[12]
Y. Kassahun, M. Edgington, J. H. Metzen, G. Sommer, and F. Kirchner. A common genetic encoding for both direct and indirect encodings of networks. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2007), pages 1029--1036, 7 2007.
[13]
Y. Kassahun and G. Sommer. Efficient reinforcement learning through evolutionary acquisition of neural topologies. In Proceedings of the 13th European Symposium on Artificial Neural Networks (ESANN 2005), pages 259--266, Bruges, Belgium, April 2005.
[14]
L. J. Lin and T. M. Mitchell. Memory approaches to reinforcement learning in non-markovian domains. Technical Report CMU-CS-92-138, Carnegie Mellon University, School of Computer Science, USA, 1992.
[15]
D. Moriarty and R. Miikkulainen. Efficient reinforcement learning through symbiotic evolution. Machine Learning, 22:11--33, 1996.
[16]
N. Saravanan and D. B. Fogel. Evolving neural control systems. IEEE Expert, 3:23--27, 1995.
[17]
H.-P. P. Schwefel. Evolution and Optimum Seeking:The Sixth Generation. John Wiley & Sons, Inc., New York, NY, USA, 1993.
[18]
N. T. Siebel and G. Sommer. Evolutionary reinforcement learning of artificial neural networks.International Journal of Hybrid Intelligent Systems, 4(3):171--183, 2007.
[19]
K. O. Stanley. Efficient Evolution of Neural Networks through Complexification. PhD thesis, Artificial Intelligence Laboratory. The University of Texas at Austin., Austin, USA, August 2004.
[20]
G. Welch and G. Bishop. An introduction to the Kalman filter. Technical Report TR95-041, University of North Carolina at Chapel Hill, Department of Computer Science, USA, 1995.
[21]
A. Wieland. Evolving controls for unstable systems. In Proceedings of the International Joint Conference on Neural Networks, pages 667--673, 1991.

Cited By

View all
  • (2024)A Training-Free Neural Architecture Search Algorithm Based on Search EconomicsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.326453328:2(445-459)Online publication date: Apr-2024
  • (2024)A spatio‐temporal graph convolutional approach to real‐time load forecasting in an edge‐enabled distributed Internet of Smart Grids energy systemConcurrency and Computation: Practice and Experience10.1002/cpe.806036:13Online publication date: 13-Mar-2024
  • (2012)Learning parameters of linear models in compressed parameter spaceProceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II10.1007/978-3-642-33266-1_14(108-115)Online publication date: 11-Sep-2012
  • Show More Cited By

Index Terms

  1. Accelerating neuroevolutionary methods using a Kalman filter

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation
    July 2008
    1814 pages
    ISBN:9781605581309
    DOI:10.1145/1389095
    • Conference Chair:
    • Conor Ryan,
    • Editor:
    • Maarten Keijzer
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 July 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Kalman filter
    2. neuroevolution

    Qualifiers

    • Research-article

    Conference

    GECCO08
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)5
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)A Training-Free Neural Architecture Search Algorithm Based on Search EconomicsIEEE Transactions on Evolutionary Computation10.1109/TEVC.2023.326453328:2(445-459)Online publication date: Apr-2024
    • (2024)A spatio‐temporal graph convolutional approach to real‐time load forecasting in an edge‐enabled distributed Internet of Smart Grids energy systemConcurrency and Computation: Practice and Experience10.1002/cpe.806036:13Online publication date: 13-Mar-2024
    • (2012)Learning parameters of linear models in compressed parameter spaceProceedings of the 22nd international conference on Artificial Neural Networks and Machine Learning - Volume Part II10.1007/978-3-642-33266-1_14(108-115)Online publication date: 11-Sep-2012
    • (2011)On Applying Neuroevolutionary Methods to Complex Robotic TasksNew Horizons in Evolutionary Robotics10.1007/978-3-642-18272-3_7(85-108)Online publication date: 2011
    • (2009)Learning complex robot control using evolutionary behavior based systemsProceedings of the 11th Annual conference on Genetic and evolutionary computation10.1145/1569901.1569920(129-136)Online publication date: 8-Jul-2009
    • (2008)EANT+KALMANProceedings of the 31st annual German conference on Advances in Artificial Intelligence10.1007/978-3-540-85845-4_30(241-248)Online publication date: 23-Sep-2008

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media