research-article

Generative encoding for multiagent learning

Authors:

David B. D'Ambrosio,

Kenneth O. StanleyAuthors Info & Claims

GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation

Pages 819 - 826

https://doi.org/10.1145/1389095.1389256

Published: 12 July 2008 Publication History

Abstract

This paper argues that multiagent learning is a potential "killer application" for generative and developmental systems (GDS) because key challenges in learning to coordinate a team of agents are naturally addressed through indirect encodings and information reuse. For example, a significant problem for multiagent learning is that policies learned separately for different agent roles may nevertheless need to share a basic skill set, forcing the learning algorithm to reinvent the wheel for each agent. GDS is a good match for this kind of problem because it specializes in ways to encode patterns of related yet varying motifs. In this paper, to establish the promise of this capability, the Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) generative approach to evolving neurocontrollers learns a set of coordinated policies encoded by a single genome representing a team of predator agents that work together to capture prey. Experimental results show that it is not only possible, but beneficial to encode a heterogeneous team of agents with an indirect encoding. The main contribution is thus to open up a significant new application domain for GDS.

References

[1]

P. J. Bentley and S. Kumar. The ways to grow designs: A comparison of embryogenies for an evolutionary design problem. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-1999), pages 35--43, 1999.

Digital Library

[2]

J. Bongard. Reducing Collective Behavioural Complexity through Heterogeneity. Arti.cial Life VII: Proceedings of the Seventh International Conference on Artificial Life, 2000.

[3]

J. Bongard. Reducing Collective Behavioural Complexity through Heterogeneity. Arti.cial Life VII: Proceedings of the Seventh International Conference on Artificial Life, 2000.

[4]

G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals, and Systems, 2(4):303--314, 1989.

[5]

D. B. D'Ambrosio and K. O. Stanley. A novel generative encoding for exploiting neural network sensor and output geometry. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), New York, NY, 2007. ACM Press.

Digital Library

[6]

T. N. Dupuy. The Evolution of Weapons and Warfare. Da Capo, New York, NY, USA, 1990.

[7]

P. Eggenberger. Evolving Morphologies of Simulated 3d Organisms Based on Differential Gene Expression. Fourth European Conference on Artificial Life, 1997.

[8]

J. Gauci and K. O. Stanley. Generating large-scale neural networks through discovering geometric regularities. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO 2007), New York, NY, 2007. ACM Press.

Digital Library

[9]

C. Green. SharpNEAT homepage. http://sharpneat.sourceforge.net/, 2003--2008.

[10]

F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 81--89. MIT Press, 1996.

Digital Library

[11]

F. Gruau, D. Whitley, and L. Pyeatt. A comparison between cellular encoding and direct encoding for genetic neural networks. In J. R. Koza, D. E. Goldberg, D. B. Fogel, and R. L. Riolo, editors, Genetic Programming 1996: Proceedings of the First Annual Conference, pages 81--89. MIT Press, 1996.

Digital Library

[12]

A. Lindenmayer. Adding continuous components to L-systems. In G. Rozenberg and A. Salomaa, editors, L Systems, Lecture Notes in Computer Science 15, pages 53--68. Springer-Verlag, Heidelberg, Germany, 1974.

Digital Library

[13]

S. Luke and L. Spector. Evolving graphs and networks with edge encoding: Preliminary report. In J. R. Koza, editor, Late-Breaking Papers of Genetic Programming 1996. Stanford Bookstore, 1996.

[14]

A. P. Martin. Increasing genomic complexity by gene duplication and the origin of vertebrates. The American Naturalist, 154(2):111--128, 1999.

[15]

J. F. Miller. Evolving a self-repairing, self-regulating, French flag organism. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), Berlin, 2004. Springer Verlag.

[16]

L. Panait and S. Luke. Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 3(11):383--434, November 2005.

Digital Library

[17]

L. Panait, R. Wiegand, and S. Luke. Improving coevolutionary search for optimal multiagent behaviors. Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI), pages 653--658, 2003.

Digital Library

[18]

M. A. Potter, K. A. De Jong, and J. J. Grefenstette. A coevolutionary approach to learning sequential decision rules. In L. J. Eshelman, editor, Proceedings of the Sixth International Conference on Genetic Algorithms, 1995.

Digital Library

[19]

K. Sims. Evolving 3D morphology and behavior by competition. pages 28--39. MIT Press, Cambridge, MA, 1994.

[20]

K. O. Stanley. Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines Special Issue on Developmental Systems, 8(2):131--162, 2007.

Digital Library

[21]

K. O. Stanley. Compositional pattern producing networks: A novel abstraction of development. Genetic Programming and Evolvable Machines Special Issue on Developmental Systems, 8(2):131--162, 2007.

Digital Library

[22]

K. O. Stanley, D. B. D'Ambrosio, and J. Gauci. A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life, 2008. To appear.

Digital Library

[23]

K. O. Stanley, D. B. D'Ambrosio, and J. Gauci. A hypercube-based indirect encoding for evolving large-scale neural networks. Artificial Life, 2008. To appear.

[24]

K. O. Stanley and R. Miikkulainen. A taxonomy for artificial embryogeny. Artificial Life, 9(2):93--130, 2003.

Digital Library

[25]

K. O. Stanley and R. Miikkulainen. Competitive coevolution through evolutionary complexification. Journal of Artifical Intelligence Research, 21:63--100, 2004.

Digital Library

[26]

P. Stone and M. Veloso. Multiagent Systems: A Survey from a Machine Learning Perspective. Autonomous Robots, 8(3):345--383, 2000.

Digital Library

[27]

R. P. Wiegand. An analysis of cooperative coevolutionary algorithms. PhD thesis, George Mason University, Fairfax, VA, USA, 2004. Director-Kenneth A. Jong.

Digital Library

[28]

C. Yong and R. Miikkulainen. Coevolution of role-based cooperation in multi-agent systems. Technical Report AI07-338, The University of Texas at Austin Department of Computer Sciences, 2007.

Cited By

Miikkulainen RLi XHandl J(2024)Evolution of Neural NetworksProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3648407(928-945)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3648407
Alcaraz-Herrera HTsompanas MBalaz IAdamatzky A(2024)Using neuroevolution for designing soft medical devicesBiomimetic Intelligence and Robotics10.1016/j.birob.2024.100205(100205)Online publication date: Dec-2024
https://doi.org/10.1016/j.birob.2024.100205
Miikkulainen RSilva SPaquete L(2023)Evolution of Neural NetworksProceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3595036(1008-1025)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3595036
Show More Cited By

Index Terms

Generative encoding for multiagent learning
1. Computing methodologies
  1. Artificial intelligence
    1. Distributed artificial intelligence
      1. Multi-agent systems
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

Evolving the placement and density of neurons in the hyperneat substrate
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computation

The Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) approach demonstrated that the pattern of weights across the connectivity of an artificial neural network (ANN) can be generated as a function of its geometry, thereby allowing ...
Evolving policy geometry for scalable multiagent learning
AAMAS '10: Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems: volume 1 - Volume 1

A major challenge for traditional approaches to multiagent learning is to train teams that easily scale to include additional agents. The problem is that such approaches typically encode each agent's policy separately. Such separation means that ...
A hypercube-based encoding for evolving large-scale neural networks

Research in neuroevolution---that is, evolving artificial neural networks (ANNs) through evolutionary algorithms---is inspired by the evolution of biological brains, which can contain trillions of connections. Yet while neuroevolution has produced ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation

July 2008

1814 pages

ISBN:9781605581309

DOI:10.1145/1389095

Conference Chair:
Conor Ryan
University of Limerick, Ireland
,
Editor:
Maarten Keijzer
Chordiant Software International B.V., The Netherlands

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO08

Sponsor:

GECCO08: Genetic and Evolutionary Computation Conference

July 12 - 16, 2008

GA, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

81
Total Citations
View Citations
381
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Miikkulainen RLi XHandl J(2024)Evolution of Neural NetworksProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3648407(928-945)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3648407
Alcaraz-Herrera HTsompanas MBalaz IAdamatzky A(2024)Using neuroevolution for designing soft medical devicesBiomimetic Intelligence and Robotics10.1016/j.birob.2024.100205(100205)Online publication date: Dec-2024
https://doi.org/10.1016/j.birob.2024.100205
Miikkulainen RSilva SPaquete L(2023)Evolution of Neural NetworksProceedings of the Companion Conference on Genetic and Evolutionary Computation10.1145/3583133.3595036(1008-1025)Online publication date: 15-Jul-2023
https://dl.acm.org/doi/10.1145/3583133.3595036
Medvet ENadizar GPigozzi FSalvato E(2023)Evolutionary Machine Learning in RoboticsHandbook of Evolutionary Machine Learning10.1007/978-981-99-3814-8_23(657-694)Online publication date: 2-Nov-2023
https://doi.org/10.1007/978-981-99-3814-8_23
Miikkulainen RWagner M(2022)Evolution of neural networksProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3520304.3533656(1020-1037)Online publication date: 9-Jul-2022
https://dl.acm.org/doi/10.1145/3520304.3533656
Papavasileiou ECornelis JJansen B(2021)A Systematic Literature Review of the Successors of “NeuroEvolution of Augmenting Topologies”Evolutionary Computation10.1162/evco_a_0028229:1(1-73)Online publication date: Mar-2021
https://doi.org/10.1162/evco_a_00282
Miikkulainen RKrawiec K(2021)Evolution of neural networksProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3449726.3461432(426-442)Online publication date: 7-Jul-2021
https://dl.acm.org/doi/10.1145/3449726.3461432
Templier PRachelson EWilson DKrawiec K(2021)A geometric encoding for neural network evolutionProceedings of the Genetic and Evolutionary Computation Conference10.1145/3449639.3459361(919-927)Online publication date: 26-Jun-2021
https://dl.acm.org/doi/10.1145/3449639.3459361
Miikkulainen RCoello Coello C(2020)Evolution of neural networksProceedings of the 2020 Genetic and Evolutionary Computation Conference Companion10.1145/3377929.3389858(495-525)Online publication date: 8-Jul-2020
https://dl.acm.org/doi/10.1145/3377929.3389858
Garciarena USantana RMendiburu A(2020)EvoFlow: A Python library for evolving deep neural network architectures in tensorflow2020 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI47803.2020.9308214(2288-2295)Online publication date: 1-Dec-2020
https://doi.org/10.1109/SSCI47803.2020.9308214
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten