skip to main content
10.1145/1102256.1102277acmconferencesArticle/Chapter ViewAbstractPublication PagesgeccoConference Proceedingsconference-collections
Article

Learning classifier system equivalent with reinforcement learning with function approximation

Published: 25 June 2005 Publication History

Abstract

We present an experimental comparison of the reinforcement process between Learning Classifier System (LCS) and Reinforcement Learning (RL) with function approximation (FA) method, regarding their generalization mechanisms. To validate our previous theoretical analysis that derived equivalence of reinforcement process between LCS and RL, we introduce a simple test environment named Gridworld, which can be applied to both LCS and RL with three different classes of generalization: (1) tabular representation; (2) state aggregation; and (3) linear approximation. From the simulation experiments comparing LCS with its GA-inactivated and corresponding RL method, all the cases regarding the class of generalization showed identical results with the criteria of performance and temporal difference (TD) error, thereby verifying the equivalence predicted from the theory.

References

[1]
Butz, M., Kovacs, T., Lanzi, P. L., Wilson, S. W.: Toward a theory of generalization and learning in xcs. IEEE Transactions on Evolutionary Computation 8 (2004) 28--46
[2]
M. V. Butz, D. E. Goldberg, and P. L. Lanzi. Bounding learning time in XCS. In Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2004), 2004.
[3]
J. H. Holland. Adaptation in Natural and Artifical Systems. The University of Michigan Press, Michigan, 1975.
[4]
P. L. Lanzi. Learning classifier systems from a reinforcement learning perspective. Soft Computing, 6:162--170, 2002.
[5]
R. Sutton and A. Barto. An introduction to reinforcement learning. MIT Press, Cambridge, MA., 1998.
[6]
S. W. Wilson. ZCS: A zeroth level classifier system. Evolutionary Computation, 2(1):1--18, 1994.
[7]
S. W. Wilson. Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149--175, 1995.

Cited By

View all

Index Terms

  1. Learning classifier system equivalent with reinforcement learning with function approximation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    GECCO '05: Proceedings of the 7th annual workshop on Genetic and evolutionary computation
    June 2005
    431 pages
    ISBN:9781450378000
    DOI:10.1145/1102256
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 June 2005

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. function approximation
    2. genetic-based machine learning
    3. learning classifier systems
    4. reinforcement learning

    Qualifiers

    • Article

    Conference

    GECCO05
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)3
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 16 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2012)Genetics-Based Machine LearningHandbook of Natural Computing10.1007/978-3-540-92910-9_30(937-986)Online publication date: 2012
    • (2008)Learning classifier systems: then and nowEvolutionary Intelligence10.1007/s12065-007-0003-31:1(63-82)Online publication date: 8-Feb-2008
    • (2007)Counter example for Q-bucket-brigade under prediction problemProceedings of the 2003-2005 international conference on Learning classifier systems10.5555/1761381.1761393(128-143)Online publication date: 1-Jan-2007
    • (2007)Empirical analysis of generalization and learning in XCS with gradient descentProceedings of the 9th annual conference on Genetic and evolutionary computation10.1145/1276958.1277321(1814-1821)Online publication date: 7-Jul-2007
    • (2007)Counter Example for Q-Bucket-Brigade Under Prediction ProblemLearning Classifier Systems10.1007/978-3-540-71231-2_10(128-143)Online publication date: 2007
    • (2005)Counter example for Q-bucket-brigade under prediction problemProceedings of the 7th annual workshop on Genetic and evolutionary computation10.1145/1102256.1102278(94-99)Online publication date: 25-Jun-2005

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media