research-article

Context-dependent predictions and cognitive arm control with XCSF

Authors:

Martin V. Butz,

Oliver HerbortAuthors Info & Claims

GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation

Pages 1357 - 1364

https://doi.org/10.1145/1389095.1389360

Published: 12 July 2008 Publication History

Abstract

While John Holland has always envisioned learning classifier systems (LCSs) as cognitive systems, most work on LCSs has focused on classification, datamining, and function approximation. In this paper, we show that the XCSF classifier system can be very suitably modified to control a robot system with redundant degrees of freedom, such as a robot arm. Inspired by recent research insights that suggest that sensorimotor codes are nearly ubiquitous in the brain and an essential ingredient for cognition in general, the XCSF system is modified to learn classifiers that encode piecewise linear sensorimotor structures, which are conditioned on prediction-relevant contextual input. In the investigated robot arm problem, we show that XCSF partitions the (contextual) posture space of the arm in such a way that accurate hand movements can be predicted given particular motor commands. Furthermore, we show that the inversion of the sensorimotor predictive structures enables accurate goal-directed closed-loop control of arm reaching movements. Besides the robot arm application, we also investigate performance of the modified XCSF system on a set of artificial functions. All results point out that XCSF is a useful tool to evolve problem space partitions that are maximally effective for the encoding of sensorimotor dependencies. A final discussion elaborates on the relation of the taken approach to actual brain structures and cognitive psychology theories of learning and behavior.

References

[1]

E. Bernadó-Mansilla and J. M. Garrell-Guiu. Accuracy-based learning classifier systems: Models, analysis, and applications to classification tasks. Evolutionary Computation, 11:209--238, 2003.]]

Digital Library

[2]

L. B. Booker, D. E. Goldberg, and J. H. Holland. Classifier systems and genetic algorithms. Artificial Intelligence, 40:235--282, 1989.]]

Digital Library

[3]

L. Bull, editor. Applications of Learning Classifier Systems. Springer-Verlag, 2004.]]

Digital Library

[4]

D. Bullock, S. Grossberg, and F. H. Guenther. A self-organizing neural model of motor equivalent reaching and tool use by a multijoint arm. Journal of Cognitive Neuroscience, 5:408--435, 1993.]]

Digital Library

[5]

M. V. Butz. Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system. GECCO 2005: Genetic and Evolutionary Computation Conference, pages 1835--1842, 2005.]]

Digital Library

[6]

M. V. Butz. Documentation of XCSFJava 1.1 plus visualization. MEDAL Report 2007008, Missouri Estimation of Distribution Algorithms Laboratory, University of Missouri in St. Louis, MO, 2007.]]

[7]

M. V. Butz, T. Kovacs, P. L. Lanzi, and S.W.Wilson. Toward a theory of generalization and learning in XCS. IEEE Transactions on Evolutionary Computation, 8:28--46, 2004.]]

Digital Library

[8]

M. V. Butz, P. L. Lanzi, and S. W. Wilson. Hyper-ellipsoidal conditions in XCS: Rotation, linear approximation, and solution structure. GECCO 2006: Genetic and Evolutionary Computation Conference, pages 1457--1464, 2006.]]

Digital Library

[9]

M. V. Butz, P. L. Lanzi, and S. W. Wilson. Function approximation with XCS: Hyperellipsoidal conditions, recursive least squares, and compaction. IEEE Transactions on Evolutionary Computation, in press.]]

Digital Library

[10]

M. V. Butz, K. Sastry, and D. E. Goldberg. Strong, stable, and reliable fitness pressure in XCS due to tournament selection. Genetic Programming and Evolvable Machines, 6:53--77, 2005.]]

Digital Library

[11]

M. V. Butz and S. W. Wilson. An algorithmic description of XCS. Soft Computing, 6:144--153, 2002.]]

[12]

J. G. Fleischer. Neural correlates of anticipation in cerebellum, basal ganglia, and hippocampus. In M. V. Butz, O. Sigaud, G. Pezzulo, and G. Baldassarre, editors, Anticipatory Behavior in Adaptive Learning Systems: From Brains to Individual and Social Behavior. Springer-Verlag, 2007.]]

Digital Library

[13]

A. P. Georgopoulus. Current issues in directional motor control. Trends in Neuroscience, 18(11):506--510, 1995.]]

[14]

M. Graziano. The organization of behavioral repertoire in motor cortex. Annual Review Neuroscience, 29:105--134, 2006.]]

[15]

A. Greenwald. Sensory feedback mechanisms in performance control: with special reference to the ideo-motor mechanism. Psychological Review, 77:73--99, 1970.]]

[16]

M. Haruno, D. M. Wolpert, and M. Kawato. Mosaic model for sensorimotor learning and control. Neural Computation, 13:2201--2220, 2001.]]

Digital Library

[17]

J. H. Holland. A cognitive system with powers of generalization and adaptation. Unpublished manuscript, 1977.]]

[18]

J. H. Holland and J. S. Reitman. Cognitive systems based on adaptive algorithms. In D. A. Waterman and F. Hayes-Roth, editors, Pattern directed inference systems, pages 313--329. Academic Press, New York, 1978.]]

[19]

J. H. Holmes, D. R. Durbin, and F. K. Winston. A new bootstrapping method to improve classification performance in learning classifier systems. Parallel Problem Solving from Nature PPSN, VI:745--754, 2000.]]

Digital Library

[20]

J. Hurst and L. Bull. A neural learning classifier system with self-adaptive constructivism for mobile robot learning. Artificial Life, 12:1--28, 2006.]]

Digital Library

[21]

T. Kovacs. Bibliography of real-world classifier systems applications. In L. Bull, editor, Applications of Learning Classifier Systems. Springer-Verlag, Berlin Heidelberg, 2004.]]

[22]

P. L. Lanzi, D. Loiacono, S. W. Wilson, and D. E. Goldberg. Extending XCSF beyond linear approximation. GECCO 2005: Genetic and Evolutionary Computation Conference, pages 1827--1834, 2005.]]

Digital Library

[23]

P. L. Lanzi, D. Loiacono, S. W. Wilson, and D. E. Goldberg. Prediction update algorithms for XCSF: RLS, kalman filter and gain adaptation. GECCO 2006: Genetic and Evolutionary Computation Conference, pages 1505--1512, 2006.]]

Digital Library

[24]

A. Maravita, C. Spence, and J. Driver. Multisensory integration and the body schema: Close to hand and within reach. Current Biology, 13:531--539, 2003.]]

[25]

F. A. Mussa-Ivaldi and E. Bizzi. Motor learning through the combination of primitives. Philosophical Transactions of the Royal Society: Biological Sciences, 355:1755--1769, 2000.]]

[26]

R. P. N. Rao and D. H. Ballard. Predictive coding in the visual cortex: A functional interpretation of some extra-classical receptive-field effects. Nature Neuroscience, 2(1):79--87, 1999.]]

[27]

G. Rizzolatti, L. Fadiga, L. Fogassi, and V. Gallese. Enhanced: The space around us. Science, 277:190--191, 1997.]]

[28]

A. B. Schwartz, D. W. Moran, and G. A. Reina. Differential representation of perception and action in the frontal cortex. Science, 303:380--383, 2004.]]

[29]

C. Stone and L. Bull. An analysis of continuous-valued representations for learning classifier systems. In L. Bull and T. Kovacs, editors, Foundations of Learning Classifier Systems, Studies in Fuzziness and Soft Computing, pages 127--175. Springer-Verlag, Berlin Heidelberg, 2005.]]

[30]

P. F. M. J. Verschure, T. Voegtlin, and R. J. Douglas. Environmentally mediated synergy between perception and behaviour in mobile robots. Nature, 425:620--624, 2003.]]

[31]

S. W. Wilson. Classifier fitness based on accuracy. Evolutionary Computation, 3(2):149--175, 1995.]]

Digital Library

[32]

S. W. Wilson. Function approximation with a classifier system. Proceedings of the Third Genetic and Evolutionary Computation Conference (GECCO-2001), pages 974--981, 2001.]]

[33]

S. W. Wilson. Classifiers that approximate functions. Natural Computing, 1:211--234, 2002.]]

Digital Library

[34]

D. M. Wolpert and M. Kawato. Multiple paired forward and inverse models for motor control. Neural Networks, 11:1317--1329, 1998.]]

Digital Library

Cited By

Urbanowicz RYang BMoore J(2018)Problem Driven Machine Learning by Co-evolving Genetic Programming Trees and Rules in a Learning Classifier SystemGenetic Programming Theory and Practice XV10.1007/978-3-319-90512-9_4(55-71)Online publication date: 6-Jul-2018
https://doi.org/10.1007/978-3-319-90512-9_4
Howard DBull LLanzi P(2016)A Cognitive Architecture Based on a Learning Classifier System with Spiking ClassifiersNeural Processing Letters10.1007/s11063-015-9451-444:1(125-147)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1007/s11063-015-9451-4
Vargas DTakano HMurata JEsparcia-Alcázar A(2015)The Relationship Between (Un)Fractured Problems and Division of Input SpaceProceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation10.1145/2739482.2768447(981-987)Online publication date: 11-Jul-2015
https://dl.acm.org/doi/10.1145/2739482.2768447
Show More Cited By

Index Terms

Context-dependent predictions and cognitive arm control with XCSF
1. Computing methodologies
  1. Artificial intelligence
    1. Planning and scheduling
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks

Recommendations

An anticipatory approach to improve XCSF
GECCO '06: Proceedings of the 8th annual conference on Genetic and evolutionary computation

XCSF is a novel version of learning classifier systems (LCS) which extends the typical concept of LCS by introducing computable classifier prediction. In XCSF Classifier prediction is computed as a linear combination of classifier inputs and a weight ...
Improving generalization in the XCSF classifier system using linear least-squares
GECCO '05: Proceedings of the 7th annual workshop on Genetic and evolutionary computation

XCSF is an extension of XCS in which classifier prediction is computed as a linear combination of classifier inputs and a weight vector associated to each classifier. XCSF can adjust the weight vector of classifiers to evolve accurate piecewise linear ...
Guided evolution in XCSF
GECCO '12: Proceedings of the 14th annual conference on Genetic and evolutionary computation

High-dimensional problems are challenging for iterative, online (Michigan-style) Learning Classifier Systems, especially because of the large size of the evolutionary search space. The present work proposes to guide mutation toward most suitable ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation

July 2008

1814 pages

ISBN:9781605581309

DOI:10.1145/1389095

Conference Chair:
Conor Ryan
University of Limerick, Ireland
,
Editor:
Maarten Keijzer
Chordiant Software International B.V., The Netherlands

Copyright © 2008 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 July 2008

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

GECCO08

Sponsor:

GECCO08: Genetic and Evolutionary Computation Conference

July 12 - 16, 2008

GA, Atlanta, USA

Acceptance Rates

Overall Acceptance Rate 1,669 of 4,410 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

22
Total Citations
View Citations
220
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Urbanowicz RYang BMoore J(2018)Problem Driven Machine Learning by Co-evolving Genetic Programming Trees and Rules in a Learning Classifier SystemGenetic Programming Theory and Practice XV10.1007/978-3-319-90512-9_4(55-71)Online publication date: 6-Jul-2018
https://doi.org/10.1007/978-3-319-90512-9_4
Howard DBull LLanzi P(2016)A Cognitive Architecture Based on a Learning Classifier System with Spiking ClassifiersNeural Processing Letters10.1007/s11063-015-9451-444:1(125-147)Online publication date: 1-Aug-2016
https://dl.acm.org/doi/10.1007/s11063-015-9451-4
Vargas DTakano HMurata JEsparcia-Alcázar A(2015)The Relationship Between (Un)Fractured Problems and Division of Input SpaceProceedings of the Companion Publication of the 2015 Annual Conference on Genetic and Evolutionary Computation10.1145/2739482.2768447(981-987)Online publication date: 11-Jul-2015
https://dl.acm.org/doi/10.1145/2739482.2768447
Vargas DTakano HMurata J(2015)Novelty-organizing team of classifiers in noisy and dynamic environments2015 IEEE Congress on Evolutionary Computation (CEC)10.1109/CEC.2015.7257254(2937-2944)Online publication date: May-2015
https://doi.org/10.1109/CEC.2015.7257254
Kneissler JStalph PDrugowitsch JButz M(2014)Filtering sensory information with xcsfEvolutionary Computation10.1162/EVCO_a_0010822:1(139-158)Online publication date: 1-Mar-2014
https://dl.acm.org/doi/10.1162/EVCO_a_00108
Vargas DTakano HMurata J(2014)Novelty-organizing team of classifiers - A team-individual multi-objective approach to reinforcement learning2014 Proceedings of the SICE Annual Conference (SICE)10.1109/SICE.2014.6935299(1785-1792)Online publication date: Sep-2014
https://doi.org/10.1109/SICE.2014.6935299
Vargas DTakano HMurata JAlba E(2013)Self organizing classifiers and niched fitnessProceedings of the 15th annual conference on Genetic and evolutionary computation10.1145/2463372.2463501(1109-1116)Online publication date: 6-Jul-2013
https://dl.acm.org/doi/10.1145/2463372.2463501
Vargas DTakano HMurata J(2013)Self organizing classifiers: first steps in structured evolutionary machine learningEvolutionary Intelligence10.1007/s12065-013-0095-x6:2(57-72)Online publication date: 26-Oct-2013
https://doi.org/10.1007/s12065-013-0095-x
Kneissler JStalph PDrugowitsch JButz MMoore J(2012)Filtering sensory information with XCSFProceedings of the 14th annual conference on Genetic and evolutionary computation10.1145/2330163.2330284(871-878)Online publication date: 7-Jul-2012
https://dl.acm.org/doi/10.1145/2330163.2330284
Marin DSigaud O(2012)Reaching optimally over the workspace: A machine learning approach2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob)10.1109/BioRob.2012.6290743(1128-1133)Online publication date: Jun-2012
https://doi.org/10.1109/BioRob.2012.6290743
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten