skip to main content
10.1145/1255047.1255073acmconferencesArticle/Chapter ViewAbstractPublication PagesesemConference Proceedingsconference-collections
Article

Motivated reinforcement learning for adaptive characters in open-ended simulation games

Published: 13 June 2007 Publication History

Abstract

Recently a new generation of virtual worlds has emerged in which users are provided with open-ended modelling tools with which they can create and modify world content. The result is evolving virtual spaces for commerce, education and social interaction. In general, these virtual worlds are not games and have no concept of winning, however the open-ended modelling capacity is nonetheless compelling. The rising popularity of open-ended virtual worlds suggests that there may also be potential for a new generation of computer games situated in open-ended environments. A key issue with the development of such games, however, is the design of non-player characters which can respond autonomously to unpredictable, open-ended changes to their environment. This paper considers the impact of open-ended modelling on character development in simulation games. Motivated reinforcement learning using context-free grammars is proposed as a means of representing unpredictable, evolving worlds for character reasoning. This technique is used to design adaptive characters for the Second Life virtual world to create a new kind of open-ended simulation game.

References

[1]
Activeworlds. "Activeworlds". www.activeworlds.com (Accessed January 2007).
[2]
Johnson, D. and Wiles, J. "Computer games with intelligence", presented at The 10th IEEE International Conference on Fuzzy Systems, pp1355--1358, 2001.
[3]
Laird, J. and van Lent, M. "Interactive computer games: human-level AI's killer application", presented at National Conference on Artificial Intelligence, pp1171--1178, 2000.
[4]
Linden. "Second Life", www.secondlife.com (accessed January, 2007).
[5]
Maher, M.-L. and Gero, J.S. "Agent models of 3D virtual worlds", ACADIA 2002: Thresholds. California State Polytechnic University, Pamona, pp. 127--138, 2002.
[6]
Merceron, A. Languages and Logic: Pearson Education Australia, 2001.
[7]
Merrick, K. and Maher, M. L. "Motivated reinforcement learning for non-player characters in persistent computer game worlds" presented at ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, 2006, CA, USA, (CD, no page numbers), 2006.
[8]
Merrick, K. 2007, "Modelling Motivation for Experience-Based Attention Focus in Reinforcement Learning", PhD Thesis, University of Sydney (manuscript).
[9]
Nilsson, N. J. Introduction to machine learning: http://ai.stanford.edu/people/nilsson/mlbook.html (accessed January, 2006), 1996.
[10]
Rex, F. "LambdaMOO: An introduction", http://www.lambdamoo.info, (accessed December, 2006).
[11]
Sutton, R. S. and Barto, A. G. Reinforcement learning: an introduction: The MIT Press, 2000.
[12]
Woodcock, S. "Games making interesting use of artificial intelligence techniques" http://www.gameai.com/games.html (accessed October 2005).

Cited By

View all
  • (2022)Toward Believable Acting for Autonomous Animated CharactersProceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3561975.3562941(1-15)Online publication date: 3-Nov-2022
  • (2019)Who Is at the Center?: Designing Playful Experiences by Using Player-Centered ApproachHCI in Games10.1007/978-3-030-22602-2_2(11-21)Online publication date: 16-Jun-2019
  • (2018)Artificial Intelligence and Virtual Worlds – Toward Human-Level AI AgentsIEEE Access10.1109/ACCESS.2018.28559706(39976-39988)Online publication date: 2018
  • Show More Cited By

Index Terms

  1. Motivated reinforcement learning for adaptive characters in open-ended simulation games

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ACE '07: Proceedings of the international conference on Advances in computer entertainment technology
    June 2007
    324 pages
    ISBN:9781595936400
    DOI:10.1145/1255047
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 June 2007

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. adaptive characters
    2. computer games
    3. context-free grammar
    4. motivated reinforcement learning

    Qualifiers

    • Article

    Conference

    ACE2007
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 36 of 90 submissions, 40%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)10
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 12 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)Toward Believable Acting for Autonomous Animated CharactersProceedings of the 15th ACM SIGGRAPH Conference on Motion, Interaction and Games10.1145/3561975.3562941(1-15)Online publication date: 3-Nov-2022
    • (2019)Who Is at the Center?: Designing Playful Experiences by Using Player-Centered ApproachHCI in Games10.1007/978-3-030-22602-2_2(11-21)Online publication date: 16-Jun-2019
    • (2018)Artificial Intelligence and Virtual Worlds – Toward Human-Level AI AgentsIEEE Access10.1109/ACCESS.2018.28559706(39976-39988)Online publication date: 2018
    • (2018)Proposal and Evaluation of an Indirect Reward Assignment Method for Reinforcement Learning by Profit Sharing MethodIntelligent Systems and Applications10.1007/978-3-030-01054-6_13(187-200)Online publication date: 9-Nov-2018
    • (2017)Proposal of PSwithEFP and its Evaluation in Multi-Agent Reinforcement LearningJournal of Advanced Computational Intelligence and Intelligent Informatics10.20965/jaciii.2017.p093021:5(930-938)Online publication date: 20-Sep-2017
    • (2017)Proposal of an Action Selection Strategy with Expected Failure Probability and Its Evaluation in Multi-agent Reinforcement LearningMulti-Agent Systems and Agreement Technologies10.1007/978-3-319-59294-7_15(172-186)Online publication date: 23-Jun-2017
    • (2016)Proposal of a Propagation Algorithm of the Expected Failure Probability and the Effectiveness on Multi-agent EnvironmentsIEEJ Transactions on Electronics, Information and Systems10.1541/ieejeiss.136.273136:3(273-281)Online publication date: 2016
    • (2013)Gamification, Serious Games, Ludic Simulation, and other Contentious CategoriesInternational Journal of Gaming and Computer-Mediated Simulations10.4018/jgcms.20130401015:2(1-19)Online publication date: 1-Apr-2013
    • (2013)Towards a Beneficial Formalization of Cyber Entities’ Interactions during the E-Learning Process in the Virtual World of “Second Life”Production and Manufacturing System Management10.4018/978-1-4666-2098-8.ch014(278-314)Online publication date: 2013
    • (2013)Soft computing for content generation: Trading market in a basketball management video game2013 IEEE Conference on Computational Inteligence in Games (CIG)10.1109/CIG.2013.6633620(1-8)Online publication date: Aug-2013
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media