skip to main content
article

Fast multi-level adaptation for interactive autonomous characters

Published: 01 April 2005 Publication History

Abstract

Adaptation (online learning) by autonomous virtual characters, due to interaction with a human user in a virtual environment, is a difficult and important problem in computer animation. In this article we present a novel multi-level technique for fast character adaptation. We specifically target environments where there is a cooperative or competitive relationship between the character and the human that interacts with that character.In our technique, a distinct learning method is applied to each layer of the character's behavioral or cognitive model. This allows us to efficiently leverage the character's observations and experiences in each layer. This also provides a convenient temporal distinction between what observations and experiences provide pertinent lessons for each layer. Thus the character can quickly and robustly learn how to better interact with any given unique human user, relying only on observations and natural performance feedback from the environment (no explicit feedback from the human). Our technique is designed to be general, and can be easily integrated into most existing behavioral animation systems. It is also fast and memory efficient.

References

[1]
Arikan, O., Forsyth, D., and O'Brien, J. 2003. Motion synthesis from annotations. ACM Trans. Graph. 22, 3, 402--408.
[2]
Blumberg, B., Downie, M., Ivanov, Y., Berlin, M., Johnson, M. P., and Tomlinson, B. 2002. Integrated learning for interactive synthetic characters. In Proceedings of SIGGRAPH 2002. ACM Press/ACM SIGGRAPH, 417--426.
[3]
Blumberg, B. and Galyean, T. 1995. Multi-level direction of autonomous creatures for real-time virtual environments. In Proceedings of SIGGRAPH 1995. ACM Press/ACM SIGGRAPH, 47--54.
[4]
Brooks, R. 1986. A robust layered control system for a mobile robot. IEEE J. Robot. Autom. 2, 14--23.
[5]
Dinerstein, J. and Egbert, P. 2004. Improved behavioral animation through regression. In Proceedings of Computer Animation and Social Agents (CASA '04). Computer Graphics Society, 231--238.
[6]
Dinerstein, J., Egbert, P., de Garis, H., and Dinerstein, N. 2004. Fast and learnable behavioral and cognitive modeling for virtual character animation. J. Comput. Anim. Virtual Worlds 15, 2, 95--108.
[7]
Egges, A., Kshirsagar, S., and Magnenat-Thalmann, N. 2004. Generic personality and emotion simulation for conversational agents. J. Comput. Anim. Virtual Worlds 15, 1--13.
[8]
Evans, R. 2002. Varieties in learning. In AI Game Programming Wisdom, E. Rabin, Ed. Charles River Media. Hingham MA, 567--578.
[9]
Faloutsos, P., van de Panne, M., and Terzopoulos, D. 2001. Composible controllers for physics-based character animation. In Proceedings of SIGGRAPH 2001. ACM Press/ACM SIGGRAPH, 39--48.
[10]
Funge, J., Tu, X., and Terzopoulos, D. 1999. Cognitive modeling: Knowledge, reasoning, and planning for intelligent characters. In Proceedings of SIGGRAPH 1999. ACM Press/ACM SIGGRAPH, 29--38.
[11]
Gadanho, S. 2003. Learning behavior-selection by emotions and cognition in a multi-goal robot task. J. Mach. Learn. Res. 4, 385--412.
[12]
Gillies, M. and Dodgson, N. 2002. Eye movements and attention for behavioural animation. J. Visual. Comput. Anim. 13, 287--300.
[13]
Gmytrasiewicz, P. and Durfee, E. 2000. Rational coordination in multi-agent environments. Autonomous Agents and Multi-Agent Systems 3, 4, 319--350.
[14]
Grzeszczuk, R., Terzopoulos, D., and Hinton, G. 1998. Neuroanimator: Fast neural network emulation and control of physics-based models. In Proceedings of SIGGRAPH 1998. ACM Press/ACM SIGGRAPH, 9--20.
[15]
Guyon, I. and Elisseeff, A. 2003. An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157--1182.
[16]
He, L. W., Cohen, M., and Salesin, D. 1996. The virtual cinematographer: A paradigm for automatic real-time camera control and directing. In Proceedings of SIGGRAPH 1996. ACM Press/ACM SIGGRAPH, 217--224.
[17]
Isla, D., Burke, R., Downie, M., and Blumberg, B. 2001. A layered brain architecture for synthetic creatures. In Proceedings of IJCAI. 1051--1058.
[18]
Kaelbling, L., Littman, M., and Moore, A. 1996. Reinforcement learning: A survey. J. Arti. Intell. Res. 4, 237--285.
[19]
Kaminka, G., Veleso, M., Schaffer, S., Sollitto, C., Adobbati, R., Marshall, A., Scholer, A., and Tejada, S. 2002. Gamebots: a flexible test bed for multiagent team research. Comm. ACM 45, 1, 43--45.
[20]
Kasper, M., Fricke, G., Steuernagel, K., and von Puttkamer, E. 2001. A behavior-based mobile robot architecture for learning from demonstration. Robotics and Autonomous Systems 34, 153--164.
[21]
Kerkez, B. and Cox, M. 2003. Incremental case-based plan recognition with local predictions. Int. J. Artif. Intell. Tools: Architectures, languages, algorithms 12, 4, 413--463.
[22]
Laird, J. 2001. It knows what you're going to do: Adding anticipation to the quakebot. In Proceedings of the Fifth International Conference on Autonomous Agents. 385--392.
[23]
Matthews, G. 1997. Personality, Emotion, and Cognitive Science. Elsevier, Amsterdam.
[24]
Meltzoff, A. and Moore, M. 1992. Early imitation within a functional framework. Infant Behavior and Development 15, 479--505.
[25]
Millar, J., Hanna, J., and Kealy, S. 1999. A review of behavioural animation. Comput. Graph. J. 23, 127--143.
[26]
Minsky, M. 1985. Society of Mind. Simon & Schuster, New York.
[27]
Mitchell, T. 1997. Machine Learning. WCB/McGraw-Hill.
[28]
Monzani, J., Caicedo, A., and Thalmann, D. 2001. Integrating behavioural animation techniques. Comput. Graph. Forum 20, 3.
[29]
Newell, A. 1990. Unified Theories of Cognition. Harvard University Press, Cambridge, MA.
[30]
Perlin, K. and Goldberg, A. 1996. Improv: A system for scripting interactive actors in virtual worlds. In Proceedings of SIGGRAPH 1996. ACM Press/ACM SIGGRAPH, 205--216.
[31]
Price, R. 2002. Accelerating Reinforcement Learning through Imitation. Ph. D. thesis, University of British Columbia.
[32]
Rao, A. and Georgeff, M. 1995. BDI agents: From theory to practice. In Proceedings of the First International Conference on Multi-Agent Systems.
[33]
Reif, J. H. and Wang, H. 1999. Social potential fields: A distributed behavioral control for autonomous robots. Robotics and Autonomous Systems 27, 171--194.
[34]
Reynolds, C. 1987. Flocks, herds, and schools: A distributed behavioral model. In Proceedings of SIGGRAPH 1987 (1987). ACM Press/ACM SIGGRAPH, 25--34.
[35]
Schyns, P., Goldstone, R., and Thilbaut, J. 1998. The development of features in object concepts. Behavioral and Brain Sciences 21, 1, 1--54.
[36]
Sen, S. and Arora, N. 1997. Learning to take risks. In Collected papers from AAAI-97 workshop on multiagent learning. 59--64.
[37]
Sims, K. 1994. Evolving virtual creatures. In Proceedings of SIGGRAPH 1994. ACM Press/ACM SIGGRAPH, 15--22.
[38]
Stone, P. 2000. Layered Learning in Multi-Agent Systems: A Winning Approach to Robotic Soccer. MIT Press, Cambridge, MA.
[39]
Stone, P. and Veloso, M. 1997. Multiagent systems: A survey from a machine learning perspective. Autonomous Robots 8, 3, 345--383.
[40]
Sutton, R. and Barto, A. 1998. Reinforcement Learning: An Introduction. MIT Press, Cambridge, Massachusetts.
[41]
Tesauro, G. 1995. Temporal difference learning in TD-Gammon. Comm. ACM 38, 3, 58--68.
[42]
Tomlinson, B. and Blumberg, B. 2002. Alphawolf: Social learning, emotion and development in autonomous virtual agents. In First GSFC/JPL Workshop on Radical Agent Concepts.
[43]
Tu, X. and Terzopoulos, D. 1994. Artificial fishes: Physics, locomotion, perception, behavior. In Proceedings of SIGGRAPH 1994. ACM Press/ACM SIGGRAPH, 43--50.
[44]
van Lent, M. and Laird, J. 2001. Learning procedural knowledge through observation. In Proceedings of International Conference On Knowledge Capture. ACM Press, 179--186.
[45]
Yoon, B. 2003. Real world learning: exploratory efforts. In Proceedings of DARPA Cognitive Systems Conference.
[46]
Zhu, T., Greiner, R., and Haubl, G. 2003. Learning a model of a web user's interests. In Proceedings of Ninth International Conference on User Modeling.

Cited By

View all
  • (2014)A Model to Incorporate Emotional Sensitivity into Human Computer InteractionsProceedings of the 2014 workshop on Emotion Representation and Modelling in Human-Computer-Interaction-Systems10.1145/2668056.2668059(25-30)Online publication date: 16-Nov-2014
  • (2013)MAS Controlled NPCs in 3D Virtual Learning EnvironmentProceedings of the 2013 International Conference on Signal-Image Technology & Internet-Based Systems10.1109/SITIS.2013.166(1026-1033)Online publication date: 2-Dec-2013
  • (2013)Character Behavior Planning and Visual Simulation in Virtual 3D SpaceIEEE MultiMedia10.1109/MMUL.2012.5420:1(49-59)Online publication date: 1-Jan-2013
  • Show More Cited By

Recommendations

Reviews

Daniel L. Chester

Competition in a virtual environment will be much more lifelike, challenging, and interesting when autonomous characters adapt to the human participant's behavior. Most machine learning techniques won't work well in this domain, however, because they require many iterations to produce substantial improvement. This paper focuses on machine learning techniques that work fast enough to produce noticeable improvements in the skills of autonomous characters within minutes of interaction. Different techniques are used for different time scales. Case-based planning (prediction) and learning are used to quickly decide the lowest level actions. With less frequency, task selection is done by simulating alternative tasks, selecting the one that appears to produce the best future outcome, and updating its expected value. Mimicking the behavior of the human participant at the action and task levels is another learning technique, which the authors observed was the second most effective technique, ranking after low-level action prediction. More time is taken to select goals; this selection is made by estimating the amount of "happiness" associated with each goal. Learning takes place by adjusting weights in linear perceptrons. This paper will interest people who model behaviors in autonomous entities, such as those found in video games and other virtual environments, but will also interest researchers in machine learning because of its emphasis on fast learning techniques. The paper summarizes experiments that evaluated the relative contributions of the learning methods studied. Brief mention is also made of how automatic camera and attention control can be achieved.

Access critical reviews of Computing literature here

Become a reviewer for Computing Reviews.

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics
ACM Transactions on Graphics  Volume 24, Issue 2
April 2005
193 pages
ISSN:0730-0301
EISSN:1557-7368
DOI:10.1145/1061347
Issue’s Table of Contents
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 April 2005
Published in TOG Volume 24, Issue 2

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. AI-based animation
  2. Computer animation
  3. behavioral modeling
  4. character animation
  5. machine learning

Qualifiers

  • Article

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 23 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2014)A Model to Incorporate Emotional Sensitivity into Human Computer InteractionsProceedings of the 2014 workshop on Emotion Representation and Modelling in Human-Computer-Interaction-Systems10.1145/2668056.2668059(25-30)Online publication date: 16-Nov-2014
  • (2013)MAS Controlled NPCs in 3D Virtual Learning EnvironmentProceedings of the 2013 International Conference on Signal-Image Technology & Internet-Based Systems10.1109/SITIS.2013.166(1026-1033)Online publication date: 2-Dec-2013
  • (2013)Character Behavior Planning and Visual Simulation in Virtual 3D SpaceIEEE MultiMedia10.1109/MMUL.2012.5420:1(49-59)Online publication date: 1-Jan-2013
  • (2013)Animation Research: Modern TechniquesModern Machine Learning Techniques and Their Applications in Cartoon Animation Research10.1002/9781118559963.ch4(131-194)Online publication date: 2-Apr-2013
  • (2012)In-game adaptation of a navigation mesh cell pathProceedings of the 2012 17th International Conference on Computer Games: AI, Animation, Mobile, Interactive Multimedia, Educational & Serious Games (CGAMES)10.1109/CGames.2012.6314580(230-236)Online publication date: 30-Jul-2012
  • (2008)Puppet MasterProceedings of the 2008 ACM SIGGRAPH/Eurographics Symposium on Computer Animation10.5555/1632592.1632619(183-191)Online publication date: 7-Jul-2008
  • (2008)DEMONSTRATION-BASED BEHAVIOR PROGRAMMING FOR EMBODIED VIRTUAL AGENTSComputational Intelligence10.1111/j.1467-8640.2008.00329.x24:4(235-256)Online publication date: 29-Oct-2008
  • (2008)Autonomy in Virtual AgentsProceedings of the 5th Hellenic conference on Artificial Intelligence: Theories, Models and Applications10.1007/978-3-540-87881-0_6(51-63)Online publication date: 2-Oct-2008
  • (2008)Intelligent Virtual Humans with Autonomy and Personality: State-of-the-ArtNew Advances in Virtual Humans10.1007/978-3-540-79868-2_2(43-84)Online publication date: 2008
  • (2007)Intelligent virtual humans with autonomy and personalityIntelligent Decision Technologies10.5555/2595898.25959001:1,2(3-15)Online publication date: 1-Jan-2007
  • Show More Cited By

View Options

Login options

Full Access

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media