skip to main content
10.1145/1349822.1349834acmconferencesArticle/Chapter ViewAbstractPublication PageshriConference Proceedingsconference-collections
research-article

Crossmodal content binding in information-processing architectures

Published: 12 March 2008 Publication History

Abstract

Operating in a physical context, an intelligent robot faces two fundamental problems. First, it needs to combine information from its different sensors to form a representation of the environment that is more complete than any representation a single sensor could provide. Second, it needs to combine high-level representations (such as those for planning and dialogue) with sensory information, to ensure that the interpretations of these symbolic representations are grounded in the situated context. Previous approaches to this problem have used techniques such as (low-level) information fusion, ontological reasoning, and (high-level) concept learning. This paper presents a framework in which these, and related approaches, can be used to form a shared representation of the current state of the robot in relation to its environment and other agents. Preliminary results from an implemented system are presented to illustrate how the framework supports behaviours commonly required of an intelligent robot.

References

[1]
M. Brenner, N. Hawes, J. Kelleher, and J. Wyatt. Mediating between qualitative and quantitative representations for task-orientated human-robot interaction. In Proc. IJCAI '07, 2007.
[2]
T. Brick and M. Scheutz. Incremental natural language processing for hri. In Proc HRI '07, pages 263--270, 2007.
[3]
R. Engel and N. Pfleger. Modality fusion. In W. Wahlster, editor, SmartKom - Foundations of Multimodal Dialogue Systems, Cognitive Technologies, pages 223--235. Springer, 7 2006.
[4]
B. Fransen, V. Morariu, E. Martinson, S. Blisard, M. Marge, S. Thomas, A. Schultz, and D. Perzanowski. Using vision, acoustics, and natural language for disambiguation. In Proc HRI '07, pages 73--80, New York, NY, USA, 2007. ACM Press.
[5]
N. Hawes, A. Sloman, J. Wyatt, M. Zillich, H. Jacobsson, G. Kruijff, M. Brenner, G. Berginc, and D. Skocaj. Towards an integrated robot with multiple cognitive functions. In Proc. AAAI '07, 2007.
[6]
N. Hawes, M. Zillich, and J. Wyatt. BALT & CAST: Middleware for cognitive robotics. In Proc. IEEE RO-MAN 2007, pages 998--1003, August 2007.
[7]
G.-J. Kruijff, J. Kelleher, and N. Hawes. Information fusion for visual reference resolution in dynamic situated dialogue. In E. Andre, L. Dybkjaer, W. Minker, H. Neumann, and M. Weber, editors, Proc. PIT '06, pages 117--128, 2006.
[8]
N. Mavridis and D. Roy. Grounded situation models for robots: Where words and percepts meet. In IEEE/RSJ International Conference on Intelligent Robots and Systems 2006, pages 4690--4697. IEEE/RSJ, October 2006.
[9]
N. J. Nilsson. Shakey the robot. Technical Report 323, AI Center, SRI International, 1984.
[10]
D. L. Parnas. On the criteria to be used in decomposing systems into modules. Commun. ACM, 15(12):1053--1058, 1972.
[11]
D. Roy. Grounding words in perception and action: Insights from computational models. Trends in Cognitive Science, 9(8):389--96, 2005.
[12]
D. Roy. Semiotic schemas: A framework for grounding language in action and perception. Artificial Intelligence, 167(1-2):170--205, 2005.
[13]
D. Roy and N. Mukherjee. Towards situated speech understanding: visual context priming of language models. Computer Speech & Language, 19(2):227--248, 2005.
[14]
D. Skocaj, G. Berginc, B. Ridge, A. Stimec, M. Jogan, O. Vanek, A. Leonardis, M. Hutter, and N. Hawes. A system for continuous learning of visual concepts. In International Conference on Computer Vision Systems ICVS 2007, Bielefeld, Germany, 2007.
[15]
A. Sloman. Varieties of affect and the CogAff architecture schema. In Proc. the AISB'01 Symposium on Emotion, Cognition and Affective Computing, pages 1--10, 2001.
[16]
L. Steels. Semiotic dynamics for embodied agents. IEEE Intelligent Systems, 21(3):32--38, 2006.
[17]
A. L. Thomaz. Socially Guided Machine Learning. PhD thesis, Massachusetts Institute of Technology, May 2006.
[18]
S. Wood. Planning and Decision Making in Dynamic Domains. Ellis Horwood, 1993.
[19]
H. Zender, P. Jensfelt, Oscar Martinez Mozos, G.-J. M. Kruijff, and W. Burgard. An integrated robotic system for spatial understanding and situated interaction in indoor environments. In Proc. AAAI '07.

Cited By

View all
  • (2021)Study of Human and Robot Social Interaction Using Artificial Cognition MethodsProceedings of Integrated Intelligence Enable Networks and Computing10.1007/978-981-33-6307-6_82(791-801)Online publication date: 24-Apr-2021
  • (2020)Study of Human and Robot Social Interaction Using Artificial Cognition MethodsInternational Journal of Scientific Research in Science and Technology10.32628/IJSRST207532(458-464)Online publication date: 2-May-2020
  • (2018)UNDERWORLDS: Cascading Situation Assessment for Robots2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS.2018.8594094(7750-7757)Online publication date: Oct-2018
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HRI '08: Proceedings of the 3rd ACM/IEEE international conference on Human robot interaction
March 2008
402 pages
ISBN:9781605580173
DOI:10.1145/1349822
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 March 2008

Permissions

Request permissions for this article.

Check for updates

Author Tag

  1. symbol grounding

Qualifiers

  • Research-article

Conference

HRI '08
HRI '08: International Conference on Human Robot Interaction
March 12 - 15, 2008
Amsterdam, The Netherlands

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Upcoming Conference

HRI '25
ACM/IEEE International Conference on Human-Robot Interaction
March 4 - 6, 2025
Melbourne , VIC , Australia

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)1
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2021)Study of Human and Robot Social Interaction Using Artificial Cognition MethodsProceedings of Integrated Intelligence Enable Networks and Computing10.1007/978-981-33-6307-6_82(791-801)Online publication date: 24-Apr-2021
  • (2020)Study of Human and Robot Social Interaction Using Artificial Cognition MethodsInternational Journal of Scientific Research in Science and Technology10.32628/IJSRST207532(458-464)Online publication date: 2-May-2020
  • (2018)UNDERWORLDS: Cascading Situation Assessment for Robots2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)10.1109/IROS.2018.8594094(7750-7757)Online publication date: Oct-2018
  • (2017)Artificial cognition for social humanrobot interactionArtificial Intelligence10.1016/j.artint.2016.07.002247:C(45-69)Online publication date: 1-Jun-2017
  • (2017)Robot task planning and explanation in open and uncertain worldsArtificial Intelligence10.1016/j.artint.2015.08.008247:C(119-150)Online publication date: 1-Jun-2017
  • (2017)A Unified Fuzzy Framework for Human Hand Motion RecognitionHuman Motion Sensing and Recognition10.1007/978-3-662-53692-6_7(147-170)Online publication date: 14-May-2017
  • (2016)Multi-Channel Bayesian ART for robot fusion perception2016 IEEE Symposium Series on Computational Intelligence (SSCI)10.1109/SSCI.2016.7850240(1-5)Online publication date: Dec-2016
  • (2013)Learning Information Acquisition for Multitasking Scenarios in Dynamic EnvironmentsIEEE Transactions on Autonomous Mental Development10.1109/TAMD.2012.22262415:1(46-61)Online publication date: 1-Mar-2013
  • (2013)ART-based fusion of multi-modal perception for robotsNeurocomputing10.1016/j.neucom.2012.08.035107(11-22)Online publication date: 1-May-2013
  • (2012)Episodic-Like Memory for Cognitive RobotsIEEE Transactions on Autonomous Mental Development10.1109/TAMD.2011.21590044:1(1-16)Online publication date: 1-Mar-2012
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media