Incremental natural language processing for HRI

ABSTRACT

Robots that interact with humans face-to-face using natural language need to be responsive to the way humans use language in those situations. We propose a psychologically-inspired natural language processing system for robots which performs incremental semantic interpretation of spoken utterances, integrating tightly with the robot's perceptual and motor systems.

REFERENCES

• 1: James F. Allen , Bradford W. Miller , Eric K. Ringger , Teresa Sikorski, A robust system for natural spoken dialogue, Proceedings of the 34th annual meeting on Association for Computational Linguistics, p.62-70, June 24-27, 1996, Santa Cruz, California [doi>10.3115/981863.981872]
• 2: C. G. Chambers, M. K. Tanenhaus, K. M. Eberhard, H. Filip, and G. N. Carlson. Circumscribing referential domains in real-time language comprehension. Journal of Memory and Language, 47(1):30--49, 2002.
• 3: D. DeVault and M. Stone. Domain inference in incremental interpretation. In Proc. ICoS, 2003.
• 4: K. M. Eberhard, M. J. Spivey-Knowlton, J. C. Sedivy, and M. K. Tanenhaus. Eye movements as a window into real-time spoken language comprehension in natural contexts. Journal of Psycholinguistic Research, 24:409--436, 1995.
• 5: Lee D. Erman , Frederick Hayes-Roth , Victor R. Lesser , D. Raj Reddy, The Hearsay-II Speech-Understanding System: Integrating Knowledge to Resolve Uncertainty, ACM Computing Surveys (CSUR), v.12 n.2, p.213-253, June 1980 [doi>10.1145/356810.356816]
• 6: C. Fellbaum, editor. Wordnet: An Electronic Lexical Database. MIT Press, Cambridge, 1998.
• 7: M. V. Ferro and B. A. Dion. Efficient incremental parsing for context-free languages. In Proceedings of the 1994 International Conference on Computer Languages, 1994.
• 8: Ian Horswill, Tagged Behavior-Based Systems: Integrating Cognition with Embodied Activity, IEEE Intelligent Systems, v.16 n.5, p.30-37, September 2001 [doi>10.1109/5254.956079 ]
• 9: Y. Kamide, G. T. M. Altmann, and S. L. Haywood. The time-course of prediction in incremental sentence processing: Evidence from anticipatory eye movements. Journal of Memory and Language, 49(1):133--156, 2003.
• 10: Karin Kipper , Hoa Trang Dang , Martha Palmer, Class-Based Construction of a Verb Lexicon, Proceedings of the Seventeenth National Conference on Artificial Intelligence and Twelfth Conference on Innovative Applications of Artificial Intelligence, p.691-696, July 30-August 03, 2000
• 11: O. Lemon, A. Bracy, A. Gruenstein, and S. Peters. Information states in a multi-modal dialogue system for human-robot conversation. In Proceedings of Bi-Dialog, 5th Workshop on Formal Semantics and Pragmatics of Dialogue, pages 57--67, 2001.
• 12: David G. Lowe, Distinctive Image Features from Scale-Invariant Keypoints, International Journal of Computer Vision, v.60 n.2, p.91-110, November 2004 [doi>10.1023/B:VISI.0000029664.99615.94]
• 13: B. Lowerre and R. Reddy. The harpy speech understanding system. In W. A. Lea, editor, Trends in Speech Recognition, pages 340--360. Pretice Hall, 1980.
• 14: D. Mori, S. Matsubara, and Y. Inagaki. Incremental parsing for interactive natural language interface. In IEEE International Conference on Systems, Man, and Cybernetics, 2001.
• 15: A. Newlands, A. H. Anderson, and J. Mullen. Adapting communicative strategies to computer-mediated communication: An analysis of task performance and dialogue structure. Applied Cognitive Psychology, 17(3):325--348, 2003.
• 16: B. O'Conaill, S. Whittaker, and S. Wilbur. Conversations over videoconferences: An evaluation of the spoken aspects of video-mediated communication. Human-Computer Interaction, 8(4):389--428, 1993.
• 17: M. Purver and M. Otsuka. Incremental generation by incremental parsing: Tactical generation in dynamic syntax. In Proceedings of the 6th annual CLUK Research Colloquium, 2003.
• 18: C. P. Rose, A. Roque, D. Bhembe, and K. VanLehn. An efficient incremental architecture for robust interpretation. In Proceedings of Human Language Technology Conference, 2002.
• 19: D. Roy. Learning visually-grounded words and syntax for a scene description task. Computer Speech and Language, 16(3), 2002.
• 20: D. Roy, P. Gorniak, N. Mukherjee, and J. Juster. A trainable spoken language understanding system. In Proceedings of the International Conference of Spoken Language Processing, 2002.
• 21: M. Scheutz. ADE - steps towards a distributed development and runtime environment for complex robotic agent architectures. Applied Artificial Intelligence, 20(4-5), 2006.
• 22: M. Scheutz, K. Eberhard, and V. Andronache. A real-time robotic model of human reference resolution using visual constraints. Connection Science Journal, 16(3):145--167, 2004.
• 23: Matthias Scheutz , Paul Schermerhorn , James Kramer , David Anderson, First steps toward natural human-like HRI, Autonomous Robots, v.22 n.4, p.411-423, May 2007 [doi>10.1007/s10514-006-9018-3]
• 24: Matthias Scheutz , Paul Schermerhorn , James Kramer, The utility of affect expression in natural language interactions in joint human-robot tasks, Proceeding of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121281]
• 25: J. C. Sedivy. Invoking discourse-based contrast sets and resolving syntactic ambiguities. Journal of Memory and Language, 46(2):341--370, 2002.
• 26: Leslie D. Setlock , Susan R. Fussell , Christine Neuwirth, Taking it out of context: collaborating within and across cultures in face-to-face settings and via instant messaging, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031712]
• 27: Cmu sphinx speech recognition engines. http://fife.speech.cs.cmu.edu/sphinx/.
• 28: L. Steels and F. Kaplan. Bootstrapping grounded word semantics. In E. J. Briscoe, editor, Linguistic evolution through language acquisition: formal and computational models, pages 53--74. Cambridge University Press, Cambridge, UK, 2002.
• 29: M. K. Tanenhaus, M. J. Spivey-Knowlton, K. M. Eberhard, and J. C. Sedivy. Integration of visual and linguistic information in spoken language comprehension. Science, 268:1632--1634, 1995.
• 30: S. Varges and M. Purver. Robust language analysis and generation for spoken dialogue systems. In Proceedings of the ECAI workshop on Development and Evaluation of Robust Spoken Dialogue Systems for Real Applications, Riva del Garda, Italy, Aug. 2006.
• 31: F. Weng, L. Cavedon, B. Raghunathan, D. Mirkovic, H. Cheng, H. Schmidt, H. Bratt, R. Mishra, S. Peters, L. Zhao, S. Upson, E. Shriberg, and C. Bergmann. A conversational dialogue system for cognitively overloaded users. In Proceedings of the 8th International Conference on Spoken Language Processing, 2004.
• 32: Terry Winograd, Understanding Natural Language, Academic Press, Inc., Orlando, FL, 1972

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.7 Natural Language Processing

General Terms:
Design, Human Factors

Keywords:
HRI, embodied NLP, incremental processing