skip to main content
10.1145/1178745.1178765acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
Article

A methodological study of situation understanding utilizing environments for multimodal observation of infant behavior

Published: 27 October 2006 Publication History

Abstract

We have developed a framework to understand situations and intentions of speakers focusing on the utterances of demonstratives. We aim at constructing a 'Multimodal Infant Behavior Corpus', which makes a valuable contribution to the elucidation of human commonsense knowledge and its acquisition mechanism. For this purpose, we have constructed environments for multimodal observation of infant behavior, in particular, environments for infant behavior recording; we have set up multiple cameras and microphones in the Cedar yurt. We have also developed a wearable speech recording device of high quality to capture infant utterances clearly. Moreover, we have developed a comment-collecting system which allows everyone to make comments easily from the multi-viewpoints. Those construction and developments make it possible to realize a framework for multimodal observation of infant behavior. Utilizing the multimodal environments, we propose a situation description model based on observation of demonstratives uttered by infants, since demonstratives appear frequently in their conversations and become a precious clue to understand situations. The proposed model, which represents the mental distances of speakers and listeners to objects on a general and simple model, enables us to predict speakers' next behavior. The consideration results enable us to conclude that the constructed environments lead to development and realization of human interaction models applicable to spoken dialog systems for elder people supporting.

References

[1]
Winograd. T, Understanding Natural Language, Academic
[2]
Stone. M, Communicative Intentions and Conversational Processes in Human-Human and Human-Computer Dialogue, World Situated Language Use: MITPress, Vol. Psycholinguistic, Linguistic and Computational Perspectives on Bridging the Product and Action Traditions, 2002.
[3]
Byron. D. K, Understanding Referring Expressions in Situated Language Some Challenges for Real-World Agents, In the First International Workshop on Language Understanding and Agents for the Real World, 2003.
[4]
Zhang. J, Huebner. K, and Knoll. A: Learning based situation recognition by sectoring omnidirectional images for robot localisation, IEEE-Press, Vol. In Proceeding of the IEEE Workshop on Omnidirectional Vision, 2001.
[5]
Roy. D: SITUATION-AWARE SPOKEN LANGUAGE PROCESSING, In Royal Institute of Acoustics Workshop on Innovation in Speech Processing, 2001.
[6]
Otake. K, A Consideration on Infant Behavior Corpus, FIT2005, Vol.2, 2005, 129--130 (in Japanese).
[7]
Kamio. A, Territory of Information: Pragmatics and Beyond New Series (John Benjamins Pub Co, 1997).
[8]
Suga. T, Solving the Riddle of Language Acquisition: How to Design a Virtual Infant (MLAS): http://www10.ocn.ne.jp/~mlas/JSLS_Paper.htm

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
HCM '06: Proceedings of the 1st ACM international workshop on Human-centered multimedia
October 2006
138 pages
ISBN:1595935002
DOI:10.1145/1178745
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 27 October 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. human interaction modeling
  2. multimodal observation
  3. situation understanding model

Qualifiers

  • Article

Conference

MM06
MM06: The 14th ACM International Conference on Multimedia 2006
October 27, 2006
California, Santa Barbara, USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 164
    Total Downloads
  • Downloads (Last 12 months)0
  • Downloads (Last 6 weeks)0
Reflects downloads up to 31 Jan 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media