skip to main content
10.1145/1067343.1067371acmotherconferencesArticle/Chapter ViewAbstractPublication PagesesemConference Proceedingsconference-collections
Article

From text to images through meanings

Published: 02 September 2004 Publication History

Abstract

Natural language is an easy and effective medium to interact with computers by humans and we foresee that it is an effective way to visualize or realize one's imagination by computer. And it makes the complex task of creating 3D scenes or images faster and easier. As a result, creation of 3D scenes will be seen as just programming the computer with human language such as English. This paper presents a meaning centric framework for text to 3D image system to visualize the meaning of text and we propose an active memory, which is based on object oriented programming technique, to construct the mental world or active database. Thus each word in the active database acts as an agent compared to just a graphic based annotation, which can be found in existing systems. The active memory incrementally acquires and organizes the lexicon of natural language, visual objects and behaviors through real time interaction with the environment (including human masters). Based on the meaning centric framework, we have started the development of a software agent called KnowNet. KnowNet is an on-line learning, tutoring and visualization system for English language, which supports bi-directional learning activities such as human learning from computer and computer learning from human. It uses VRML browser to render 3D images. Most importantly, KnowNet acts as an entertainer for people who is proficient in English language and it act as an English language teacher for those who wants to learn English language.

References

[1]
N. Badler, W. Becket, B. Di Eugenio, C. Geib, L. Levison, M. Moore, B. Webber, M. White, and X. Zhao.: "Intentions and expectations in animating instructions: the AnimNL project," in Intentions in Animation and Action, Institute for Research in Cognitive Science, University of Pennsylvania, March 11-12 (1993)
[2]
B. Coyne and R. Sproat. WordsEye.: An Automatic text-to-scene conversion system. In SIGGRAPH 2001, Los Angeles, CA(2001)
[3]
S. R. Clay and J. Wilhelms. Put: Language-based interactive manipulation of Objects. IEEE Computer Graphics and Applications, pages 31--39, March 1996
[4]
P. R. Cohen, M. Johnston, D. McGee, S. Oviatt, J. Pittman, I. Smith, L. Chen, J. Clow. "QuickSet": Multimodal Interaction for Distributed Applications. " ACM International Multimedia Conference, New York: ACM, pp. 31--40 (1997)
[5]
S. Dupuy, A. Egges, V. Legendre and P. Nugues.: Generating a 3D simulation of a car accident from a written description in natural language: the CarSim system. In: ACL Workshop proceedings: Temporal and Spatial Information Processing, July 7th 2001, Association for Computational Linguistics (ACL) (2001) 1--8
[6]
R. Moore, John Dowding, Harry Bratt, J. Mark Gawron, Yonael Gorfu and Adam Cheyer, in " Proceedings of the Fifth Conference on Applied Natural Language Processing", "CommandTalk: A Spoken-Language Interface for Battlefield Simulations", Association for Computational Linguistics, by Washington, DC, (1997)pp. 1--7
[7]
Y. Shinyama, Takenobu Tokunaga, and Hozumi Tanaka.: "Kairai - Software Robots Understanding Natural Language", Third International Workshop on Human-Computer Conversation, (Jul. 2000)
[8]
M. Xie, Jayakumar S. Kandhasamy, and H. F. Chia. Robot Intelligence: Towards machines that understand meanings.International Symposium of Santa Caterina on Challenges in the Internet and Interdisciplinary research, Amalfi Coast, Italy, January 2004

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
ACE '04: Proceedings of the 2004 ACM SIGCHI International Conference on Advances in computer entertainment technology
September 2004
368 pages
ISBN:1581138822
DOI:10.1145/1067343
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 September 2004

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Article

Conference

ACE04

Acceptance Rates

Overall Acceptance Rate 36 of 90 submissions, 40%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 461
    Total Downloads
  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media