skip to main content
10.1145/1216295.1216336acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
Article

Making sense of virtual environments: action representation, grounding and common sense

Published: 28 January 2007 Publication History

Abstract

The development of complex interactive 3D systems raises the need for representations supporting more abstract descriptions of world objects, their behaviour and the world dynamics. The inclusion of Artificial Intelligence representations and their use within 3D graphic worlds face both fundamental and technical issues due to the difference in representational logic between computer graphics and knowledge-based systems. We present a framework for such an integration illustrated by a first prototype.

References

[1]
André, E., Herzog, G and Rist, T. On the Simultaneous Interpretation of Real World Image Sequences and their Natural Language Description: The System SOCCER. Proceedings of the 8th European Conference on Artificial Intelligence, Munich, 1988, pp. 449--454.
[2]
Aylett, R. and Luck, M. Applying Artificial Intelligence to Virtual Reality: Intelligent Virtual Environments, Applied Artificial Intelligence, 14(1), 2000, pp. 3--23.
[3]
Bicici, E. and St. Amant, R. Reasoning about the functionality of tools and physical artifacts. Technical Report TR-2003-22, Department of Computer Science, North Carolina State University, April, 2003.
[4]
Bindiganavale, R. Schuler, W. Allbeck, J. Badler, N., Joshi, A. and Palmer, M. Dynamically Altering Agent Behaviors Using Natural Language Instructions. Autonomous Agents 2000, ACM Press, pp. 293--300.
[5]
Campos, J., Hornsby, K., and Egenhofer, M. A Model for Exploring Virtual Reality Environments. Journal of Visual Languages and Computing 14 (5), 2003, pp. 469--492.
[6]
Cavazza, M. and Palmer, I. High-Level Interpretation in Virtual Environments. Applied Artificial Intelligence, 14:1, 1999, pp. 125--144.
[7]
Cavazza, M., Hartley, S., Lugrin, J. L., and Le Bras, M. Qualitative Physics in Virtual Environments. Proceedings of the 9th international conference on Intelligent User Interfaces, Funchal, Madeira, Portugal, ACM Press, 2004, pp. 54--61.
[8]
Clay S. R. and Wilhelms, J. Language-Based Interactive Manipulation of Objects, IEEE Computer Graphics and Applications, Vol. 16, Number 2, March 1996, pp. 31--39.
[9]
Erignac, C., Interactive Semi-Qualitative Simulation in Virtual Environments. PhD Thesis, University of Pennsylvania, 2001.
[10]
Falkenhainer, B., and Forbus, K. Setting up large scale qualitative models. In Proceedings of the Seventh National Conference on Artificial Intelligence, Menlo Park, Calif.: AAAI Press, 1988.
[11]
Far, B.H. Functional Reasoning, Explanation and Analysis, Technical Report JAERI-M 91-225, Japan Atomic Energy Research Institute, Jan. 1992.
[12]
Forbus, K.D., Qualitative Process Theory, Artificial Intelligence, 24, 1984, pp. 85--168.
[13]
Hayes, P. Naive Physics I: Ontology for Liquids. In: J. Hobbs and It. Moore (Eds.), Formal Theories of the Commonsense World. Norwood NJ: ABLEX Publishing, 1985.
[14]
Hayes, P. The naive physics manifesto. In D. Mitche, Editor, Expert Systems in the Micro-Electronic Age. Edinburg University Press, 1978, pp. 242--270.
[15]
Hecker, C. Physics in computer games. Communications of the ACM, 43(7), 2000, pp. 34--39.
[16]
Jiang, H., Kessler, G.D. and Nonnemaker, J. DEMIS: A Dynamic Event Model for Interactive Systems, Proceedings of the ACM VRST 2002 Conference, Hong Kong, China, 2002. pp. 97--104.
[17]
Kallmann, M. and Thalmann, D. Modeling Behaviors of Interactive Objects for Virtual Reality Applications, Journal of Visual Languages and Computing, 13(2), 2002, pp. 177--195.
[18]
Kalogerakis V., Christodoulakis S., Moumoutzis N., 2006. Coupling Ontologies with Graphics Content for knowledge driven visualization, Proceedings of the IEEE Virtual Reality Conference 2006.
[19]
Kleinermann, F., De Troyer, O., Mansouri, H., Romero, R., Pellens, B., Bille, W. Designing Semantic Virtual Reality Applications, Proceedings of the 2nd INTUITION International Workshop, Senlis, France, 2005.
[20]
Latoschik, M. E., Biermann P., and Wachsmuth, I. Knowledge in the Loop: Semantics Representation for Multimodal Simulative Environments. Proceedings of the 5th International Symposium on Smart Graphics 2005 Frauenwoerth Cloister, Germany, pp. 25--39.
[21]
Lee, C. H. J, Bonanni, L., Espinosa, J.H., Lieberman, H. and Selker, T. Augmenting kitchen appliances with a shared context using knowledge about daily events. Proceedings of the 11th international conference on Intelligent user interfaces, Sydney, Australia, 2006, pp. 348--350.
[22]
Lenat, D.B. CYC: A Large-Scale Investment in Knowledge Infrastructure. Communications of the ACM, 38(11), 1995, pp. 33--38.
[23]
Lewis, M. and Jacobson, J. Games Engines in Scientific Research. Communications of ACM, Vol. 45, No. I, 2002. pp 27--31.
[24]
Lieberman, H. and Espinosa, J. A goal-oriented interface to consumer electronics using planning and commonsense reasoning, Proceedings of the 11th international conference on Intelligent User Interfaces, Sydney, Australia, ACM Press, 2006, pp. 226--233.
[25]
Lieberman, H., Liu, H., Singh, P., and Barry, B. Beating Common Sense into Interactive Applications. AI Magazine 25(4), 2004, pp. 63--76.
[26]
Lugrin, J-L. and Cavazza, M. AI-based World Behaviour for Emergent Narratives. Third ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, Hollywood, California, USA, 2006.
[27]
Minsky, M. Commonsense-based interfaces. Communications of the ACM, 43(8), 2000, pp. 66--73.
[28]
Müller-Tomfelde, C., Paris, C. and Stevenson, D. Interactive Landmarks: Linking Virtual Environments with Knowledge-Based Systems. In: Proceedings of the OZCHI, Wollongong, Australia, November 2004.
[29]
Roy, D. Semiotic Schemas: A Framework for Grounding Language in the Action and Perception. Artificial Intelligence, 167(1-2), 2005, pp. 170--205.
[30]
Singh, P. and Barry, B. Collecting commonsense experiences. Proceedings of the Second International Conference on Knowledge Capture (K-CAP 2003), Florida, USA, 2003.
[31]
Vaina, L. and Jaulent, M.-C. Object structure and action requirements: A compatibility model for functional recognition. International Journal of Intelligent Systems, 6, 1991, pp. 313--336.
[32]
Van Leeuwen, L., Smitsman, A., & Van Leeuwen, C. Affordances, perceptual complexity, and the development of tool use. Journal of Experimental Psychology: Human Perception and Performance, 20, 1994. pp. 174--191.
[33]
Weld, D., and De Kleer, J., Reading in Qualitative Reasoning about Physical Systems, Morgan Kaufmann, 1990.
[34]
Winograd, T. Understanding Natural Language, Academic Press, New York, 1972.
[35]
Winston, P. H., Katz, B., Binford, T.O. and Lowry, M. R. Learning Physical Descriptions From Functional Definitions, Examples, and Precedents. Proceedings of the American Association for Artificial Intelligence Conference 1983, pp. 433--439.
[36]
Zhou, S. and Ting, S. P. Qualitative Physics for Movable Objects in MOUT. Annual Simulation Symposium, IEEE Press, 2006, pp. 320--3.

Cited By

View all
  • (2023)Disentangled counterfactual learning for physical audiovisual commonsense reasoningProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666668(12476-12488)Online publication date: 10-Dec-2023
  • (2020)Unveiling the implicit knowledge, one scenario at a timeThe Visual Computer10.1007/s00371-020-01904-7Online publication date: 12-Jul-2020
  • (2020)Action Sequencing in VR, a No-Code ApproachTransactions on Computational Science XXXVII10.1007/978-3-662-61983-4_4(57-76)Online publication date: 14-Jul-2020
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
IUI '07: Proceedings of the 12th international conference on Intelligent user interfaces
January 2007
388 pages
ISBN:1595934812
DOI:10.1145/1216295
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 January 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. intelligent virtual environments
  2. knowledge representation
  3. qualitative reasoning

Qualifiers

  • Article

Conference

IUI07

Acceptance Rates

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)4
  • Downloads (Last 6 weeks)0
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Disentangled counterfactual learning for physical audiovisual commonsense reasoningProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666668(12476-12488)Online publication date: 10-Dec-2023
  • (2020)Unveiling the implicit knowledge, one scenario at a timeThe Visual Computer10.1007/s00371-020-01904-7Online publication date: 12-Jul-2020
  • (2020)Action Sequencing in VR, a No-Code ApproachTransactions on Computational Science XXXVII10.1007/978-3-662-61983-4_4(57-76)Online publication date: 14-Jul-2020
  • (2018)Understanding everything NPCs can doProceedings of the 13th International Conference on the Foundations of Digital Games10.1145/3235765.3235776(1-10)Online publication date: 7-Aug-2018
  • (2018)VAnnotatoRProceedings of the 29th on Hypertext and Social Media10.1145/3209542.3209572(150-154)Online publication date: 3-Jul-2018
  • (2018)Query-based composition of animations for 3D web applicationsProceedings of the 23rd International ACM Conference on 3D Web Technology10.1145/3208806.3208828(1-9)Online publication date: 20-Jun-2018
  • (2018)Explorable Representation of Interaction in VR/AR EnvironmentsAugmented Reality, Virtual Reality, and Computer Graphics10.1007/978-3-319-95282-6_42(589-609)Online publication date: 14-Jul-2018
  • (2017)Knowledge-based representation of 3D content behavior in a service-oriented virtual environmentProceedings of the 22nd International Conference on 3D Web Technology10.1145/3055624.3075959(1-10)Online publication date: 5-Jun-2017
  • (2017)Ontology‐Based Representation and Modelling of Synthetic 3D Content: A State‐of‐the‐Art ReviewComputer Graphics Forum10.1111/cgf.1308336:8(329-353)Online publication date: 23-Feb-2017
  • (2017)ALET: Agents Learning their Environment through TextComputer Animation and Virtual Worlds10.1002/cav.175928:3-4Online publication date: 21-Apr-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media