skip to main content
10.1145/1322192.1322243acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
poster

Natural multimodal dialogue systems: a configurable dialogue and presentation strategies component

Published: 12 November 2007 Publication History

Abstract

In the context of natural multimodal dialogue systems, we address the challenging issue of the definition of cooperative answers in an appropriate multimodal form. Highlighting the intertwined relation of multimodal outputs with the content, we focus on the Dialogic strategy component, a component that defines from the set of possible contents to answer a user's request, the content to be presented to the user and its multimodal presentation. The content selection and the presentation allocation managed by the Dialogic strategy component are based on various constraints such as the availability of a modality and the user's preferences. Considering three generic types of dialogue strategies and their corresponding handled types of information as well as three generic types of presentation tasks, we present a first implementation of the Dialogic strategy component based on rules. By providing a graphical interface to configure the component by editing the rules, we show how the component can be easily modified by ergonomists at design time for exploring different solutions. In further work we envision letting the user modify the component at runtime.

References

[1]
Berton, A., Bühler, D. and Minker, W. SmartKom-mobile car: User interaction with mobile services in a car environment. In Wahlster, W. (ed.), SmartKom: foundations of multimodal dialogue systems, 523--537, Springer, 2006.
[2]
Bordegoni, M., Faconti, G., Feiner, S., Maybury, M. T., Rist, T., Ruggieri, S., Trahanias, P. and Wilson, M. A standard reference model for intelligent multimedia presentation systems. Computers standards and interfaces, 18, 6-7, 477--496, Elsevier, 1997.
[3]
Bunt, H., Kipp, M., Maybury, M. and Wahlster, W. Fusion and coordination for multimodal interactive information presentation. In Stock, O. and Zancanaro, M. (ed.) Multimodal intelligent information presentation, 325--339, Springer, 2005.
[4]
Elting, C., Rapp, S., Möhler, G. and Strube M. Architecture and implementation of multimodal plug and play. In Proc. ICMI'03, 93--100, ACM Press, 2003.
[5]
Fréard, D., Jamet, E., Le Bohec, O., Poulain, G. and Botherel, V. Subjective measurement of workload related to a multimodal interaction task: Nasa-tlx vs. Workload profile. In Proc. HCII'07, Springer, 2007.
[6]
Horchani, M., Nigay, L. and Panaget, F. A platform for output dialogic strategies in natural multimodal dialogue systems. In Proc. IUI'07, 206--215, ACM Press, 2007.
[7]
Horchani, M., Fréard, D., Jamet, E., Nigay, L. and Panaget, F. A platform for designing multimodal dialogic and presentation strategies. In Proc. of the 2007 Workshop on the Semantics and Pragmatics of Dialogue DECALOG, 169--170, 2007.
[8]
JADE, Java Agent DEvelopment Framework, jade.tilab.com.
[9]
Louis, V. and Martinez T. JADE semantics framework. In Bellifemine, F., Caire, G. and Greenwood, D. (ed.), Developping multi-agent systems with JADE, John Wiley & Sons, 2007.
[10]
Mansoux, B., Nigay, L., and Troccaz, J. Output Multimodal Interaction: The Case of Augmented Surgery. In Proc. of the 20th British Human Computer Interaction Group conference HCI'06, 177--192, BCS Conference Series, Springer-Verlag and ACM Press, 2006.
[11]
Roth, F. and Hefley, W. E. Intelligent multimedia presentation systems: research and principles, In Maybury, M. (ed.), Intelligent multimedia interfaces, AAAI Press/The MIT Press, 1993.
[12]
Rousseau, C., Bellik, Y., Vernier, F. and Bazalgette, D. A framework for the intelligent multimodal presentation information, Signal Processing Journal, Special issue on Multimodal Interfaces, 86, 12, 3696--3713, Elsevier, 2006.
[13]
Sadek, D. Design considerations on dialogue systems: from theory to technology. In IDS'99, 173--187, 1999.
[14]
Sinha, A. and Landay, J. Capturing user tests in a multimodal, multidevice informal prototing tool. In Proc. ICMI'03, 117--124, ACM Press, 2003.
[15]
UIMS. A metamodel for the runtime architecture of an interactive system. SIGCHI bulletin, 24, 32--37, 1992.
[16]
Wahlster, W. Dialogue systems go multimodal: the SmartKom experience. In Wahlster, W. (ed.), SmartKom: foundations of multimodal dialogue systems, 3--27, Springer, 2006.
[17]
Zhou, M., Wen, Z., and Aggarwal, V. A graph-matching approach to dynamic media allocation in intelligent multimedia interfaces. In Proc. IUI'05, 114--121, ACM Press, 2005.

Cited By

View all
  • (2013)Prediction of interface preferences with a classifier selection approachJournal on Multimodal User Interfaces10.1007/s12193-013-0127-y7:4(321-349)Online publication date: 26-Sep-2013
  • (2010)Dynamic user interface distribution for flexible multimodal interactionInternational Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction10.1145/1891903.1891930(1-8)Online publication date: 8-Nov-2010
  • (2007)Subjective Measurement of Workload Related to a Multimodal Interaction Task: NASA-TLX vs. Workload ProfileHuman-Computer Interaction. HCI Intelligent Multimodal Interaction Environments10.1007/978-3-540-73110-8_7(60-69)Online publication date: 2007

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces
November 2007
402 pages
ISBN:9781595938176
DOI:10.1145/1322192
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. development tool
  2. dialogue and presentation strategies
  3. multimodal output

Qualifiers

  • Poster

Conference

ICMI07
Sponsor:
ICMI07: International Conference on Multimodal Interface
November 12 - 15, 2007
Aichi, Nagoya, Japan

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)1
  • Downloads (Last 6 weeks)0
Reflects downloads up to 19 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2013)Prediction of interface preferences with a classifier selection approachJournal on Multimodal User Interfaces10.1007/s12193-013-0127-y7:4(321-349)Online publication date: 26-Sep-2013
  • (2010)Dynamic user interface distribution for flexible multimodal interactionInternational Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction10.1145/1891903.1891930(1-8)Online publication date: 8-Nov-2010
  • (2007)Subjective Measurement of Workload Related to a Multimodal Interaction Task: NASA-TLX vs. Workload ProfileHuman-Computer Interaction. HCI Intelligent Multimodal Interaction Environments10.1007/978-3-540-73110-8_7(60-69)Online publication date: 2007

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media