poster

Simultaneous prediction of dialog acts and address types in three-party conversations

Authors:

Yosuke Matsusaka,

Yasuharu DenAuthors Info & Claims

ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces

Pages 66 - 73

https://doi.org/10.1145/1322192.1322207

Published: 12 November 2007 Publication History

Abstract

This paper reports on automatic prediction of dialog acts and address types in three-party conversations. In multi-party interaction, dialog structure becomes more complex compared to one-to-one case, because there is more than one hearer for an utterance. To cope with this problem, we predict dialog acts and address types simultaneously on our framework. Prediction of dialog act labels has gained to 68.5% by considering both context and address types. CART decision tree analysis has also been applied to examine useful features to predict those labels.

References

[1]

David Traum. Issues in multi-party dialogues. In Frank Dignum, editor, Advances in Agent Communication, volume 2922 of Lecture Notes in Artificial Intelligence, pages 201--211. Springer, Berlin, 2004.

[2]

Adam Kendon. Some functions of gaze direction in social interaction. Acta Psychologica, 26:22--63, 1967.

[3]

Starkey Duncan and Donald W. Fiske. Face-to-Face Interaction: Research, Methods and Theory. Lawrence Erlbaum, Hillsdale, NJ, 1977.

[4]

Paul Ekman and Wallace V. Friesen. The repertoire of non-verbal behaviour: Categories, origins, usage and coding. Semiotica, 1:49--98, 1969.

[5]

Roel Vertegaal, Harro Vons, and Robert Slagter. Look who's talking: the gaze groupware system. In CHI '98: CHI 98 conference summary on Human factors in computing systems, pages 293--294, New York, NY, USA, 1998. ACM Press.

Digital Library

[6]

R. P. H. Vertegaal, R. Slagter, G. C. van der Veer, and A. Nijholt. Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes. In Proceedings ACM SIGCHI Conference CHI 2001, pages 301--308, New York, March 2001. ACM Press.

Digital Library

[7]

Y. Matsusaka, T. Tojo, and T. Kobayashi. Conversation robot participating in group conversation. IEICE Trans. of Information and System, E86--D, No.1:26--36, 2003.

[8]

I. McCowan, D. Gatica-Perez, S. Bengio, and G. Lathoud. Automatic analysis of multimodal group actions in meetings, 2003.

[9]

M. Mast, R. Kompe, S. Harbeck, A. Kiessling, and V. Warnke. Dialog act classification with the help of prosody. In Proc. ICSLP '96, volume~3, pages 1732--1735, Philadelphia, PA, 1996.

[10]

E. Shriberg, R. Bates, P. Taylor, A. Stolcke, D. Jurafsky, K. Ries, N. Coccaro, R. Martin, M. Meteer, and C. van Ess-Dykema. Can prosody aid the automatic classification of dialog acts in conversational speech. Language and Speech, 41(3--4):439--487, 1998.

[11]

Anand Venkataraman, Luciana Ferrer, Andreas Stolcke, and Elizabeth Shriberg. Training a prosody based dialog act tagger from unlabeled data. In International Conference of Acoustics Speech and Signal Processing, volume 1, pages 272--275, Hong Kong, April 2003.

[12]

J. Ang, Y. Liu, and E. Shriberg. Automatic dialog act segmentation and classification in multiparty meetings. In International Conference of Acoustics Speech and Signal Processing, 2005.

[13]

N. Jovanovic and H. J. A. op den Akker. Towards automatic addressee identification in multi-party dialogues. In Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue, Boston, MA, USA, pages 89--92, Pennsylvania, USA, 2004. Association for Computational Linguistics.

[14]

Yasuharu Den and Mika Enomoto. A scientific approach to conversational informatics: Description, analysis, and modeling of human conversation. In Toyoaki Nishida, editor, Conversational Informatics: An Engineering Approach, chapter 17. John Wiley & Sons, Hoboken, NJ, in press.

[15]

R. M. Coulthard. An Introduction to Discourse Analysis. Longman, 2nd edition, 1985.

[16]

Mika Enomoto, Yasuharu Den, and Yosuke Matsusaka. Annotation of dialog acts and addressees in three-party conversations and its preliminary analysis. SIG Report of the Japanese Society for Artificial Intelligence, SIG-SLUD-A502:7-12, 2005.

[17]

Mika Enomoto, Yasuharu Den, and Yosuke Matsusaka. Annotation of dialog acts and addressees in three-party conversations and its preliminary analysis (2). SIG Report of the Japanese Society for Artificial Intelligence, SIG-SLUD-A503:3--8, 2006.

[18]

Michael Kipp. Gesture Generation by Imitation: From Human Behavior to Computer Character A nimation. Dissertation.com, Boca Raton, FL, 2004.

[19]

L. Breiman, J. Friedman, R. A. Olshen, and C. J. Stone. Classification and Regression Trees. Wadsworth, 1984.

[20]

S. Siegel and N. John Jr. Castellan. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, New York, 2nd edition, 1988.

[21]

Charles Goodwin. Restarts, pauses and the achievement of a state of mutual gaze at turn beginnings. Sociological Inquiry, 3--4: 272--302, 1980.

[22]

Christial Heath. Body Movement and speech in medical interaction. Cambridge University Press, Cambridge, UK, 1986.

Cited By

Leite IHajishirzi HAndrist SLehman JEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Managing chaosProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2522871(43-50)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2522871
Hajishirzi HLehman JHodgins JLee GGinzburg J(2012)Using group history to identify character-directed utterances in multi-child interactionsProceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue10.5555/2392800.2392838(207-216)Online publication date: 5-Jul-2012
https://dl.acm.org/doi/10.5555/2392800.2392838

Index Terms

Simultaneous prediction of dialog acts and address types in three-party conversations
1. Computing methodologies
  1. Modeling and simulation

Recommendations

Dialog acts in greeting and leavetaking in social talk
ISIAA 2017: Proceedings of the 1st ACM SIGCHI International Workshop on Investigating Social Interactions with Artificial Agents

Conversation proceeds through dialogue moves or acts, and dialog act annotation can aid the design of artificial dialog. While many dialogs are task-based or instrumental, with clear goals, as in the case of a service encounter or business meeting, ...
Facilitating multiparty dialog with gaze, gesture, and speech
ICMI-MLMI '10: International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction

We study how synchronized gaze, gesture and speech rendered by an embodied conversational agent can influence the flow of conversations in multiparty settings. We begin by reviewing a computational framework for turn-taking that provides the foundation ...
Multi-party interaction with a virtual character and a human-like robot
VRST '13: Proceedings of the 19th ACM Symposium on Virtual Reality Software and Technology

Research on interactive virtual characters and social robots focuses mainly on one-to-one interactions and multi-party interactions concept are rather less explored. As we are developing these characters to be helpful to us in our daily lives as guides, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces

November 2007

402 pages

ISBN:9781595938176

DOI:10.1145/1322192

General Chairs:
Kenji Mase
Nagoya University, Japan
,
Dominic Massaro
UC Santa Cruz, USA
,
Program Chairs:
Kazuya Takeda
Nagoya University, Japan
,
Deb Roy
MIT, USA
,
Alexandros Potamianos
Technical University of Crete, Greece

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Poster

Conference

ICMI07

Sponsor:

ICMI07: International Conference on Multimodal Interface

November 12 - 15, 2007

Aichi, Nagoya, Japan

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
176
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Leite IHajishirzi HAndrist SLehman JEpps JChen FOviatt SMase KSears AJokinen KSchuller B(2013)Managing chaosProceedings of the 15th ACM on International conference on multimodal interaction10.1145/2522848.2522871(43-50)Online publication date: 9-Dec-2013
https://dl.acm.org/doi/10.1145/2522848.2522871
Hajishirzi HLehman JHodgins JLee GGinzburg J(2012)Using group history to identify character-directed utterances in multi-child interactionsProceedings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue10.5555/2392800.2392838(207-216)Online publication date: 5-Jul-2012
https://dl.acm.org/doi/10.5555/2392800.2392838

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten