Multimodal agent interface based on dynamical dialogue model: MAICO: multimodal agent interface for communication

ABSTRACT

In this paper, we describe a multimodal interface prototype system based on Dynamical Dialogue Model. This system not only integrates information of speech and gestures, but also controls the response timing in order to realize a smooth interaction between user and computer. Our approach consists of human-human dialogue analysis, and computational modeling of dialogue.

REFERENCES

• 1: S.Seki, et al:Human Interface with Rhythm based on Dynamical Dialogue Interface, IPSJ SIG-HS, 98-HI- 80, pp.39-40, 1998(in Japanese).
• 2: K.Sakamoto et al: Multimodal Interface Prototype Based on Rhythm and Timing of Interaction", Proc. RWC'97, pp.31-36, 1997.
• 3: K.Watanuki, et al: Analysis of Multi-Modal Interaction Data in Human Communication, Proc. ASJ Spring Conference, 1-7-20, pp.39-40, 1994(in Japanese).
• 4: K.Watanuki, et al:Study of Listener's Back Channels, ASJ Spring Conference, 3-6-15, pplll-l12,1998(in Japanese).
• 5: Kenji Sakamoto , Haruo Hinode , Keiko Watanuki , Susumu Seki , Jiro Kiyama , Fumio Togawa, A response model for a CG character based on timing of interactions in a multimodal human interface, Proceedings of the 2nd international conference on Intelligent user interfaces, p.257-260, January 06-09, 1997, Orlando, Florida, United States [doi>10.1145/238218.238342]
• 6: N.Ward: In Japanese a Low Pitch Means 'back- Channel Feedback Please', SIG-SLP- 11-2,1996.
• 7: R.A.Bolt : The Integrated Multi-Modal Interface, The Transactions of the Institute of Electronics Information and Communication Engineers, Vol. J-70- D,No. 11,1987.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

Additional Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

General Terms:
Experimentation, Theory

Keywords:
back channels, dynamical dialogue model, dynamics, multimodal interface, nonverbal information, response timing