Multi-party interaction with a virtual character and a human-like robot

ABSTRACT

Research on interactive virtual characters and social robots focuses mainly on one-to-one interactions and multi-party interactions concept are rather less explored. As we are developing these characters to be helpful to us in our daily lives as guides, companions, assistants or receptionists, they should be aware of the existence of multiple people and address their requirements in a natural way and act according to the social rules and norms. In contrast with previous work, we are interested in multi-party and multi-modal interactions between 3D virtual characters, real humans and social robots. This means that any of these participants can interact with each other. In this paper we present our on-going work, provide a discussion on multi-party interaction, describe the overall system architecture and mention our future work.

References

1. Bohus, D., and Horvitz, E. 2009. Open-world dialog: Challenges, directions and a prototype. In IJCAI Workshop on Knowledge and Reasoning in Practical Dialog Systems,.
2. Cassell, J., Vilhjálmsson, H., Högni, and Bickmore, T. 2001. BEAT: the behavior expression animation toolkit. In Proceedings of the 28th annual conference on Computer graphics and interactive techniques, SIGGRAPH '01, 477--486.
3. D. Traum, and Larsson, S. 2003. The information state approach to dialogue management. In Current and New Directions in Discourse and Dialogue, 325--353.
4. Foster, M., Gaschler, A., Giuliani, M., Isard, A., Pateraki, M., and Petrick, R. 2012. Two people walk into a bar: Dynamic multi-party social interaction with a robot agent. In Proceedings of the ACM International Conference on Multi-modal Interaction (ICMI 2012), 3--10.
5. Kasap, Z., and Magnenat-Thalmann, N. 2012. Building long-term relationships with virtual and robotic characters: The role of remembering. The Visual Computer 28, 1, 87--97.
6. Kondo, Y., Takemura, K., Takamatsu, J., and Ogasawara, T. 2013. A gesture-centric android system for multi-party human-robot interaction. Journal of Human-Robot Interaction 2, 1, 133--151.
7. Lee, C., Jung, S., Kim, K., Lee, D., and Lee, G. 2010. Recent approaches to dialogue management for spoken dialog systems. Journal of Computing Science and Engineering 4, 1, 1--22.
8. Matsuyama, Y., Taniyama, H., F., and Kobayashi, T. 2010. Framework of communication activation robot participating in multiparty conversation. In AAAI Fall Symposium.
9. Rashobh, R. S., and Khong, A. W. H. 2012. A variable step-size adaptive algorithm for room acoustics equalization exploiting the sparseness constraint. In IEEE Int. Symp. Circuits and Systems.
10. Ren, J., Jiang, X., and Yu, J. 2013. Relaxed local ternary pattern for face recognition. In IEEE Int. Conf. Image Processing.
11. Traum, D. R., and Morency, L. P. 2010. Integration of visual perception in dialogue understanding for virtual humans in multi-party interaction. In AAMAS International Workshop on Interacting with ECAs as Virtual Characters, 1723--1726.
12. Traum, D. 2004. Issues in multi-party dialoguess. In: F. Dignum, editor. Advances in Agent Communication (November), 201--211.
13. Wu, K., Goh, S. T., and Khong, A. W. H. 2013. Speaker localization and tracking in the presence of sound interference by exploiting speech harmonicity. In IEEE. Int. Conf. Acoust., Speech, Signal Process. (ICASSP 13).
14. Yang, X., Yuan, J., and Thalmann, D. 2013. Human-virtual human interaction by upper body gesture understanding. In The 19th ACM Symposium on Virtual Reality Software and Technology (VRST2013).

Index Terms

• Computing methodologies

  • Computer graphics

    • Graphics systems and interfaces

      • Perception

      • Virtual reality

• Human-centered computing

  • Human computer interaction (HCI)

    • Interaction paradigms

      • Mixed / augmented reality

      • Virtual reality