Interaction debugging: an integral approach to analyze human-robot interaction

ABSTRACT

Along with the development of interactive robots, controlled experiments and field trials are regularly conducted to stage human-robot interaction. Experience in this field has shown that analyzing human-robot interaction for evaluation purposes fosters the development of improved systems and the generation of new knowledge. In this paper, we present the interaction debugging approach, which is based on the collection and analysis of data from robots and their environment. Considering the multimodality of robotic technology, often only audio and video are insufficient for detailed analysis of human-robot interaction. Therefore, in our analysis we integrate multimodal information using audio, video, sensory data, and intermediate variables. An important aspect of the interaction debugging approach is using a tool called Interaction Debugger to analyze data. By supporting user-friendly data presentation, annotation and navigation, Interaction Debugger enables fine-grained inspection of human-robot interaction. The main goal of this paper is to address how an integral approach to the analysis of human-robot interaction can be adopted. This is demonstrated by three case studies.

REFERENCES

• 1: Sakagami, R., Watanabe, C., Aoyama, S., Matsunaga, N., Higaki, N., and Fujimura, K., The intelligent ASIMO: System overview and integration, in Proc. IEEE/RSJ Int. Conf. Intelligent Robots and Systems, 2002, pp. 2478--2483.
• 2: Fujita, M., AIBO: Toward the era of digital creatures, Int. J. Robot. Res., vol. 20, no. 10, pp. 781--794, 2001.
• 3: Ishiguro, H., Ono, T., Imai, M., Maeda, T., Kanda, T., and Nakatsu, R., Robovie: an interactive humanoid robot", Int. J. Industrial Robot, Vol. 28, No. 6, pp.498--503, 2001.
• 4: Dautenhahn, K. and Werry, I., A Quantitative Technique for Analysing Robot-Human Interactions, in Proc. of the 2002 IEEE/RSJ Int. Conf. on Intelligent Robots and Systems, 2002, pp. 1132--1138.
• 5: Kanda, T., Hirano, T., Eaton, D., and Ishiguro, H., Interactive Robots as Social Partners and Peer Tutors for Children: A Field Trial, in Human Computer Interaction, Vol. 19, No. 1-2, pp. 61--84, 2004.
• 6: Siegwart, R., Arras, K.O., Bouabdallah, S., Burnier, D., Froidevaux, G., Greppin, X., Jensen, B., Lorotte, A., Mayor, L., Meisser, M., Philippsen, R., Piguet, R., Ramel, G., Terrien, G., and Tomatis, N., "Robox at Expo.02: A Large-Scale Installation of Personal Robots," in Robotics and Autonomous Systems, vol. 42, issue 3-4, pp. 203--222, 2003.
• 7: Quek, F., Shi, Y., Kirbas, C., and Wu, S., VisSTA: A Tool for Analyzing Multimodal Discourse Data, in Seventh International Conference on Spoken Language Processing, 2002, Denver, CO.
• 8: Kipp, M. and Anvil, A., Generic Annotation Tool for Multimodal Dialogue, in Proceedings of the 7th European Conference on Speech Communication and Technology (Eurospeech), pp. 1367--1370, Aalborg, 2001.
• 9: Kanda, T., Ishiguro, H., Imai, M., and Ono, T., Development and Evaluation of Interactive Humanoid Robots, In Proceedings of the IEEE, 2004, pp. 1839--1850.
• 10: Sumi, Y., Ito, S., Matsuguchi, T., Fels, S., and Mase, K., Collaborative Capturing and Interpretation of Interactions, Pervasive 2004 Workshop on Memory and Sharing of Experiences, pp. 1--7, 2004.
• 11: Nomura, T., Tasaki, T., Kanda, T., Shiomi, M, Ishiguro, H., and Hagita, N., Questionnaire-Based Research on Opinions of Visitors for Communication Robots at an Exhibition in Japan, International Conference on Human-Computer Interaction (Interact 2005), 2005.
• 12: Ekman, P. and Friensen, W. V., Manual for the Facial Action Coding System and Action Unit Photographs, Consulting Psychologists Press, 1978.
• 13: Shogo Nabe , Takayuki Kanda , Kazuo Hiraki , Hiroshi Ishiguro , Kiyoshi Kogure , Norihiro Hagita, Analysis of human behavior to a communication robot in an open field, Proceeding of the 1st ACM SIGCHI/SIGART conference on Human-robot interaction, March 02-03, 2006, Salt Lake City, Utah, USA [doi>10.1145/1121241.1121282]
• 14: Nabe, S., Kanda, T., Hiraki, K., Ishiguro, H., and Hagita, N., Human Friendship Estimation Model for Communication Robots, 2005 IEEE-RAS International Conference on Humanoid Robots, 2005.
• 15: Jenny Preece , Yvonne Rogers , Helen Sharp, Interaction Design, John Wiley & Sons, Inc., New York, NY, 2002
• 16: Ben Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1997
• 17: Donald A. Norman , Stephen W. Draper, User Centered System Design; New Perspectives on Human-Computer Interaction, Lawrence Erlbaum Associates, Inc., Mahwah, NJ, 1986

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Evaluation/methodology

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.1 Multimedia Information Systems
          Subjects: Video (e.g., tape, disk, DVI)

  J. Computer Applications
  J.4 SOCIAL AND BEHAVIORAL SCIENCES
      Subjects: Psychology

General Terms:
Design, Experimentation, Human Factors, Measurement

Keywords:
analysis tool, integral approach, interaction, multimodal data