Interacting with embodied agents that can see: how vision-enabled agents can assist in spatial tasks

ABSTRACT

In this paper, we describe user experiences with a system equipped with cognitive vision that interacts with the user in the context of personal assistance in the office. A cognitive vision computer can see the user and user responses and react to situations that happen in the environment, crossing the boundary between the virtual and the physical world. How should such a seeing computer interact with its users? Three different interface styles -- a traditional GUI, a cartoon-like embodied agent and a realistic embodied agent -- are tested in two tasks where users are actively observed by a (simulated) cognitive vision system. The system assists them in problem solving. Both the non-embodied and the embodied interaction styles offer the user certain advantages and the pros and cons based on the experiment results are discussed in terms of performance, intelligence, trust, comfort, and social presence.

REFERENCES

• 1: Elisabeth André , Thomas Rist , Jochen Müller, Integrating reactive and scripted behaviors in a life-like presentation agent, Proceedings of the second international conference on Autonomous agents, p.261-268, May 10-13, 1998, Minneapolis, Minnesota, United States [doi>10.1145/280765.280842]
• 2: Gene Ball , Jack Breese, Emotion and personality in a conversational agent, Embodied conversational agents, MIT Press, Cambridge, MA, 2001
• 3: Bickmore, T. W., 2003. Relational Agents: Effecting Change through Human-Computer Relationships, Ph.D. Thesis, MIT.
• 4: Timothy W. Bickmore , Rosalind W. Picard, Establishing and maintaining long-term human-computer relationships, ACM Transactions on Computer-Human Interaction (TOCHI), v.12 n.2, p.293-327, June 2005 [doi>10.1145/1067860.1067867]
• 5: Susan E. Brennan , Justina O. Ohaeri, Effects of message style on users' attributions toward agents, Conference companion on Human factors in computing systems, p.281-282, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/259963.260492]
• 6: Cole, R., Vuuren, S. v., Pellom, B., Hacioglu, K., Ma, J., Movellan, J., Schwartz, S., Wadestein, D., Ward, W. and Yan, J., 2003. Perceptive Animated Interfaces: First Steps Toward a New Paradigm for Human--Computer Interaction, Proc. IEEE, 91(9), 1391--1405.
• 7: Justine Cassell, Embodied conversational agents: representation and intelligence in user interfaces, AI Magazine, v.22 n.4, p.67-83, Winter 2001
• 8: Nils Dahlbäck , Arne Jönsson , Lars Ahrenberg, Wizard of Oz studies: why and how, Proceedings of the 1st international conference on Intelligent user interfaces, p.193-200, January 04-07, 1993, Orlando, Florida, United States [doi>10.1145/169891.169968]
• 9: ECVision, online at http://www.ecvision.org. Last accessed: January 2006.
• 10: Furnham, A., 1997. The psychology of behaviour at work, Psychology Press, Publishers, East Sussex, UK.
• 11: de Greef, H. P. and IJsselsteijn, W. A., 2000. Social Presence in the PhotoShare Tele-Application. Proc. PRESENCE 2000, Delft, The Netherlands
• 12: Keirsey Online Temperament Sorter (German), http://www.keirsey.com/ Last accessed: January 2006.
• 13: King, W. J., and Ohya, J., 1995. The representation of agents: a study of phenomena in virtual environments, Proc. RO-MAN'95.
• 14: Koda, T. and Maes, P., 1996. Agents with Faces: The Effects of Personification of Agents, Proc. HCI '96 UK, 98--103.
• 15: Jaron Lanier, Agents of alienation, interactions, v.2 n.3, p.66-72, July 1995 [doi>10.1145/208666.208684]
• 16: Kwan Min Lee , Clifford Nass, Designing social presence of social actors in human computer interaction, Proceedings of the SIGCHI conference on Human factors in computing systems, April 05-10, 2003, Ft. Lauderdale, Florida, USA [doi>10.1145/642611.642662]
• 17: James C. Lester , Sharolyn A. Converse , Susan E. Kahler , S. Todd Barlow , Brian A. Stone , Ravinder S. Bhogal, The persona effect: affective impact of animated pedagogical agents, Proceedings of the SIGCHI conference on Human factors in computing systems, p.359-366, March 22-27, 1997, Atlanta, Georgia, United States [doi>10.1145/258549.258797]
• 18: Mori, M., 1970. The Uncanny Valley. Energy, 7, 33--35.
• 19: Mori, M., 2005. On the Uncanny Valley. Proc. workshop Humanoids-2005.
• 20: Clifford Nass , Jonathan Steuer , Ellen R. Tauber, Computers are social actors, Proceedings of the SIGCHI conference on Human factors in computing systems: celebrating interdependence, p.72-78, April 24-28, 1994, Boston, Massachusetts, United States [doi>10.1145/191666.191703]
• 21: Rosalind W. Picard, Affective computing, MIT Press, Cambridge, MA, 1997
• 22: Power, G., Wills, G. and Hall, W., 2002. User Perception of Anthropomorphic Characters with Varying Levels of Interaction. Proc. HCI '02 UK, 37--51.
• 23: Byron Reeves , Clifford Nass, The media equation: how people treat computers, television, and new media like real people and places, Cambridge University Press, New York, NY, 1996
• 24: Corina Sas , P. O’Hare , Ronan Reilly, Presence and task performance: an approach in the light of cognitive style, Cognition, Technology and Work, v.6 n.1, p.53-56, February 2004 [doi>10.1007/s10111-003-0145-8]
• 25: Short, J. A., Williams, E., & Christie, B., 1976. The social psychology of telecommunications, New York: John Wiley & Sons.
• 26: Sproull, L., Walker, J., Subramani, R., Kiesler, S., and Waters, K., 1996. When the Interface is a Face, Human Computer Interaction, 11, 97--124.
• 27: Vernon, D., 2004. Cognitive Vision -- The Development of a Discipline, Proc. IST 2004 Event 'Participate in your future'.
• 28: Xiao, J., Catrambone, R. and Stasko, J., 2003. Be Quiet? Evaluating Proactive and Reactive User Interface Assistants, Proc. INTERACT 2003, 383--390.

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.2 User Interfaces (D.2.2, H.1.2, I.3.6)

Keywords:
cognitive vision, embodied agent, intelligent systems, personal assistant, personality type, user experience