skip to main content
10.1145/1753326.1753481acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Lie tracking: social presence, truth and deception in avatar-mediated telecommunication

Published: 10 April 2010 Publication History

Abstract

The success of visual telecommunication systems depends on their ability to transmit and display users' natural nonverbal behavior. While video-mediated communication (VMC) is the most widely used form of interpersonal remote interaction, avatar-mediated communication (AMC) in shared virtual environments is increasingly common. This paper presents two experiments investigating eye tracking in AMC. The first experiment compares the degree of social presence experienced in AMC and VMC during truthful and deceptive discourse. Eye tracking data (gaze, blinking, and pupil size) demonstrates that oculesic behavior is similar in both mediation types, and uncovers systematic differences between truth telling and lying. Subjective measures show users' psychological arousal to be greater in VMC than AMC. The second experiment demonstrates that observers of AMC can more accurately detect truth and deception when viewing avatars with added oculesic behavior driven by eye tracking. We discuss implications for the design of future visual telecommunication media interfaces.

References

[1]
M. Argyle and M. Cook. Gaze and Mutual Gaze. Cambridge University Press Cambridge, 1976.
[2]
M. Argyle and R. Ingham. Gaze, mutual gaze and proximity. Semiotica, 6(1):32--50, 1972.
[3]
N. Badler, M. Hollick, and J. Granieri. Real-time control of a virtual human using minimal sensors. Presence, 2(1):82--86, 1993.
[4]
J. Bailenson, A. Beall, J. Loomis, J. Blascovich, and M. Turk. Transformed social interaction: Decoupling representation from behavior and form in collaborative virtual environments. Presence: Teleoperators & Virtual Environments, 13(4):428--441, 2004.
[5]
S. Benford, J. Bowers, L. Fahlén, C. Greenhalgh, and D. Snowdon. User embodiment in collaborative virtual environments. Proc. SIGCHI conference on Human factors in computing systems, pages 242--249, 1995.
[6]
J. Burns and B. Kintz. Eye contact while lying during an interview. Bulletin of the Psychonomic Society, 7(1):87--89, 1976.
[7]
J. Carlson, J. George, J. Burgoon, M. Adkins, and C. White. Deception in computer-mediated communication. Group Decision and Negotiation, 13(1):5--28, 2004.
[8]
M. Chen. Leveraging the asymmetric sensitivity of eye contact for videoconference. Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, pages 49--56, 2002.
[9]
P. Costa Jr and R. McCrae. Revised NEO Personality Inventory (NEOPI-R). The Sage handbook of personality theory and assessment: Personality measurement and testing, page 179, 2008.
[10]
R. Daft and R. Lengel. Information richness: A new approach to managerial behavior and organizational design. Research in organizational behavior, 6:191--233, 1984.
[11]
A. Dennis and J. Valacich. Rethinking media richness: Towards a theory of media synchronicity. In Proc. 32nd Hawaii International Conference on System Sciences, volume 1, pages 1--10. Citeseer, 1999.
[12]
B. DePaulo. Humans as lie detectors. Journal of Communication, 30(2):129--39, 1980.
[13]
B. DePaulo. Nonverbal behavior and self-presentation. Psychological Bulletin, 111(2):203--243, 1992.
[14]
B. DePaulo and D. Kashy. Everyday lies in close and casual relationships. Journal of Personality and Social Psychology, 74(1):63--79, 1998.
[15]
B. DePaulo, J. Lindsay, B. Malone, L. Muhlenbruck, K. Charlton, and H. Cooper. Cues to deception. Psychological Bulletin, 129(1):74--118, 2003.
[16]
J. Draper, D. Kaber, and J. Usher. Telepresence. Human Factors: The Journal of the Human Factors and Ergonomics Society, 40(3):354--375, 1998.
[17]
N. Ducheneaut, N. Yee, E. Nickell, and R. Moore. Alone together?: exploring the social dynamics of massively multiplayer online games. In Proceedings of the SIGCHI conference on Human Factors in computing systems, page 416. ACM, 2006.
[18]
C. Evinger, K. Manning, and P. Sibony. Eyelid movements. Mechanisms and normal data. Investigative ophthalmology & visual science, 32(2):387--400, 1991.
[19]
M. Garau. The Impact of Avatar Fidelity on Social Interaction in Virtual Environments. PhD thesis, University College London, 2003.
[20]
M. Garau, M. Slater, V. Vinayagamoorthy, A. Brogni, A. Steed, and M. Sasse. The impact of avatar realism and eye gaze control on perceived quality of communication in a shared immersive virtual environment. Human factors in computing systems, pages 529--536, 2003.
[21]
M. Hartwig, P. Granhag, L. Stromwall, and A. Vrij. Police officers' lie detection accuracy: Interrogating freely versus observing video. Police Quarterly, 7(4):429, 2004.
[22]
E. Isaacs and J. Tang. What video can and cannot do for collaboration: A case study. Multimedia Systems, 2(2):63--73, 1994.
[23]
H. Ishii, M. Kobayashi, and J. Grudin. Integration of interpersonal space and shared workspace: ClearBoard design and experiments. ACM Transactions on Information Systems (TOIS), 11(4):349--375, 1993.
[24]
M. Knapp, R. Hart, and H. Dennis. An exploration of deception as a communication construct. Human Communication Research, 1(1):15--29, 1974.
[25]
S. Leal and A. Vrij. Blinking During and After Lying. Journal of Nonverbal Behavior, 32(4):187--194, 2008.
[26]
T. Levine, H. Park, and S. McCornack. Accuracy in detecting truths and lies: Documenting the veracity effect. Communication Monographs, 66(2):125--144, 1999.
[27]
D. McNair, M. Lorr, and L. Droppleman. Profile of mood states. San Diego, California, 53:6, 1992.
[28]
T. Partala and V. Surakka. Pupil size variation as an indication of affective processing. International Journal of Human-Computer Studies, 59(1-2):185--198, 2003.
[29]
D. Roberts, R. Wolff, O. Otto, and A. Steed. Constructing a Gazebo: Supporting Teamwork in a Tightly Coupled, Distributed Task in Virtual Reality. Presence: Teleoperators & Virtual Environments, 12(6):644--657, 2003.
[30]
J. Short, E. Williams, and B. Christie. The social psychology of telecommunications. John Wiley & Sons, 1976.
[31]
W. Steptoe, O. Oyekoya, A. Murgia, R. Wolff, J. Rae, E. Guimaraes, D. Roberts, and A. Steed. Eye Tracking for Avatar Eye Gaze Control During Object-Focused Multiparty Interaction in Immersive Collaborative Virtual Environments. In Proc. 2009 IEEE Virtual Reality Conference, pages 83--90. IEEE Computer Society Washington, DC, USA, 2009.
[32]
W. Steptoe, R. Wolff, A. Murgia, E. Guimaraes, J. Rae, P. Sharkey, D. Roberts, and A. Steed. Eye-tracking for avatar eye-gaze and interactional analysis in immersive collaborative virtual environments. In Proc. ACM 2008 conference on Computer supported cooperative work, pages 197--200. ACM New York, NY, USA, 2008.
[33]
D. Thalmann. The Role of Virtual Humans in Virtual Environment Technology and Interfaces. In Proc. of Joint EC-NSF Advanced Research Workshop, Bonas, France. ACM New York, NY, USA, 1999.
[34]
A. Vrij. The impact of information and setting on detection of deception by police detectives. Journal of Nonverbal Behavior, 18(2):117--136, 1994.
[35]
J. Walczyk, J. Schwartz, R. Clifton, B. Adams, M. Wei, and P. Zha. Lying person-to-person about life events: A cognitive framework for lie detection. Personnel Psychology, 58(1):141--170, 2005.
[36]
J. Wang, M. Spezio, and C. Camerer. Pinocchios pupil: Using eyetracking and pupil dilation to understand truth-telling and deception in games. forthcoming in American Economic Review, 2006.
[37]
E. Williams. Experimental comparisons of face-to-face and mediated communication: A review. Psychological Bulletin, 84(5):963--976, 1977.
[38]
R. Wolff, D. Roberts, A. Murgia, N. Murray, J. Rae, W. Steptoe, A. Steed, and P. Sharkey. Communicating Eye Gaze across a Distance without Rooting Participants to the Spot. In Proc. IEEE/ACM Symposium on Distributed Simulation and Real-Time Applications-Volume 00, pages 111--118. IEEE Computer Society Washington, DC, USA, 2008.
[39]
M. Zuckerman, B. DePaulo, and R. Rosenthal. Verbal and nonverbal communication of deception. Advances in experimental social psychology, 14:1--59, 1981.
[40]
M. Zuckerman, R. Koestner, and R. Driver. Beliefs about cues associated with deception. Journal of Nonverbal Behavior, 6(2):105--114, 1981.

Cited By

View all
  • (2024)The Influence of Environmental Context on the Creation of Cartoon-like Avatars in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00052(298-308)Online publication date: 16-Mar-2024
  • (2023)How Gaze Visualization Facilitates Initiation of Informal Communication in 3D Virtual SpacesACM Transactions on Computer-Human Interaction10.1145/361736831:1(1-32)Online publication date: 29-Nov-2023
  • (2023)Work vs. Leisure – Differences in Avatar Characteristics Depending on Social SituationsProceedings of the 16th International Symposium on Visual Information Communication and Interaction10.1145/3615522.3615537(1-9)Online publication date: 22-Sep-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI '10: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
April 2010
2690 pages
ISBN:9781605589299
DOI:10.1145/1753326
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 10 April 2010

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. avatar-mediated communication
  2. deception
  3. eye tracking
  4. social presence
  5. trust
  6. video-mediated communication
  7. virtual environments

Qualifiers

  • Research-article

Conference

CHI '10
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)74
  • Downloads (Last 6 weeks)5
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)The Influence of Environmental Context on the Creation of Cartoon-like Avatars in Virtual Reality2024 IEEE Conference Virtual Reality and 3D User Interfaces (VR)10.1109/VR58804.2024.00052(298-308)Online publication date: 16-Mar-2024
  • (2023)How Gaze Visualization Facilitates Initiation of Informal Communication in 3D Virtual SpacesACM Transactions on Computer-Human Interaction10.1145/361736831:1(1-32)Online publication date: 29-Nov-2023
  • (2023)Work vs. Leisure – Differences in Avatar Characteristics Depending on Social SituationsProceedings of the 16th International Symposium on Visual Information Communication and Interaction10.1145/3615522.3615537(1-9)Online publication date: 22-Sep-2023
  • (2023)"A feeling of déjà vu": The Effects of Avatar Appearance-Similarity on Persuasiveness in Social Virtual RealityProceedings of the ACM on Human-Computer Interaction10.1145/36101677:CSCW2(1-31)Online publication date: 4-Oct-2023
  • (2023)"\"Hello I am here\": Proximal Nonverbal Cues Role in Initiating Social Interactions in VR"Proceedings of the 2023 ACM International Symposium on Wearable Computers10.1145/3594738.3611356(11-16)Online publication date: 8-Oct-2023
  • (2023)What And How Together: A Taxonomy On 30 Years Of Collaborative Human-Centered XR Tasks2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)10.1109/ISMAR59233.2023.00047(322-335)Online publication date: 16-Oct-2023
  • (2023)The assessment of presence and performance in an AR environment for motor imitation learningComputers in Human Behavior10.1016/j.chb.2023.107810146:COnline publication date: 26-Jul-2023
  • (2023)'You have to put a lot of trust in me': autonomy, trust, and trustworthiness in the context of mobile apps for mental healthMedicine, Health Care and Philosophy10.1007/s11019-023-10146-y26:3(313-324)Online publication date: 30-Mar-2023
  • (2023)Eye Tracking in Virtual Reality: a Broad Review of Applications and ChallengesVirtual Reality10.1007/s10055-022-00738-z27:2(1481-1505)Online publication date: 18-Jan-2023
  • (2022)The Eye in Extended Reality: A Survey on Gaze Interaction and Eye Tracking in Head-worn Extended RealityACM Computing Surveys10.1145/349120755:3(1-39)Online publication date: 25-Mar-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media