skip to main content
10.1145/1452392.1452404acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

Multimodal recognition of personality traits in social interactions

Published: 20 October 2008 Publication History

Abstract

This paper targets the automatic detection of personality traits in a meeting environment by means of audio and visual features; information about the relational context is captured by means of acoustic features designed to that purpose. Two personality traits are considered: Extraversion (from the Big Five) and the Locus of Control. The classification task is applied to thin slices of behaviour, in the form of 1-minute sequences. SVM were used to test the performances of several training and testing instance setups, including a restricted set of audio features obtained through feature selection. The outcomes improve considerably over existing results, provide evidence about the feasibility of the multimodal analysis of personality, the role of social context, and pave the way to further studies addressing different features setups and/or targeting different personality traits.

References

[1]
André, E., Klense, M., Gebhard, P., Allen, S., Rist, T. Integrating models of personality and emotions into lifelike characters. In Proceedings of the Workshop on Affect in Interaction - Towards a New Generation of Interfaces, pp 136--149, 1999.
[2]
Ambady, N., and Rosenthal, R. Thin slices of expressive behaviors as predictors of interpersonal consequences: A meta-analysis. Psychological Bulletin, 111, pp. 256--274
[3]
Argamon, S., Dhawle, S., Koppel, M., Pennbaker, J. Lexical predictors of personality type. In Proceedings of Interface and the Classification Society of North America. 2005.
[4]
Blanck, P. D., Rosenthal, R., Vannicelli, M., and Lee, T. D. Therapists' tone of voice: descriptive, psychometric, interactional and competence analyses. Journal of Social and Clinical Psychology, 4, pp. 154--178. 1986
[5]
Burges, C. J. C., A Tutorial on Support Vector Machines for Pattern Recognition. In Data Mining and Knowledge Discovery, Vol. 2, No. 2, pp. 121--167, 1998.
[6]
Carrere, S., and Gottman, J. M. Predicting divorce among newlyweds from the first three minutes of a marital conflict discussion. Family Process, 38, pp. 293--301
[7]
Castelfranchi, F. D. Rosis. How can personality factors contribute to make agents more 'believable', Proceedings of the Workshop on Behavior Planning for Lifelike Characters and Avatars, Sitges, Spain, 25--35, 1999.
[8]
Chippendale, P. Towards Automatic Body Language Annotation. Proceedings of the 7th International Conference on Automatic Face and Gesture Recognition - FG2006 (IEEE), pp. 487--492, Southampton, UK, 2006.
[9]
Curhan, J. R., and Pentland, A. Thin slices of Negotiation: predicting outcomes from conversational dynamics within the first 5 minutes. Journal of Applied Psychology, Vol. 92, No. 3, pp. 802--811, 2007
[10]
Farma, T., and Cortivonis, I. Un Questionario sul "Locus of Control": Suo Utilizzo nel Contesto Italiano (A Questionnaire on the Locus of Control: Its Use in the Italian Context). Ricerca in Psicoterapia. Vol. 2, 2000.
[11]
Furnham, D. Language and Personality. In Giles, H., & Robinson, W. (Eds.), Handbook of Language and Social Psychology. Winley, 1990.
[12]
Giannoutsos, V. 'Stress, Attitudes and Personality in Computing'. In Proceedings of ETHICOMP 2004, Syros, Gr. 2004
[13]
Gottman, J. M, and Notarius, C. I. Decade review: observing marital interaction. Journal of Marriage and the Family, 62, pp. 927--947, 2000
[14]
Goren-Bar, D., Graziola, I., Pianesi, F., Zancanaro, M. Influence of Personality Factors on Visitors' Attitudes towards Adaptivity Dimensions for Mobile Museum Guides'. In User Modeling and User Adapted Interaction: The Journal of Personalization Research, 16 (1), 31--62. 2006
[15]
Hall, J. W., and Watson, W. H. The Effects of a normative intervention on group decision-making performance. In Human Relations, 23(4), 299--317, 1970.
[16]
Heider, F. The psychology of interpersonal relations. Wiley. New York. 1957
[17]
Hsu, C. W. A. Hsu C.-W., Lin C.-J. A Comparison of Methods for Multi-Class Support Vector Machines. IEEE Transactions on Neural Networks, 13, pp. 415--425, 2002.
[18]
John, O. P., Srivastava, S. The Big five trait taxonomy: History, measurement and theoretical perspectives. In Pervian, L. A. & John, O. P., (Eds.) Handbook of personality theory and research. Guilford Press. New York, 1999.
[19]
Johnson, R. D., Marakas, G., Plamer, J. W. Individual Perceptions Regarding the Capabilities and Roles of Computing Technology: Development of The Computing Technology Continuum of Perspective. Ms. 2002.
[20]
Kenny, D. A. Interpersonal perception: A social relations analysis. New York: Guilford Press. 1994.
[21]
Kressel, U. Pairwise Classification and Support Vector Machines. In B. Scholkopf, C. J. C. Burges, and A. J. Smola (eds.) Advances in Kernel Methods - Support Vector Learning. MIT Press, Cambridge, MA, 1999.
[22]
Mairesse, F., Walker, M. Words marks the nerds: computational models of personality recognition through language. In Proceedings of the 28th Annual Conference of the Cognitive Science Society, pp 543--548, 2006.
[23]
Mairesse F., Walker M. A., Mehl M. R., and Moore R. K. Using Linguistic Cues for the Automatic Recognition of Personality in Conversation and Text. In Journal of Artificial Intelligence Research 30, pp.457--500. 2007
[24]
Mallory P., and Miller V. A possible basis for the association of voice characteristics and personality traits. Speech Monograph, 25, pp. 255--260, 1958.
[25]
Mana, N., Lepri, B., Chippendale, P., Cappelletti, A., Pianesi, F., Svaizer, P, and Zancanaro, M. Multimodal Corpus of Multi-Party Meetings for Automatic Social Behavior Analysis and Personality Traits Detection. In Proceedings of Workshop on Tagging, Mining and Retrieval of Human-Related Activity Information, at ICMI07, International Conference on Multimodal Interfaces, Nagoya, Japan, 2007.
[26]
Oberlander, J. and Nowson, S. 2006. Whose thumb is it anyway? Classifying author personality from weblog text. In Proceedings of the Annual Meeting of the ACL. Association for Computational Linguistics, Morristown, NJ, 627--634, 2006.
[27]
Pentland, A. Socially Aware Computation and Communication. IEEE Computer, May 2005.
[28]
Pentland, A. A Computational Model of Social Signaling. Proceedings of the 18th International Conference on Pattern Recognition (ICPR'06). Volume 1, Issue 2006, Page(s):1080--1083, 2006.
[29]
Pentland, A. Honest Signals: how they shape our world. (In press). MIT Press, September 2008.
[30]
Perugini, M. and Di Blas L. Analyzing Personality-Related Adjectives from an Eticemic Perspective: the Big Five Marker Scale (BFMS) and the Italian AB5C Taxonomy. In De Raad, B., & Perugini, M. (Eds.), Big Five Assessment, Hogrefe und Huber Publishers. Gööttingen, 281--304, 2002.
[31]
Rotter J. B. Generalized Expectancies for Internal versus External Control of Reinforcment. In Psychological Monographs, 80 (1, Whole N. 609), 1965
[32]
Siegman A., and Pope B. Personality variable associated with productivity and verbal fluency in the initial interview. Proceedings of the 73rd Annual Conference of the American Psychological Association, 1965.
[33]
Scherer K. R. Personality Inference from Voice Quality: the Loud Voice of Extraversion. European Journal of Social Psychology, vol. 8, pp. 467--487, 1978.
[34]
Scherer K. R. Personality markers in speech. In Scherer K. R. and Giles H. (eds.) Social Markers in Speech, pp. 147--209 Cambridge University Press, 1979.
[35]
Stoltzman, W. Toward a Social Signaling Framework: Activity and Emphasis in Speech. MEng Thesis, MIT, 2006
[36]
Zhou, X. and Conati, C. Inferring user goals from personality and behavior in a causal model of user affect. In Proceedings of the 8th international Conference on intelligent User interfaces (Miami, Florida, USA, January 12 - 15, 2003). IUI '03. ACM, New York, NY, 211--218. DOI= http://doi.acm.org/10.1145/604045.604078, 2003.

Cited By

View all
  • (2024)Social Battery as a Design MetaphorSocial Battery as a Design MetaphorAvances en Interacción Humano-Computadora10.47756/aihc.y9i1.1479:1(56-64)Online publication date: 30-Nov-2024
  • (2024)Multimodal Personality Prediction: A Real-Time Recognition System for Social Robots with Data Acquisition2024 21st International Conference on Ubiquitous Robots (UR)10.1109/UR61395.2024.10597440(673-676)Online publication date: 24-Jun-2024
  • (2024) VyaktitvaNirdharan : Multimodal Assessment of Personality and Trait Emotional Intelligence IEEE Transactions on Affective Computing10.1109/TAFFC.2024.340424315:4(2139-2153)Online publication date: Oct-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
ICMI '08: Proceedings of the 10th international conference on Multimodal interfaces
October 2008
322 pages
ISBN:9781605581989
DOI:10.1145/1452392
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 October 2008

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. group interaction
  2. intelligent environments
  3. personality modeling
  4. support vector machines

Qualifiers

  • Research-article

Conference

ICMI '08
Sponsor:
ICMI '08: INTERNATIONAL CONFERENCE ON MULTIMODAL INTERFACES
October 20 - 22, 2008
Crete, Chania, Greece

Acceptance Rates

Overall Acceptance Rate 453 of 1,080 submissions, 42%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)94
  • Downloads (Last 6 weeks)4
Reflects downloads up to 09 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Social Battery as a Design MetaphorSocial Battery as a Design MetaphorAvances en Interacción Humano-Computadora10.47756/aihc.y9i1.1479:1(56-64)Online publication date: 30-Nov-2024
  • (2024)Multimodal Personality Prediction: A Real-Time Recognition System for Social Robots with Data Acquisition2024 21st International Conference on Ubiquitous Robots (UR)10.1109/UR61395.2024.10597440(673-676)Online publication date: 24-Jun-2024
  • (2024) VyaktitvaNirdharan : Multimodal Assessment of Personality and Trait Emotional Intelligence IEEE Transactions on Affective Computing10.1109/TAFFC.2024.340424315:4(2139-2153)Online publication date: Oct-2024
  • (2024)Wearable Sensor Systems to Detect Biomarkers of Personality Traits for Healthy Aging: A ReviewIEEE Sensors Journal10.1109/JSEN.2024.342935724:17(27061-27075)Online publication date: 1-Sep-2024
  • (2024)Hyper-personalized employment in urban hubs: multimodal fusion architectures for personality-based job matchingNeural Computing and Applications10.1007/s00521-024-10587-3Online publication date: 4-Dec-2024
  • (2023)Estimating and Visualizing Persuasiveness of Participants in Group DiscussionsJournal of Information Processing10.2197/ipsjjip.31.3431(34-44)Online publication date: 2023
  • (2023)EMP: Emotion-guided Multi-modal Fusion and Contrastive Learning for Personality Traits RecognitionProceedings of the 2023 ACM International Conference on Multimedia Retrieval10.1145/3591106.3592243(243-252)Online publication date: 12-Jun-2023
  • (2023)The Art of Inquiry: Toward Robots that Infer Speech and Movement Characteristics2023 11th International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW)10.1109/ACIIW59127.2023.10388090(1-8)Online publication date: 10-Sep-2023
  • (2023)Personality trait estimation in group discussions using multimodal analysis and speaker embeddingJournal on Multimodal User Interfaces10.1007/s12193-023-00401-017:2(47-63)Online publication date: 8-Feb-2023
  • (2022)Automatic Personality Assessment through Movement AnalysisSensors10.3390/s2210394922:10(3949)Online publication date: 23-May-2022
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media