skip to main content
10.1145/2668024.2668028acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
research-article

Automatic Recognition of Personality Traits: A Multimodal Approach

Published: 12 November 2014 Publication History

Abstract

A system being capable of recognize personality traits may be utilized in an enormous number of applications. Adding personality-dependency may be useful to build speaker-adaptive models, e.g., to improve Spoken Dialogue Systems (SDSs) or to monitor agents in call-centers. Therefore, the First Audio/Visual Mapping Personality Traits Challenge (MAPTRAITS 2014) focuses on estimating personality traits. In this context, this study presents the results for multimodal recognition of personality traits using support vector machines. As only small portions of the data is used for personality estimation at a time (which are later combined to a final estimate), different segmentation methods (and how to derive a final hypothesis) are analyzed regarding the task as both a regression and a classification problem.

References

[1]
T. Ahonen, A. Hadid, and M. Pietikainen. Face description with local binary patterns: Application to face recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 28(12):2037--2041, 2006.
[2]
T. R. Almaev and M. F. Valstar. Local gabor binary patterns from three orthogonal planes for automatic facial expression recognition. In Affective Computing and Intelligent Interaction (ACII), 2013 Humaine Association Conference on, pages 356--361. IEEE, 2013.
[3]
C. Busso, Z. Deng, S. Yildirim, M. Bulut, C. M. Lee, A. Kazemzadeh, S. Lee, U. Neumann, and S. Narayanan. Analysis of emotion recognition using facial expressions, speech and multimodal information. In Proceedings of the 6th international conference on Multimodal interfaces, pages 205--211. ACM, 2004.
[4]
O. Celiktutan, F. Eyben, E. Sariyanidi, H. Gunes, and B. Schuller. Maptraits 2014: Introduction to the audio/visual mapping personality traits challenge. In Proceedings of the 16th ACM on International conference on multimodal interaction. ACM, 2014.
[5]
A. Dhall, A. Asthana, R. Goecke, and T. Gedeon. Emotion recognition using phog and lpq features. In Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pages 878--883. IEEE, 2011.
[6]
A. Dhall, R. Goecke, J. Joshi, M. Wagner, and T. Gedeon. Emotion recognition in the wild challenge 2013. In Proceedings of the 15th ACM on International conference on multimodal interaction, pages 509--516. ACM, 2013.
[7]
F. Eyben, M. Wöllmer, and B. Schuller. Opensmile: the munich versatile and fast open-source audio feature extractor. In Proceedings of the international conference on Multimedia, pages 1459--1462. ACM, 2010.
[8]
G. W. Flake and S. Lawrence. Efficient svm regression training with smo. Machine Learning, 46(1-3):271--290, 2002.
[9]
Y.-L. Lin and G. Wei. Speech emotion recognition based on hmm and svm. In Machine Learning and Cybernetics, 2005. Proceedings of 2005 International Conference on, volume 8, 2005.
[10]
G. McKeown, M. Valstar, R. Cowie, M. Pantic, and M. Schroder. The semaine database: Annotated multimodal records of emotionally colored conversations between a person and a limited agent. Affective Computing, IEEE Transactions on, 3(1):5--17, 2012.
[11]
R. W. Picard. Affective computing. MIT press, 2000.
[12]
E. Sánchez-Lozano, P. Lopez-Otero, L. Docio-Fernandez, E. Argones-Rúa, and J. L. Alba-Castro. Audiovisual three-level fusion for continuous estimation of russell's emotion circumplex. In Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge, pages 31--40. ACM, 2013.
[13]
E. Sariyanidi, H. Gunes, M. Gökmen, and A. Cavallaro. Local zernike moment representation for facial affect recognition. In Proceedings of the British Machine Vision Conference, pages 1--4. Springer Berlin Heidelberg, 2013.
[14]
B. Schuller, S. Steidl, and A. Batliner. The interspeech 2009 emotion challenge. In INTERSPEECH, volume 2009, pages 312--315. Citeseer, 2009.
[15]
B. Schuller, S. Steidl, A. Batliner, F. Burkhardt, L. Devillers, C. A. Müller, and S. S. Narayanan. The interspeech 2010 paralinguistic challenge. In INTERSPEECH, pages 2794--2797, 2010.
[16]
B. Schuller, S. Steidl, A. Batliner, E. Nöth, A. Vinciarelli, F. Burkhardt, R. Van Son, F. Weninger, F. Eyben, T. Bocklet, et al. The interspeech 2012 speaker trait challenge. In INTERSPEECH, 2012.
[17]
B. Schuller, S. Steidl, A. Batliner, A. Vinciarelli, K. Scherer, F. Ringeval, M. Chetouani, F. Weninger, F. Eyben, E. Marchi, et al. The interspeech 2013 computational paralinguistics challenge: social signals, conflict, emotion, autism. 2013.
[18]
B. Schuller, M. Valstar, F. Eyben, G. McKeown, R. Cowie, and M. Pantic. Avec 2011 - the first international audio/visual emotion challenge. In Affective Computing and Intelligent Interaction, pages 415--424. Springer, 2011.
[19]
B. Schuller, M. Valster, F. Eyben, R. Cowie, and M. Pantic. Avec 2012: the continuous audio/visual emotion challenge. In Proceedings of the 14th ACM international conference on Multimodal interaction, pages 449--456. ACM, 2012.
[20]
N. Sebe, I. Cohen, and T. S. Huang. Multimodal emotion recognition. Handbook of Pattern Recognition and Computer Vision, 4:387--419, 2005.
[21]
S. K. Shevade, S. S. Keerthi, C. Bhattacharyya, and K. R. K. Murthy. Improvements to the smo algorithm for svm regression. Neural Networks, IEEE Transactions on, 11(5):1188--1193, 2000.
[22]
M. Sidorov, S. Ultes, and A. Schmitt. Emotions are a personal thing: Towards speaker-adaptive emotion recognition. In IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 4836--4840, May 2014.
[23]
X. Tan and B. Triggs. Fusing gabor and lbp feature sets for kernel-based face recognition. In Analysis and Modeling of Faces and Gestures, pages 235--249. Springer, 2007.
[24]
M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic. Avec 2013: the continuous audio/visual emotion and depression recognition challenge. In Proceedings of the 3rd ACM international workshop on Audio/visual emotion challenge, pages 3--10. ACM, 2013.
[25]
C.-H. Wu and W.-B. Liang. Emotion recognition of affective speech based on multiple classifiers using acoustic-prosodic information and semantic labels. Affective Computing, IEEE Transactions on, 2(1):10--21, 2011.
[26]
C. Xu, P. Du, Z. Feng, Z. Meng, T. Cao, and C. Dong. Multi-modal emotion recognition fusing video and audio. Appl. Math, 7(2):455--462, 2013.

Cited By

View all
  • (2025)Explainable human-centered traits from head motion and facial expression dynamicsPLOS ONE10.1371/journal.pone.031388320:1(e0313883)Online publication date: 17-Jan-2025
  • (2024)Wearable Sensor Systems to Detect Biomarkers of Personality Traits for Healthy Aging: A ReviewIEEE Sensors Journal10.1109/JSEN.2024.342935724:17(27061-27075)Online publication date: 1-Sep-2024
  • (2023)Detection of mild cognitive impairment from non-semantic, acoustic voice features: the Framingham Heart Study (Preprint)JMIR Aging10.2196/55126Online publication date: 3-Dec-2023
  • Show More Cited By

Index Terms

  1. Automatic Recognition of Personality Traits: A Multimodal Approach

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MAPTRAITS '14: Proceedings of the 2014 Workshop on Mapping Personality Traits Challenge and Workshop
    November 2014
    38 pages
    ISBN:9781450339568
    DOI:10.1145/2668024
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 November 2014

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. audio-visual features
    2. feature-based fusion
    3. personality traits
    4. support vector machine

    Qualifiers

    • Research-article

    Conference

    ICMI '14
    Sponsor:

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)22
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Explainable human-centered traits from head motion and facial expression dynamicsPLOS ONE10.1371/journal.pone.031388320:1(e0313883)Online publication date: 17-Jan-2025
    • (2024)Wearable Sensor Systems to Detect Biomarkers of Personality Traits for Healthy Aging: A ReviewIEEE Sensors Journal10.1109/JSEN.2024.342935724:17(27061-27075)Online publication date: 1-Sep-2024
    • (2023)Detection of mild cognitive impairment from non-semantic, acoustic voice features: the Framingham Heart Study (Preprint)JMIR Aging10.2196/55126Online publication date: 3-Dec-2023
    • (2022)First Impressions: A Survey on Vision-Based Apparent Personality Trait AnalysisIEEE Transactions on Affective Computing10.1109/TAFFC.2019.293005813:1(75-95)Online publication date: 1-Jan-2022
    • (2020)Automatic Personality Identification Using Students’ Online Learning BehaviorIEEE Transactions on Learning Technologies10.1109/TLT.2019.292422313:1(26-37)Online publication date: 1-Jan-2020
    • (2020)Single-Modal Video Analysis of Personality Traits using Low-Level Visual Features2020 Tenth International Conference on Image Processing Theory, Tools and Applications (IPTA)10.1109/IPTA50016.2020.9286620(1-6)Online publication date: 9-Nov-2020
    • (2020)Multi-Scenario Fusion for More Accurate Classifications of Personal Characteristics2020 IEEE Intl Conf on Dependable, Autonomic and Secure Computing, Intl Conf on Pervasive Intelligence and Computing, Intl Conf on Cloud and Big Data Computing, Intl Conf on Cyber Science and Technology Congress (DASC/PiCom/CBDCom/CyberSciTech)10.1109/DASC-PICom-CBDCom-CyberSciTech49142.2020.00059(300-305)Online publication date: Aug-2020
    • (2018)Multimodal Personality Trait Analysis for Explainable Modeling of Job Interview DecisionsExplainable and Interpretable Models in Computer Vision and Machine Learning10.1007/978-3-319-98131-4_10(255-275)Online publication date: 30-Nov-2018
    • (2017)Multi-modal Score Fusion and Decision Trees for Explainable Automatic Job Candidate Screening from Video CVs2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)10.1109/CVPRW.2017.210(1651-1659)Online publication date: Jul-2017
    • (2016)Multimodal fusion of audio, scene, and face features for first impression estimation2016 23rd International Conference on Pattern Recognition (ICPR)10.1109/ICPR.2016.7899605(43-48)Online publication date: Dec-2016
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media