skip to main content
10.1145/2733373.2806328acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
short-paper

Emotion Distribution Recognition from Facial Expressions

Published: 13 October 2015 Publication History

Abstract

Most existing facial expression recognition methods assume the availability of a single emotion for each expression in the training set. However, in practical applications, an expression rarely expresses pure emotion, but often a mixture of different emotions. To address this problem, this paper deals with a more common case where multiple emotions are associated to each expression. The key idea is to learn the specific description degrees of all basic emotions for each expression and the mapping from the expression images to the emotion distributions by the proposed emotion distribution learning (EDL) method.The databases used in the experiments are the s-JAFFE database and the s-BU\_3DFE database as they are the databases with explicit scores for each emotion on each expression image. Experimental results show that EDL can effectively deal with the emotion distribution recognition problem and perform remarkably better than the state-of-the-art multi-label learning methods.

References

[1]
T. Ahonen, A. Hadid, and M. Pietikainen. Face description with local binary patterns: Application to face recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12):2037--2041, 2006.
[2]
A. L. Berger, V. J. D. Pietra, and S. A. D. Pietra. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39--71, 1996.
[3]
P. Ekman and W. V. Friesen. Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2):124, 1971.
[4]
X. Geng and R. Ji. Label distribution learning. In Proceedings of the 13th IEEE International Conference on Data Mining Workshops, pages 377--383, 2013.
[5]
X. Geng, C. Yin, and Z.-H. Zhou. Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(10):2401--2412, 2013.
[6]
S.-J. Huang and Z.-H. Zhou. Multi-label learning by exploiting label correlations locally. In Proceedings of the 26th AAAI Conference on Artificial Intelligence, pages 949--955, Toronto, Canada, 2012.
[7]
D. C. Liu and J. Nocedal. On the limited memory bfgs method for large scale optimization. Mathematical Programming, 45(1--3):503--528, 1989.
[8]
P. Liu, S. Han, Z. Meng, and Y. Tong. Facial expression recognition via a boosted deep belief network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1805--1812, Columbus, OH, 2014.
[9]
M. J. Lyons, S. Akamatsu, M. Kamachi, J. Gyoba, and J. Budynek. The japanese female facial expression (JAFFE) database. http://www.kasrl.org/jaffe.html, 1998.
[10]
V. J. Mistry and M. M. Goyani. A literature survey on facial expression recognition using global features. International Journal of Engineering and Advanced Technology, 2:653--657, 2013.
[11]
R. Plutchik. A general psychoevolutionary theory of emotion. Theories of emotion, 1, 1980.
[12]
J. Read, B. Pfahringer, G. Holmes, and E. Frank. Classifier chains for multi-label classification. Machine Learning, 85(3):333--359, 2011.
[13]
O. Rudovic, V. Pavlovic, and M. Pantic. Multi-output laplacian dynamic ordinal regression for facial expression recognition and intensity estimation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2634--2641, 2012.
[14]
C. Song, W. Liu, and Y. Wang. Facial expression recognition based on hessian regularized support vector machine. In Proceedings of the Fifth International Conference on Internet Multimedia Computing and Service, pages 264--267. ACM, 2013.
[15]
T.-H. Wang and J.-J. J. Lien. Facial expression recognition system based on rigid and non-rigid motion separation and 3d pose estimation. Pattern Recognition, 42(5):962--977, 2009.
[16]
L. Yin, X. Wei, Y. Sun, J. Wang, and M. J. Rosato. A 3d facial expression database for facial behavior research. In 7th IEEE international conference on automatic face and gesture recognition, pages 211--216, 2006.
[17]
M.-L. Zhang. Ml-rbf: Rbf neural networks for multi-label learning. Neural Processing Letters, 29(2):61--74, 2009.
[18]
M.-L. Zhang. Lift: Multi-label learning with label-specific features. In Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pages 1609--1614, Barcelona, Spain, 2011.
[19]
M.-L. Zhang and Z.-H. Zhou. Multilabel neural networks with applications to functional genomics and text categorization. IEEE Transactions on Knowledge and Data Engineering, 18(10):1338--1351, 2006.
[20]
M.-L. Zhang and Z.-H. Zhou. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering, 26(8):1819--1837, 2014.

Cited By

View all
  • (2025)Label Distribution Learning by Partitioning Label Distribution ManifoldIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.334180736:2(3786-3796)Online publication date: Feb-2025
  • (2025)Learning Fuzzy Label-Distribution-Specific Features for Data ProcessingIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.341914433:1(365-376)Online publication date: 1-Jan-2025
  • (2025)Jointly Learning From Unimodal and Multimodal-Rated Labels in Audio-Visual Emotion RecognitionIEEE Open Journal of Signal Processing10.1109/OJSP.2025.35302746(165-174)Online publication date: 2025
  • Show More Cited By

Index Terms

  1. Emotion Distribution Recognition from Facial Expressions

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '15: Proceedings of the 23rd ACM international conference on Multimedia
    October 2015
    1402 pages
    ISBN:9781450334594
    DOI:10.1145/2733373
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 October 2015

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. description degree
    2. emotion distribution learning
    3. facial expression recognition

    Qualifiers

    • Short-paper

    Funding Sources

    • Jiangsu Natural Science Funds for Distinguished Young Scholar
    • National Natural Science Foundation of China
    • Natural Science Foundation of Jiangsu Province of China

    Conference

    MM '15
    Sponsor:
    MM '15: ACM Multimedia Conference
    October 26 - 30, 2015
    Brisbane, Australia

    Acceptance Rates

    MM '15 Paper Acceptance Rate 56 of 252 submissions, 22%;
    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)52
    • Downloads (Last 6 weeks)9
    Reflects downloads up to 19 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Label Distribution Learning by Partitioning Label Distribution ManifoldIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.334180736:2(3786-3796)Online publication date: Feb-2025
    • (2025)Learning Fuzzy Label-Distribution-Specific Features for Data ProcessingIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.341914433:1(365-376)Online publication date: 1-Jan-2025
    • (2025)Jointly Learning From Unimodal and Multimodal-Rated Labels in Audio-Visual Emotion RecognitionIEEE Open Journal of Signal Processing10.1109/OJSP.2025.35302746(165-174)Online publication date: 2025
    • (2025)Self‐learning weight network based on label distribution training for facial expression recognitionIET Image Processing10.1049/ipr2.1332619:1Online publication date: 15-Jan-2025
    • (2025)Hierarchical feature selection via joint local label enhancement and neighborhood label distribution correlationKnowledge-Based Systems10.1016/j.knosys.2025.113123311(113123)Online publication date: Feb-2025
    • (2024)PortraitEmotion3D: A Novel Dataset and 3D Emotion Estimation Method for Artistic Portraiture AnalysisApplied Sciences10.3390/app14231123514:23(11235)Online publication date: 2-Dec-2024
    • (2024)Exploiting multi-label correlation in label distribution learningProceedings of the Thirty-Third International Joint Conference on Artificial Intelligence10.24963/ijcai.2024/478(4326-4334)Online publication date: 3-Aug-2024
    • (2024)Label distribution learning for compound facial expression recognition in‐the‐wild: A comparative studyExpert Systems10.1111/exsy.13724Online publication date: 10-Sep-2024
    • (2024)MIDAS: Mixing Ambiguous Data with Soft Labels for Dynamic Facial Expression Recognition2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV57701.2024.00642(6538-6548)Online publication date: 3-Jan-2024
    • (2024)Adaptive Weighted Ranking-Oriented Label Distribution LearningIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2023.325897635:8(11302-11316)Online publication date: Aug-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media