skip to main content
10.1145/1526709.1526738acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

Large scale multi-label classification via metalabeler

Published: 20 April 2009 Publication History

Abstract

The explosion of online content has made the management of such content non-trivial. Web-related tasks such as web page categorization, news filtering, query categorization, tag recommendation, etc. often involve the construction of multi-label categorization systems on a large scale. Existing multi-label classification methods either do not scale or have unsatisfactory performance. In this work, we propose MetaLabeler to automatically determine the relevant set of labels for each instance without intensive human involvement or expensive cross-validation. Extensive experiments conducted on benchmark data show that the MetaLabeler tends to outperform existing methods. Moreover, MetaLabeler scales to millions of multi-labeled instances and can be deployed easily. This enables us to apply the MetaLabeler to a large scale query categorization problem in Yahoo!, yielding a significant improvement in performance.

References

[1]
L. Cai and T. Hofmann. Hierarchical document categorization with support vector machines. In CIKM '04: Proceedings of the thirteenth ACM international conference on Information and knowledge management, pages 78--87, New York, NY, USA, 2004.
[2]
R. Caruana and A. Niculescu-Mizil. An empirical comparison of supervised learning algorithms. In ICML '06: Proceedings of the 23rd international conference on Machine learning, pages 161--168, New York, NY, USA, 2006. ACM.
[3]
S. Chakrabarti, B. Dom, R. Agrawal, and P. Raghavan. Scalable feature selection, classification and signature generation for organizing large text databases into hierarchical topic taxonomies. The VLDB Journal, 7(3):163--178, 1998.
[4]
S. Dumais and H. Chen. Hierarchical classification of web content. In SIGIR '00: Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, pages 256--263, New York, NY, USA, 2000.
[5]
S. Dzeroski and B. Zenko. Is combining classifiers with stacking better than selecting the best one? Mach. Learn., 54(3):255--273, 2004.
[6]
R.-E. Fan, K.-W. Chang, C.-J. Hsieh, X.-R. Wang, and C.-J. Lin. LIBLINEAR: A library for large linear classification. Journal of Machine Learning Research, 9:1871--1874, 2008.
[7]
R.-E. Fan and C.-J. Lin. A study on threshold selection for multi-label classication. 2007.
[8]
N. Ghamrawi and A. McCallum. Collective multi-label classification. In CIKM '05: Proceedings of the 14th ACM international conference on Information and knowledge management, pages 195--200, New York, NY, USA, 2005. ACM Press.
[9]
N. Japkowicz and S. Stephen. The class imbalance problem: A systematic study. Intell. Data Anal., 6(5):429--449, 2002.
[10]
S. Ji, L. Tang, S. Yu, and J. Ye. Extracting shared subspace for multi-label classification. In KDD ’08: Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 381--389, New York, NY, USA, 2008. ACM.
[11]
T. Joachims. Text categorization with support vector machines: learning with many relevant features. In C. Nédellec and C. Rouveirol, editors, Proceedings of ECML-98, 10th European Conference on Machine Learning, pages 137--142, Heidelberg et al., 1998.
[12]
I. Katakis, G. Tsoumakas, and I. Vlahavas. Multilabel text classification for automated tag suggestion. In Proceedings of the ECML/PKDD 2008 Discovery Challenge, 2008.
[13]
S. S. Keerthi, S. Sundararajan, K.-W. Chang, C.-J. Hsieh, and C.-J. Lin. A sequential dual method for large scale multi-class linear svms. In KDD, pages 408--416, 2008.
[14]
D. Koller and M. Sahami. Hierarchically classifying documents using very few words. In ICML '97: Proceedings of the Fourteenth International Conference on Machine Learning, pages 170--178, San Francisco, CA, USA, 1997. Morgan Kaufmann Publishers Inc.
[15]
D. D. Lewis, Y. Yang, T. G. Rose, and F. Li. Rcv1: A new benchmark collection for text categorization research. J. Mach. Learn. Res., 5:361--397, 2004.
[16]
T.-Y. Liu, Y. Yang, H. Wan, H.-J. Zeng, Z. Chen, and W.-Y. Ma. Support vector machines classification with a very large-scale taxonomy. SIGKDD Explor. Newsl., 7(1):36--43, 2005.
[17]
T.-Y. Liu, Y. Yang, H. Wan, Q. Zhou, B. Gao, H.-J. Zeng, Z. Chen, and W.-Y. Ma. An experimental study on large-scale web categorization. In WWW ’05: Special interest tracks and posters of the 14th international conference on World Wide Web, pages 1106--1107, New York, NY, USA, 2005. ACM.
[18]
A. McCallum, R. Rosenfeld, T. M. Mitchell, and A. Y. Ng. Improving text classification by shrinkage in a hierarchy of classes. In ICML '98: Proceedings of the Fifteenth International Conference on Machine Learning, pages 359--367, San Francisco, CA, USA, 1998. Morgan Kaufmann Publishers Inc.
[19]
K. Punera and J. Ghosh. Enhanced hierarchical classification via isotonic smoothing. In WWW '08: Proceeding of the 17th international conference on World Wide Web, pages 151--160, 2008.
[20]
R. Rifkin and A. Klautau. In defense of one-vs-all classification. JMLR, 5:101--141, 2004.
[21]
J. Rousu, C. Saunders, S. Szedmak, and J. Shawe-Taylor. Kernel-based learning of hierarchical multilabel classification models. J. Mach. Learn. Res., 7:1601--1626, 2006.
[22]
L. Tang and H. Liu. Bias analysis in text classification for highly skewed data. In ICDM '05: Proceedings of the Fifth IEEE International Conference on Data Mining, pages 781--784, 2005. IEEE Computer Society.
[23]
L. Tang, H. Liu, J. Zhang, N. Agarwal, and J. J. Salerno. Topic taxonomy adaptation for group profiling. ACM Trans. Knowl. Discov. Data, 1(4):1--28, 2008.
[24]
L. Tang, J. Zhang, and H. Liu. Acclimatizing taxonomic semantics for hierarchical content classification. In KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 384--393, 2006.
[25]
I. Tsochantaridis, T. Hofmann, T. Joachims, and Y. Altun. Support vector machine learning for interdependent and structured output spaces. In ICML '04: Proceedings of the twenty-first international conference on Machine learning, page 104, New York, NY, USA, 2004. ACM.
[26]
G. Tsoumakas and K. Ioannis. Multi label classification: An overview. International Journal of Data Warehousing and Mining, 3:1--13, 2007.
[27]
G. Tsoumakas and K. Ioannis. Random k-labelsets: An ensemble method for multilabel classification. In ECML, 2007.
[28]
N. Ueda and K. Saito. Parametric mixture models for multi-labeled text. In NIPS, pages 721--728, 2002.
[29]
R. Yan, J. Tesic, and J. R. Smith. Model-shared subspace boosting for multi-label classification. In KDD '07: Proceedings of the 13th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 834--843, 2007.
[30]
Y. Yang. A study of thresholding strategies for text categorization. In SIGIR '01: Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval, pages 137--145, New York, NY, USA, 2001. ACM.
[31]
K. Yu, S. Yu, and V. Tresp. Multi-label informed latent semantic indexing. In SIGIR '05: Proceedings of the 28th annual international ACM SIGIR conference on Research and development in information retrieval, pages 258--265, New York, NY, USA, 2005. ACM.
[32]
B. Zadrozny and C. Elkan. Transforming classifier scores into accurate multiclass probability estimates. In KDD '02: Proceedings of the eighth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 694--699, 2002.
[33]
M.-L. Zhang and Z.-H. Zhou. Ml-knn: A lazy learning approach to multi-label learning. Pattern Recogn., 40(7):2038--2048, 2007.
[34]
S. Zhu, X. Ji, W. Xu, and Y. Gong. Multi-labelled classification using maximum entropy method. In SIGIR, 2005.

Cited By

View all
  • (2025)Revisiting multi-dimensional classification from a dimension-wise perspectiveFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-023-3272-919:1Online publication date: 1-Jan-2025
  • (2024)EvoImp: Multiple Imputation of Multi-label Classification data with a genetic algorithmPLOS ONE10.1371/journal.pone.029714719:1(e0297147)Online publication date: 19-Jan-2024
  • (2024)Multi-View Multi-Label Fine-Grained Emotion Decoding From Human Brain ActivityIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.321776735:7(9026-9040)Online publication date: Jul-2024
  • Show More Cited By

Index Terms

  1. Large scale multi-label classification via metalabeler

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WWW '09: Proceedings of the 18th international conference on World wide web
    April 2009
    1280 pages
    ISBN:9781605584874
    DOI:10.1145/1526709

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 20 April 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. hierarchical classification
    2. large scale
    3. meta model
    4. metalabeler
    5. multi-label classification
    6. query categorization

    Qualifiers

    • Research-article

    Conference

    WWW '09
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)36
    • Downloads (Last 6 weeks)2
    Reflects downloads up to 22 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Revisiting multi-dimensional classification from a dimension-wise perspectiveFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-023-3272-919:1Online publication date: 1-Jan-2025
    • (2024)EvoImp: Multiple Imputation of Multi-label Classification data with a genetic algorithmPLOS ONE10.1371/journal.pone.029714719:1(e0297147)Online publication date: 19-Jan-2024
    • (2024)Multi-View Multi-Label Fine-Grained Emotion Decoding From Human Brain ActivityIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.321776735:7(9026-9040)Online publication date: Jul-2024
    • (2024)Bottleneck Identification in Cloudified Mobile Networks based on Distributed TelemetryIEEE Transactions on Mobile Computing10.1109/TMC.2023.3312051(1-18)Online publication date: 2024
    • (2024)Incomplete multi-view partial multi-label classification via deep semantic structure preservationComplex & Intelligent Systems10.1007/s40747-024-01562-510:6(7661-7679)Online publication date: 27-Jul-2024
    • (2023)Joint optimization of scoring and thresholding models for online multi-label classificationPattern Recognition10.1016/j.patcog.2022.109167136:COnline publication date: 1-Apr-2023
    • (2023)Local-based k values for multi-label k-nearest neighbors ruleEngineering Applications of Artificial Intelligence10.1016/j.engappai.2022.105487116:COnline publication date: 20-Jan-2023
    • (2023)Multi-Label Ranking: Mining Multi-Label and Label Ranking DataMachine Learning for Data Science Handbook10.1007/978-3-031-24628-9_23(511-535)Online publication date: 26-Feb-2023
    • (2022)Self-Supervised Multi-Label Transformation Prediction for Video Representation LearningJournal of Circuits, Systems and Computers10.1142/S021812662250159631:09Online publication date: 18-Feb-2022
    • (2022)Regularized Matrix Factorization for Multilabel Learning With Missing LabelsIEEE Transactions on Cybernetics10.1109/TCYB.2020.301689752:5(3710-3721)Online publication date: May-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media