Kai Yu
ABSTRACT
In this paper we propose a non-greedy active learning method for text categorization using least-squares support vector machines (LSSVM). Our work is based on transductive experimental design (TED), an active learning formulation that effectively explores the information of unlabeled data. Despite its appealing properties, the optimization problem is however NP-hard and thus--like most of other active learning methods--a greedy sequential strategy to select one data example after another was suggested to find a suboptimum. In this paper we formulate the problem into a continuous optimization problem and prove its convexity, meaning that a set of data examples can be selected with a guarantee of global optimum. We also develop an iterative algorithm to efficiently solve the optimization problem, which turns out to be very easy-to-implement. Our text categorization experiments on two text corpora empirically demonstrated that the new active learning algorithm outperforms the sequential greedy algorithm, and is promising for active text categorization applications.
- A. C. Atkinson and A. N. Donev. Optimum experiment designs. Oxford Statistical Science Series. Oxford University Press, 1992.Google Scholar
- O. Chapelle. Active learning for Parzen window classifier. In Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics, pages 49--56, 2005.Google Scholar
- D. Cohn and Z. Ghahramani. Active learning with statistical models. Journal of Arti¯cial Intelligence Research, 4:129--145, 1996. Google ScholarDigital Library
- D. Donoho. For most large underdetermined systems of linear equations, the minimal l1-norm solution is also the sparsest solution. Communications on Pure and Applied Mathematics, 59(6), 2006.Google Scholar
- Y. Freund, H. S. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Machine Learning, 28(2-3):133--168, 1997. Google ScholarDigital Library
- C. Guestrin, A. Krause, and A. Singh. Near-optimal sensor placements in gaussian processes. In Proc. of the International Conference on Machine Learning (ICML), 2005. Google ScholarDigital Library
- X. He, W. Min, D. Cai, and K. Zhou. Laplacian optimal design for image retrieval. In ACM SIGIR Conference, 2007. Google ScholarDigital Library
- S. C. H. Hoi, R. Jin, J. Zhu, and M. R. Lyu. Batch mode active learning and its application to medical image classi¯cation. In International Conference on Machine Learning (ICML), 2006. Google ScholarDigital Library
- D. D. Lewis, Y. Yang, T. Rose, and F. Li. RCV1: A new benchmark collection for text categorization research. Journal of Machine Learning Research, 2005. Google ScholarDigital Library
- D. MacKay. Information-based objective functions for active data selection. Neural Computation, 4(4):590--604, 1992. Google ScholarDigital Library
- D. MacKay. Information-based objective functions for active data selection. Neural Computation, 4(4):590--604, 1992. Google ScholarDigital Library
- A. Schein and L. Ungar. Optimality for active learning of logistic regression classi¯ers. Technical Report Technical Report MS-CIS-04-07, The University of Pennsylvania, Department of Computer and Information Science, 2004.Google Scholar
- G. Schohn and D. Cohn. Less is more: Active learning with support vector machines. In International Conference on Machine Learning, 2000. Google ScholarDigital Library
- J. Suykens and J. Vandewalle. Least squares support vector machine classifiers. Neural Processing Letters, 1999. Google ScholarDigital Library
- R. Tibshirani. Regression shrinkage and selection via the lasso. J. Royal. Statist. Soc B, 58(1), 1996.Google Scholar
- S. Tong and D. Koller. Support vector machine active learning with applications to text classification. Journal of Machine Learning Research, 2, 2001. Google ScholarDigital Library
- K. Yu, J. Bi, and V. Tresp. Active learning via transductive experimental design. In International Conference on Machine Learning (ICML), 2006. Google ScholarDigital Library
- J. Zhang and Y. Yang. Robustness of regularized linear classifcation methods in text categorization. In The 26th Annual International SIGIR Conference (SIGIR'99), 2003. Google ScholarDigital Library
- T. Zhang and F. J. Oles. Text categorization based on regularized linear classi¯cation methods. Information Retrieval, (4):5--31, 2001. Google ScholarDigital Library
- W. V. Zhang, X. He, B. Rey, and R. Jones. Query rewritting using active learning for sponsored search. In ACM SIGIR Conference, 2007. Google ScholarDigital Library
Index Terms
- trNon-greedy active learning for text categorization using convex ansductive experimental design
Recommendations
Large-scale text categorization by batch mode active learning
WWW '06: Proceedings of the 15th international conference on World Wide WebLarge-scale text categorization is an important research topic for Web data mining. One of the challenges in large-scale text categorization is how to reduce the human efforts in labeling text documents for building reliable classification models. In ...
A Novel Active Learning Method Using SVM for Text Classification
Support vector machines (SVMs) are a popular class of supervised learning algorithms, and are particularly applicable to large and high-dimensional classification problems. Like most machine learning methods for data classification and information ...
Batch Mode Active Learning with Applications to Text Categorization and Image Retrieval
Most machine learning tasks in data classification and information retrieval require manually labeled data examples in the training stage. The goal of active learning is to select the most informative examples for manual labeling in these learning ...
Comments