skip to main content
10.1145/1150402.1150426acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
Article

Learning the unified kernel machines for classification

Published: 20 August 2006 Publication History

Abstract

Kernel machines have been shown as the state-of-the-art learning techniques for classification. In this paper, we propose a novel general framework of learning the Unified Kernel Machines (UKM) from both labeled and unlabeled data. Our proposed framework integrates supervised learning, semi-supervised kernel learning, and active learning in a unified solution. In the suggested framework, we particularly focus our attention on designing a new semi-supervised kernel learning method, i.e., Spectral Kernel Learning (SKL), which is built on the principles of kernel target alignment and unsupervised kernel design. Our algorithm is related to an equivalent quadratic programming problem that can be efficiently solved. Empirical results have shown that our method is more effective and robust to learn the semi-supervised kernels than traditional approaches. Based on the framework, we present a specific paradigm of unified kernel machines with respect to Kernel Logistic Regresions (KLR), i.e., Unified Kernel Logistic Regression (UKLR). We evaluate our proposed UKLR classification scheme in comparison with traditional solutions. The promising results show that our proposed UKLR paradigm is more effective than the traditional classification approaches.

References

[1]
M. Belkin and I. M. and P. Niyogi. Regularization and semi-supervised learning on large graphs. In COLT, 2004.
[2]
M. Belkin and P. Niyogi. Semi-supervised learning on riemannian manifolds. Machine Learning, 2004.
[3]
E. Chang, S. C. Hoi, X. Wang, W.-Y. Ma, and M. Lyu. A unified machine learning framework for large-scale personalized information management. In The 5th Emerging Information Technology Conference, NTU Taipei, 2005.
[4]
E. Chang and M. Lyu. Unified learning paradigm for web-scale mining. In Snowbird Machine Learning Workshop, 2006.
[5]
O. Chapelle, A. Zien, and B. Scholkopf. Semi-supervised learning. MIT Press, 2006.
[6]
F. R. K. Chung. Spectral Graph Theory. American Mathematical Society, 1997.
[7]
D. A. Cohn, Z. Ghahramani, and M. I. Jordan. Active learning with statistical models. In NIPS, volume 7, pages 705--712, 1995.
[8]
N. Cristianini, J. Shawe-Taylor, and A. Elisseeff. On kernel-target alignment. JMLR, 2002.
[9]
S. Fine, R. Gilad-Bachrach, and E. Shamir. Query by committee, linear separation and random walks. Theor. Comput. Sci., 284(1):25--51, 2002.
[10]
Y. Freund, H. S. Seung, E. Shamir, and N. Tishby. Selective sampling using the query by committee algorithm. Mach. Learn., 28(2-3):133--168, 1997.
[11]
S. C. Hoi, R. Jin, and M. R. Lyu. Large-scale text categorization by batch mode active learning. In WWW2006, Edinburg, 2006.
[12]
S. B. C. M. J. A. K. Suykens, G. Horvath and J. Vandewalle. Advances in Learning Theory: Methods, Models and Applications. NATO Science Series: Computer & Systems Sciences, 2003.
[13]
R. Kondor and J. Lafferty. Diffusion kernels on graphs and other discrete structures. 2002.
[14]
G. Lanckriet, N. Cristianini, P. Bartlett, L. E. Ghaoui, and M. Jordan. Learning the kernel matrix with semi-definite programming. JMLR, 5:27--72, 2004.
[15]
G. Lanckriet, L. Ghaoui, C. Bhattacharyya, and M. Jordan. Minimax probability machine. In Advances in Neural Information Processing Systems 14, 2002.
[16]
R. Liere and P. Tadepalli. Active learning with committees for text categorization. In Proceedings 14th Conference of the American Association for Artificial Intelligence (AAAI), pages 591--596, MIT Press, 1997.
[17]
R. Meir and G. Ratsch. An introduction to boosting and leveraging. In In Advanced Lectures on Machine Learning (LNAI2600), 2003.
[18]
A. Ng, M. Jordan, and Y. Weiss. On spectral clustering: Analysis and an algorithm. In In Advances in Neural Information Processing Systems 14, 2001.
[19]
N. Roy and A. McCallum. Toward optimal active learning through sampling estimation of error reduction. In 18th ICML, pages 441--448, 2001.
[20]
B. Scholkopf, A. Smola, and K.-R. Muller. Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10:1299--1319, 1998.
[21]
A. Smola and R. Kondor. Kernels and regularization on graphs. In Intl. Conf. on Learning Theory, 2003.
[22]
M. Szummer and T. Jaakkola. Partially labeled classification with markov random walks. In Advances in Neural Information Processing Systems, 2001.
[23]
S. Tong and E. Chang. Support vector machine active learning for image retrieval. In Proc ACM Multimedia Conference, pages 107--118, New York, 2001.
[24]
S. Tong and D. Koller. Support vector machine active learning with applications to text classification. In Proc. 17th ICML, pages 999--1006, 2000.
[25]
V. N. Vapnik. Statistical Learning Theory. John Wiley & Sons, 1998.
[26]
G. Wu, Z. Zhang, and E. Y. Chang. Kronecker factorization for speeding up kernel machines. In SIAM Int. Conference on Data Mining (SDM), 2005.
[27]
T. Zhang and R. K. Ando. Analysis of spectral kernel design based semi-supervised learning. In NIPS, 2005.
[28]
D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Schlkopf. Learning with local and global consistency. In NIPS'16, 2005.
[29]
J. Zhu and T. Hastie. Kernel logistic regression and the import vector machine. In NIPS 14, pages 1081--1088, 2001.
[30]
X. Zhu. Semi-supervised learning literature survey. Technical report, Computer Sciences TR 1530, University of Wisconsin - Madison, 2005.
[31]
X. Zhu, Z. Ghahramani, and J. Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proc. ICML' 2003, 2003.
[32]
X. Zhu, J. Kandola, Z. Ghahramani, and J. Lafferty. Nonparametric transforms of graph kernels for semi-supervised learning. In NIPS 2005, 2005.

Cited By

View all
  • (2024)Decomposition and Symmetric Kernel Deep Neural Network Fuzzy Support Vector MachineSymmetry10.3390/sym1612158516:12(1585)Online publication date: 27-Nov-2024
  • (2022)Kernel Alignment Inspired Linear Discriminant AnalysisMachine Learning and Knowledge Discovery in Databases10.1007/978-3-662-44845-8_26(401-416)Online publication date: 10-Mar-2022
  • (2019)Large Scale Online Multiple Kernel Regression with Application to Time-Series PredictionACM Transactions on Knowledge Discovery from Data10.1145/329987513:1(1-33)Online publication date: 23-Jan-2019
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '06: Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining
August 2006
986 pages
ISBN:1595933395
DOI:10.1145/1150402
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 20 August 2006

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. active learning
  2. classification
  3. kernel logistic regressions
  4. kernel machines
  5. semi-supervised learning
  6. spectral kernel learning
  7. supervised learning
  8. unsupervised kernel design

Qualifiers

  • Article

Conference

KDD06

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)1
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Decomposition and Symmetric Kernel Deep Neural Network Fuzzy Support Vector MachineSymmetry10.3390/sym1612158516:12(1585)Online publication date: 27-Nov-2024
  • (2022)Kernel Alignment Inspired Linear Discriminant AnalysisMachine Learning and Knowledge Discovery in Databases10.1007/978-3-662-44845-8_26(401-416)Online publication date: 10-Mar-2022
  • (2019)Large Scale Online Multiple Kernel Regression with Application to Time-Series PredictionACM Transactions on Knowledge Discovery from Data10.1145/329987513:1(1-33)Online publication date: 23-Jan-2019
  • (2016)Kernel methods for word sense disambiguationArtificial Intelligence Review10.1007/s10462-015-9455-546:1(41-58)Online publication date: 1-Jun-2016
  • (2015)Domain Invariant Transfer Kernel LearningIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2014.237337627:6(1519-1532)Online publication date: 1-Jun-2015
  • (2015)An overview of kernel alignment and its applicationsArtificial Intelligence Review10.1007/s10462-012-9369-443:2(179-192)Online publication date: 1-Feb-2015
  • (2014)Extracting certainty from uncertaintyProceedings of the 28th International Conference on Neural Information Processing Systems - Volume 110.5555/2968826.2968856(262-270)Online publication date: 8-Dec-2014
  • (2014)Online multiple kernel regressionProceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining10.1145/2623330.2623712(293-302)Online publication date: 24-Aug-2014
  • (2014)Low-Density Cut Based Tree Decomposition for Large-Scale SVM ProblemsProceedings of the 2014 IEEE International Conference on Data Mining10.1109/ICDM.2014.127(839-844)Online publication date: 14-Dec-2014
  • (2014)Kernel-Distance Target AlignmentPattern Recognition10.1007/978-3-662-45646-0_11(101-110)Online publication date: 2014
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media