skip to main content
10.1145/1014052.1014089acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
Article

Incorporating prior knowledge with weighted margin support vector machines

Published: 22 August 2004 Publication History

Abstract

Like many purely data-driven machine learning methods, Support Vector Machine (SVM) classifiers are learned exclusively from the evidence presented in the training dataset; thus a larger training dataset is required for better performance. In some applications, there might be human knowledge available that, in principle, could compensate for the lack of data. In this paper, we propose a simple generalization of SVM: Weighted Margin SVM (WMSVMs) that permits the incorporation of prior knowledge. We show that Sequential Minimal Optimization can be used in training WMSVM. We discuss the issues of incorporating prior knowledge using this rather general formulation. The experimental results show that the proposed methods of incorporating prior knowledge is effective.

References

[1]
K. Bennett and A. Demiriz. Semi-supervised support vector machines. In Advances in Neural Information Processing Systems 11, 1998.
[2]
C. Chang and C. Lin. LIBSVM: a library for support vector machines (version 2.3), 2001.
[3]
G. Fung and O. Mangasarian. Semi-supervised support vector machines for unlabeled data classification. Optimization Methods and Software, 15, 2001.
[4]
G. Fung, O. L. Mangasarian, and J. Shavlik. Knowledge-based support vector machine classifiers. In Data Mining Institute Technical Report 01-09, Nov 2001.
[5]
G. H. Golub and C. F. V. Loan. Matrix Computation. Johns Hopkins Univ Press, 1996.
[6]
W. R. Hersh, C. Buckley, T. J. Leone, and D. H. Hickam. Ohsumed: An interactive retrieval evaluation and new large test collection for research, 1994.
[7]
T. Joachims. Text categorization with support vector machines: learning with many relevant features. In C. Nedellec and C. Rouveirol, editors, Proceedings of ECML-98, 10th European Conference on Machine Learning, number 1398, pages 137--142, Chemnitz, DE, 1998. Springer Verlag, Heidelberg, DE.
[8]
T. Joachims. Transductive inference for text classification using support vector machines. In Proc. 16th International Conf. on Machine Learning, pages 200--209. Morgan Kaufmann, San Francisco, CA, 1999.
[9]
T. Joachims. Learning To Classify Text Using Support Vector Machines. Kluwer Academic Publishers, Boston, 2002.
[10]
S. Keerthi, S. Shevade, C. Bhattacharyya, and K. Murthy. Improvements to platt's smo algorithm for svm classifier design, 1999.
[11]
W. Lam and C. Ho. Using a generalized instance set for automatic text categorization. In W. B. Croft, A. Moffat, C. J. van Rijsbergen, R. Wilkinson, and J. Zobel, editors, Proceedings of SIGIR-98, 21st ACM International Conference on Research and Development in Information Retrieval, pages 81--89, Melbourne, AU, 1998. ACM Press, New York, US.
[12]
J. Platt. Fast training of support vector machines using sequential minimal optimization. In B. Scholkopf, C. Burges, and A. Smola, editors, Advances in kernel methods - support vector learning. MIT Press, 1998.
[13]
R. Schapire, M. Rochery, M. Rahim, and N. Gupta. Incorporating prior knowledge into boosting. In Proceedings of the Nineteenth International Conference In Machine Learning, 2002.
[14]
B. Scholkopf, P. Simard, A. Smola, and V. Vapnik. Prior knowledge in support vector kernels. In B. Scholkopf, C. Burges, and A. Smola, editors, Advances in kernel methods - support vector learning. MIT Press, 1998.
[15]
S. Tong and D. Koller. Support vector machine active learning with applications to text classification. In P. Langley, editor, Proceedings of ICML-00, 17th International Conference on Machine Learning, pages 999--1006, Stanford, US, 2000. Morgan Kaufmann Publishers, San Francisco, US.
[16]
V. N. Vapnik. Statistical learning theory. John Wiley & Sons, New York, NY, 1998.
[17]
V. N. Vapnik. The nature of statistical learning theory, 2nd Edition. Springer Verlag, Heidelberg, DE, 1999.
[18]
Y. Yang and X. Liu. A re-examination of text categorization methods. In M. A. Hearst, F. Gey, and R. Tong, editors, Proceedings of SIGIR-99, 22nd ACM International Conference on Research and Development in Information Retrieval, pages 42--49, Berkeley, US, 1999. ACM Press, New York, US.
[19]
J. Zhang and Y. Yang. Robustness of regularized linear classification methods in text categorization. In Proceedings of SIGIR-2003, 26st ACM International Conference on Research and Development in Information Retrieval. ACM Press, 2003.

Cited By

View all
  • (2023)LIMEADE: From AI Explanations to Advice TakingACM Transactions on Interactive Intelligent Systems10.1145/358934513:4(1-29)Online publication date: 28-Mar-2023
  • (2022)Support Vector Machine-Based Soft Decision for Consecutive-Symbol-Expanded 4-Dimensional Constellation in Underwater Visible Light Communication SystemPhotonics10.3390/photonics91108049:11(804)Online publication date: 26-Oct-2022
  • (2022)Signal timing parameters inference method at intersections using license plate recognition dataIET Intelligent Transport Systems10.1049/itr2.1219816:8(1092-1107)Online publication date: 3-May-2022
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
KDD '04: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining
August 2004
874 pages
ISBN:1581138881
DOI:10.1145/1014052
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 August 2004

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. incorporating prior knowledge
  2. support vector machines
  3. text categorization

Qualifiers

  • Article

Conference

KDD04

Acceptance Rates

Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

Upcoming Conference

KDD '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)9
  • Downloads (Last 6 weeks)0
Reflects downloads up to 09 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2023)LIMEADE: From AI Explanations to Advice TakingACM Transactions on Interactive Intelligent Systems10.1145/358934513:4(1-29)Online publication date: 28-Mar-2023
  • (2022)Support Vector Machine-Based Soft Decision for Consecutive-Symbol-Expanded 4-Dimensional Constellation in Underwater Visible Light Communication SystemPhotonics10.3390/photonics91108049:11(804)Online publication date: 26-Oct-2022
  • (2022)Signal timing parameters inference method at intersections using license plate recognition dataIET Intelligent Transport Systems10.1049/itr2.1219816:8(1092-1107)Online publication date: 3-May-2022
  • (2020)Link-based traffic state estimation and prediction for arterial networks using license-plate recognition dataTransportation Research Part C: Emerging Technologies10.1016/j.trc.2020.102660117(102660)Online publication date: Aug-2020
  • (2020)Fault diagnosis of the train communication network based on weighted support vector machineIEEJ Transactions on Electrical and Electronic Engineering10.1002/tee.2315315:7(1077-1088)Online publication date: 27-May-2020
  • (2019)The OCS-SVM: An Objective-Cost-Sensitive SVM With Sample-Based Misclassification Cost InvarianceIEEE Access10.1109/ACCESS.2019.29334377(118931-118942)Online publication date: 2019
  • (2019)Feature weighted confidence to incorporate prior knowledge into support vector machines for classificationKnowledge and Information Systems10.1007/s10115-018-1165-258:2(371-397)Online publication date: 1-Feb-2019
  • (2018)Deep Belief Network Based Hybrid Model for Building Energy Consumption PredictionEnergies10.3390/en1101024211:1(242)Online publication date: 19-Jan-2018
  • (2018)Topics and Label Propagation: Best of Both Worlds for Weakly Supervised Text ClassificationComputational Linguistics and Intelligent Text Processing10.1007/978-3-319-75487-1_35(446-459)Online publication date: 21-Mar-2018
  • (2017)The folded k-spectrum kernel: A machine learning approach to detecting transcription factor binding sites with gapped nucleotide dependenciesPLOS ONE10.1371/journal.pone.018557012:10(e0185570)Online publication date: 5-Oct-2017
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media