Selecting parameters of SVM using meta-learning and kernel matrix-based meta-features

ABSTRACT

The Support Vector Machine (SVM) algorithm is sensitive to the choice of parameter settings, which makes it hard to use by non-experts. It has been shown that meta-learning can be used to support the selection of SVM parameter values. Previous approaches have used general statistical measures as meta-features. Here we propose a new set of meta-features that are based on the kernel matrix. We test them on the problem of setting the width of the Gaussian kernel for regression problems. We obtain significant improvements in comparison to earlier meta-learning results. We expect that with better support in the selection of parameter values, SVM becomes accessible to a wider range of users.

REFERENCES

• 1: Pavel B. Brazdil , Carlos Soares , Joaquim Pinto Da Costa, Ranking Learning Algorithms: Using IBL and Meta-Learning on Accuracy and Time Results, Machine Learning, v.50 n.3, p.251-277, March 2003 [doi>10.1023/A:1021713901879 ]
• 2: Christopher J. C. Burges, A Tutorial on Support Vector Machines for Pattern Recognition, Data Mining and Knowledge Discovery, v.2 n.2, p.121-167, June 1998 [doi>10.1023/A:1009715923555 ]
• 3: Olivier Chapelle , Vladimir Vapnik , Olivier Bousquet , Sayan Mukherjee, Choosing Multiple Parameters for Support Vector Machines, Machine Learning, v.46 n.1-3, p.131-159, 2002
• 4: Ronan Collobert , Samy Bengio, SVMTorch: support vector machines for large-scale regression problems, The Journal of Machine Learning Research, 1, p.143-160, 9/1/2001
• 5: Nello Cristianini , Colin Campbell , John Shawe-Taylor, Dynamically adapting kernels in support vector machines, Proceedings of the 1998 conference on Advances in neural information processing systems II, p.204-210, July 1999
• 6: N. Cristianini, J. Shawe-Taylor, A. Elisseeff, and J. Kandola. On kernel-target alignment. In T. G. Dietterich, S. Becker, and Z. Ghahramani, editors, Advances in Neural Information Processing Systems 14, Cambridge, MA, 2002. MIT Press.
• 7: R. Ihaka and R. Gentleman. R: A language for data analysis and graphics. J. of Computational and Graphical Statistics, 5(3):299--314, 1996.
• 8: Tommi Jaakkola , Mark Diekhans , David Haussler, Using the Fisher Kernel Method to Detect Remote Protein Homologies, Proceedings of the Seventh International Conference on Intelligent Systems for Molecular Biology, p.149-158, August 06-10, 1999
• 9: J. Kandola and J. Shawe-Taylor. Refining kernels for regression and uneven classification problems. In C. M. Bishop and B. J. Frey, editors, Proc. of the Ninth Int. Workshop on Artificial Intelligence and Statistics, 2003.
• 10: METAL Project. Esprit project METAL (#26.357). http://www.metal-kdd.org, 2002.
• 11: K.-R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Schölkopf. An introduction to kernel-based learning algorithms. IEEE Transactions on Neural Networks, 12(2):181--201, 2001.
• 12: H. Neave and P. Worthington. Distribution-Free Tests. Routledge, 1992.
• 13: J. Petrak. MLEE: Machine Learning Experimentation Environment. Austrian Institute for Artificial Intelligence, 2002. http://www.ai.univie.ac.at/~johann/mlee.html.
• 14: Alexander J. Smola , Bernhard Schölkopf, From regularization operators to support vector kernels, Proceedings of the 1997 conference on Advances in neural information processing systems 10, p.343-349, July 1998, Denver, Colorado, United States
• 15: Carlos Soares , Pavel B. Brazdil , Petr Kuba, A Meta-Learning Method to Select the Kernel Width in Support Vector Regression, Machine Learning, v.54 n.3, p.195-209, March 2004 [doi>10.1023/B:MACH.0000015879.28004.9b]

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE
      I.2.6 Learning

Additional Classification:
  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.2 Design Methodology
          Subjects: Classifier design and evaluation

Keywords:
meta-learning, parameter selection, support vector machines