Collaborative ordinal regression

ABSTRACT

Ordinal regression has become an effective way of learning user preferences, but most research focuses on single regression problems. In this paper we introduce collaborative ordinal regression, where multiple ordinal regression tasks are handled simultaneously. Rather than modeling each task individually, we explore the dependency between ranking functions through a hierarchical Bayesian model and assign a common Gaussian Process (GP) prior to all individual functions. Empirical studies show that our collaborative model outperforms the individual counterpart in preference learning applications.

REFERENCES

• 1: Chris Burges , Tal Shaked , Erin Renshaw , Ari Lazier , Matt Deeds , Nicole Hamilton , Greg Hullender, Learning to rank using gradient descent, Proceedings of the 22nd international conference on Machine learning, p.89-96, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102363]
• 2: Caruana, R., Baluja, S., & Mitchell, T. (1996). Using the future to "sort out" the present: Rankprop and multitask learning for medical risk evaluation. NIPS'8.
• 3: Wei Chu , Zoubin Ghahramani, Gaussian Processes for Ordinal Regression, The Journal of Machine Learning Research, 6, p.1019-1041, 9/1/2005
• 4: Wei Chu , Zoubin Ghahramani, Preference learning with Gaussian processes, Proceedings of the 22nd international conference on Machine learning, p.137-144, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102369]
• 5: Cohen, W. W., Schapire, R. E., & Singer, Y. (1999). Learning to order things. Journal of Artificial Intelligence Research, 10, 243--270.
• 6: Crammer, K., & Singer, Y. (2002). Pranking with ranking. NIPS'14.
• 7: Herbrich, R. (2005). On Gaussian expectation propagation. http://research.microsoft.com/~rherb/papers/EP.pdf.
• 8: Herbrich, R., Graepel, T., & Obermayer, K. (2000). Large margin rank boundaries for ordinal regression. Advances in Large Margin Classifiers (pp. 115--132). MIT Press.
• 9: Kalervo Järvelin , Jaana Kekäläinen, IR evaluation methods for retrieving highly relevant documents, Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, p.41-48, July 24-28, 2000, Athens, Greece [doi>10.1145/345508.345545]
• 10: Kim, H., & Ghahramani, Z. (2003). The EM-EP algorithm for Gaussian process classification. Proceedings of the Workshop on Probabilistic Graphical Models for Classification at ECML.
• 11: McCullagh, P. (1980). Regression models for ordinal data. Journal of the Royal Statistical Society B, 42, 109--142.
• 12: Thomas P. Minka , Rosalind Picard, A family of algorithms for approximate bayesian inference, 2001
• 13: Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian processes for machine learning. MIT Press.
• 14: Jasson D. M. Rennie , Nathan Srebro, Fast maximum margin matrix factorization for collaborative prediction, Proceedings of the 22nd international conference on Machine learning, p.713-719, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102441]
• 15: Schwaighofer, A., Tresp, V., & Yu, K. (2005). Hierarchical bayesian modelling with gaussian processes. NIPS'17.
• 16: Seeger, M. (2002). Notes on Minka's expectation propagation for Gaussian process classification (Technical Report). University of Edinburgh.
• 17: Kai Yu , Volker Tresp , Anton Schwaighofer, Learning Gaussian processes from multiple tasks, Proceedings of the 22nd international conference on Machine learning, p.1012-1019, August 07-11, 2005, Bonn, Germany [doi>10.1145/1102351.1102479]

INDEX TERMS

Primary Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Correlation and regression analysis

Additional Classification:
  G. Mathematics of Computing
  G.3 PROBABILITY AND STATISTICS
      Subjects: Statistical computing

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Statistical

General Terms:
Theory