Combining predictions for accurate recommender systems

ABSTRACT

We analyze the application of ensemble learning to recommender systems on the Netflix Prize dataset. For our analysis we use a set of diverse state-of-the-art collaborative filtering (CF) algorithms, which include: SVD, Neighborhood Based Approaches, Restricted Boltzmann Machine, Asymmetric Factor Model and Global Effects. We show that linearly combining (blending) a set of CF algorithms increases the accuracy and outperforms any single CF algorithm. Furthermore, we show how to use ensemble methods for blending predictors in order to outperform a single blending algorithm. The dataset and the source code for the ensemble blending are available online.

References

1. R. M. Bell and Y. Koren. Scalable collaborative filtering with jointly derived neighborhood interpolation weights. In ICDM '07: Proceedings of the 2007 Seventh IEEE International Conference on Data Mining, pages 43--52, Washington, DC, USA, 2007. IEEE Computer Society.
2. J. Bennet and S. Lanning. The netflix prize. KDD Cup workshop, 2007. "http://www.netflixprize.com".
3. L. Breiman. Bagging predictors. In Machine Learning, pages 123--140, 1996.
4. L. Breiman. Random forests. Machine Learning, 45:5--32, 2001.
5. R. Caruana, A. Niculescu-Mizil, G. Crew, and A. Ksikes. Ensemble selection from libraries of models. In In Proceedings of the 21st International Conference on Machine Learning, pages 137--144. ACM Press, 2004.
6. D. A. Davis, N. V. Chawla, N. A. Christakis, and A.-L. Barabási. Time to CARE: a collaborative engine for practical disease prediction. Springer, November 2009.
7. J. Friedman. Stochastic gradient boosting. Computational Statistics and Data Analysis, 2002.
8. P. Geurts, D. Ernst, and L. Wehenkel. Extremely randomized trees. Mach. Learn., 63(1):3--42, 2006.
9. M. Jahrer. ELF - Ensemble Learning Framework. An open source C++ framework for supervised learning. http://elf-project.sourceforge.net, 2010.
10. Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In KDD '08: Proceeding of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 426--434. ACM, 2008.
11. Y. Koren. The BellKor solution to the Netflix Grand Prize, 2009.
12. Y. Koren. Collaborative filtering with temporal dynamics. In KDD '09: Proceeding of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 2009.
13. Y. Koren. Factor in the neighbors: Scalable and accurate collaborative filtering. In KDD: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, 2009.
14. A. Paterek. Improving regularized singular value decomposition for collaborative filtering. Proceedings of KDD Cup and Workshop, 2007.
15. M. Piotte and M. Chabbert. The Pragmatic theory solution to the Netflix Grand Prize, 2009.
16. R. Salakhutdinov, A. Mnih, and G. E. Hinton. Restricted boltzmann machines for collaborative filtering. In ICML, pages 791--798, 2007.
17. J. Sill, G. Takacs, L. Mackey, and D. Lin. Feature-weighted linear stacking. arXiv:0911.0460v2, 2009.
18. G. Takács, I. Pilászy, B. Németh, and D. Tikk. Matrix factorization and neighbor based algorithms for the netflix prize problem. In RecSys '08: Proceedings of the 2008 ACM conference on Recommender systems, pages 267--274. ACM, 2008.
19. A. Töscher and M. Jahrer. The BigChaos solution to the Netflix Prize 2008. Technical report, commendo research & consulting, October 2008.
20. A. Töscher, M. Jahrer, and R. M. Bell. The BigChaos solution to the Netflix Grand Prize, 2009.
21. A. Töscher, M. Jahrer, and R. Legenstein. Improved neighborhood-based algorithms for large-scale recommender systems. In KDD Workshop at SIGKDD 08, August 2008.

Index Terms

• Information systems

  • Information systems applications

    • Data mining