Collaborative Deep Learning for Recommender Systems

ABSTRACT

Collaborative filtering (CF) is a successful approach commonly used by many recommender systems. Conventional CF-based methods use the ratings given to items by users as the sole source of information for learning to make recommendation. However, the ratings are often very sparse in many applications, causing CF-based methods to degrade significantly in their recommendation performance. To address this sparsity problem, auxiliary information such as item content information may be utilized. Collaborative topic regression (CTR) is an appealing recent method taking this approach which tightly couples the two components that learn from two different sources of information. Nevertheless, the latent representation learned by CTR may not be very effective when the auxiliary information is very sparse. To address this problem, we generalize recently advances in deep learning from i.i.d. input to non-i.i.d. (CF-based) input and propose in this paper a hierarchical Bayesian model called collaborative deep learning (CDL), which jointly performs deep representation learning for the content information and collaborative filtering for the ratings (feedback) matrix. Extensive experiments on three real-world datasets from different domains show that CDL can significantly advance the state of the art.

References

1. D. Agarwal and B.-C. Chen. Regression-based latent factor models. In KDD, pages 19--28, 2009.
2. P. Baldi and P. J. Sadowski. Understanding dropout. In NIPS, pages 2814--2822, 2013.
3. Y. Bengio, L. Yao, G. Alain, and P. Vincent. Generalized denoising auto-encoders as generative models. In NIPS, pages 899--907, 2013.
4. C. M. Bishop. Pattern Recognition and Machine Learning. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2006.
5. D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent Dirichlet allocation. JMLR, 3:993--1022, 2003.
6. J. Bobadilla, F. Ortega, A. Hernando, and A. Gutiérrez. Recommender systems survey. Knowledge Based Systems, 46:109--132, 2013.
7. M. Chen, Z. E. Xu, K. Q. Weinberger, and F. Sha. Marginalized denoising autoencoders for domain adaptation. In ICML, pages 767--774, 2012.
8. T. Chen, W. Zhang, Q. Lu, K. Chen, Z. Zheng, and Y. Yu. Svdfeature: a toolkit for feature-based collaborative filtering. JMLR, 13:3619--3622, 2012.
9. K. Georgiev and P. Nakov. A non-iid framework for collaborative filtering with restricted boltzmann machines. In ICML, pages 1148--1156, 2013.
10. A. Graves, S. Fernández, F. J. Gomez, and J. Schmidhuber. Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks. In ICML, pages 369--376, 2006.
11. G. E. Hinton, N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Improving neural networks by preventing co-adaptation of feature detectors. CoRR, abs/1207.0580, 2012.
12. L. Hu, J. Cao, G. Xu, L. Cao, Z. Gu, and C. Zhu. Personalized recommendation via cross-domain triadic factorization. In WWW, pages 595--606, 2013.
13. Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In ICDM, pages 263--272, 2008.
14. M. I. Jordan, Z. Ghahramani, T. Jaakkola, and L. K. Saul. An introduction to variational methods for graphical models. Machine Learning, 37(2):183--233, 1999.
15. N. Kalchbrenner, E. Grefenstette, and P. Blunsom. A convolutional neural network for modelling sentences. ACL, pages 655--665, 2014.
16. A. Krizhevsky, I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. In NIPS, pages 1106--1114, 2012.
17. K. Lang. Newsweeder: Learning to filter netnews. In ICML, pages 331--339, 1995.
18. W.-J. Li, D.-Y. Yeung, and Z. Zhang. Generalized latent factor models for social network analysis. In IJCAI, pages 1705--1710, 2011.
19. D. J. C. MacKay. A practical Bayesian framework for backpropagation networks. Neural Computation, 4(3):448--472, 1992.
20. T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111--3119, 2013.
21. A. V. D. Oord, S. Dieleman, and B. Schrauwen. Deep content-based music recommendation. In NIPS, pages 2643--2651, 2013.
22. S. Purushotham, Y. Liu, and C.-C. J. Kuo. Collaborative topic regression with social matrix factorization for recommendation systems. In ICML, pages 759--766, 2012.
23. S. Rendle, C. Freudenthaler, Z. Gantner, and L. Schmidt-Thieme. BPR: Bayesian personalized ranking from implicit feedback. In UAI, pages 452--461, 2009.
24. T. N. Sainath, B. Kingsbury, V. Sindhwani, E. Arisoy, and B. Ramabhadran. Low-rank matrix factorization for deep neural network training with high-dimensional output targets. In ICASSP, pages 6655--6659, 2013.
25. R. Salakhutdinov and G. E. Hinton. Deep Boltzmann machines. In AISTATS, pages 448--455, 2009.
26. R. Salakhutdinov and G. E. Hinton. Semantic hashing. Int. J. Approx. Reasoning, 50(7):969--978, 2009.
27. R. Salakhutdinov and A. Mnih. Probabilistic matrix factorization. In NIPS, pages 1257--1264, 2007.
28. R. Salakhutdinov, A. Mnih, and G. E. Hinton. Restricted boltzmann machines for collaborative filtering. In ICML, pages 791--798, 2007.
29. S. G. Sevil, O. Kucuktunc, P. Duygulu, and F. Can. Automatic tag expansion using visual similarity for photo sharing websites. Multimedia Tools Appl., 49(1):81--99, 2010.
30. A. P. Singh and G. J. Gordon. Relational learning via collective matrix factorization. In KDD, pages 650--658, 2008.
31. R. S. Strichartz. A Guide to Distribution Theory and Fourier Transforms. World Scientific, 2003.
32. J. Tang, R. W. White, and P. Bailey. Recommending interesting activity-related local entities. In SIGIR, pages 1161--1162, 2011.
33. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. JMLR, 11:3371--3408, 2010.
34. S. Wager, S. Wang, and P. Liang. Dropout training as adaptive regularization. In NIPS, pages 351--359, 2013.
35. C. Wang and D. M. Blei. Collaborative topic modeling for recommending scientific articles. In KDD, pages 448--456, 2011.
36. H. Wang, B. Chen, and W.-J. Li. Collaborative topic regression with social regularization for tag recommendation. In IJCAI, pages 2719--2725, 2013.
37. H. Wang and W. Li. Relational collaborative topic regression for recommender systems. TKDE, 27(5):1343--1355, 2015.
38. H. Wang, X. Shi, and D. Yeung. Relational stacked denoising autoencoder for tag recommendation. In AAAI, pages 3052--3058, 2015.
39. N. Wang and D.-Y. Yeung. Learning a deep compact image representation for visual tracking. In NIPS, pages 809--817, 2013.
40. X. Wang and Y. Wang. Improving content-based and hybrid music recommendation using deep learning. In ACM MM, pages 627--636, 2014.
41. W. Zhang, H. Sun, X. Liu, and X. Guo. Temporal qos-aware web service recommendation via non-negative tensor factorization. In WWW, pages 585--596, 2014.
42. K. Zhou and H. Zha. Learning binary codes for collaborative filtering. In KDD, pages 498--506, 2012.

Index Terms

• Applied computing

  • Law, social and behavioral sciences

• Computing methodologies

  • Modeling and simulation

    • Simulation theory

      • Systems theory

• Mathematics of computing

  • Information theory