skip to main content
10.1145/1571941.1571979acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
research-article

Fast nonparametric matrix factorization for large-scale collaborative filtering

Published: 19 July 2009 Publication History

Abstract

With the sheer growth of online user data, it becomes challenging to develop preference learning algorithms that are sufficiently flexible in modeling but also affordable in computation. In this paper we develop nonparametric matrix factorization methods by allowing the latent factors of two low-rank matrix factorization methods, the singular value decomposition (SVD) and probabilistic principal component analysis (pPCA), to be data-driven, with the dimensionality increasing with data size. We show that the formulations of the two nonparametric models are very similar, and their optimizations share similar procedures. Compared to traditional parametric low-rank methods, nonparametric models are appealing for their flexibility in modeling complex data dependencies. However, this modeling advantage comes at a computational price--it is highly challenging to scale them to large-scale problems, hampering their application to applications such as collaborative filtering. In this paper we introduce novel optimization algorithms, which are simple to implement, which allow learning both nonparametric matrix factorization models to be highly efficient on large-scale problems. Our experiments on EachMovie and Netflix, the two largest public benchmarks to date, demonstrate that the nonparametric models make more accurate predictions of user ratings, and are computationally comparable or sometimes even faster in training, in comparison with previous state-of-the-art parametric matrix factorization models.

References

[1]
J. Abernethy, F. Bach, T. Evgeniou, and J.-P. Vert. Low-rank matrix factorization with attributes. Technical report, Ecole des Mines de Paris, 2006.
[2]
R. M. Bell, Y. Koren, and C. Volinsky. The BellKor solution to the Netflix prize. Technical report, AT&T Labs, 2007.
[3]
E. J. Cand`es and T. Tao. The power of convex relaxation: Near-optimal matrix completion. Submitted for publication, 2009.
[4]
D. DeCoste. Collaborative prediction using ensembles of maximum margin matrix factorization. In The 23rd International Conference on Machine Learning (ICML), 2006.
[5]
M. Kurucz, A. A. Benczur, and K. Csalogany. Methods for large scale SVD with missing values. In Proceedings of KDD Cup and Workshop, 2007.
[6]
Y. J. Lim and Y. W. Teh. Variational Bayesian approach to movie rating prediction. In Proceedings of KDD Cup and Workshop, 2007.
[7]
C. E. Rasmussen and C. K. I. Williams. Gaussian Processes for Machine Learning. The MIT Press, 2006.
[8]
J. D. M. Rennie and N. Srebro. Fast maximum margin matrix factorization for collaborative prediction. In The 22nd International Conference on Machine Learning (ICML), 2005.
[9]
S. Roweis and Z. Ghahramani. A unifying review of linear Gaussian models. Neural Computation, 11:305--345, 1999.
[10]
R. Salakhutdinov and A. Mnih. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In The 25th International Conference on Machine Learning (ICML), 2008.
[11]
B. Schölkopf and A. J. Smola. Learning with Kernels. MIT Press, 2002.
[12]
N. Srebro, J. D. M. Rennie, and T. S. Jaakola. Maximum-margin matrix factorization. In Advances in Neural Information Processing Systems 18 (NIPS), 2005.
[13]
G. Takacs, I. Pilaszy, B. Nemeth, and D. Tikk. On the gravity recommendation system. In Proceedings of KDD Cup and Workshop, 2007.
[14]
M. E. Tipping and C. M. Bishop. Probabilistic principal component analysis. Journal of the Royal Statisitical Scoiety, B(61):611--622, 1999.
[15]
M. Wu. Collaborative filtering via ensembles of matrix factorizations. In Proceedings of KDD Cup and Workshop, 2007.
[16]
K. Yu, J. Lafferty, S. Zhu, and Y. Gong. Large-scale collaborative prediction using a nonparametric random effects model. In The 25th International Conference on Machine Learning (ICML), 2009.
[17]
K. Yu and V. Tresp. Learning to learn and collaborative filtering. In NIPS workshop on "Inductive Transfer: 10 Years Later", 2005.
[18]
Y. Zhang and J. Koren. Efficient Bayesian hierarchical user modeling for recommendation systems. In The 30th ACM SIGIR Conference, 2007.

Cited By

View all
  • (2024)Pseudo Gradient-Adjusted Particle Swarm Optimization for Accurate Adaptive Latent Factor AnalysisIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2023.334091954:4(2213-2226)Online publication date: Apr-2024
  • (2023)Diversified Regularization Enhanced Training for Effective Manipulator CalibrationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.315303934:11(8778-8790)Online publication date: Nov-2023
  • (2023)Charging Time Prediction of Electric Vehicle Charging Pile via Momentum-incorporated Non-negative Latent-factorization-of-tensors with Swish-regularization2023 IEEE 6th International Conference on Pattern Recognition and Artificial Intelligence (PRAI)10.1109/PRAI59366.2023.10332010(1097-1103)Online publication date: 18-Aug-2023
  • Show More Cited By

Index Terms

  1. Fast nonparametric matrix factorization for large-scale collaborative filtering

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
      July 2009
      896 pages
      ISBN:9781605584836
      DOI:10.1145/1571941
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 19 July 2009

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. collaborative filtering
      2. matrix factorization
      3. nonparametric models

      Qualifiers

      • Research-article

      Conference

      SIGIR '09
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 792 of 3,983 submissions, 20%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)17
      • Downloads (Last 6 weeks)2
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)Pseudo Gradient-Adjusted Particle Swarm Optimization for Accurate Adaptive Latent Factor AnalysisIEEE Transactions on Systems, Man, and Cybernetics: Systems10.1109/TSMC.2023.334091954:4(2213-2226)Online publication date: Apr-2024
      • (2023)Diversified Regularization Enhanced Training for Effective Manipulator CalibrationIEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2022.315303934:11(8778-8790)Online publication date: Nov-2023
      • (2023)Charging Time Prediction of Electric Vehicle Charging Pile via Momentum-incorporated Non-negative Latent-factorization-of-tensors with Swish-regularization2023 IEEE 6th International Conference on Pattern Recognition and Artificial Intelligence (PRAI)10.1109/PRAI59366.2023.10332010(1097-1103)Online publication date: 18-Aug-2023
      • (2023)A Well-Designed Regularization Scheme for Latent Factorization of High-Dimensional and Incomplete Water-Quality Tensors from Sensor Networks2023 19th International Conference on Mobility, Sensing and Networking (MSN)10.1109/MSN60784.2023.00089(596-603)Online publication date: 14-Dec-2023
      • (2023)Multiple Biases-Incorporated Latent Factorization of TensorsDynamic Network Representation Based on Latent Factorization of Tensors10.1007/978-981-19-8934-6_2(11-26)Online publication date: 8-Mar-2023
      • (2022)TD-DNN: A Time Decay-Based Deep Neural Network for Recommendation SystemApplied Sciences10.3390/app1213639812:13(6398)Online publication date: 23-Jun-2022
      • (2022)Advancing Non-Negative Latent Factorization of Tensors With Diversified Regularization SchemesIEEE Transactions on Services Computing10.1109/TSC.2020.298876015:3(1334-1344)Online publication date: 1-May-2022
      • (2022) An L 1 -and- L 2 -Norm-Oriented Latent Factor Model for Recommender Systems IEEE Transactions on Neural Networks and Learning Systems10.1109/TNNLS.2021.307139233:10(5775-5788)Online publication date: Oct-2022
      • (2022)Modeling Dynamic User Preference via Dictionary Learning for Sequential RecommendationIEEE Transactions on Knowledge and Data Engineering10.1109/TKDE.2021.305040734:11(5446-5458)Online publication date: 1-Nov-2022
      • (2022)IntroductionLatent Factor Analysis for High-dimensional and Sparse Matrices10.1007/978-981-19-6703-0_1(1-10)Online publication date: 16-Nov-2022
      • Show More Cited By

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media