Paired Restricted Boltzmann Machine for Linked Data

ABSTRACT

Restricted Boltzmann Machines (RBMs) are widely adopted unsupervised representation learning methods and have powered many data mining tasks such as collaborative filtering and document representation. Recently, linked data that contains both attribute and link information has become ubiquitous in various domains. For example, social media data is inherently linked via social relations and web data is networked via hyperlinks. It is evident from recent work that link information can enhance a number of real-world applications such as clustering and recommendations. Therefore, link information has the potential to advance RBMs for better representation learning. However, the majority of existing RBMs have been designed for independent and identically distributed data and are unequipped for linked data. In this paper, we aim to design a new type of Restricted Boltzmann Machines that takes advantage of linked data. In particular, we propose a paired Restricted Boltzmann Machine (pRBM), which is able to leverage the attribute and link information of linked data for representation learning. Experimental results on real-world datasets demonstrate the effectiveness of the proposed framework pRBM.

References

1. Jonathan Chang and David M Blei. Relational topic models for document networks. In International conference on artificial intelligence and statistics, pages 81--88, 2009.
2. Shiyu Chang, Wei Han, Jiliang Tang, Guo-Jun Qi, Charu C Aggarwal, and Thomas S Huang. Heterogeneous network embedding via deep architectures. In Proceedings of KDD. ACM, 2015.
3. Asja Fischer and Christian Igel. An introduction to restricted boltzmann machines. In Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, pages 14--36. Springer, 2012.
4. Alexei Vázqueza Alessandro Flamminia, Amos Maritana, and Alessandro Vespignanib. Modeling of protein interaction networks. ComPlexUs, 1:38--44, 2003.
5. Asela Gunawardana and Christopher Meek. Tied boltzmann machines for cold start recommendations. In Proceedings of RecSys, pages 19--26. ACM, 2008.
6. Geoffrey Hinton. Training products of experts by minimizing contrastive divergence. Neural computation, 14(8):1771--1800, 2002.
7. Geoffrey E Hinton and Ruslan R Salakhutdinov. Replicated softmax: an undirected topic model. In NIPS, pages 1607--1614, 2009.
8. Xia Hu, Lei Tang, Jiliang Tang, and Huan Liu. Exploiting social relations for sentiment analysis in microblogging. In Proceedings of WSDM, pages 537--546. ACM, 2013.
9. Ian Jolliffe. Principal component analysis. Wiley Online Library, 2002.
10. Ryan Kiros, Axel J Soto, Evangelos Milios, and Vlado Keselj. Representation learning for sparse, high dimensional multi-label classification, 2012.
11. Kang Li, Jing Gao, Suxin Guo, Nan Du, Xiaoyi Li, and Aidong Zhang. Lrbm: A restricted boltzmann machine based approach for representation learning on linked data. In ICDM. IEEE, 2014.
12. Xiaoyi Li, Nan Du, Hui Li, Kang Li, Jing Gao, and Aidong Zhang. A deep learning approach to link prediction in dynamic networks. In Proceedings of SDM, pages 289--297, 2014.
13. Jon D Mcauliffe and David M Blei. Supervised topic models. In NIPS, pages 121--128, 2008.
14. Kevin P Murphy. Machine learning: a probabilistic perspective. MIT press, 2012.
15. Panagiotis Papadimitriou, Ali Dasdan, and Hector Garcia-Molina. Web graph similarity for anomaly detection. Journal of Internet Services and Applications, 1(1):19--30, 2010.
16. NhatHai Phan, Dejing Dou, Brigitte Piniewski, and David Kil. Social restricted boltzmann machine: Human behavior prediction in health social networks. In Proceedings of ASONAM. ACM, 2015.
17. Ruslan Salakhutdinov and Geoffrey E Hinton. Deep boltzmann machines. In AISTATS, pages 448--455, 2009.
18. Ruslan Salakhutdinov, Andriy Mnih, and Geoffrey Hinton. Restricted boltzmann machines for collaborative filtering. In Proceedings of ICML, pages 791--798. ACM, 2007.
19. Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93, 2008.
20. Ilya Sutskever and Tijmen Tieleman. On the convergence properties of contrastive divergence. In AISTAS, pages 789--795, 2010.
21. Jiliang Tang, Yi Chang, Aggarwal Charu, and Huan Liu. A survey of signed network mining in social media. ACM Computing Survey, 2016.
22. Jiliang Tang and Huan Liu. Feature selection with linked data in social media. In SDM, pages 118--128. SIAM, 2012.
23. Jiliang Tang and Huan Liu. Unsupervised feature selection for linked social media data. In Proceedings of KDD, pages 904--912. ACM, 2012.
24. Ben Taskar, Pieter Abbeel, and Daphne Koller. Discriminative probabilistic models for relational data. In Proceedings of UAI, pages 485--492. Morgan Kaufmann Publishers Inc., 2002.
25. Tijmen Tieleman. Training restricted boltzmann machines using approximations to the likelihood gradient. In Proceedings of ICML, pages 1064--1071. ACM, 2008.
26. Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of ICML, pages 1096--1103. ACM, 2008.
27. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. The Journal of Machine Learning Research, 11:3371--3408, 2010.
28. Suhang Wang, Jiliang Tang, Charu Aggarwal, and Huan Liu. Liked docuent embedding for classification. In Proceedings of the 25th ACM International Conference on Information and Knowledge Management. ACM, 2015.
29. Yilin Wang, Suhang Wang, Jiliang Tang, Huan Liu, and Baoxin Li. Unsupervised sentiment analysis for social media images. In IJCAI, 2015.
30. Yuhao Wang and Jianyang Zeng. Predicting drug-target interactions using restricted boltzmann machines. Bioinformatics, 29(13):i126--i134, 2013.
31. Pengtao Xie, Yuntian Deng, and Eric Xing. Diversifying restricted boltzmann machine for document modeling. In Proceedings of KDD, pages 1315--1324. ACM, 2015.
32. Cheng Yang, Zhiyuan Liu, Deli Zhao, Maosong Sun, and Edward Y Chang. Network representation learning with rich text information. In Proceedings of IJCAI, pages 2111--2117, 2015.

Index Terms

• Information systems

  • Information systems applications

    • Data mining