A Sparse Topic Model for Extracting Aspect-Specific Summaries from Online Reviews

ABSTRACT

Online reviews have become an inevitable part of a consumer's decision making process, where the likelihood of purchase not only depends on the product's overall rating, but also on the description of its aspects. Therefore, e-commerce websites such as Amazon and Walmart constantly encourage users to write good quality re- views and categorically summarize different facets of the products. However, despite such attempts, it takes a significant effort to skim through thousands of reviews and look for answers that address the query of consumers. For example, a gamer might be interested in buying a monitor with fast refresh rates and support for Gsync and Freesync technologies, while a photographer might be interested in aspects such as color depth and accuracy. To address these chal- lenges, in this paper, we propose a generative aspect summarization model called APSUM that is capable of providing fine-grained sum- maries of online reviews. To overcome the inherent problem of aspect sparsity, we impose dual constraints: (a) a spike-and-slab prior over the document-topic distribution and (b) a linguistic su- pervision over the word-topic distribution. Using a rigorous set of experiments, we show that the proposed model is capable of out- performing the state-of-the-art aspect summarization model over a variety of datasets and deliver intuitive fine-grained summaries that could simplify the purchase decisions of consumers.

References

1. Sasha Blair-Goldensohn, Kerry Hannan, Ryan McDonald, Tyler Neylon, George A Reis, and Jeff Reynar. 2008. Building a sentiment summarizer for local service reviews. In WWW workshop on NLP in the information explosion era, Vol. 14. 339--348.
2. David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. Journal of machine Learning research 3, Jan (2003), 993--1022.
3. Samuel Brody and Noemie Elhadad. 2010. An unsupervised aspectsentiment model for online reviews. In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 804--812.
4. Jonathan Chang, Sean Gerrish, Chong Wang, Jordan L Boyd-Graber, and David M Blei. 2009. Reading tea leaves: How humans interpret topic models. In Advances in neural information processing systems. 288--296.
5. Zhiyuan Chen and Bing Liu. 2014. Topic Modeling using Topics from Many Domains, Lifelong Learning and Big Data. In Proceedings of the 31st International Conference on Machine Learning (ICML-14). 703--711.
6. Orphée De Clercq, Marjan Van de Kauter, Els Lefever, and Véronique Hoste. 2015. Applying hybrid terminology extraction to aspect-based sentiment analysis. In International Workshop on Semantic Evaluation (SemEval 2015). Association for Computational Linguistics, 719--724.
7. Marie-Catherine De Marneffe and Christopher D Manning. 2008. Stanford typed dependencies manual. Technical Report. Technical report, Stanford University.
8. Qiming Diao, Minghui Qiu, Chao-Yuan Wu, Alexander J Smola, Jing Jiang, and Chong Wang. 2014. Jointly modeling aspects, ratings and sentiments for movie recommendation (jmars). In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 193--202.
9. Aitor Garcia-Pablos, Montse Cuadros, and German Rigau. 2015. V3: unsupervised aspect based sentiment analysis for SemEval-2015 Task 12. SemEval-2015 (2015), 714.
10. Zhen Hai, Gao Cong, Kuiyu Chang, Peng Cheng, and Chunyan Miao. 2017. Analyzing Sentiments in One Go: A Supervised Joint Topic Modeling Approach. IEEE Transactions on Knowledge and Data Engineering 29, 6 (2017), 1172--1185.
11. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 168--177.
12. Yohan Jo and Alice H Oh. 2011. Aspect and sentiment unification model for online review analysis. In Proceedings of the fourth ACM international conference on Web search and data mining. ACM, 815-- 824.
13. Tianyi Lin, Wentao Tian, Qiaozhu Mei, and Hong Cheng. 2014. The dual-sparse topic model: mining focused topics and focused terms in short text. In Proceedings of the 23rd international conference on World wide web. ACM, 539--550.
14. Qian Liu, Bing Liu, Yuanlin Zhang, Doo Soon Kim, and Zhiqiang Gao. 2016. Improving Opinion Aspect Extraction Using Semantic Similarity and Aspect Associations.. In AAAI. 2986--2992.
15. David Mimno, Hanna M Wallach, Edmund Talley, Miriam Leenders, and Andrew McCallum. 2011. Optimizing semantic coherence in topic models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 262-- 272.
16. Samaneh Moghaddam and Martin Ester. 2012. On the design of LDA models for aspect-based opinion mining. In Proceedings of the 21st ACM international conference on Information and knowledge management. ACM, 803--812.
17. David Newman, Youn Noh, Edmund Talley, Sarvnaz Karimi, and Timothy Baldwin. 2010. Evaluating topic models for digital libraries. In Proceedings of the 10th annual joint conference on Digital libraries. ACM, 215--224.
18. James Petterson, Wray Buntine, Shravan M Narayanamurthy, Tibério S Caetano, and Alex J Smola. 2010. Word features for latent dirichlet allocation. In Advances in Neural Information Processing Systems. 1921-- 1929.
19. Ana-Maria Popescu and Orena Etzioni. 2007. Extracting product features and opinions from reviews. In Natural language processing and text mining. Springer, 9--28.
20. Soujanya Poria, Erik Cambria, Lun-Wei Ku, Chen Gui, and Alexander Gelbukh. 2014. A rule-based approach to aspect extraction from product reviews. In Proceedings of the second workshop on natural language processing for social media (SocialNLP). 28--37.
21. Md Mustafizur Rahman and Hongning Wang. 2016. Hidden topic sentiment model. In Proceedings of the 25th International Conference on World Wide Web. 155--165.
22. José Saias. 2015. Sentiue: Target and aspect based sentiment analysis in semeval-2015 task 12. Association for Computational Linguistics.
23. Kim Schouten and Flavius Frasincar. 2016. Survey on aspect-level sentiment analysis. IEEE Transactions on Knowledge and Data Engineering 28, 3 (2016), 813--830.
24. SEMVAL. {n. d.}. International Workshop on Semantic Evaluation. http://alt.qcri.org/semeval2016/
25. Ivan Titov and Ryan McDonald. 2008. Modeling online reviews with multi-grain topic models. In Proceedings of the 17th international conference on World Wide Web. ACM, 111--120.
26. Chong Wang and David M Blei. 2009. Decoupling sparsity and smoothness in the discrete hierarchical dirichlet process. In Advances in neural information processing systems. 1982--1989.
27. Shuai Wang, Zhiyuan Chen, Geli Fei, Bing Liu, and Sherry Emery. 2016. Targeted Topic Modeling for Focused Analysis. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. ACM.
28. Shuai Wang, Zhiyuan Chen, and Bing Liu. 2016. Mining AspectSpecific Opinion using a Holistic Lifelong Topic Model. In Proceedings of the 25th International Conference on World Wide Web. 167--176.
29. Yao Wu and Martin Ester. 2015. Flame: A probabilistic model combining aspect based opinion mining and collaborative filtering. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining. ACM, 199--208.
30. Zaihan Yang, Alexander Kotov, Aravind Mohan, and Shiyong Lu. 2015. Parametric and non-parametric user-aware sentiment topic models. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 413--422.
31. Lei Zhang and Bing Liu. 2014. Aspect and entity extraction for opinion mining. In Data mining and knowledge discovery for big data. Springer, 1--40.
32. Wayne Xin Zhao, Jing Jiang, Hongfei Yan, and Xiaoming Li. 2010. Jointly modeling aspects and opinions with a MaxEnt-LDA hybrid. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 56--65.
33. Yuan Zuo, Junjie Wu, Hui Zhang, Deqing Wang, Hao Lin, Fei Wang, and Ke Xu. 2015. Complementary Aspect-based Opinion Mining Across Asymmetric Collections. In Data Mining (ICDM), 2015 IEEE International Conference on. IEEE, 669--678.

Index Terms

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Unsupervised learning

        • Topic modeling

• Information systems

  • Information retrieval

    • Document representation

      • Document topic models

    • Retrieval tasks and goals

      • Summarization

  • Information systems applications

    • Data mining