skip to main content
10.1145/2959100.2959153acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
research-article
Public Access

Crowd-Based Personalized Natural Language Explanations for Recommendations

Published: 07 September 2016 Publication History

Abstract

Explanations are important for users to make decisions on whether to take recommendations. However, algorithm generated explanations can be overly simplistic and unconvincing. We believe that humans can overcome these limitations. Inspired by how people explain word-of-mouth recommendations, we designed a process, combining crowdsourcing and computation, that generates personalized natural language explanations. We modeled key topical aspects of movies, asked crowdworkers to write explanations based on quotes from online movie reviews, and personalized the explanations presented to users based on their rating history. We evaluated the explanations by surveying 220 MovieLens users, finding that compared to personalized tag-based explanations, natural language explanations: 1) contain a more appropriate amount of information, 2) earn more trust from users, and 3) make users more satisfied. This paper contributes to the research literature by describing a scalable process for generating high quality and personalized natural language explanations, improving on state-of-the-art content-based explanations, and showing the feasibility and advantages of approaches that combine human wisdom with algorithmic processes.

Supplementary Material

MP4 File (p175.mp4)

References

[1]
S. Bedathur, K. Berberich, J. Dittrich, N. Mamoulis, and G. Weikum. Interesting-phrase mining for ad-hoc text analytics. Proceedings of the VLDB Endowment, 3(1--2):1348--1357, 2010.
[2]
M. S. Bernstein, G. Little, R. C. Miller, B. Hartmann, M. S. Ackerman, D. R. Karger, D. Crowell, and K. Panovich. Soylent: a word processor with a crowd inside. In UIST, 2010.
[3]
S. Bostandjiev, J. O'Donovan, and T. Höllerer. TasteWeights: a visual interactive hybrid recommender system. In RecSys, 2012.
[4]
S. Chang, P. Dai, J. Chen, and E. H. Chi. Got Many Labels?: Deriving Topic Labels from Multiple Sources for Social Media Posts using Crowdsourcing and Ensemble Learning. In WWW, 2015.
[5]
S. Chang, P. Dai, L. Hong, S. Cheng, Z. Tianjiao, and C. Ed. AppGrouper: Knowledge-based Interactive Clustering Tool for App Search Results. In IUI, 2016.
[6]
S. Chang, M. F. Harper, L. He, and L. G. Terveen. CrowdLens: Experimenting with Crowd-Powered Recommendation and Explanation . In ICWSM, 2016.
[7]
J. Cheng and M. S. Bernstein. Flock: Hybrid crowd-machine learning classifiers. In CSCW, 2015.
[8]
A. El-Kishky, Y. Song, C. Wang, C. R. Voss, and J. Han. Scalable topical phrase mining from text corpora. Proceedings of the VLDB Endowment, 8(3):305--316, 2014.
[9]
B. J. Frey and D. Dueck. Clustering by passing messages between data points. Science, 315:972--976, 2007.
[10]
F. M. Harper, D. Raban, S. Rafaeli, and J. A. Konstan. Predictors of answer quality in online Q&A sites. In CHI, 2008.
[11]
J. L. Herlocker, J. A. Konstan, and J. Riedl. Explaining collaborative filtering recommendations. In CSCW, 2000.
[12]
A. Jameson, M. C. Willemsen, A. Felfernig, M. Gemmis, P. Lops, G. Semeraro, and L. Chen. Recommender Systems Handbook, chapter Human Decision Making and Recommender Systems, pages 611--648. Springer US, Boston, MA, 2015.
[13]
A. Kittur, B. Smus, S. Khamkar, and R. Kraut. Crowdforge: Crowdsourcing complex work. UIST, 2011.
[14]
J. Liu, J. Shang, C. Wang, X. Ren, and J. Han. Mining quality phrases from massive text corpora. 2015.
[15]
S. M. McNee, J. Riedl, and J. A. Konstan. Being accurate is not enough. In CHI EA, 2006.
[16]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111--3119, 2013.
[17]
P. Organisciak, J. Teevan, S. Dumais, R. C. Miller, and A. T. Kalai. A Crowd of Your Own: Crowdsourcing for On-Demand Personalization. In HCOMP, 2014.
[18]
R. Sinha and K. Swearingen. The role of transparency in recommender systems. In CHI EA, 2002.
[19]
C. A. Thompson, M. H. Göker, and P. Langley. A personalized system for conversational recommendations. Journal of Artificial Intelligence Research, 21(1):393--428, Mar. 2004.
[20]
N. Tintarev and J. Masthoff. Evaluating the effectiveness of explanations for recommender systems. User Modeling and User-Adapted Interaction, 22:399--439, 2012.
[21]
N. Tintarev and J. Masthoff. Recommender Systems Handbook, chapter Explaining Recommendations: Design and Evaluation, pages 353--382. Springer US, Boston, MA, 2015.
[22]
J. Vig, S. Sen, and J. Riedl. Tagsplanations. In IUI, 2008.
[23]
J. Vig, S. Sen, and J. Riedl. The Tag Genome. ACM Trans. on Interactive Intelligent Systems, 2(3):1--44, sep 2012.
[24]
W. Wang and I. Benbasat. Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. J. Manage. Inf. Syst., 23(4):217--246, May 2007.
[25]
S.-H. Yang, A. Kolcz, A. Schlaikjer, and P. Gupta. Large-scale high-precision topic modeling on twitter. In KDD, Aug 2014.

Cited By

View all
  • (2024)Improving Faithfulness and Factuality with Contrastive Learning in Explainable RecommendationACM Transactions on Intelligent Systems and Technology10.1145/365398416:1(1-23)Online publication date: 26-Dec-2024
  • (2024)Beyond Relevance: Factor-level Causal Explanation for User Travel Decisions with Counterfactual Data AugmentationACM Transactions on Information Systems10.1145/365367342:5(1-31)Online publication date: 29-Apr-2024
  • (2024)Leveraging ChatGPT for Automated Human-centered Explanations in Recommender SystemsProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645171(597-608)Online publication date: 18-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
RecSys '16: Proceedings of the 10th ACM Conference on Recommender Systems
September 2016
490 pages
ISBN:9781450340359
DOI:10.1145/2959100
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 September 2016

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. clustering
  2. crowdsourcing
  3. natural language processing
  4. recommendation explanations
  5. word2vec

Qualifiers

  • Research-article

Funding Sources

Conference

RecSys '16
Sponsor:
RecSys '16: Tenth ACM Conference on Recommender Systems
September 15 - 19, 2016
Massachusetts, Boston, USA

Acceptance Rates

RecSys '16 Paper Acceptance Rate 29 of 159 submissions, 18%;
Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)186
  • Downloads (Last 6 weeks)26
Reflects downloads up to 09 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Improving Faithfulness and Factuality with Contrastive Learning in Explainable RecommendationACM Transactions on Intelligent Systems and Technology10.1145/365398416:1(1-23)Online publication date: 26-Dec-2024
  • (2024)Beyond Relevance: Factor-level Causal Explanation for User Travel Decisions with Counterfactual Data AugmentationACM Transactions on Information Systems10.1145/365367342:5(1-31)Online publication date: 29-Apr-2024
  • (2024)Leveraging ChatGPT for Automated Human-centered Explanations in Recommender SystemsProceedings of the 29th International Conference on Intelligent User Interfaces10.1145/3640543.3645171(597-608)Online publication date: 18-Mar-2024
  • (2024)Less is More: Towards Sustainability-Aware Persuasive Explanations in Recommender SystemsProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3691708(1108-1112)Online publication date: 8-Oct-2024
  • (2024)Finding Paths for Explainable MOOC Recommendation: A Learner PerspectiveProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636898(426-437)Online publication date: 18-Mar-2024
  • (2024)On the Negative Perception of Cross-domain Recommendations and ExplanationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657735(2102-2113)Online publication date: 10-Jul-2024
  • (2024)Bring Privacy To The Table: Interactive Negotiation for Privacy Settings of Shared Sensing DevicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642897(1-22)Online publication date: 11-May-2024
  • (2024)Predicting and Presenting Task Difficulty for Crowdsourcing Food Rescue PlatformsProceedings of the ACM Web Conference 202410.1145/3589334.3648155(4686-4696)Online publication date: 13-May-2024
  • (2024)Evaluating Trust in Recommender Systems: A User Study on the Impacts of Explanations, Agency Attribution, and Product TypesInternational Journal of Human–Computer Interaction10.1080/10447318.2024.2313921(1-13)Online publication date: 14-Feb-2024
  • (2023)Leveraging Large Language Models for Goal-driven Interactive RecommendationsProceedings of the 11th International Conference on Human-Agent Interaction10.1145/3623809.3623965(464-466)Online publication date: 4-Dec-2023
  • Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media