skip to main content
10.1145/3172944.3172959acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
research-article

Beyond the Ranked List: User-Driven Exploration and Diversification of Social Recommendation

Published: 05 March 2018 Publication History

Abstract

The beyond-relevance objectives of recommender systems have been drawing more and more attention. For example, a diversity-enhanced interface has been shown to associate positively with overall levels of user satisfaction. However, little is known about how users adopt diversity-enhanced interfaces to accomplish various real-world tasks. In this paper, we present two attempts at creating a visual diversity-enhanced interface that presents recommendations beyond a simple ranked list. Our goal was to design a recommender system interface to help users explore the different relevance prospects of recommended items in parallel and to stress their diversity. Two within-subject user studies in the context of social recommendation at academic conferences were conducted to compare our visual interfaces. Results from our user study show that the visual interfaces significantly reduced the exploration efforts required for given tasks and helped users to perceive the recommendation diversity. We show that the users examined a diverse set of recommended items while experiencing an improvement in overall user satisfaction. Also, the users» subjective evaluations show significant improvement in many user-centric metrics. Experiences are discussed that shed light on avenues for future interface designs.

References

[1]
Jisun An, Daniele Quercia, and Jon Crowcroft. 2013. Why individuals seek diverse opinions (or why they don't). In Proceedings of the 5th Annual ACM Web Science Conference. ACM, 15--18.
[2]
Dirk Bollen, Bart P Knijnenburg, Martijn C Willemsen, and Mark Graus. 2010. Understanding choice overload in recommender systems. In Proceedings of the fourth ACM conference on Recommender systems. ACM, 63--70.
[3]
Svetlin Bostandjiev, John O'Donovan, and Tobias Höllerer. 2012. TasteWeights: a visual interactive hybrid recommender system. In Proceedings of the sixth ACM conference on Recommender systems. ACM, 35--42.
[4]
Engin Bozdag and Jeroen van den Hoven. 2015. Breaking the filter bubble: democracy and design. Ethics and Information Technology 17, 4 (2015), 249--265.
[5]
Peter Brusilovsky, Jung Sun Oh, Claudia López, Denis Parra, and Wei Jeng. 2016. Linking information and people in a social system for academic conferences. New Review of Hypermedia and Multimedia (2016), 1--31.
[6]
Jilin Chen, Werner Geyer, Casey Dugan, Michael Muller, and Ido Guy. 2009. Make new friends, but keep the old: recommending people on social networking sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 201--210.
[7]
Henriette Cramer, Vanessa Evers, Satyan Ramlal, Maarten Van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction 18, 5 (2008), 455.
[8]
Paolo Cremonesi, Franca Garzotto, and Roberto Turrin. 2012. Investigating the persuasion potential of recommender systems from a quality perspective: An empirical study. ACM Transactions on Interactive Intelligent Systems (TiiS) 2, 2 (2012), 11.
[9]
Cecilia di Sciascio, Vedran Sabol, and Eduardo E Veas. 2016. Rank As You Go: User-Driven Exploration of Search Results. In Proceedings of the 21st International Conference on Intelligent User Interfaces. ACM, 118--129.
[10]
Michael D Ekstrand, Daniel Kluver, F Maxwell Harper, and Joseph A Konstan. 2015. Letting users choose recommender algorithms: An experimental study. In Proceedings of the 9th ACM Conference on Recommender Systems. ACM, 11--18.
[11]
Siamak Faridani, Ephrat Bitton, Kimiko Ryokai, and Ken Goldberg. 2010. Opinion space: a scalable tool for browsing online comments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1175--1184.
[12]
Dorota Glowacka, Tuukka Ruotsalo, Ksenia Konuyshkova, Samuel Kaski, Giulio Jacucci, and others. 2013. Directing exploratory search: Reinforcement learning from user interactions with keywords. In Proceedings of the 2013 international conference on Intelligent user interfaces. ACM, 117--128.
[13]
Brynjar Gretarsson, John O'Donovan, Svetlin Bostandjiev, Christopher Hall, and Tobias Höllerer. 2010. Smallworlds: visualizing social recommendations. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 833--842.
[14]
Jonathan L Herlocker, Joseph A Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM conference on Computer supported cooperative work. ACM, 241--250.
[15]
Rong Hu and Pearl Pu. 2011. Helping Users Perceive Recommendation Diversity. In DiveRS@ RecSys. 43--50.
[16]
Marius Kaminskas and Derek Bridge. 2016. Diversity, Serendipity, Novelty, and Coverage: A Survey and Empirical Analysis of Beyond-Accuracy Objectives in Recommender Systems. ACM Transactions on Interactive Intelligent Systems (TiiS) 7, 1 (2016), 2.
[17]
Hannah Kim, Jaegul Choo, Haesun Park, and Alex Endert. 2016. Interaxis: Steering scatterplot axes via observation-level interaction. IEEE transactions on visualization and computer graphics 22, 1 (2016), 131--140.
[18]
Khalil Klouche, Tuukka Ruotsalo, Diogo Cabral, Salvatore Andolina, Andrea Bellucci, and Giulio Jacucci. 2015. Designing for exploratory search on touch devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 4189--4198.
[19]
Khalil Klouche, Tuukka Ruotsalo, Luana Micallef, Salvatore Andolina, and Giulio Jacucci. 2017. Visual Re-Ranking for Multi-Aspect Information Retrieval. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. ACM, 57--66.
[20]
Bart P. Knijnenburg, Svetlin Bostandjiev, John O'Donovan, and Alfred Kobsa. 2012. Inspectability and Control in Social Recommenders. In 6th ACM Conference on Recommender System. 43--50.
[21]
Q Vera Liao and Wai-Tat Fu. 2013. Beyond the filter bubble: interactive effects of perceived threat and topic involvement on selective exposure to information. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 2359--2368.
[22]
Jennifer Moody and David H Glass. 2016. A Novel Classification Framework for Evaluating Individual and Aggregate Diversity in Top-N Recommendations. ACM Transactions on Intelligent Systems and Technology (TIST) 7, 3 (2016), 42.
[23]
Sean A Munson and Paul Resnick. 2010. Presenting diverse political opinions: how and how much. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1457--1466.
[24]
Mark EJ Newman. 2001. Clustering and preferential attachment in growing networks. Physical Review E 64, 2 (2001), 025102.
[25]
Tien T Nguyen, Pik-Mai Hui, F Maxwell Harper, Loren Terveen, and Joseph A Konstan. 2014. Exploring the filter bubble: the effect of using recommender systems on content diversity. In Proceedings of the 23rd international conference on World wide web. ACM, 677--686.
[26]
John O'Donovan, Brynjar Gretarsson, Svetlin Bostandjiev, Tobias Hollerer, and Barry Smyth. 2009. A visual interface for social information filtering. In Computational Science and Engineering, 2009. CSE'09. International Conference on, Vol. 4. IEEE, 74--81.
[27]
John O'Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. PeerChooser: visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1085--1088.
[28]
Valeria Orso, Tuukka Ruotsalo, Jukka Leino, Luciano Gamberini, and Giulio Jacucci. 2017. Overlaying social information: The effects on users search and information-selection behavior. Information Processing & Management 53, 6 (2017), 1269--1286.
[29]
Denis Parra and Peter Brusilovsky. 2015. User-controllable personalization: A case study with SetFusion. International Journal of Human-Computer Studies 78 (2015), 43--67.
[30]
Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems. ACM, 157--164.
[31]
Tuukka Ruotsalo, Jaakko Peltonen, Manuel Eugster, Dorota Głowacka, Ksenia Konyushkova, Kumaripaba Athukorala, Ilkka Kosunen, Aki Reijonen, Petri Myllym"aki, Giulio Jacucci, and others. 2013. Directing exploratory search with interactive intent modeling. In Proceedings of the 22nd ACM international conference on Conference on information & knowledge management. ACM, 1759--1764.
[32]
J Ben Schafer, Joseph A Konstan, and John Riedl. 2002. Meta-recommendation systems: user-controlled integration of diverse recommendations. In Proceedings of the eleventh international conference on Information and knowledge management. ACM, 43--51.
[33]
Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 990--998.
[34]
Robert Tarjan. 1972. Depth-first search and linear graph algorithms. SIAM journal on computing 1, 2 (1972), 146--160.
[35]
Choon Hui Teo, Houssam Nassif, Daniel Hill, Sriram Srinivasan, Mitchell Goodman, Vijai Mohan, and SVN Vishwanathan. 2016. Adaptive, Personalized Diversity for Visual Discovery. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 35--38.
[36]
Nina Tintarev. 2017. Presenting Diversity Aware Recommendations: Making Challenging News Acceptable. (2017).
[37]
Nava Tintarev and Judith Masthoff. 2012. Evaluating the effectiveness of explanations for recommender systems. User Modeling and User-Adapted Interaction 22, 4--5 (1 Oct. 2012), 399--439.Peter Brusilovsky. 2016. A personalized people recommender system using global search approach. IConference 2016 Proceedings (2016).
[38]
Chun-Hua Tsai and Peter Brusilovsky. 2017a. Enhancing Recommendation Diversity Through a Dual Recommendation Interface. In Workshop on Interfaces and Human Decision Making for Recommender Systems.
[39]
Chun-Hua Tsai and Peter Brusilovsky. 2017b. Leveraging Interfaces to Improve Recommendation Diversity. In Adjunct Publication of the 25th Conference on User Modeling, Adaptation and Personalization. ACM, 65--70.
[40]
Chun-Hua Tsai and Peter Brusilovsky. 2017c. Providing Control and Transparency in a Social Recommender System for Academic Conferences. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization. ACM, 313--317.
[41]
Chun-Hua Tsai and Yu-Ru Lin. 2016. Tracing and Predicting Collaboration for Junior Scholars. In Proceedings of the 25th International Conference Companion on World Wide Web. International World Wide Web Conferences Steering Committee, 375--380.
[42]
Katrien Verbert, Denis Parra, Peter Brusilovsky, and Erik Duval. 2013. Visualizing recommendations to support exploration, transparency and controllability. In Proceedings of the 2013 international conference on Intelligent user interfaces. ACM, 351--362.
[43]
Weiquan Wang and Izak Benbasat. 2007. Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. Journal of Management Information Systems 23, 4 (2007), 217--246.
[44]
David Wong, Siamak Faridani, Ephrat Bitton, Björn Hartmann, and Ken Goldberg. 2011. The diversity donut: enabling participant control over the diversity of recommended responses. In CHI'11 Extended Abstracts on Human Factors in Computing Systems. ACM, 1471--1476.

Cited By

View all
  • (2024)Investigating Characteristics of Media Recommendation Solicitation in r/ifyoulikeblankProceedings of the ACM on Human-Computer Interaction10.1145/36870418:CSCW2(1-23)Online publication date: 8-Nov-2024
  • (2024)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 7-Mar-2024
  • (2024)Towards Human-Centered Explainable AI: A Survey of User Studies for Model ExplanationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.333184646:4(2104-2122)Online publication date: 1-Apr-2024
  • Show More Cited By

Index Terms

  1. Beyond the Ranked List: User-Driven Exploration and Diversification of Social Recommendation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    IUI '18: Proceedings of the 23rd International Conference on Intelligent User Interfaces
    March 2018
    698 pages
    ISBN:9781450349451
    DOI:10.1145/3172944
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 March 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. diversification
    2. diversity
    3. social recommendation
    4. user interface
    5. user-driven exploration

    Qualifiers

    • Research-article

    Conference

    IUI'18
    Sponsor:

    Acceptance Rates

    IUI '18 Paper Acceptance Rate 43 of 299 submissions, 14%;
    Overall Acceptance Rate 746 of 2,811 submissions, 27%

    Upcoming Conference

    IUI '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)60
    • Downloads (Last 6 weeks)5
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Investigating Characteristics of Media Recommendation Solicitation in r/ifyoulikeblankProceedings of the ACM on Human-Computer Interaction10.1145/36870418:CSCW2(1-23)Online publication date: 8-Nov-2024
    • (2024)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 7-Mar-2024
    • (2024)Towards Human-Centered Explainable AI: A Survey of User Studies for Model ExplanationsIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.333184646:4(2104-2122)Online publication date: 1-Apr-2024
    • (2023)Selection Interface for Promoting User Selection Diversity by Presenting Positive/Negative Review Text and Video to Evoke Product Impression and User EmotionElectronics10.3390/electronics1212261112:12(2611)Online publication date: 9-Jun-2023
    • (2023)LIMEADE: From AI Explanations to Advice TakingACM Transactions on Interactive Intelligent Systems10.1145/358934513:4(1-29)Online publication date: 8-Dec-2023
    • (2023)Enabling Goal-Focused Exploration of Podcasts in Interactive Recommender SystemsProceedings of the 28th International Conference on Intelligent User Interfaces10.1145/3581641.3584032(142-155)Online publication date: 27-Mar-2023
    • (2022)Bursting Scientific Filter Bubbles: Boosting Innovation via Novel Author DiscoveryProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3501905(1-13)Online publication date: 29-Apr-2022
    • (2022)User-controllable Recommendation Against Filter BubblesProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3532075(1251-1261)Online publication date: 6-Jul-2022
    • (2022)Intra-list similarity and human diversity perceptions of recommendations: the details matterUser Modeling and User-Adapted Interaction10.1007/s11257-022-09351-w33:4(769-802)Online publication date: 12-Dec-2022
    • (2022)“Knowing me, knowing you”: personalized explanations for a music recommender systemUser Modeling and User-Adapted Interaction10.1007/s11257-021-09304-932:1-2(215-252)Online publication date: 1-Apr-2022
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media