Exploring Social Recommendations with Visual Diversity-Promoting Interfaces

Abstract

The beyond-relevance objectives of recommender systems have been drawing more and more attention. For example, a diversity-enhanced interface has been shown to associate positively with overall levels of user satisfaction. However, little is known about how users adopt diversity-enhanced interfaces to accomplish various real-world tasks. In this article, we present two attempts at creating a visual diversity-enhanced interface that presents recommendations beyond a simple ranked list. Our goal was to design a recommender system interface to help users explore the different relevance prospects of recommended items in parallel and to stress their diversity. Two within-subject user studies in the context of social recommendation at academic conferences were conducted to compare our visual interfaces. Results from our user study show that the visual interfaces significantly reduced the exploration efforts required for given tasks and helped users to perceive the recommendation diversity. We show that the users examined a diverse set of recommended items while experiencing an improvement in overall user satisfaction. Also, the users’ subjective evaluations show significant improvement in many user-centric metrics. Experiences are discussed that shed light on avenues for future interface designs.

References

1. Jisun An, Daniele Quercia, and Jon Crowcroft. 2013. Why individuals seek diverse opinions (or why they don’t). In Proceedings of the 5th Annual ACM Web Science Conference. ACM, 15--18.
2. Fabiano M. Belém, Carolina S. Batista, Rodrygo L. T. Santos, Jussara M. Almeida, and Marcos A. Gonçalves. 2016. Beyond relevance: Explicitly promoting novelty and diversity in tag recommendation. ACM Transactions on Intelligent Systems and Technology (TIST) 7, 3 (2016), 26.
3. Toine Bogers. 2018. Tag-based recommendation. In Social Information Access. Springer, 441--479.
4. Dirk Bollen, Bart P. Knijnenburg, Martijn C. Willemsen, and Mark Graus. 2010. Understanding choice overload in recommender systems. In Proceedings of the 4th ACM Conference on Recommender Systems. ACM, 63--70.
5. Svetlin Bostandjiev, John O’Donovan, and Tobias Höllerer. 2012. TasteWeights: A visual interactive hybrid recommender system. In Proceedings of the 6th ACM Conference on Recommender Systems. ACM, 35--42.
6. Engin Bozdag and Jeroen van den Hoven. 2015. Breaking the filter bubble: Democracy and design. Ethics and Information Technology 17, 4 (2015), 249--265.
7. Peter Brusilovsky, Jung Sun Oh, Claudia López, Denis Parra, and Wei Jeng. 2016. Linking information and people in a social system for academic conferences. New Review of Hypermedia and Multimedia (2016), 1--31.
8. Bruno Cardoso, Gayane Sedrakyan, Francisco Gutiérrez, Denis Parra, Peter Brusilovsky, and Katrien Verbert. 2018. IntersectionExplorer, a multi-perspective approach for exploring recommendations. International Journal of Human-Computer Studies (2018).
9. Jilin Chen, Werner Geyer, Casey Dugan, Michael Muller, and Ido Guy. 2009. Make new friends, but keep the old: Recommending people on social networking sites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 201--210.
10. Henriette Cramer, Vanessa Evers, Satyan Ramlal, Maarten Van Someren, Lloyd Rutledge, Natalia Stash, Lora Aroyo, and Bob Wielinga. 2008. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction 18, 5 (2008), 455.
11. Paolo Cremonesi, Franca Garzotto, and Roberto Turrin. 2012. Investigating the persuasion potential of recommender systems from a quality perspective: An empirical study. ACM Transactions on Interactive Intelligent Systems (TiiS) 2, 2 (2012), 11.
12. Tuan Nhon Dang and Leland Wilkinson. 2014. Scagexplorer: Exploring scatterplots by their scagnostics. In Visualization Symposium (PacificVis), 2014 IEEE Pacific. IEEE, 73--80.
13. Stavin Deeswe and Raymond Kosala. 2015. An integrated search interface with 3D visualization. Procedia Computer Science 59 (2015), 483--492.
14. Cecilia di Sciascio, Vedran Sabol, and Eduardo E. Veas. 2016. Rank as you go: User-driven exploration of search results. In Proceedings of the 21st International Conference on Intelligent User Interfaces. ACM, 118--129.
15. Michael D. Ekstrand, F. Maxwell Harper, Martijn C. Willemsen, and Joseph A. Konstan. 2014. User perception of differences in recommender algorithms. In Proceedings of the 8th ACM Conference on Recommender Systems. ACM, 161--168.
16. Michael D. Ekstrand, Daniel Kluver, F. Maxwell Harper, and Joseph A. Konstan. 2015. Letting users choose recommender algorithms: An experimental study. In Proceedings of the 9th ACM Conference on Recommender Systems. ACM, 11--18.
17. Siamak Faridani, Ephrat Bitton, Kimiko Ryokai, and Ken Goldberg. 2010. Opinion space: A scalable tool for browsing online comments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1175--1184.
18. Minos N. Garofalakis, Rajeev Rastogi, and Kyuseok Shim. 1999. SPIRIT: Sequential pattern mining with regular expression constraints. In VLDB, Vol. 99. 7--10.
19. Mouzhi Ge, Carla Delgado-Battenfeld, and Dietmar Jannach. 2010. Beyond accuracy: Evaluating recommender systems by coverage and serendipity. In Proceedings of the 4th ACM Conference on Recommender Systems. ACM, 257--260.
20. Dorota Glowacka, Tuukka Ruotsalo, Ksenia Konuyshkova, Samuel Kaski, and Giulio Jacucci. 2013. Directing exploratory search: Reinforcement learning from user interactions with keywords. In Proceedings of the 2013 International Conference on Intelligent User Interfaces. ACM, 117--128.
21. Brynjar Gretarsson, John O’Donovan, Svetlin Bostandjiev, Christopher Hall, and Tobias Höllerer. 2010. Smallworlds: Visualizing social recommendations. In Computer Graphics Forum, Vol. 29. Wiley Online Library, 833--842.
22. Julio Guerra, Shaghayegh Sahebi, Yu-Ru Lin, and Peter Brusilovsky. 2014. The problem solving genome: Analyzing sequential patterns of student work with parameterized exercises. In Proceedings of the 7th International Conference on Educational Data Mining (EDM'14).
23. Pankaj Gupta, Ashish Goel, Jimmy Lin, Aneesh Sharma, Dong Wang, and Reza Zadeh. 2013. WTF: The who to follow service at Twitter. In Proceedings of the 22nd International Conference on World Wide Web. ACM, 505--514.
24. Ido Guy. 2015. Social recommender systems. In Recommender Systems Handbook. Springer, 511--543.
25. Ido Guy. 2018. People Recommendation on Social Media. Springer International Publishing, Cham, 570--623.
26. Ido Guy, Inbal Ronen, and Eric Wilcox. 2009. Do you know?: Recommending people to invite into your social network. In Proceedings of the 14th International Conference on Intelligent User Interfaces. ACM, 77--86.
27. Ido Guy, Sigalit Ur, Inbal Ronen, Adam Perer, and Michal Jacovi. 2011. Do you want to know?: Recommending strangers in the enterprise. In Proceedings of the ACM 2011 Conference on Computer Supported Cooperative Work. ACM, 285--294.
28. Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work. ACM, 241--250.
29. Rong Hu and Pearl Pu. 2011. Helping users perceive recommendation diversity. In DiveRS@ RecSys. 43--50.
30. Marius Kaminskas and Derek Bridge. 2016. Diversity, serendipity, novelty, and coverage: A survey and empirical analysis of beyond-accuracy objectives in recommender systems. ACM Trans. Interact. Intell. Syst. 7, 1, Article 2 (Dec. 2016), 42 pages.
31. Marius Kaminskas and Derek Bridge. 2016. Diversity, serendipity, novelty, and coverage: A survey and empirical analysis of beyond-accuracy objectives in recommender systems. ACM Transactions on Interactive Intelligent Systems (TiiS) 7, 1 (2016), 2.
32. Hannah Kim, Jaegul Choo, Haesun Park, and Alex Endert. 2016. Interaxis: Steering scatterplot axes via observation-level interaction. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2016), 131--140.
33. Khalil Klouche, Tuukka Ruotsalo, Diogo Cabral, Salvatore Andolina, Andrea Bellucci, and Giulio Jacucci. 2015. Designing for exploratory search on touch devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems. ACM, 4189--4198.
34. Khalil Klouche, Tuukka Ruotsalo, Luana Micallef, Salvatore Andolina, and Giulio Jacucci. 2017. Visual re-ranking for multi-aspect information retrieval. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval. ACM, 57--66.
35. Daniel Kluver, Michael D. Ekstrand, and Joseph A. Konstan. 2018. Rating-based collaborative filtering: Algorithms and evaluation. In Social Information Access. Springer, 344--390.
36. Bart P. Knijnenburg, Svetlin Bostandjiev, John O’Donovan, and Alfred Kobsa. 2012. Inspectability and control in social recommenders. In Proceedings of the 6th ACM Conference on Recommender System. 43--50.
37. Pigi Kouki, James Schaffer, Jay Pujara, John O’Donovan, and Lise Getoor. 2017. User preferences for hybrid explanations. In Proceedings of the 11th ACM Conference on Recommender Systems. ACM, 84--88.
38. Johannes Kunkel, Benedikt Loepp, and Jürgen Ziegler. 2017. A 3D item space visualization for presenting and manipulating user preferences in collaborative filtering. In Proceedings of the 22nd International Conference on Intelligent User Interfaces. ACM, 3--15.
39. Danielle Lee and Peter Brusilovsky. 2018. Recommendations based on social links. In Social Information Access. Springer, 391--440.
40. Q. Vera Liao and Wai-Tat Fu. 2013. Beyond the filter bubble: Interactive effects of perceived threat and topic involvement on selective exposure to information. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 2359--2368.
41. Sean M. McNee, John Riedl, and Joseph A. Konstan. 2006. Being accurate is not enough: How accuracy metrics have hurt recommender systems. In CHI’06 Extended Abstracts on Human Factors in Computing Systems. ACM, 1097--1101.
42. Jennifer Moody and David H. Glass. 2016. A novel classification framework for evaluating individual and aggregate diversity in top-N recommendations. ACM Transactions on Intelligent Systems and Technology (TIST) 7, 3 (2016), 42.
43. Sean A. Munson and Paul Resnick. 2010. Presenting diverse political opinions: How and how much. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1457--1466.
44. Mark E. J. Newman. 2001. Clustering and preferential attachment in growing networks. Physical Review E 64, 2 (2001), 025102.
45. Tien T. Nguyen, Pik-Mai Hui, F. Maxwell Harper, Loren Terveen, and Joseph A. Konstan. 2014. Exploring the filter bubble: The effect of using recommender systems on content diversity. In Proceedings of the 23rd International Conference on World Wide Web. ACM, 677--686.
46. John O’Donovan, Brynjar Gretarsson, Svetlin Bostandjiev, Tobias Hollerer, and Barry Smyth. 2009. A visual interface for social information filtering. In Proceedings of the International Conference on Computational Science and Engineering, 2009 (CSE’09). Vol. 4. IEEE, 74--81.
47. John O’Donovan, Barry Smyth, Brynjar Gretarsson, Svetlin Bostandjiev, and Tobias Höllerer. 2008. PeerChooser: Visual interactive recommendation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1085--1088.
48. Michael P. O’Mahony and Barry Smyth. 2018. From opinions to recommendations. In Social Information Access. Springer, 480--509.
49. Valeria Orso, Tuukka Ruotsalo, Jukka Leino, Luciano Gamberini, and Giulio Jacucci. 2017. Overlaying social information: The effects on users’ search and information-selection behavior. Information Processing 8 Management 53, 6 (2017), 1269--1286.
50. Anshul Vikram Pandey, Josua Krause, Cristian Felix, Jeremy Boy, and Enrico Bertini. 2016. Towards understanding human similarity perception in the analysis of large sets of scatter plots. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. ACM, 3659--3669.
51. Denis Parra and Peter Brusilovsky. 2015. User-controllable personalization: A case study with SetFusion. International Journal of Human-Computer Studies 78 (2015), 43--67.
52. Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the 5th ACM Conference on Recommender Systems. ACM, 157--164.
53. Tuukka Ruotsalo, Jaakko Peltonen, Manuel Eugster, Dorota Głowacka, Ksenia Konyushkova, Kumaripaba Athukorala, Ilkka Kosunen, Aki Reijonen, Petri Myllymäki, Giulio Jacucci, et al. 2013. Directing exploratory search with interactive intent modeling. In Proceedings of the 22nd ACM International Conference on Conference on Information 8 Knowledge Management. ACM, 1759--1764.
54. J. Ben Schafer, Joseph A. Konstan, and John Riedl. 2002. Meta-recommendation systems: User-controlled integration of diverse recommendations. In Proceedings of the 11th International Conference on Information and Knowledge Management. ACM, 43--51.
55. Rory L. L. Sie, Hendrik Drachsler, Marlies Bitter-Rijpkema, and Peter Sloep. 2012. To whom and why should I connect? Co-author recommendation based on powerful and similar peers. International Journal on Technology Enhanced Learning 4, 1/2 (2012), 121--137.
56. Jie Tang, Jing Zhang, Limin Yao, Juanzi Li, Li Zhang, and Zhong Su. 2008. Arnetminer: Extraction and mining of academic social networks. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 990--998.
57. Robert Tarjan. 1972. Depth-first search and linear graph algorithms. SIAM Journal on Computing 1, 2 (1972), 146--160.
58. Choon Hui Teo, Houssam Nassif, Daniel Hill, Sriram Srinivasan, Mitchell Goodman, Vijai Mohan, and S. V. N. Vishwanathan. 2016. Adaptive, personalized diversity for visual discovery. In Proceedings of the 10th ACM Conference on Recommender Systems. ACM, 35--38.
59. Nina Tintarev. 2017. Presenting diversity aware recommendations: Making challenging news acceptable. The FATRec Workshop on Responsible Recommendation (FATRec'17).
60. Nava Tintarev and Judith Masthoff. 2012. Evaluating the effectiveness of explanations for recommender systems. User Modeling and User-Adapted Interaction 22, 4–5 (1 Oct. 2012), 399--439.
61. Chun-Hua Tsai. 2017. An interactive and interpretable interface for diversity in recommender systems. In Proceedings of the 22nd International Conference on Intelligent User Interfaces Companion (IUI’17 Companion). ACM, New York, 225--228.
62. Chun-Hua Tsai and Peter Brusilovsky. 2016. A personalized people recommender system using global search approach. IConference 2016 Proceedings (2016).
63. Chun-Hua Tsai and Peter Brusilovsky. 2017. Enhancing recommendation diversity through a dual recommendation interface. In Proceedings of the Workshop on Interfaces and Human Decision Making for Recommender Systems.
64. Chun-Hua Tsai and Peter Brusilovsky. 2017. Leveraging interfaces to improve recommendation diversity. In Adjunct Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization. ACM, 65--70.
65. Chun-Hua Tsai and Peter Brusilovsky. 2017. Providing control and transparency in a social recommender system for academic conferences. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization. ACM, 313--317.
66. Chun-Hua Tsai and Yu-Ru Lin. 2016. Tracing and predicting collaboration for junior scholars. In Proceedings of the 25th International Conference Companion on World Wide Web. International World Wide Web Conferences Steering Committee, 375--380.
67. Katrien Verbert, Denis Parra, Peter Brusilovsky, and Erik Duval. 2013. Visualizing recommendations to support exploration, transparency and controllability. In Proceedings of the 2013 International Conference on Intelligent User Interfaces. ACM, 351--362.
68. Jesse Vig, Shilad Sen, and John Riedl. 2012. The tag genome: Encoding community knowledge to support novel interaction. ACM Transactions on Interactive Intelligent Systems (TiiS) 2, 3 (2012), 13.
69. Weiquan Wang and Izak Benbasat. 2007. Recommendation agents for electronic commerce: Effects of explanation facilities on trusting beliefs. Journal of Management Information Systems 23, 4 (2007), 217--246.
70. David Wong, Siamak Faridani, Ephrat Bitton, Björn Hartmann, and Ken Goldberg. 2011. The diversity donut: Enabling participant control over the diversity of recommended responses. In CHI’11 Extended Abstracts on Human Factors in Computing Systems. ACM, 1471--1476.
71. Bo Xiao and Izak Benbasat. 2007. E-commerce product recommendation agents: Use, characteristics, and impact. MIS Quarterly 31, 1 (2007), 137--209.

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI design and evaluation methods