Deliberation and Resolution on Wikipedia: A Case Study of Requests for Comments

Abstract

Resolving disputes in a timely manner is crucial for any online production group. We present an analysis of Requests for Comments (RfCs), one of the main vehicles on Wikipedia for formally resolving a policy or content dispute. We collected an exhaustive dataset of 7,316 RfCs on English Wikipedia over the course of 7 years and conducted a qualitative and quantitative analysis into what issues affect the RfC process. Our analysis was informed by 10 interviews with frequent RfC closers. We found that a major issue affecting the RfC process is the prevalence of RfCs that could have benefited from formal closure but that linger indefinitely without one, with factors including participants' interest and expertise impacting the likelihood of resolution. From these findings, we developed a model that predicts whether an RfC will go stale with 75.3% accuracy, a level that is approached as early as one week after dispute initiation.

References

1. Pablo Aragón, Andreas Kaltenbrunner, Antonio Calleja-López, Andrés Pereira, Arnau Monterde, Xabier E Barandiaran, and Vicenç Gómez. 2017. Deliberative Platform Design: The case study of the online discussions in Decidim Barcelona. In International Conference on Social Informatics. Springer, 277--287.
2. Yochai Benkler. 2002. Coase's Penguin, or, Linux and" The Nature of the Firm". Yale law journal (2002), 369--446.
3. Ivan Beschastnikh, Travis Kriplean, and David W McDonald. 2008. Wikipedian Self-Governance in Action: Motivating the Policy Lens. In ICWSM. 27--35.
4. Tegawendé F Bissyandé, David Lo, Lingxiao Jiang, Laurent Réveillere, Jacques Klein, and Yves Le Traon. 2013. Got issues? who cares about it? a large scale investigation of issue trackers from github. In Software Reliability Engineering (ISSRE), 2013 IEEE 24th International Symposium on. IEEE, 188--197.
5. LauraWBlack, Howard TWelser, Dan Cosley, and JocelynMDeGroot. 2011. Self-governance through group discussion in Wikipedia: Measuring deliberation in online groups. Small Group Research 42, 5 (2011), 595--634.
6. Luciana Buriol, Carlos Castillo, Debora Donato, Stefano Leonardi, and Stefano Millozzi. 2006. Temporal evolution of the wikigraph. (2006).
7. Moira Burke and Robert Kraut. 2008. Mopping up: modeling wikipedia promotion decisions. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. ACM, 27--36.
8. Brian Butler, Elisabeth Joyce, and Jacqueline Pike. 2008. Don't look now, but we've created a bureaucracy: the nature and roles of policies and rules in wikipedia. In Proceedings of the SIGCHI conference on human factors in computing systems. ACM, 1101--1110.
9. Kathy Charmaz and Linda Liska Belgrave. 2007. Grounded theory. The Blackwell encyclopedia of sociology (2007).
10. Justin Cranshaw and Aniket Kittur. 2011. The polymath project: lessons from a successful online collaboration in mathematics. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1865--1874.
11. Cristian Danescu-Niculescu-Mizil, Lillian Lee, Bo Pang, and Jon Kleinberg. 2012. Echoes of power: Language effects and power differences in social interaction. In Proceedings of the 21st international conference on World Wide Web. ACM, 699--708.
12. Cristian Danescu-Niculescu-Mizil, Moritz Sudhof, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. A computational approach to politeness with application to social factors. arXiv preprint arXiv:1306.6078 (2013).
13. Siamak Faridani, Ephrat Bitton, Kimiko Ryokai, and Ken Goldberg. 2010. Opinion space: a scalable tool for browsing online comments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1175--1184.
14. Oliver Ferschke, Iryna Gurevych, and Yevgen Chebotar. 2012. Behind the article: Recognizing dialog acts in Wikipedia talk pages. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics. Association for Computational Linguistics, 777--786.
15. Andrea Forte and Amy Bruckman. 2008. Scaling consensus: Increasing decentralization in Wikipedia governance. In Hawaii International Conference on System Sciences, Proceedings of the 41st Annual. IEEE, 157--157.
16. R Stuart Geiger and Heather Ford. 2011. Participation in Wikipedia's article deletion processes. In Proceedings of the 7th international symposium on Wikis and open collaboration. ACM, 201--202.
17. Marit Hinnosaar, Toomas Hinnosaar, Michael Kummer, and Olga Slivko. 2017. Wikipedia Matters: a significant impact of user-generated content on real-life choices. (2017).
18. Joshua Introne, Robert Laubacher, Gary Olson, and Thomas Malone. 2011. The Climate CoLab: Large scale model-based collaborative planning. In Collaboration Technologies and Systems (CTS), 2011 International Conference on. IEEE, 40--47.
19. Sirkka L Jarvenpaa and Dorothy E Leidner. 1998. Communication and trust in global virtual teams. Journal of Computer-Mediated Communication 3, 4 (1998), JCMC346.
20. Brian Keegan and Casey Fiesler. 2017. The Evolution and Consequences of Peer Producing Wikipedia's Rules. (2017).
21. Aniket Kittur and Robert E Kraut. 2008. Harnessing the wisdom of crowds in wikipedia: quality through coordination. In Proceedings of the 2008 ACM conference on Computer supported cooperative work. ACM, 37--46.
22. Aniket Kittur and Robert E Kraut. 2010. Beyond Wikipedia: coordination and conflict in online production groups. In Proceedings of the 2010 ACM conference on Computer supported cooperative work. ACM, 215--224.
23. Aniket Kittur, Bongwon Suh, Bryan A Pendleton, and Ed H Chi. 2007. He says, she says: conflict and coordination in Wikipedia. In Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, 453--462.
24. Travis Kriplean, Ivan Beschastnikh, David W McDonald, and Scott A Golder. 2007. Community, consensus, coercion, control: cs* w or how policy mediates mass participation. In Proceedings of the 2007 international ACM conference on Supporting group work. ACM, 167--176.
25. Travis Kriplean, Michael Toomim, Jonathan Morgan, Alan Borning, and Andrew Ko. 2012. Is this what you meant?: promoting listening on the web with reflect. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1559--1568.
26. David Laniado, Riccardo Tasso, Yana Volkovich, and Andreas Kaltenbrunner. 2011. When the wikipedians talk: Network and tree structure of wikipedia discussion pages.. In ICWSM. 177--184.
27. Christoph Lattemann and Stefan Stieglitz. 2005. Framework for governance in open source communities. In System Sciences, 2005. HICSS'05. Proceedings of the 38th Annual Hawaii International Conference on. IEEE, 192a--192a.
28. Keith Maki, Michael Yoder, Yohan Jo, and Carolyn Rosé. 2017. Roles and Success in Wikipedia Talk Pages: Identifying Latent Patterns of Behavior. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), Vol. 1. 1026--1035.
29. Elinor Ostrom. 2015. Governing the commons. Cambridge university press.
30. Zizi Papacharissi. 2004. Democracy online: Civility, politeness, and the democratic potential of online political discussion groups. New media & society 6, 2 (2004), 259--283.
31. James W Pennebaker, Ryan L Boyd, Kayla Jordan, and Kate Blackburn. 2015. The development and psychometric properties of LIWC2015. Technical Report.
32. Jenny Preece. 2000. Online communities: Designing usability and supporting sociability. John Wiley & Sons, Inc.
33. Jodi Schneider, John G Breslin, and Alexandre Passant. 2010. A content analysis: How Wikipedia talk pages are used. (2010).
34. Jodi Schneider, Alexandre Passant, and John G Breslin. 2011. Understanding and improving Wikipedia article discussion spaces. In Proceedings of the 2011 ACM Symposium on Applied Computing. ACM, 808--813.
35. Tiziano Squartini, Francesco Picciolo, Franco Ruzzenenti, and Diego Garlaschelli. 2013. Reciprocity of weighted networks. Scientific reports 3 (2013), 2729.
36. Igor Steinmacher, Tayana Conte, Marco Aurélio Gerosa, and David Redmiles. 2015. Social barriers faced by newcomers placing their first contribution in open source software projects. In Proceedings of the 18th ACM conference on Computer supported cooperative work & social computing. ACM, 1379--1392.
37. Hans Stiegler and Menno DT de Jong. 2015. Facilitating personal deliberation online: Immediate effects of two ConsiderIt variations. Computers in human behavior 51 (2015), 461--469.
38. Besiki Stvilia, Michael B Twidale, Linda C Smith, and Les Gasser. 2008. Information quality work organization in Wikipedia. Journal of the Association for Information Science and Technology 59, 6 (2008), 983--1001.
39. Róbert Sumi, Taha Yasseri, et al. 2011. Edit wars in Wikipedia. In Privacy, Security, Risk and Trust (PASSAT) and 2011 IEEE Third International Conference on Social Computing (SocialCom), 2011 IEEE Third International Conference on. IEEE, 724--727.
40. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2016. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th international conference on world wide web. International World Wide Web Conferences Steering Committee, 613--624.
41. Dario Taraborelli and Giovanni Luca Ciampaglia. 2010. Beyond notability. Collective deliberation on content inclusion in Wikipedia. In Self-Adaptive and Self-Organizing Systems Workshop (SASOW), 2010 Fourth IEEE International Conference on. IEEE, 122--125.
42. Fernanda B Viegas, Martin Wattenberg, Jesse Kriss, and Frank Van Ham. 2007. Talk before you type: Coordination in Wikipedia. In System sciences, 2007. HICSS 2007. 40th annual Hawaii international conference on. IEEE, 78--78.
43. Fernanda B Viégas, MartinWattenberg, and Matthew M McKeon. 2007. The hidden order of Wikipedia. In International conference on Online communities and social computing. Springer, 445--454.
44. Howard T Welser, Dan Cosley, Gueorgi Kossinets, Austin Lin, Fedor Dokshin, Geri Gay, and Marc Smith. 2011. Finding social roles in Wikipedia. In Proceedings of the 2011 iConference. ACM, 122--129.
45. Dennis M Wilkinson and Bernardo A Huberman. 2007. Cooperation and quality in wikipedia. In Proceedings of the 2007 international symposium on Wikis. ACM, 157--164.
46. Diyi Yang, Aaron Halfaker, Robert E Kraut, and Eduard H Hovy. 2016. Who Did What: Editor Role Identification in Wikipedia. In ICWSM. 446--455.
47. Taha Yasseri, Robert Sumi, András Rung, András Kornai, and János Kertész. 2012. Dynamics of conflicts in Wikipedia. PLoS ONE 7, 6 (2012), e38869.
48. Amy X Zhang, Lea Verou, and David R Karger. 2017. Wikum: Bridging Discussion Forums and Wikis Using Recursive Summarization. In CSCW. 2082--2096.

Index Terms

• Human-centered computing

  • Collaborative and social computing

    • Empirical studies in collaborative and social computing