Umashanthi Pavalanathan
Skip Abstract Section
Abstract
Wikipedia has a strong norm of writing in a "neutral point of view" (NPOV). Articles that violate this norm are tagged, and editors are encouraged to make corrections. But the impact of this tagging system has not been quantitatively measured. Does NPOV tagging help articles to converge to the desired style? Do NPOV corrections encourage editors to adopt this style? We study these questions using a corpus of NPOV-tagged articles and a set of lexicons associated with biased language. An interrupted time series analysis shows that after an article is tagged for NPOV, there is a significant decrease in biased language in the article, as measured by several lexicons. However, for individual editors, NPOV corrections and talk page discussions yield no significant change in the usage of words in most of these lexicons, including Wikipedia's own list of "words to watch." This suggests that NPOV tagging and discussion does improve content, but has less success enculturating editors to the site's linguistic norms.
- Khalid Al Khatib, Hinrich Schütze, and Cathleen Kantner. 2012. Automatic Detection of Point of View Differences in Wikipedia. In Proceedings of the 24th International Conference on Computational Linguistics (COLING 2012). Association for Computational Linguistics, 33--50.Google Scholar
- Maik Anderka and Benno Stein. 2012. A Breakdown of Quality Flaws in Wikipedia. In Proceedings of the 2nd Joint WICOW/AIRWeb Workshop on Web Quality. ACM, 11--18. Google ScholarDigital Library
- Maik Anderka, Benno Stein, and Nedim Lipka. 2012. Predicting Quality Flaws in User-Generated Content: The Case of Wikipedia. In Proceedings of the 35th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 981--990. Google ScholarDigital Library
- Ofer Arazy, Felipe Ortega, Oded Nov, Lisa Yeo, and Adam Balila. 2015. Functional Roles and Career Paths in Wikipedia. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing. ACM, 1092--1105. Google ScholarDigital Library
- Yoav Benjamini and Yosef Hochberg. 1995. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. Journal of the Royal Statistical Society. Series B (Methodological) (1995), 289--300.Google Scholar
- Yochai Benkler. 2006. The Wealth of Networks: How Social Production Transforms Markets and Freedom. Yale University Press. Google ScholarDigital Library
- Kelly Bergstrom. 2011. 'Don't Feed the Troll": Shutting Down Debate About Community Expectations on Reddit.com. First Monday 16, 8 (2011).Google Scholar
- James Lopez Bernal, Steven Cummins, and Antonio Gasparrini. 2017. Interrupted Time Series Regression for the Evaluation of Public Health Interventions: A Tutorial. International Journal of Epidemiology 46, 1 (2017), 348--355.Google Scholar
- Ivan Beschastnikh, Travis Kriplean, and David W McDonald. 2008. Wikipedian Self-Governance in Action: Motivating the Policy Lens.. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM).Google Scholar
- Pierre Bourdieu. 1991. Language and Symbolic Power. Harvard University PressGoogle Scholar
- Susan L Bryant, Andrea Forte, and Amy Bruckman. 2005. Becoming Wikipedian: Transformation of Participation in a Collaborative Online Encyclopedia. In Proceedings of the 2005 ACM Conference on Supporting Group Work. ACM, 1--10. Google ScholarDigital Library
- Gary Burnett and Laurie Bonnici. 2003. Beyond the FAQ: Explicit and Implicit Norms in Usenet Newsgroups. Library & Information Science Research 25, 3 (2003), 333--351.Google ScholarCross Ref
- Ewa S Callahan and Susan C Herring. 2011. Cultural Bias in Wikipedia Content on Famous Persons. Journal of the Association for Information Science and Technology 62, 10 (2011), 1899--1915. Google ScholarDigital Library
- Joshua McCann Kyle Frye Jed R. Brubaker Casey Fiesler, Jialun 'Aaron" Jiang. 2018. Reddit Rules! Characterizing an Ecosystem of Governance. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM).Google Scholar
- Stevie Chancellor, Andrea Hu, and Munmun De Choudhury. 2018. Norms Matter: Contrasting Social Support Around Behavior Change in Online Weight Loss Communities. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM. Google ScholarDigital Library
- Eshwar Chandrasekharan, Umashanthi Pavalanathan, Anirudh Srinivasan, Adam Glynn, Jacob Eisenstein, and Eric Gilbert. 2017. You Can't Stay Here: The Efficacy of Reddit's 2015 Ban Examined Through Hate Speech. Proceedings of the ACM on Human-Computer Interaction 1, CSCW, Article 31 (Dec. 2017), 22 pages. Google ScholarDigital Library
- Justin Cheng, Cristian Danescu-Niculescu-Mizil, and Jure Leskovec. 2015. Antisocial Behavior in Online Discussion Communities.. In Proceedings of the International AAAI Conference on Web and Social Media(ICWSM). 61--70.Google Scholar
- Thomas Chesney. 2006. An Empirical Examination of Wikipedia's Credibility. First Monday 11, 11 (2006).Google Scholar
- Andrea Ciffolilli. 2003. Phantom Authority, Self-Selective Recruitment and Retention of Members in Virtual Communities: The Case of Wikipedia. First Monday 8, 12 (2003).Google Scholar
- Dan Cosley, Dan Frankowski, Loren Terveen, and John Riedl. 2007. SuggestBot: Using Intelligent Task Routing to Help People Find Work in Wikipedia. In Proceedings of the 12th International Conference on Intelligent User Interfaces. ACM, 32--41. Google ScholarDigital Library
- Cristian Danescu-Niculescu-Mizil, Robert West, Dan Jurafsky, Jure Leskovec, and Christopher Potts. 2013. No Country for Old Members: User Lifecycle and Linguistic Change in Online Communities. In Proceedings of the 22nd International Conference on World Wide Web. ACM, 307--318. Google ScholarDigital Library
- Sanmay Das, Allen Lavoie, and Malik Magdon-Ismail. 2016. Manipulation Among the Arbiters of Collective Intelligence: How Wikipedia Administrators Mold Public Opinion. ACM Transactions on the Web (TWEB) 10, 4 (2016), 24. Google ScholarDigital Library
- Luca De Alfaro and Michael Shavlovsky. 2013. Attributing authorship of revisioned content. In Proceedings of the 22nd international conference on World Wide Web. ACM, 343--354. Google ScholarDigital Library
- Angelo Di Iorio, Fabio Vitali, and Stefano Zacchiroli. 2008. Wiki Content Templating. In Proceedings of the 17th International Conference on World Wide Web. ACM, 615--624. Google ScholarDigital Library
- Nora A Draper. 2018. Distributed intervention: Networked Content Moderation in Anonymous Mobile Spaces. Feminist Media Studies (2018), 1--17.Google Scholar
- Michelle Broder Van Dyke. 2015. Reddit Users Revolt After Site Bans 'Fat People Hate" And Other Communities. https://www.buzzfeed.com/mbvd/reddit-users-revolt-after-site-bans-fat-people-hate-and-othe. Buzzfeed News (2015).Google Scholar
- Antonella Elia. 2006. An Analysis of Wikipedia Digital Writing. In Proceedings of the Workshop on NEW TEXT Wikis and Blogs and Other Dynamic Text Sources.Google Scholar
- William Emigh and Susan C Herring. 2005. Collaborative Authoring on the Web: A Genre Analysis of Online Encyclopedias. In System Sciences, 2005. HICSS'05. Proceedings of the 38th Annual Hawaii International Conference on. IEEE, 99a--99a. Google ScholarDigital Library
- Oliver Ferschke, Iryna Gurevych, and Marc Rittberger. 2013. The Impact of Topic Bias on Quality Flaw Prediction in Wikipedia. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Vol. 1. 721--730.Google Scholar
- Fabian Flöck and Maribel Acosta. 2014. WikiWho: Precise and Efficient Attribution of Authorship of Revisioned Content. In Proceedings of the 23rd International Conference on World Wide Web. ACM, 843--854. Google ScholarDigital Library
- Andrea Forte and Amy Bruckman. 2005. Why Do People Write for Wikipedia? Incentives to Contribute to Open-- Content Publishing. Proceedings of the 2005 ACM Conference on Supporting Group Work 5 (2005), 6--9.Google Scholar
- Andrea Forte and Amy Bruckman. 2006. From Wikipedia to the Classroom: Exploring Online Publication and Learning. In Proceedings of the 7th International Conference on Learning Sciences. International Society of the Learning Sciences, 182--188. Google ScholarDigital Library
- R Stuart Geiger, Aaron Halfaker, Maryana Pinchuk, and Steven Walling. 2012. Defense Mechanism or Socialization Tactic? Improving. In Proceedings of the Sixth International AAAI Conference on Weblogs and Social Media.Google Scholar
- R Stuart Geiger and David Ribes. 2010. The Work of Sustaining Order in Wikipedia: The Banning of a Vandal. In Proceedings of the 2010 ACM Conference on Computer Supported Cooperative Work. ACM, 117--126. Google ScholarDigital Library
- Shane Greenstein and Feng Zhu. 2012. Collective Intelligence and Neutral Point of View: the Case of Wikipedia. Technical Report. National Bureau of Economic Research.Google Scholar
- James Grimmelmann. 2015. The Virtues of Moderation. Yale JL & Tech. 17 (2015), 42.Google Scholar
- Aaron Halfaker, R Stuart Geiger, Jonathan T Morgan, and John Riedl. 2013. The Rise and Decline of an Open Collaboration System: How Wikipedia's Reaction to Popularity is Causing its Decline. American Behavioral Scientist 57, 5 (2013), 664--688.Google ScholarCross Ref
- Aaron Halfaker, Aniket Kittur, and John Riedl. 2011. Don't Bite the Newbies: How Reverts Affect the Quantity and Quality of Wikipedia Work. In Proceedings of the 7th International Symposium on Wikis and Open Collaboration. ACM, 163--172. Google ScholarDigital Library
- Manoj Harpalani, Michael Hart, Sandesh Singh, Rob Johnson, and Yejin Choi. 2011. Language of Vandalism: Improving Wikipedia Vandalism Detection Via Stylometric Analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short papers-Volume 2. Association for Computational Linguistics, 83--88. Google ScholarDigital Library
- Livnat Herzig, Alex Nunes, and Batia Snir. 2011. An Annotation Scheme for Automated Bias Detection in Wikipedia. In Proceedings of the 5th Linguistic Annotation Workshop. Association for Computational Linguistics, 47--55. Google ScholarDigital Library
- Michael A Hogg and Scott A Reid. 2006. Social Identity, Self-Categorization, and the Communication of Group Norms. Communication Theory 16, 1 (2006), 7--30.Google ScholarCross Ref
- Joan B Hooper. 1974. On Assertive Predicates. Indiana University Linguistics Club.Google Scholar
- Ken Hyland. 2005. Metadiscourse: Exploring Interaction in Writing. A&C Black.Google Scholar
- Dell Hymes. 1972. On Communicative Competence. Sociolinguistics 269293 (1972), 269--293.Google Scholar
- Mikolaj Jan Piskorski and Andreea Gorbatâi. 2017. Testing Coleman's Social-Norm Enforcement Mechanism: Evidence from Wikipedia. Amer. J. Sociology 122, 4 (2017), 1183--1222.Google ScholarCross Ref
- Lauri Karttunen. 1971. Implicative Verbs. Language (1971), 340--358.Google Scholar
- Brian Keegan, Darren Gergle, and Noshir Contractor. 2013. Hot Off the Wiki: Structures and Dynamics of Wikipedia's Coverage of Breaking News Events. American Behavioral Scientist 57, 5 (2013), 595--622.Google ScholarCross Ref
- Sara Kiesler, Robert Kraut, Paul Resnick, and Aniket Kittur. 2012. Regulating Behavior in Online Communities. Building Successful Online Communities: Evidence-Based Social Design. MIT Press, Cambridge, MA (2012).Google Scholar
- Amy Jo Kim. 2000. Community Building on the Web: Secret Strategies for Successful Online Communities. Addison-Wesley Longman Publishing Co., Inc. Google ScholarDigital Library
- Paul Kiparsky and Carol Kiparsky. 1970. Fact. ed. M. Bierwisch and K. Heidolph. Progress in Linguistics (1970).Google Scholar
- Travis Kriplean, Ivan Beschastnikh, and David W McDonald. 2008. Articulations of Wikiwork: Uncovering Valued Work in wikipedia Through Barnstars. In Proceedings of the 2008 ACM conference on Computer Supported Cooperative Work. ACM, 47--56. Google ScholarDigital Library
- Cliff Lampe and Paul Resnick. 2004. Slash (dot) and Burn: Distributed Moderation in a Large Online Conversation Space. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 543--550. Google ScholarDigital Library
- Cliff Lampe, Paul Zube, Jusil Lee, Chul Hyun Park, and Erik Johnston. 2014. Crowdsourcing Civility: A Natural Experiment Examining the Effects of Distributed Moderation in Online Forums. Government Information Quarterly 31, 2 (2014), 317--326.Google ScholarCross Ref
- Andrew Lih. 2004. Wikipedia as Participatory Journalism: Reliable Sources? Metrics for Evaluating Collaborative Media as a News Resource. Nature 3, 1 (2004).Google Scholar
- Nedim Lipka and Benno Stein. 2010. Identifying Featured Articles in Wikipedia: Writing Style Matters. In Proceedings of the 19th International Conference on World Wide Web. ACM, 1147--1148. Google ScholarDigital Library
- Bing Liu, Minqing Hu, and Junsheng Cheng. 2005. Opinion Observer: Analyzing and Comparing Opinions on the Web. In Proceedings of the 14th International Conference on World Wide Web. ACM, 342--351. Google ScholarDigital Library
- Tanushree Mitra Mattia Samory. 2018. Conspiracies Online: User Discussions in a Conspiracy Community Following Dramatic Events. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM).Google Scholar
- Uwe Matzat and G Rooks. 2014. Styles of Moderation in Online Health and Support Communities: An Experimental Comparison of Their Acceptance and Effectiveness. Computers in Human Behavior 36 (2014), 65--75. Google ScholarDigital Library
- Hacker News. 2018. Hacker News FAQ. https://news.ycombinator.com/newsfaq.html {Online; accessed 22-July-2018}.Google Scholar
- Oded Nov. 2007. What Motivates Wikipedians? Commun. ACM 50, 11 (2007), 60--64. Google ScholarDigital Library
- Katherine Panciera, Aaron Halfaker, and Loren Terveen. 2009. Wikipedians Are Born, Not Made: A Study of Power Editors on Wikipedia. In Proceedings of the ACM 2009 International Conference on Supporting Group Work. ACM, 51--60. Google ScholarDigital Library
- Martin Potthast, Benno Stein, and Robert Gerling. 2008. Automatic Vandalism Detection in Wikipedia. In European Conference on Information Retrieval. Springer, 663--668. Google ScholarDigital Library
- Reid Priedhorsky, Jilin Chen, Shyong Tony K Lam, Katherine Panciera, Loren Terveen, and John Riedl. 2007. Creating, Destroying, and Restoring Value in Wikipedia. In Proceedings of the 2007 International ACM Conference on Supporting Group Work. ACM, 259--268. Google ScholarDigital Library
- Sheizaf Rafaeli and Yaron Ariel. 2008. Online Motivational Factors: Incentives for Participation and Contribution in Wikipedia. Psychological Aspects of Cyberspace: Theory, Research, Applications (2008), 243--267.Google Scholar
- Mahmudur Rahman, Bogdan Carbunar, Jaime Ballesteros, George Burri, and Duen Horng Chau. 2014. Turning the Tide: Curbing Deceptive Yelp Behaviors. In Proceedings of the 2014 SIAM International Conference on Data Mining. SIAM, 244--252.Google ScholarCross Ref
- Joseph M Reagle Jr. 2010. 'Be Nice": Wikipedia Norms for Supportive Communication. New Review of Hypermedia and Multimedia 16, 1--2 (2010), 161--180. Google ScholarDigital Library
- Marta Recasens, Cristian Danescu-Niculescu-Mizil, and Dan Jurafsky. 2013. Linguistic Models for Analyzing and Detecting Biased Language. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).Google Scholar
- Yuqing Ren, F Maxwell Harper, Sara Drenner, Loren Terveen, Sara Kiesler, John Riedl, and Robert E Kraut. 2012. Building Member Attachment in Online Communities: Applying Theories of Group Identity and Interpersonal Bonds. Mis Quarterly (2012), 841--864. Google ScholarDigital Library
- Ellen Riloff and Janyce Wiebe. 2003. Learning Extraction Patterns for Subjective Expressions. In Proceedings of the 2003 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 105--112. Google ScholarDigital Library
- Joachim Schroer and Guido Hertel. 2009. Voluntary Engagement in an Open Web-based Encyclopedia: Wikipedians and Why They Do It. Media Psychology 12, 1 (2009), 96--120.Google ScholarCross Ref
- Kay Kyeongju Seo. 2007. Utilizing Peer Moderating in Online Discussions: Addressing the Controversy Between Teacher Moderation and Nonmoderation. The American Journal of Distance Education 21, 1 (2007), 21--36.Google ScholarCross Ref
- Pnina Shachaf and Noriko Hara. 2010. Beyond Vandalism: Wikipedia Trolls. Journal of Information Science 36, 3 (2010), 357--370. Google ScholarDigital Library
- Sara Sood, Judd Antin, and Elizabeth Churchill. 2012. Profanity Use in Online Communities. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1481--1490. Google ScholarDigital Library
- Jakob Voss. 2005. Measuring Wikipedia. In Proceedings of 10th International Conference of the International Society for Scientometrics and Informetrics.Google Scholar
- Claudia Wagner, David Garcia, Mohsen Jadidi, and Markus Strohmaier. 2015. It's a Man's Wikipedia? Assessing Gender Inequality in an Online Encyclopedia.. In Proceedings of the International AAAI Conference on Web and Social Media(ICWSM). 454--463.Google Scholar
- Howard T Welser, Dan Cosley, Gueorgi Kossinets, Austin Lin, Fedor Dokshin, Geri Gay, and Marc Smith. 2011. Finding Social Roles in Wikipedia. In Proceedings of the 2011 iConference. ACM, 122--129. Google ScholarDigital Library
- Wikipedia. 2018. Neutral Point of View. https://en.wikipedia.org/wiki/Wikipedia:Neutral_point_of_view {Online; accessed 15-April-2018}.Google Scholar
- Wikipedia. 2018. Wikipedia. https://en.wikipedia.org/wiki/Wikipedia {Online; accessed 15-April-2018}.Google Scholar
- Wikipedia. 2018. Wikipedia Words to Watch. https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style/Words_to_ watch {Online; accessed 15-April-2018}.Google Scholar
- Wikipedia. 2018. Wikipedia:Expectations and Norms of the Wikipedia Community. https://en.wikipedia.org/wiki/ Wikipedia: Expectations_and_norms_of_the_Wikipedia_community {Online; accessed 15-April-2018}.Google Scholar
- Wikipedia. 2018. Wikipedia:Manual of Style. https://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style {Online; accessed 15-April-2018}.Google Scholar
- Kevin Wise, Brian Hamman, and Kjerstin Thorson. 2006. Moderation, Response Rate, and Message Interactivity: Features of Online Communities and Their Effects on Intent to Participate. Journal of Computer-Mediated Communication 12, 1 (2006), 24--41.Google ScholarCross Ref
- Diyi Yang, Aaron Halfaker, Robert E Kraut, and Eduard H Hovy. 2016. Who Did What: Editor Role Identification in Wikipedia. In Proceedings of the International AAAI Conference on Web and Social Media (ICWSM). 446--455.Google Scholar
- Heng-Li Yang and Cheng-Yu Lai. 2010. Motivations of Wikipedia Content Contributors. Computers in Human Behavior 26, 6 (2010), 1377--1383. Google ScholarDigital Library
Index Terms
- Mind Your POV: Convergence of Articles and Editors Towards Wikipedia's Neutrality Norm
Recommendations
The Internet's Hidden Rules: An Empirical Study of Reddit Norm Violations at Micro, Meso, and Macro Scales
Norms are central to how online communities are governed. Yet, norms are also emergent, arise from interaction, and can vary significantly between communities---making them challenging to study at scale. In this paper, we study community norms on Reddit ...
DAWT: Densely Annotated Wikipedia Texts Across Multiple Languages
WWW '17 Companion: Proceedings of the 26th International Conference on World Wide Web CompanionIn this work, we open up the DAWT dataset - Densely Annotated Wikipedia Texts across multiple languages. The annotations include labeled text mentions mapping to entities (represented by their Freebase machine ids) as well as the type of the entity. The ...
Learning multilingual named entity recognition from Wikipedia
We automatically create enormous, free and multilingual silver-standard training annotations for named entity recognition (ner) by exploiting the text and structure of Wikipedia. Most ner systems rely on statistical models of annotated data to identify ...
Comments