Technologies, Methods, and Values: Changes in Empirical Research at CSCW 1990 - 2015

Abstract

The field of Computer Supported Cooperative Work has constantly evolved to meet the changing needs of individuals at home, at work, and online. To understand how these changes impacted CSCW research, we systematically reviewed 1209 papers and notes published at the ACM Conference on Computer Supported Cooperative Work between 1990 and 2015. When considered with results from two previous literature reviews, covering 1990 - 1998 and 1998 - 2004 respectively, our analysis provides perspective on 25 years of groupware research. We show that the field has responded to, not anticipated, changes in the computing landscape, long-term trends away from 'systems' and explanatory research, and a lack of bibliographic research that synthesizes findings. Finally, we discuss implications of these trends for CSCW research: how results are synthesized across the field, what kinds of research we value, and how multi-device ecologies are studied.

References

1. James "Bo" Begole, John C. Tang, Randall B. Smith, and Nicole Yankelovich. 2002. Work Rhythms: Analyzing Visualiza- tions of Awareness Histories of Distributed Groups. In Proceedings of the 2002 ACM Conference on Computer Supported Cooperative Work (CSCW '02). ACM, New York, NY, USA, 334--343.
2. Jeanette Blomberg and Helena Karasti. 2013. Re ections on 25 Years of Ethnography in CSCW. In Journal of Computer- Supported Cooperative Work, Vol. 22. 373--423.
3. Justin Cheng and Michael Bernstein. 2014. Catalyst: Triggering Collective Action with Thresholds. In Proceedings of the 17th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '14). ACM, New York, NY, USA, 1211--1221.
4. Parmit K. Chilana, Andrew J. Ko, and Jacob Wobbrock. 2015. From User-Centered to Adoption-Centered Design: A Case Study of an HCI Research Innovation Becoming a Product. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 1749--1758.
5. Peter Dalsgaard and Kim Halskov. 2010. Designing Urban Media Façades: Cases and Challenges. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '10). ACM, New York, NY, USA, 2277--2286.
6. Horace Dediu. 2013. When Apple Reached Parity with Windows. (2013). http://www.asymco.com/2014/01/13/ when-apple-reached-parity-with-windows/
7. Gerardine Desanctis and R. Brent Gallupe. 1987. A Foundation for the Study of Group Decision Support Systems. Manage. Sci. 33, 5 (May 1987), 589--609.
8. Paul Dourish and Victoria Bellotti. 1992. Awareness and Coordination in Shared Workspaces. In Proceedings of the 1992 ACM Conference on Computer-supported Cooperative Work (CSCW '92). ACM, New York, NY, USA, 107--114. Proc. ACM Hum.-Comput. Interact., Vol. 1, No. 1, Article 106. Publication date: January 2017. Changes in Empirical Research at CSCW 106:17
9. Mary Elwart-Keys, David Halonen, Marjorie Horton, Robert Kass, and Paul Scott. 1990. User interface requirements for face to face groupware. In CHI '90: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, New York, NY, USA, 295--301.
10. JoelFischer,MartinPorcheron,AndrésLucero,AaronQuigley,StaceyScott,LuiginaCiol, JohnRooksby,andNemanja Memarovic. 2016. Collocated Interaction: New Challenges in Same Time, Same Place Research. In Proceedings of the 19th ACM Conference on Computer Supported Cooperative Work and Social Computing Companion (CSCW '16 Companion). ACM, New York, NY, USA, 465--472.
11. GeraldineFitzpatrickandGunnarEllingsen.2012.AReviewof25YearsofCSCWResearchinHealthcare:Contributions, Challenges and Future Agendas. In Journal of Computer-Supported Cooperative Work, Vol. 22. 609--665.
12. Saul Greenberg and Bill Buxton. 2008. Usability Evaluation Considered Harmful (Some of the Time). In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '08). ACM, New York, NY, USA, 111--120.
13. Jonathan Grudin. 1994. Groupware and social dynamics: eight challenges for developers. Commun. ACM 37, 1 (1994), 92--105.
14. Jonathan Grudin. 2017. From Tool to Partner: The Evolution of Human-Computer Interaction. Synthesis Lectures on Human-Centered Interaction 10, 1 (2017), i--183.
15. Robert Johansen, David Sibbet, Suzyan Benson, Alixia Martin, Robert Mittman, and Paul Sa o. 1991. Leading Business Teams: How Teams Can Use Technology and Group Process Tools to Enhance Performance. Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA.
16. Joseph 'Jofish' Kaye and Phoebe Sengers. 2007. The Evolution of Evaluation. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, Article 2.
17. Brian Keegan, Dan Horn, Thomas A Finholt, Joseph Kaye, and others. 2013. Structure and Dynamics of Coauthorship, Citation, and Impact within CSCW. arXiv preprint arXiv:1307.7172 (2013).
18. Barbara Kitchenham, O Pearl Brereton, David Budgen, Mark Turner, John Bailey, and Stephen Linkman. 2009. System- atic Literature Reviews in Software Engineering -- A Systematic Literature Review. Information and software technology 51, 1 (2009), 7--15.
19. Adam DI Kramer, Jamie E Guillory, and Je rey T Hancock. 2014. Experimental evidence of massive-scale emotional contagion through social networks. Proceedings of the National Academy of Sciences 111, 24 (2014), 8788--8790.
20. Charlotte P. Lee and Drew Paine. 2015. From The Matrix to a Model of Coordinated Action (MoCA): A Conceptual Framework of and for CSCW. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, New York, NY, USA, 179--194.
21. H Lieberman. 2003. The Tyranny of Evaluation. ACM CHI Fringe. http://web.media.mit.edu/~lieber/Misc/ TyrannyEvaluation.html
22. Gloria Mark, Shamsi T. Iqbal, Mary Czerwinski, and Paul Johns. 2014. Bored Mondays and Focused Afternoons: The Rhythm of Attention and Online Activity in the Workplace. In Proceedings of the 32Nd Annual ACM Conference on Human Factors in Computing Systems (CHI '14). ACM, New York, NY, USA, 3025--3034.
23. Preben Mogensen and Mike Robinson. 1995. Triggering Artefacts. In AI & Society, Vol. 9. Springer-Verlag, London, 373--388.
24. Dan R. Olsen, Jr. 2007. Evaluating User Interface Systems Research. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (UIST '07). ACM, New York, NY, USA, 251--258.
25. Wanda J. Orlikowski. 1992. Learning from Notes: Organizational Issues in Groupware Implementation. In Proceedings of the 1992 ACM Conference on Computer-supported Cooperative Work (CSCW '92). ACM, New York, NY, USA, 362--369.
26. David Pinelle and Carl Gutwin. 2000. A Review of Groupware Evaluations. In Proceedings of the 9th IEEE International Workshops on Enabling Technologies: Infrastructure for Collaborative Enterprises (WETICE '00). IEEE, Washington, DC, USA, 86--91.
27. Stacey D. Scott, Karen D. Grant, and Regan L. Mandryk. 2003. System Guidelines for Co-located, Collaborative Work on a Tabletop Display. In Proceedings of the eighth conference on European Conference on Computer Supported Cooperative Work (ECSCW'03). Kluwer Academic Publishers, Norwell, MA, USA, 159--178.
28. R. Sommer. 1969. Personal Space; The Behavioral Basis of Design. Prentice-Hall, Englewood Cli s, New Jersey.
29. Murray Turo.1985. Information, Value, and the Internal Marketplace. Technological Forecasting and Social Change 27, Proc. ACM Hum.-Comput. Interact., Vol. 1, No. 1, Article 106. Publication date: January 2017. 106:18 James R. Wallace, Saba Oji, and Craig Anslow 4 (1985), 357--373.
30. Jacques Wainer and Claudia Barsottini. 2007. Empirical Research in CSCW -- a Review of the ACM/CSCW Conferences from 1998 to 2004. Journal of the Brazilian Computer Society 13, 3 (2007), 27--35.
31. James R. Wallace and Stacey D. Scott. 2008. Contextual Design Considerations for Co-located, Collaborative Tables. In IEEE Symposium on Tabletops and Interactive Surfaces (TABLETOP 2008). IEEE, Amsterdam, The Netherlands, 57--64.
32. Shuman Zhai. 2003. Evaluation is the Worst Form of HCI Research Except All Those Other Forms that Have Been Tried. CHI Place Essay (2003). http://www.shuminzhai.com/papers/EvaluationDemocracy.htm

Index Terms

• Human-centered computing

  • Human computer interaction (HCI)

    • HCI theory, concepts and models