Benefits of synchronous collaboration support for an application-centered analysis team working on complex problems: a case study

ABSTRACT

A month-long quasi-experiment was conducted using a distributed team responsible for modeling, simulation, and analysis. Six experiments of three different time durations (short, medium, and long) were performed. The primary goal was to discover if synchronous collaboration capability through a particular application improved the ability of the team to form a common mental model of the analysis problem(s) and solution(s). The results indicated that such collaboration capability did improve the formation of common mental models, both in terms of time and quality (i.e., depth of understanding), and that the improvement did not vary by time duration. In addition, common mental models were generally formed by interaction around a shared graphical image, the progress of collaboration was not linear but episodic, and tasks that required drawing and conversing at the same time were difficult to do.

REFERENCES

• 1: Dominik Buszko , Wei-Hsing (Dan) Lee , Abdelsalam (Sumi) Helal, Decentralized ad-hoc groupware API and framework for mobile collaboration, Proceedings of the 2001 International ACM SIGGROUP Conference on Supporting Group Work, September 30-October 03, 2001, Boulder, Colorado, USA [doi>10.1145/500286.500291]
• 2: Campbell, Donald T., & Julian C. Stanley. (1966). Experimental and Quasi-Experimental Designs for Research. Boston: Houghton Mifflin Company.
• 3: Annie Chabert , Ed Grossman , Larry S. Jackson , Stephen R. Pietrowiz , Chris Seguin, Java object-sharing in Habanero, Communications of the ACM, v.41 n.6, p.69-76, June 1998 [doi>10.1145/276609.276622]
• 4: George Chin, Jr. , Carina S. Lansing, Capturing and supporting contexts for scientific data sharing via the biological sciences collaboratory, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031677]
• 5: Disz, Terry. (2001). "The Access Grid Collaboration Environment." Syllabus (April 2001). Available at http://www.campus-technology.com/article.asp?id=3238.
• 6: Doser, Adele B., Peter B. Merkle, Curtis Johnson, Wendell Jones, David Warner, & Tim Murphy. (2003). "Enabling Technology for Human Collaboration." Sandia National Laboratories Technical Report SAND 2003-4225. Available at http://www.bodymedia.com/pdf/sandia-4225.pdf.
• 7: Eidson, Eric D., & Mark A. Ehlen. (2005). "NISAC Agent-Based Laboratory for Economics (N-ABLE™™): Overview of Agent and Simulation Architectures." Sandia National Laboratories Technical Report SAND2005-0263.
• 8: Espinosa, J. Alberto, Robert E. Kraut, Sandra A. Slaughter, Javier F. Lerch, James D. Herbsleb, & Audris Mockus. (2001). "Shared Mental Models and Coordination in Large-Scale, Distributed Software Development." Proceedings of the International Conference on Information Systems (ICIS 2001), 16-19 December, New Orleans, LA, USA, 513--518.
• 9: Espinosa, J. Alberto, Robert E. Kraut, Sandra A. Slaughter, Javier F. Lerch, James D. Herbsleb, & Audris Mockus. (2002). "Shared Mental Models, Familiarity, and Coordination: A Multi-Method Study of Distributed Software Teams." Proceedings of the Twenty-Third International Conference on Information Systems (ICIS 2002), 15-18 December, Barcelona, Spain, 425--433.
• 10: Chris Furmanski , David Payton , Mike Daily, Quantitative Evaluation Methodology for Dynamic, Web-Based Collaboration Tools, Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS'04) - Track 5, p.50132.1, January 05-08, 2004
• 11: Saul Greenberg , Carl Gutwin , Andy Cockburn, Awareness through fisheye views in relaxed-WYSIWIS groupware, Proceedings of the conference on Graphics interface '96, p.28-38, May 1996, Toronto, Ontario, Canada
• 12: Carl Gutwin , Saul Greenberg , Mark Roseman, Workspace Awareness in Real-Time Distributed Groupware: Framework, Widgets, and Evaluation, Proceedings of HCI on People and Computers XI, p.281-298, January 1996
• 13: http://www.lotus.com/products/product3.nsf/wdocs/ homepage http://www.lotus.com/products/product3.nsf/wdocs/ltwhome
• 14: http://www.webex.com/
• 15: http://www.microsoft.com/windows/netmeeting/ http://www.microsoft.com/office/livemeeting/prodinfo/default.mspx
• 16: Susanne Hupfer , Li-Te Cheng , Steven Ross , John Patterson, Introducing collaboration into an application development environment, Proceedings of the 2004 ACM conference on Computer supported cooperative work, November 06-10, 2004, Chicago, Illinois, USA [doi>10.1145/1031607.1031611]
• 17: Ellen Isaacs , Alan Walendowski , Steve Whittaker , Diane J. Schiano , Candace Kamm, The character, functions, and styles of instant messaging in the workplace, Proceedings of the 2002 ACM conference on Computer supported cooperative work, November 16-20, 2002, New Orleans, Louisiana, USA [doi>10.1145/587078.587081]
• 18: Klimoski, Richard, & Susan Mohammed. (1994). "Team Mental Model: Construct or Metaphor?" Journal of Management 20(2), 403--437.
• 19: Langan-Fox, Janice, Sharon Code, & Kim Langfield-Smith. (2000). "Team Mental Models: Techniques, Methods, and Analytic Approaches." Human Factors 42(2), 242--271.
• 20: Levesque, Laurie L., & Jeanne M. Wilson. (2001). "Cognitive divergence and shared mental models in software development project teams." Journal of Organizational Behavior 22(2), 135--144.
• 21: Linebarger, John M., Christopher D. Janneck, & G. Drew Kessler. (2005). "'Leaving the World Behind:' Supporting Group Collaboration Patterns in a Shared Virtual Environment for Product Design." Scheduled for publication in Presence: Teleoperators and Virtual Environments 14(6).
• 22: Mathieu, John E., Gerald F. Goodwin, Tonia S. Heffner, Eduardo Salas, & Janis A. Cannon-Bowers. (2000). "The Influence of Shared Mental Models on Team Process and Performance." Journal of Applied Psychology 85(2), 273--283.
• 23: Joseph Edward McGrath , Andrea B. Hollingshead, Groups Interacting with Technology: Ideas, Evidence, Issues and an Agenda, Sage Publications, Inc., Thousand Oaks, CA, 1993
• 24: McLeod, Poppy Lauretta. (1992). "An Assessment of the Experimental Literature on the Electronic Support of Group Work: Results of a Meta-Analysis." Human-Computer Interaction 7(3), 257--280.
• 25: Montgomery, Douglas C. (2001). Design and Analysis of Experiments. 5th Edition. New York: John Wiley & Sons.
• 26: Bonnie A. Nardi , Steve Whittaker , Erin Bradner, Interaction and outeraction: instant messaging in action, Proceedings of the 2000 ACM conference on Computer supported cooperative work, p.79-88, December 2000, Philadelphia, Pennsylvania, United States [doi>10.1145/358916.358975]
• 27: Noble, David, & Michael Letsky. (2001). "Cognitive-Based Metrics to Evaluate Collaboration Effectiveness." Available at http://www.onr.navy.mil/sci_tech/personnel/342/342_ckm/ pubs/docs/33_mgc_ebr/cognitive_based_metrics.doc.
• 28: Noble, David, Diana Buck, & Jim Yeargain. (2001). "Metrics for Evaluation of Cognitive-Based Collaboration Tools." Proceedings of the 6th International Command and Control Research and Technology Symposium, 19-21 June, Annapolis, MD, USA. Available at http://www.dodccrp.org/events/2001/6th_ICCRTS/Cd/Tracks/Papers/Track6/106_tr6.pdf.
• 29: Procopio, Michael J. (2002). "YCab.NET: Decentralized Collaboration Groupware for Mobile Devices using the Microsoft .NET Framework." Master's of Science Thesis, University of Florida. Available at http://etd.fcla.edu/UF/UFE1000155/procopio_m.pdf
• 30: Rentsch, J. R., & R. J. Hall. (1994). "Members of Great Teams Think Alike: A Model of Team Effectiveness and Schema Similarity Among Team Members." In M. M. Beyerlein & D. A. Johnson, eds., Advances in Interdisciplinary Studies of Work Teams: Theories of Self-Managing Work Teams, Vol. 1 (Greenwich, CT, USA: JAI Press), 223--261.
• 31: Rentsch, Joan R., & Richard J. Klimoski. (2001). "Why do 'great minds' think alike?: antecedents of team member schema agreement." Journal of Organizational Behavior 22(2), 107--120.
• 32: Rouse, William B., & Nancy M. Morris. (1986). "On Looking Into the Black Box: Prospects and Limits in the Search for Mental Models." Psychological Bulletin 100(3), 349--363.
• 33: Shadish, William R., Thomas D. Cook, & Donald T. Campbell. (2002). Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Boston: Houghton Mifflin Company.
• 34: M. Stefik , D. G. Bobrow , G. Foster , S. Lanning , D. Tatar, WYSIWIS revised: early experiences with multiuser interfaces, ACM Transactions on Information Systems (TOIS), v.5 n.2, p.147-167, April 1987 [doi>10.1145/27636.28056]
• 35: Judith D. Wilson , Nathan Hoskin , John T. Nosek, The benefits of collaboration for student programmers, Proceedings of the twenty-fourth SIGCSE technical symposium on Computer science education, p.160-164, February 18-19, 1993, Indianapolis, Indiana, United States

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Collaborative computing

Additional Classification:
  H. Information Systems
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)
      H.5.3 Group and Organization Interfaces
          Subjects: Computer-supported cooperative work

General Terms:
Design, Experimentation, Human Factors, Measurement

Keywords:
benefits of collaboration, collaboration experiments, collaboration frameworks, common mental models, synchronous collaboration