Article

Challenges in the analysis of multimodal messaging

Authors:

Elizabeth D. MynattAuthors Info & Claims

CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work

Pages 427 - 430

https://doi.org/10.1145/1180875.1180939

Published: 04 November 2006 Publication History

Abstract

New forms of computer-mediated communication are increasingly multimodal, providing capabilities for communicating with some combination of text, image, audio, and video. In this paper, we point to the need to develop better methods for studying multimodal communication -- more specifically, for studying thencommunicative role of and relationships among differentnmodalities within their increasingly complex, multimodal semioticnlandscapes. We present two challenges in the analysis of multimodal communication, point of view and unit of analysis, both encountered in the context of our study of the use of photo-enhanced instant messaging.

References

[1]

Battarbee, K. (2003). Defining co-experience. In Proceedings of the Conference on Designing Pleasurable Products and Interfaces. New York: ACM Press, pp. 109--113.

Digital Library

[2]

Wimsatt, W.K. & Beardsley, M.C. (1987). The intentional fallacy. Reprinted in J. Margolis (Ed.), Philosophy looks at the arts, 3rd ed. Philadelphia: Temple University Press.

[3]

Crabtree, A., Rodden, T., & Mariani, J. (2004). Collaborating around collections: Informing the continued development of photoware. In Proceedings of the ACM Conference on Computer Supported Cooperative Work. New York: ACM Press, pp. 396--405.

Digital Library

[4]

Erickson, R. (2004). Origins: A brief intellectual and technological history of the emergence of multimodal discourse analysis. In P. LeVine & R. Scollon (Eds.), Discourse and technology: Multimodal discourse analysis. Washington, D.C.: Georgetown University Press, pp. 196--207.

[5]

Fabersjö, H., Windt, E., Wridell, Y., & Sanneblad, J. (2003). Amigo - wireless image based instant messaging for handheld computers. In Extended Abstracts of the ACM Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 910--911.

Digital Library

[6]

Gombrich, E.H. (1960). Art and illusion: A study of the psychology of pictorial representation. Princeton, NJ: Princeton University Press.

[7]

Gombrich, E.H. (1982). The visual image: Its place in communication. In E.H. Gombrich (Ed.), The image and the eye. Oxford, UK: Phaidon Press, pp. 137--161.

[8]

Goodwin, C. (2003). The semiotic body in its environment. In J. Coupland & R. Gwyn (Eds.), Discourses of the body. New York: Palgrave/Macmillan, pp. 19--42.

[9]

Heath, C. & Luff, P. (1992). Explicating face-to-face interaction. In G.N. Gilbert (Ed.), Researching social life. London: Sage, pp. 306--327.

[10]

Kindberg, T., Spasojevic, M., Fleck, R., & Sellen, A. (2005). The ubiquitous camera: An in-depth study of camera phone use. IEEE Pervasive Computing, 4(2), 42--50.

Digital Library

[11]

Koskinen, I., Kurvinen, E., & Lehtonen, T. (2002). Mobile Image. Edita, Finalnd: IT Press.

[12]

Kress, G. & van Leeuwen, T. (1996). Reading images: The grammar of visual design. London: Routledge.

[13]

Kress, G. & van Leeuwen, T. (2001). Multimodal discourse: The modes and media of contemporary communication. New York: Oxford University Press.

[14]

Mäkelä, A., Giller, V., Tscheligi, M., & Sefelin, R. (2000). Joking, storytelling, artsharing, expressing affection: A field trial of how children and their social network communicate with digital images in leisure time. In Proceedings of the ACM Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 548--555.

Digital Library

[15]

McCloud, S. (1993). Understanding comics: The invisible art. New York: Harper Collins.

[16]

Voida, A. & Mynatt, E.D. (2005). Six themes of the communicative appropriation of photographic images. In Proceedings of the ACM Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 171--180.

Digital Library

[17]

White, M. (1957). Criticism. Aperture, 2(2), 29--30.

Cited By

Jourde FLaurillau YNigay LSukaviriya NVanderdonckt JHarrison M(2010)COMM notation for specifying collaborative and multimodal interactive systemsProceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems10.1145/1822018.1822038(125-134)Online publication date: 19-Jun-2010
https://dl.acm.org/doi/10.1145/1822018.1822038

Index Terms

Challenges in the analysis of multimodal messaging
1. Human-centered computing
  1. Collaborative and social computing
2. Information systems
  1. Information systems applications
    1. Multimedia information systems
  2. World Wide Web
    1. Web applications
      1. Internet communications tools

Recommendations

Awareness of presence, instant messaging and WebWho

We report from a study of how awareness of presence can affect instant messaging behavior. WebWho is a web based awareness system that visualizes where people are located in a large university computer lab. It allows students to virtually locate one ...
Communication characteristics of instant messaging: effects and predictions of interpersonal relationships
CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work

Instant Messaging is a popular medium for both social and work-related communication. In this paper we report an investigation of the effect of interpersonal relationship on underlying basic communication characteristics (such as messaging rate and ...
RFC 6121: Extensible Messaging and Presence Protocol (XMPP): Instant Messaging and Presence

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CSCW '06: Proceedings of the 2006 20th anniversary conference on Computer supported cooperative work

November 2006

548 pages

ISBN:1595932496

DOI:10.1145/1180875

General Chairs:
Pamela Hinds
Stanford University, USA
,
David Martin
SMART Technologies Inc., Canada

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 November 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

CSCW06

Sponsor:

CSCW06: Computer Supported Cooperative Work

November 4 - 8, 2006

Alberta, Banff, Canada

Acceptance Rates

Overall Acceptance Rate 2,235 of 8,521 submissions, 26%

Upcoming Conference

CSCW '25

Sponsor:
sigchi

Computer-Supported Cooperative Work and Social Computing

October 18 - 22, 2025

Bergen , Norway

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
560
Total Downloads

Downloads (Last 12 months)1
Downloads (Last 6 weeks)0

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Jourde FLaurillau YNigay LSukaviriya NVanderdonckt JHarrison M(2010)COMM notation for specifying collaborative and multimodal interactive systemsProceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems10.1145/1822018.1822038(125-134)Online publication date: 19-Jun-2010
https://dl.acm.org/doi/10.1145/1822018.1822038

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten