skip to main content
10.1145/1639642.1639672acmconferencesArticle/Chapter ViewAbstractPublication PagesassetsConference Proceedingsconference-collections
research-article

Speaking through pictures: images vs. icons

Published: 25 October 2009 Publication History

Abstract

People with aphasia, a condition that impairs the ability to understand or generate written or spoken language, are aided by assistive technology that helps them communicate through a vocabulary of icons. These systems are akin to language translation systems, translating icon arrangements into spoken or written language and vice versa. However, these icon-based systems have little vocabulary breadth or depth, making it difficult for people with aphasia to apply their usage to multiple real world situations. Pictures from the web are numerous, varied, and easily accessible and thus, could potentially address the small size issues of icon-based systems. We present results from two studies that investigate this potential and demonstrate that images can be as effective as icons when used as a replacement for English language communication. The first study uses elderly subjects to investigate the efficacy of images vs. icons in conveying word meaning; the second study examines the retention of word-level meaning by both images and icons with a population of aphasics. We conclude that images collected from the web are as functional as icons in conveying information and thus, are feasible to use in assistive technology that supports people with aphasia.

References

[1]
Ageless Project. http://jenett.org/ageless/. 2006
[2]
Allen, M., McGrenere, J., and Purves, B. The Field Evaluation of a Mobile Digital Image Communication Application Designed for People with Aphasia, ACM Transactions on Accessible Computing, 1(1), 1--26, 2008
[3]
Boyd-Graber, J., Nikolova, S., Moffatt, K., Kin, K., Lee, J., Mackey, L., Tremaine, M., and Klawe. M. Participatory design with proxies: Developing a desktop-PDA system to support people with aphasia. In Proc. CHI 2006, 151--160. ACM Press, 2006.
[4]
Danielsson, H. and Jonsson, B. Pictures as language. In Proc. International Conference on Language and Visualization 2001, Stockholm, Sweden, 2001.
[5]
Deng, J., Dong, W., Socher, R., Li, L-J., Li, K. and Fei-Fei, L. ImageNet: A Large-Scale Hierarchical Image Database. In Proc. CVPR 2009, IEEE Press, 2009.
[6]
DynaVox. http://www.dynavoxtech.com/. 2006
[7]
Fellbaum, C. WordNet: Electronic Lexical Database, A semantic network of English verbs. MIT Press, 1998.
[8]
General Text Matcher. http://nlp.cs.nyu.edu/GTM/. 2006
[9]
Goodglass, H., Kaplan, E. and Barresi, B. The assessment of aphasia and related disorders (3rd Ed.). Philadelphia: Lippincott Williams&Wilkins, 2001.
[10]
Hochberg, J. and Brooks, V. Pictorial recognition as an unlearned ability: a study of one child's performance", American Journal of Psychology, 75, 624--628. 1962
[11]
Kjelldahl, L. A Survey of Some Perceptual Features for Computer Graphics and Visualization. In Proc. Linkoping Electronic, 2003
[12]
Lingraphica. http://www.lingraphicare.com/. 2005
[13]
Lucia, C. Toward Developing a Model of Music Therapy Intervention in the Rehabilitation of Head Trauma Patients. Music Therapy Perspectives. 4, 34--39.
[14]
Moffatt, K., McGrenere, J., Purves, B., and Klawe, M. The Participatory Design of a Sound and Image Enhanced Daily Planner for People with Aphasia. In Proc. CHI 2004, 407--414. ACM Press 2004.
[15]
Nadel, S. A field experiment in racial psychology. British Journal of Psychology, 28, 195--211. 1937
[16]
National Aphasia Association. http://www.aphasia.org.
[17]
Prentke Romich Company. http://www.prentrom.com/. 2006
[18]
Ryan, T, and Schwartz, C. Speed of perception as a function of mode of representation. American Journal of Psychology, 69, 193--199. 1956
[19]
Rubin, E. Figure and ground. In DCW Beardslee, M. (Ed.), Readings in perception. New York: Van Nostrand, 1958.
[20]
Steele R., Weinrich M, Wertz RT, Kleczewska MK, Carlson GS. Computer-based visual communication in aphasia. Neuropsychologia; 27(4):409--26. 1989
[21]
Thorburn, L., Newhoff, M., and Rubin, S. Ability of Subjects with Aphasia to Visually Analyze Written Language, Pantomime, and Iconographic Symbols. American Journal of Speech Language Pathology, 4(4): 174--179, 1995
[22]
Van de Sandt-Koenderman, M., Wiegers, M., and Hardy, P. A Computerized Communication Aid for People with Aphasia. Disability Rehabilitation, 27(9): 529--533, 2005
[23]
Von Ahn, L. and Dabbish, L. Labeling Images with a Computer Game. In Proc. CHI 2004, 319--326. ACM Press, 2004

Cited By

View all
  • (2023)Identifying an Appropriate Picture Stimulus for a Bangla Picture Description TaskJournal of Speech, Language, and Hearing Research10.1044/2022_JSLHR-22-0015266:4(1334-1350)Online publication date: 12-Apr-2023
  • (2022)Accessibility-Related Publication Distribution in HCI Based on a Meta-AnalysisExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519701(1-28)Online publication date: 27-Apr-2022
  • (2022)A Survey of Technologies Facilitating Home and Community-Based Stroke RehabilitationInternational Journal of Human–Computer Interaction10.1080/10447318.2022.205054539:5(1016-1042)Online publication date: 20-Apr-2022
  • Show More Cited By

Index Terms

  1. Speaking through pictures: images vs. icons

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    Assets '09: Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility
    October 2009
    290 pages
    ISBN:9781605585581
    DOI:10.1145/1639642
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 October 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. aphasia
    2. computerized visual communication (C-VIC)
    3. visual communication (VIC)

    Qualifiers

    • Research-article

    Conference

    ASSETS09
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 436 of 1,556 submissions, 28%

    Upcoming Conference

    ASSETS '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)32
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2023)Identifying an Appropriate Picture Stimulus for a Bangla Picture Description TaskJournal of Speech, Language, and Hearing Research10.1044/2022_JSLHR-22-0015266:4(1334-1350)Online publication date: 12-Apr-2023
    • (2022)Accessibility-Related Publication Distribution in HCI Based on a Meta-AnalysisExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491101.3519701(1-28)Online publication date: 27-Apr-2022
    • (2022)A Survey of Technologies Facilitating Home and Community-Based Stroke RehabilitationInternational Journal of Human–Computer Interaction10.1080/10447318.2022.205054539:5(1016-1042)Online publication date: 20-Apr-2022
    • (2022)Naming Images in Aphasia: Effects of Graphic Representations and Photographs on Naming Performance in Persons With and Without AphasiaAphasiology10.1080/02687038.2022.206442137:7(993-1015)Online publication date: 3-May-2022
    • (2019)Participatory design for selection of icons to represent daily activities of living for a vision-based rehabilitation-cum-assistance system for locked-in patientsDisability and Rehabilitation: Assistive Technology10.1080/17483107.2019.156859315:3(282-291)Online publication date: 18-Feb-2019
    • (2015)TalkingTilesProceedings of the 2015 International Conference on Interactive Tabletops & Surfaces10.1145/2817721.2817723(63-72)Online publication date: 15-Nov-2015
    • (2015)Accessibility to electronic communication for people with cognitive disabilitiesUniversal Access in the Information Society10.1007/s10209-014-0351-614:4(547-562)Online publication date: 1-Nov-2015
    • (2015)Visual Predictions of Traffic ConditionsAdvances in Artificial Intelligence10.1007/978-3-319-18356-5_11(122-129)Online publication date: 29-Apr-2015
    • (2014)Augmenting text with multiple pictures can facilitate online information processing across language barriersProceedings of the Second International Symposium of Chinese CHI10.1145/2592235.2592247(80-86)Online publication date: 26-Apr-2014
    • (2013)Expressing through digital photographsUniversal Access in the Information Society10.1007/s10209-012-0286-812:3(309-326)Online publication date: 1-Aug-2013
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media