skip to main content
10.1145/1322192.1322195acmconferencesArticle/Chapter ViewAbstractPublication Pagesicmi-mlmiConference Proceedingsconference-collections
keynote

Just in time learning: implementing principles of multimodal processing and learning for education

Published: 12 November 2007 Publication History

Abstract

Baldi, a 3-D computer-animated tutor has been developed to teach speech and language. I review this technology and pedagogy and describe evaluation experiments that have substantiated the effectiveness of our language-training program, Timo Vocabulary, to teach vocabulary and grammar. With a new Lesson Creator, teachers, parents, and even students can build original lessons that allow concepts, vocabulary, animations, and pictures to be easily integrated. The Lesson Creator application facilitates the specialization and individualization of lessons by allowing teachers to create customized vocabulary lists Just in Time as they are needed. The Lesson Creator allows the coach to give descriptions of the concepts as well as corrective feedback, which allows errorless learning and encourages the child to think as they are learning. I describe the Lesson Creator, illustrate it, and speculate on how its evaluation can be accomplished.

References

[1]
Anderson, R. C., & Freebody, P. (1981). Vocabulary knowledge. In J. T. Guthrie (Ed.), Comprehension and teaching: Research perspectives (pp. 71--117). Newark, DE: International Reading Association.
[2]
Beck, I. L., McKeown, M. G., & Kucan, L. (2002). Bringing words to life: Robust vocabulary instruction. New York: The Guilford Press.
[3]
Bosseler, A. and Massaro, D. W. (2003). Development and Evaluation of a Computer-Animated Tutor for Vocabulary and Language Learning for Children with Autism. Journal of Autism and Developmental Disorders, 33, 653--672.
[4]
Cooke, N. J. (1994). Varieties of knowledge elicitation techniques. International Journal of Human-Computer Studies, 41, 801--849.
[5]
Dubois, M., & Vial, I. (2000) Multimedia design: the effects of relating multimodal information. Journal of Computer Assisted Learning, 16, 157--165.
[6]
Ericsson, A. (2006). Protocol analysis and expert thought: Concurrent verbalizations of thinking during experts' performance on representative tasks. In A. Ericsson, N, Charness, P. J. Feltovich, & R. R. Hoffman (Eds.) The Cambridge Handbook of Expertise and Expert Performance (pp. 223--241). Cambridge: Cambridge University Press, 2006.
[7]
Gupta, P., & MacWhinney, B. (1997) Vocabulary acquisition and verbal short-term memory: computation and neural bases. Brain and Language, 59, 267--333.
[8]
Hoffman, R. R., & Lintern, G. (2006). Eliciting and representing the knowledge of experts. In A. Ericsson, N, Charness, P. J. Feltovich, & R. R. Hoffman (Eds.) The Cambridge Handbook of Expertise and Expert Performance (pp. 203--222). Cambridge: Cambridge University Press, 2006.
[9]
Gee, J. P. (2003). What video games have to teach us about learning and literacy. New York: Palgrave/Macmillan.
[10]
Horner, R. D., & Baer, D. M. (1978). Multiple-probe technique: A variation of the multiple baseline. Journal of Applied Behavior Analysis, 11, 189--196.
[11]
Lesson Creator. Animated Speech Corporation. http://animatedspeech.com/Products/products_lessoncreator.html
[12]
Massaro, D. W. (1987). Speech perception by ear and eye: A Paradigm for psychological inquiry. Hillsdale, NJ: Erlbaum.
[13]
Massaro, D. W. (1998). Perceiving talking faces: From speech perception to a behavioral principle. Cambridge, Massachusetts: MIT Press.
[14]
Massaro, D. W. (2003). A computer-animated tutor for spoken and written language learning. . Proceedings of the 5th international conference on Multimodal interfacesall(pp. 172--175).Vancouver, British Columbia, Canada. November 05--07, 2003.
[15]
Massaro, D. W. (2004). Symbiotic Value of an Embodied Agent in Language Learning. In R.H. Sprague, Jr.(Ed.), Proceedings of 37th Annual Hawaii International Conference on System Sciences,(HICCS'04) (CD--ROM,10 pages). Los Alimitos, CA: IEEE Computer Society Press. Best paper in Emerging Technologies.
[16]
Massaro, D. W. (2006). Embodied Agents in Language Learning for Children with Language Challenges. Proceedings of the 10th International Conference on Computers Helping People with Special Needs, ICCHP 2006 pp.809--816. University of Linz, Austria. Berlin, Germany: Springer.
[17]
Massaro, D. W. & Bosseler, A. (2003). Perceiving Speech by Ear and Eye: Multimodal Integration by Children with Autism. Journal of Developmental and Learning Disorders, 7, 111--144.
[18]
Massaro, D. W., & Light, J. (2004). Improving the vocabulary of children with hearing loss. Volta Review, 104(3), 141--174.
[19]
Massaro, D. W., Ouni, S., Cohen, M. M., & Clark, R. (2005). A Multilingual Embodied Conversational Agent. In R.H. Sprague (Ed.), Proceedings of 38th Annual Hawaii International Conference on System Sciences, (HICCS'05) (CD--ROM,10 pages). Los Alimitos, CA: IEEE Computer Society Press.
[20]
Meyer, D., Princiotta, D., and Lanahan, L., (2004). The Summer After Kindergarten: Children's Activities and Library Use by Household Socioeconomic Status, National Center for Education Statistics, Institute of Education Sciences, U.S. Dept. of Education, Education Statistics Quarterly, 6:3.
[21]
Nagy, W., & Herman, P. A. (1987). Breadth and depth of vocabulary knowledge: Implications for acquisition and instruction. In M. G. McKeown & M. E. Curtis (Eds.), The nature of vocabulary acquisition (pp. 19--36). Hillsdale, NJ: Erlbaum.
[22]
National Reading Panel. (2000). Teaching children to read: An evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. Washington, DC: National Institute of Child Health and Human Development.
[23]
Prinz, J. J. (2002). Furnishing the mind: Concepts and their perceptual basis. Cambridge, Massachusetts: MIT Press.
[24]
Qian D.D. (2002). Investigating the Relationship Between Vocabulary Knowledge and Academic Reading Performance: An Assessment Perspective. Language Learning, 52, 513--536.
[25]
Stahl, S. (1983). Differential word knowledge and reading comprehension. Journal of Reading Behavior, 15(4), 33--50.
[26]
Team up with Timo Vocabulary. http://animatedspeech.com/products/products_vocabulary.html
[27]
U.S. Department of Education, (2003). Table AA9, Number of Children Served Under IDEA by Disability and Age Group, 1994 Through 2003, Office of Special Education Programs, Washington, DC. http://www.ideadata.org/tables2aa94th/ar_.htm .
[28]
Vermeer, A. (2001). Breadth and depth of vocabulary in relation to L1/L2 acquisition and frequency of input. Applied Psycholinguistics, 22, 217--234.
[29]
Vygotsky, L. (1986). Thought and Language. Cambridge, Massachusetts: The MIT Press.
[30]
Waxman, S. R. (2002). Early word-learning and conceptual development: Everything had a name, and each name gave birth to a new thought. In U. Goswami (Ed.) Blackwell Handbook of childhood cognitive development (pp. 102--126). Malden, MA: Blackwell Publishishing.

Cited By

View all
  • (2015)Review of Research on Children with Autism Spectrum Disorder and the Use of TechnologyJournal of Special Education Technology10.1177/01626434150300010230:1(13-27)Online publication date: 1-Mar-2015
  • (2011)A Multiagent Approach to Teaching Complex Systems DevelopmentMulti-Agent Systems for Education and Interactive Entertainment10.4018/978-1-60960-080-8.ch004(70-87)Online publication date: 2011

Index Terms

  1. Just in time learning: implementing principles of multimodal processing and learning for education

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICMI '07: Proceedings of the 9th international conference on Multimodal interfaces
    November 2007
    402 pages
    ISBN:9781595938176
    DOI:10.1145/1322192
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 12 November 2007

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. education
    2. language learning
    3. multisensory integration
    4. speech
    5. vocabulary

    Qualifiers

    • Keynote

    Conference

    ICMI07
    Sponsor:
    ICMI07: International Conference on Multimodal Interface
    November 12 - 15, 2007
    Aichi, Nagoya, Japan

    Acceptance Rates

    Overall Acceptance Rate 453 of 1,080 submissions, 42%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)4
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2015)Review of Research on Children with Autism Spectrum Disorder and the Use of TechnologyJournal of Special Education Technology10.1177/01626434150300010230:1(13-27)Online publication date: 1-Mar-2015
    • (2011)A Multiagent Approach to Teaching Complex Systems DevelopmentMulti-Agent Systems for Education and Interactive Entertainment10.4018/978-1-60960-080-8.ch004(70-87)Online publication date: 2011

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media