Using annotated video as an information retrieval interface

ABSTRACT

The ability to deliver appropriate information to learners at the most appropriate time is an essential component of good instruction. In the best learning environments, this information is received in the context of the performance of the skills that are being acquired. This paper explores a technological approach to situated information retrieval by linking materials to segments of a video recording a skill performance. An interface is described where users navigate through a video performance and are presented with information relevant to the current video location. An approach to algorithmically generating interfaces of this type is then presented. The system takes as input annotations that describe a video recording of a performance, translates these annotations into subject terms used to catalog information resources, and then retrieves materials from online database servers using the Z39.50 information retrieval protocol. As an example application, the system was used to generate online teacher professional development materials by linking annotated video of classroom teaching with resources cataloged in the ERIC database.

REFERENCES

• 1: Robin Douglas Burke, Representation, storage and retrieval of tutorial stories in a social simulation, Northwestern University, Evanston, IL, 1993
• 2: Christel, M., Kanade, T., Mauldin, M., Reddy, R., Stevens, S., Wactlar, H. (1996) Techniques for the creation and exploration of Digital Video Libraries. In Furht, B. (Ed) Multimedia Tools and Applications (Volume 2). Boston, MA: Kluwer Academic Publishers, 1996.
• 3: Collins, A. (1991) Cognitive Apprenticeship and Instructional Technology. In L. Idol & B. F. Jones (Eds.), Educational values and cognitive instruction: Implications for reform. Hillsdale, NJ: Lawrence Erlbaum Associates, 1991.
• 4: Collins, A., Brown, J. S., and Newman, S. (1989) Cognitive Apprenticeship: Teaching the crafts of reading, writing, and mathematics. In L. B. Resnick (Ed.) Kowing, learning and instruction: Essays in honor of Robert Glaser. Hillsdale, NJ: Lawrence Erlbaum Associates.
• 5: Educational Resources Information Center (1998). ERIC Annual Report 1998. National Library of Education, Office of Educational Research and Improvement, U.S. Department of Education.
• 6: Kenneth D. Forbus , Peter B. Whalley, Using qualitative physics to build articulate software for thermodynamics education, Proceedings of the twelfth national conference on Artificial intelligence (vol. 2), p.1175-1182, October 1994, Seattle, Washington, United States
• 7: Forbus, K., Everett, J., Ureel, L., Brokowski, M., Baher, J., and Kuehne, S. (1998) Distributed Coaching for an Intelligent Learning Environment. Proceedings of the Qualitative Reasoning Workshop 1998, May 1998, Cape Cod, MA. Menlo Park, CA: AAAI Press.
• 8: Edward A. Fox, Lexical relations: enhancing effectiveness of information retrieval systems, ACM SIGIR Forum, v.15 n.3, p.5-36, Winter 1980 [doi>10.1145/1095403.1095404]
• 9: Gordon, Andrew S. (1995) Automated Video Assessment of Human Performance. In J. ,Greer (ed) Proceedings of AI-ED 95 - World Conference on Artificial Intelligence in Education, Washington, DC; August 16-19, 1995. Charlottesville, VA: AACE Press. pp. 541-546.
• 10: Johnson, J.R.V. (1990) Educational Research and Educational Practice: Bridging the Gap. Journal of Education for Teaching; 16( 1):83-90. March 1990.
• 11: Menachem Yaakov Jona, Representing and applying teaching strategies in computer-based learning-by-doing tutors, Northwestern University, Evanston, IL, 1996
• 12: Young Whan Kim , Jin H. Kim, A model of knowledge based information retrieval with hierarchical concept, Journal of Documentation, v.46 n.2, p.113-136, Jun. 1990 [doi>10.1108/eb026857]
• 13: Lampert, Magdalene and Jan Hawkins (1998). Report to the National Science Foundation: New Technologies for the Study of Teaching. Workshop held June 9-l 1, 1998, AM Arbor, MI.
• 14: McDonough, Jo and Steven McDonough (1990) What's the Use of Research? ELT Journal, 44(2):102-09, April 1990.
• 15: Murray, Tom (1999) Authoring Intelligent Tutoring Systems: An analysis of the state of the art. International Journal of Artificial Intelligence in Education, lO:loO-133, 1999.
• 16: Rada, Roy an Ellen Bicknell. (1989) Ranking Documents with a Thesaurus. Journal of the American Society of Information Science, 40:304-3 10.
• 17: Schank, Roger and Chip Cleary (1994) Engines for Education. Hillsdale, NJ: Lawrence Erlbaum Associates.
• 18: Spaeth, Jeanne (1994) So What Good is Research? Teaching Music, 2( 1):42-43. August 1994.
• 19: Stigler, J.W., P.A. Gonzales, T. Kawanaka, S. Knoll, and A. Serrano (1999) The TIMSS Videotape Classroom Study: Methods and Findings from an exploratory research project on eighth-grade mathematics instruction in Germany, Japan, and the United States. Jessup, MD: Education Publications Center.
• 20: Yih-Chen Wang , James Vandendorpe , Martha Evens, Relational thesauri in information retrieval, Journal of the American Society for Information Science, v.36 n.1, p.15-27, Jan. 1985
• 21: CORPORATE National Information Standards Organization Staff, Information Retrieval: Application Service Definition and Protocol Specification, Z39.50-1995, NISO Press, 1995

INDEX TERMS

Primary Classification:
  I. Computing Methodologies
  I.2 ARTIFICIAL INTELLIGENCE

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
  H.5 INFORMATION INTERFACES AND PRESENTATION (I.7)

General Terms:
Design, Experimentation, Human Factors

Keywords:
computer learning environments, digital libraries, inforamtion retrieval, multimedia interfaces