Article

Multimedia and human-in-the-loop: interaction as content enrichment

Author:

Bruno EmondAuthors Info & Claims

HCM '07: Proceedings of the international workshop on Human-centered multimedia

Pages 77 - 84

https://doi.org/10.1145/1290128.1290141

Published: 28 September 2007 Publication History

Abstract

The current work is part of the broadband visual research program at the Institute for Information Technology (National Research Council Canada). The research program is currently focused on developing human-centered multimedia technology to support large group visual communication and collaboration. This paper outlines some conceptual foundations for the development of a human-centered multimedia research tool to capture interaction data, which could be linked to users cognitive processing. The approach is based on the notion of multimedia interaction as content enrichment and on cognitive modeling methodology.

References

[1]

A. James and N. Dimitrova, "Human-centered multimedia: culture, deployment, and access," IEEE Multimedia Magazine, vol. 13, pp. 12--19, 2006.

Digital Library

[2]

A. James, M. Christel, S. Gilles, R. Sarukkai, and W.-Y. Ma, "Multimedia information retrieval: What is it, and why isn't anyone using it?," in Proceedings of the 7th ACM SIGMM international workshop on Multimedia information retrieval. New York, NY, USA: ACM Press, 2005, pp. 3--8.

Digital Library

[3]

A. Newell, Unified theories of Cognition. Cambridge, MA: Harvard University Press, 1990.

Digital Library

[4]

A. Jaimes, N. Sebe, and D. Gatica-Perez, "Human--centered computing: a multimedia perspective," in Proceedings of the 14th annual ACM international conference on Multimedia, 2006, pp. 855--864.

Digital Library

[5]

C. S. Peirce, "On a new list of categories," Proceedings of the American Academy of Arts and Sciences, vol. 7, 1868.

[6]

B. Emond, "Pour une grammaire formelle de la classification des signes chez Charles S. Peirce," Semiotica, vol. 72, 1988.

[7]

J. Hunter and J. Newmarch, "An indexing, browsing, search and retrieval system for audiovisual libraries," in Lecture Notes in Computer Science, vol. 1696, 1999, pp. 76--91.

Digital Library

[8]

D. Zhang and J. F. Nunamaker, "A natural language approach to content--based indexing and retreival for interactive e-learning.," IEEE Transactions on Multimedia, vol. 6, pp. 450--458, 2004.

Digital Library

[9]

C. G. M. Snoek and M. Worring, "Multimodal video indexing: a review of the state-of-the-art," Multimedia Tools and Applications, vol. 25, pp. 5--35, 2005.

Digital Library

[10]

D. O. Gorodnichy and A. Yogeswaran, "Detection and tracking of pianist hands and fingers," presented at Proc. of the Canadian conference Computer & Robot Vision (CRV'06), Quèèbec, Canada, 2006.

Digital Library

[11]

M. M. Yeung and B. L. Yeo, "Video visualization for compact presentation and fast browsing of picturial content," IEEE Transactions on Circuits Systems and Video Technology, vol. 7, pp. 771--785, 1997.

Digital Library

[12]

G. Comeau, M. Brooks, and J. Spence, "Video and broadband video conference in professional development," presented at Biennial conference of the International Consortium for Educational Development (ICED), University of Ottawa, Ottawa, Ontario., 2004.

[13]

J. W. Stigler, P. Gonzales, T. Kawanaka, S. Knoll, and A. Serano, "The TIMSS videotape classroom study: methods and findings from an exploratory research project on eighth-grade mathematics instruction in Germany, Japan, and the United States," National Center for Education Statistics Report, US Department of Education, Washington 1999.

[14]

I. I. JTC1/SC29/WG11, "MPEG-4 Overview--(V.21-Jeju Version)," 2002.

[15]

International_MIDI_Association, MIDI Musical Instrument Digitial Interface Specification 1.0. Los Angeles, CA: International MIDI Association, 1983.

[16]

International_MIDI_Association, Standard MIDI Files 1.0. Los Angeles, CA: International MIDI Association, 1988.

[17]

Dublin_Core_Metadata_Initiative, "Dublin Core Metadata Element Set, Version 1.1: Reference Description," 2004.

[18]

C. S. L. T. S. Committee, "IEEE Standard for Learning Object Metadata (1484.12.1-2002)," IEEE Learning Technology Standards Committee, 2002.

[19]

World_Wide_Web_Consortium, "Synchronized Multimedia Integration Language (SMIL 2.0)," 2005.

[20]

J. Brook, "The Development of an Asynchronous Video Reference Database to Support Suzuki Piano Instruction," presented at Graduate Music Students' Association (GMSA) Conference, University of Ottawa, Ottawa, 2006.

[21]

B. Emond, M. Brooks, and A. Smith, "A broadband Web-based Application for Video Sharing and Annotation," in Proceedings of Ninth International ACM Conference on Multimedia, 2001, pp. 603--604.

Digital Library

[22]

B. Emond and M. Brooks, "The Private Video Sharing and Annotation Server: A broadband application for teacher training and music education," presented at International Lisp Conference, New York, NY, 2003.

[23]

H. Masum, M. Brooks, and J. Spence, "MusicGrid: A case study in broadband video collaboration," First Monday, vol. 10, 2005.

[24]

B. Emond and R. L. West, "Cyberpsychology: A human-interaction perspective based on cognitive modelling," Cyberpsychology and Behavior, vol. 6, pp. 527--536, 2003.

[25]

A. Pentland, "Socially aware computation and communication," Computer, vol. 38, pp. 33--40, 2005.

Digital Library

[26]

I. McCowan, D. Gatica-Perez, S. Bengio, G. Lathoud, M. Barnard, and D. Zhang, "Automatic analysis of multimodal group actions in meetings," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, pp. 305--317, 2005.

Digital Library

[27]

B. Emond, N. G. Vinson, J. A. Singer, M. A. Barfurth, and M. Brooks, "ReView: A Digital Video Player to Support Music Practice and Learning," Journal of Technology in Music Learning, Accepted.

[28]

G. Geisler, G. Marchionini, B. M. Wildemuth, A. Hughes, M. Yang, T. Wilkens, and R. Spinks, "Video browsing interfaces for the open video project," in CHI '02 extended abstracts on Human factors in computer systems, 2002, pp. 514--515.

Digital Library

[29]

G. Geisler, S. Giersch, D. McArthur, and M. McClelland, "Creating virtual collections in digital libraries: benefits and implementation issues," in Proceedings of the second ACM/IEEE-CS joint conference on Digital libraries, 2002, pp. 210--218.

Digital Library

[30]

C.-W. Lin, J. Zhou, J. Youn, and S. M.-T., "MPEG video streaming with vcr functionality," IEEE Transactions on Circuits and Systems for Video Technology, vol. 11, pp. 415--425, 2001.

Digital Library

[31]

F. C. Li, A. Gupta, A. Sanocki, L. He, and Y. Rui, "Browsing digital video," presented at Paper presented at the ACM Conference on Human Factors in Computing Systems, 2000.

Digital Library

[32]

C. Drake and C. Palmer, "Skill acquisition in music performance: Relations between planning and temporal control," Cognition, vol. 74, pp. 1--32, 2000.

[33]

G. Mantovani, "The psychological construction of the internet; from information foraging to social gathering to cultural mediation," Cyberpsychology and behavior, vol. 4, pp. 47--56, 2001.

[34]

P. Pirolli and S. K. Card, "Information foraging," Psychological Review, vol. 106, pp. 643--675, 1999.

[35]

E. Wenger, Communities of practices. Learning, meaning, and identity. Cambridge: Cambridge University Press, 1998.

[36]

G. Riva and C. Galimberti, "The mind in the web: Psychology in the internet age," Cyberpsychology and behavior, vol. 4, pp. 1--5, 2001.

[37]

M. A. Barfurth, J. Singer, B. Emond, N. Vinson, M. Brooks, and J. Spence, "Evaluation factors for multi-stakeholder broadband visual communication projects," presented at Eleventh IEEE International Workshops on Enabling Technologies: Infrastrcutures for Collaborative Entreprises-WET-ICE, 2002.

Digital Library

[38]

C. Galimberti, S. Ignazi, P. Vercesi, and G. Riva, "Communication and cooperation in networked environments: an experimental analysis," Cyberpsychology and behavior, vol. 4, pp. 131--146, 2001.

[39]

G. Riva, "The mind over the web: the quest for the definition of a method for internet research," Cyberpsychology and behavior, vol. 4, pp. 7--16, 2001.

[40]

C. Goldspink, "Methodological implications of complex systems approaches to sociality: simulation as a foundation for knowledge," Journal of Artificial Societies and Social Simulation, vol. 5, 2002.

[41]

J. M. Epstein and R. Axtell, Growing Artificial Societies. Cambridge, MA: MIT Press, 1996.

Digital Library

[42]

M. Minsky, The Society of Mind. New York: Simon and Schuster, 1986.

Digital Library

[43]

C. Goldspink, "Modelling social systems as complex: towards a social simulation meta-model," Journal of Artificial Societies and Social Simulation, vol. 3, 2000.

[44]

R. Conte and M. Paolucci, "Intelligent social learning," Journal of Artificial Societies and Social Simulation, vol. 4, 2001.

[45]

M. D. Byrne and J. R. Anderson, "Serial modules in parallel: The psychological refractory period and perfect time-sharing," Psychological Review, vol. 108, pp. 847--869, 2001.

[46]

J. R. Anderson and M. P. Matessa, "A production system theory of serial memory," Psychological Review, vol. 104, pp. 728--748, 1997.

[47]

J. R. Anderson and L. M. Reder, "The fan effect: New results and new theories," Journal of Experimental Psychology: General, vol. 128, pp. 186--197, 1999.

[48]

J. R. Anderson, D. Bothell, C. Lebiere, and M. P. Matessa, "An integrated theory of list memory," Journal of Memory and Language, pp. 341--380, 1998.

[49]

E. M. Altmann and J. G. Trafton, "Memory for goals: An activation--based model," Cognitive Science, vol. 26, pp. 39--83, 2002.

[50]

B. D. Ehret, W. D. Gray, and S. S. Kirschenbaum, "Contending with complexity: Developing and using a scaled world in applied cognitive research," Human Factors, vol. 42, pp. 8--23, 2000.

[51]

J. R. Anderson, R. Budiu, and L. M. Reder, "A theory of sentence memory as part of a general theory of memory," Journal of Memory and Language, vol. 45, pp. 337--367, 2001.

[52]

B. Emond, "Modeling natural language comprehension and anaphora resolution with ACT-R," presented at Fourth Annual ACT-R Workshop, Pittsburgh, PA, 1997.

[53]

J. Leitão, "An ACT-R Model of Syllogistic Reasoning," presented at Fourth International Conference on Cognitive Modeling, 2001.

[54]

B. Emond, "Cognitive representations and processes in syllogistic reasoning: existential graphs and cognitive modelling," Psychologica, vol. 32, pp. 311--340, 2003.

[55]

R. L. West and C. Lebiere, "Simple games as dynamic, coupled systems: Randomness and other emergent properties," Cognitive Systems Research, vol. 1, pp. 221--239, 2001.

Digital Library

[56]

J. R. Anderson, Y. Qin, M.-H. Sohn, V. A. Stenger, and C. S. Carter, "An information-processing model of the BOLD response in symbol manipulation tasks," Psychonomic Bulletin and Review, in press.

[57]

M. P. Matessa and J. R. Anderson, "Modeling Focused Learning in Role Assignment," Language and Cognitive Processes, vol. 15, pp. 263--292, 2000.

[58]

D. Fum and F. Del Missier, "Modeling counter offer behavior in dyadic distributive negotiation," in Proceedings of the Fourth International Conference on Cognitive Modeling: Lawrence Erlbaum Associates, 2001, pp. 79--84.

[59]

C. Lebiere, "Modeling group decision making in the ACT-R cognitive architecture," presented at Computational Social and Organizational Science (CASOS), Pittsburgh, PA, 2002.

[60]

J. R. Anderson, "Spanning seven orders of magnitude: a challenge for cognitive modeling," Cognitive Science, vol. 26, pp. 85--112, 2002.

[61]

J. R. Anderson, The architecture of cognition. Cambridge, MA, USA: Harvard University Press, 1983.

Digital Library

[62]

J. R. Anderson and C. Lebiere, The Atomic Components of Thought. Mahwah, NJ, USA: Lawrence Erlbaum Associates, 1998.

[63]

Z. W. Pylyshyn, Computation and cognition: toward a foundation for cognitive science. Cambridge, MA, USA: MIT Press, 1984.

Digital Library

[64]

D. E. Kieras and D. E. Meyer, "An overview of the EPIC architecture for cognition and performance with application to human-computer interaction," Human-Computer Interaction, vol. 12, 1997.

[65]

W. Kintsch, Comprehension. Cambridge, UK: Cambridge University Press, 1998.

[66]

W. D. Gray and E. M. Altmann, "Cognitive modeling and human-computer interaction," in International encyclopedia of ergonomics and human factors, vol. 1, W. Karwowski, Ed., 2001, pp. 387--391.

[67]

M. D. Byrne, "ACT-R/PM and menu selection: Applying a cognitive architecture to HCI," International Journal of Human-Computer Studies, vol. 55, pp. 41--84, 2001.

Digital Library

[68]

B. Emond and R. L. West, "Using cognitive modelling simulations for user interface design decisions," in Innovations in Applied Artificial Intelligence: Proceedings of the 17th International Conference on Industrial & Engineering Applications of Artificial Intelligence & Expert Systems, B. Orchard, C. Yang, and M. Ali, Eds. Berlin: Springer-Verlag, 2004, pp. 305--314.

Digital Library

Cited By

Mithun NPanda RRoy-Chowdhury A(2020)Construction of Diverse Image Datasets From Web Collections With Limited LabelingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2019.289889930:4(1147-1161)Online publication date: Apr-2020
https://doi.org/10.1109/TCSVT.2019.2898899
Emond BScobie DAllen MPostma MMcIver W(2008)An H.323 Broadband Virtual Camera for supporting asynchronous visual communication in large groups2008 IEEE International Symposium on Technology and Society10.1109/ISTAS.2008.4559765(1-4)Online publication date: Jun-2008
https://doi.org/10.1109/ISTAS.2008.4559765

Index Terms

Multimedia and human-in-the-loop: interaction as content enrichment

Recommendations

Quantified self and modeling of human cognition
UbiComp/ISWC'15 Adjunct: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers

This paper tackles an important issue of how to use quantified self technologies for modeling human cognition. At the same time, it provides some insights into benefits user modeling and personalization offer for Quantified Self community, especially in ...
Toward a descriptive cognitive model of human learning

The majority of previous computational models of high-order human cognition incorporate gradient descent algorithms for their learning mechanisms and strict error minimization as the sole objective of learning. Recently, however, the validity of ...
When Eye-Tracking Meets Cognitive Modeling: Applications to Cyber Security Systems
Human Aspects of Information Security, Privacy and Trust
Abstract
Human cognitive modeling techniques and related software tools have been widely used by researchers and practitioners to evaluate the effectiveness of user interface (UI) designs and related human performance. However, they are rarely used in the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HCM '07: Proceedings of the international workshop on Human-centered multimedia

September 2007

112 pages

ISBN:9781595937810

DOI:10.1145/1290128

General Chairs:
Alejandro Jaimes
IDIAP, Switzerland
,
Nicu Sebe
University of Amsterdam, The Netherlands

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM07

Sponsor:

MM07: The 15th ACM International Conference on Multimedia 2007

September 28, 2007

Bavaria, Augsburg, Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
433
Total Downloads

Downloads (Last 12 months)3
Downloads (Last 6 weeks)0

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Mithun NPanda RRoy-Chowdhury A(2020)Construction of Diverse Image Datasets From Web Collections With Limited LabelingIEEE Transactions on Circuits and Systems for Video Technology10.1109/TCSVT.2019.289889930:4(1147-1161)Online publication date: Apr-2020
https://doi.org/10.1109/TCSVT.2019.2898899
Emond BScobie DAllen MPostma MMcIver W(2008)An H.323 Broadband Virtual Camera for supporting asynchronous visual communication in large groups2008 IEEE International Symposium on Technology and Society10.1109/ISTAS.2008.4559765(1-4)Online publication date: Jun-2008
https://doi.org/10.1109/ISTAS.2008.4559765

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten