Article

Towards to an automatic semantic annotation for multimedia learning objects

Authors:

Serge Linckels,

Christoph MeinelAuthors Info & Claims

Emme '07: Proceedings of the international workshop on Educational multimedia and multimedia education

Pages 19 - 26

https://doi.org/10.1145/1290144.1290149

Published: 28 September 2007 Publication History

Abstract

The number of digital video recordings has increased dramatically. The idea of recording lectures, speeches, and other academic events is not new. But, the accessibility and traceability of its content for further use is rather limited. Searching multimedia data, in particular audiovisual data, is still a challenging task to overcome. We describe and evaluate a new approach to generate asemantic annotation for multimedia resources, i.e., recorded university lectures. Speech recognition is applied to create atentative and deficient transliteration of the video recordings. We show that the imperfect transliteration is sufficient to generate semantic metadata serialized in an OWL file. The semantic annotation process based on textual material and deficient transliterations of lecture recordings are discussed and evaluated.

References

[1]

J. Allen. Natural Language Understanding. Addison Wesley, 1994.

[2]

F. Baader, D. Calvanese, D. L. McGuinness, D. Nardi, and P. F. Patel-Schneider, editors. The Description Logic Handbook: Theory, Implementation, and Applications. Cambridge University Press, 2003.

Digital Library

[3]

R. A. Baeza-Yates and B. A. Ribeiro-Neto. Modern Information Retrieval. ACM Press / Addison-Wesley, 1999.

Digital Library

[4]

M. Bertini, A. D. Bimbo, C. Torniai, R. Cucchiara, and C. Grana. Mom: Multimedia ontology manager. a framework for automatic annotation and semantic retrieval of video sequences. In ACM SIGMM, pages 787--788, 2006.

Digital Library

[5]

Y. Chen and W. J. Heng. Automatic synchronization of speech transcript and slides in presentation. In International Symposium on Circuits and Systems (ISCAS), pages 568--571, 2003.

[6]

H. S. Christopher D. Manning. Foundations of Statistical Natural Language Processing. The MIT Press, 1999.

Digital Library

[7]

M. Engelhardt, A. Hildebrand, D. Lange, and T. C. Schmidt. Reasoning about eLearning Multimedia Objects. In International Workshop on Semantic Web Annotations for Multimedia (SWAMM), 2006.

[8]

A. Haubold and J. R. Kender. Augmented segmentation and visualization for presentation videos, 2005.

Digital Library

[9]

W. HÄurst, T. Kreuzer, and M. WiesenhÄutter. A qualitative study towards using large vocabulary automatic speech recognition to index recorded presentations for search and access over the web. In IADIS Internatinal Conference WWW/Internet (ICWI), pages 135--143, 2002.

[10]

A. Jaimes, T. Nagamine, J. Liu, K. Omura, and N. Sebe. Affective meeting video analysis. In IEEE Multimedia and Expo, pages 1412--1415, 2005.

[11]

N. Karam, S. Linckels, and C. Meinel. Semantic composition of lecture subparts for a personalized e-learning. In European Semantic Web Conference, volume 4519 of Lecture Notes in Computer Science, pages 716--728, 2007.

Digital Library

[12]

S. Linckels and C. Meinel. Resolving ambiguities in the semantic interpretation of natural language questions. In Intelligent Data Engineering and Automated Learning (IDEAL), volume 4224 of LNCS, pages 612--619, 2006.

Digital Library

[13]

R. Mertens, H. Schneider, O. Mller, and O. Vornberger. Hypermedia navigation concepts for lecture recordings. In E-Learn: World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education, pages 2480--2847, 2004.

[14]

R. Mitkov, editor. The Oxford Handbook of Computational Linguistics. Oxford University Press, 2004.

Digital Library

[15]

C.-W. Ngo, F. Wang, and T.-C. Pong. Structuring lecture videos for distance learning applications. In Multimedia Software Engineering, pages 215--222, 2003.

[16]

S. Repp and C. Meinel. Segmenting of recorded lecture videos - the algorithm voiceseg. In Signal Processing and Multimedia Applications (SIGMAP), pages 317--322, 2006.

[17]

S. Repp and C. Meinel. Semantic indexing for recorded educational lecture videos. In International Conference on Pervasive Computing and Communications Workshops (PERCOMW), page 240, 2006.

Digital Library

[18]

H. Sack and J. Waitelonis. Automated annotations of synchronized multimedia presentations. In Workshop on Mastering the Gap: From Information Extraction to Semantic Representation, CEUR Workshop Proceedings, 2006.

[19]

H. Sack and J. Waitelonis. Integrating social tagging and document annotation for content-based search in multimedia data. In Semantic Authoring and Annotation Workshop (SAAW), 2006.

[20]

R. A. Schmidt. Terminological representation, natural language & relation algebra. In German AI Conference (GWAI), volume 671 of LNCS, pages 357--371, 1993.

Digital Library

[21]

J. Tejedor, R. Garca, M. Fernndez, F. J. Lpez-Colino, F. Perdrix, J. A. Macas, R. M. Gil, M. Oliva, D. Moya, J. Cols, and P. Castells. Ontology-based retrieval of human speech. In Workshop on Web Semantics (WebS 2007), 2007.

Digital Library

[22]

W. W. W. C. W3C. OWL Web Ontology Language. http://www.w3.org/TR/owl-features/, 2004.

[23]

F. Wang, C.-W. Ngo, and T.-C. Pong. Prediction-based gesture detection in lecture videos by combining visual, speech and electronic slides. In IEEE Multimedia and Expo, pages 653--656, 2006.

[24]

P. Wolf, W. Putz, A. Stewart, A. Steinmetz, M. Hemmje, and E. Neuhold. Lecturelounge - experience education beyond the borders of the classroom. International Journal on Digital Libraries, 4(1):39--41, 2004.

Digital Library

[25]

N. Yamamoto, J. Ogata, and Y. Ariki. Topic segmentation and retrieval system for lecture videos based on spontaneous speech recognition. In European Conference on Speech Communication and Technology, pages 961--964, 2003.

[26]

Y. Zhu and D. Zhou. Video browsing and retrieval based on multimodal integration. In Web Intelligence, pages 650--653, 2003.

Digital Library

Cited By

Repp SLinckels SMeinel C(2019)Question answering from lecture videos based on an automatic semantic annotationACM SIGCSE Bulletin10.1145/1597849.138427840:3(17-21)Online publication date: 28-Feb-2019
https://dl.acm.org/doi/10.1145/1597849.1384278
El Guemmat KOuahabi S(2018)A Literature Review of Indexing and Searching Techniques Implementation in Educational Search EnginesInternational Journal of Information and Communication Technology Education10.4018/IJICTE.201804010614:2(72-83)Online publication date: Apr-2018
https://doi.org/10.4018/IJICTE.2018040106
Friedland GHürst WKnipping L(2018)Educational MultimediaIEEE MultiMedia10.1109/MMUL.2008.7115:3(54-56)Online publication date: 23-Dec-2018
https://dl.acm.org/doi/10.1109/MMUL.2008.71
Show More Cited By

Index Terms

Towards to an automatic semantic annotation for multimedia learning objects
1. Information systems
  1. Information retrieval
2. Theory of computation
  1. Semantics and reasoning
    1. Program reasoning
      1. Abstraction

Recommendations

Question answering from lecture videos based on an automatic semantic annotation
ITiCSE '08

The number of digital lecture video recordings has increased dramatically. The accessibility, usability and the traceability of their content for students-use is limited. Therefore retrieval of audiovisual lecture recordings is a complex task. Speech ...
Question answering from lecture videos based on an automatic semantic annotation
ITiCSE '08: Proceedings of the 13th annual conference on Innovation and technology in computer science education

The number of digital lecture video recordings has increased dramatically. The accessibility, usability and the traceability of their content for students-use is limited. Therefore retrieval of audiovisual lecture recordings is a complex task. Speech ...
Discovering salient prosodic cues and their interactions for automatic story segmentation in Mandarin broadcast news

This paper investigates speech prosody for automatic story segmentation in Mandarin broadcast news. Prosodic cues effectively used in English story segmentation deserve a re-investigation since the lexical tones of Mandarin may complicate the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

Emme '07: Proceedings of the international workshop on Educational multimedia and multimedia education

September 2007

138 pages

ISBN:9781595937834

DOI:10.1145/1290144

General Chairs:
Gerald Friedland
ICSI Berkeley, USA
,
Wolfgang Hürst
University of Freiburg, Germany
,
Lars Knipping
Berlin University of Technology, Germany
,
Program Chair:
Max Mühlhäuser
Darmstadt University of Technology, Germany

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 28 September 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

MM07

Sponsor:

MM07: The 15th ACM International Conference on Multimedia 2007

September 28, 2007

Bavaria, Augsburg, Germany

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

17
Total Citations
View Citations
645
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Repp SLinckels SMeinel C(2019)Question answering from lecture videos based on an automatic semantic annotationACM SIGCSE Bulletin10.1145/1597849.138427840:3(17-21)Online publication date: 28-Feb-2019
https://dl.acm.org/doi/10.1145/1597849.1384278
El Guemmat KOuahabi S(2018)A Literature Review of Indexing and Searching Techniques Implementation in Educational Search EnginesInternational Journal of Information and Communication Technology Education10.4018/IJICTE.201804010614:2(72-83)Online publication date: Apr-2018
https://doi.org/10.4018/IJICTE.2018040106
Friedland GHürst WKnipping L(2018)Educational MultimediaIEEE MultiMedia10.1109/MMUL.2008.7115:3(54-56)Online publication date: 23-Dec-2018
https://dl.acm.org/doi/10.1109/MMUL.2008.71
Johnson Lim SLiu YLee W(2018)Multi-facet product information search and retrieval using semantically annotated product family ontologyInformation Processing and Management: an International Journal10.1016/j.ipm.2009.09.00146:4(479-493)Online publication date: 29-Dec-2018
https://dl.acm.org/doi/10.1016/j.ipm.2009.09.001
Ostreika AVasiu RGudoniene DMaskeliunas RRutkauskiene D(2016)An Ontology Oriented Approach for E-Learning Objects Design and ImprovementInformation and Software Technologies10.1007/978-3-319-24770-0_13(138-150)Online publication date: 10-Jan-2016
https://doi.org/10.1007/978-3-319-24770-0_13
Maratea APetrosino AManzo M(2013)Generation of description metadata for video filesProceedings of the 14th International Conference on Computer Systems and Technologies10.1145/2516775.2516795(262-269)Online publication date: 28-Jun-2013
https://dl.acm.org/doi/10.1145/2516775.2516795
Kitayama DSumiya K(2010)Retrieving system of presentation contents based on user's operations and semantic contextsProceedings of the 15th international conference on Database Systems for Advanced Applications - Volume Part II10.1007/978-3-642-12098-5_50(460-463)Online publication date: 1-Apr-2010
https://dl.acm.org/doi/10.1007/978-3-642-12098-5_50
Lim SLiu YLee WAlonso OZaragoza H(2009)Faceted search and retrieval based on semantically annotated product family ontologyProceedings of the WSDM '09 Workshop on Exploiting Semantic Annotations in Information Retrieval10.1145/1506250.1506254(15-24)Online publication date: 9-Feb-2009
https://dl.acm.org/doi/10.1145/1506250.1506254
Fogarolli ARonchetti M(2009)Domain Independent Semantic Representation of Multimedia PresentationsProceedings of the 2009 International Conference on Intelligent Networking and Collaborative Systems10.1109/INCOS.2009.80(31-38)Online publication date: 4-Nov-2009
https://dl.acm.org/doi/10.1109/INCOS.2009.80
Repp SMeinel C(2009)Automatic Extraction of Semantic Descriptions from the Lecturer's SpeechProceedings of the 2009 IEEE International Conference on Semantic Computing10.1109/ICSC.2009.17(513-520)Online publication date: 14-Sep-2009
https://dl.acm.org/doi/10.1109/ICSC.2009.17
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten