skip to main content
10.1145/1101149.1101305acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
Article

Image annotations by combining multiple evidence & wordNet

Published: 06 November 2005 Publication History

Abstract

The development of technology generates huge amounts of non-textual information, such as images. An efficient image annotation and retrieval system is highly desired. Clustering algorithms make it possible to represent visual features of images with finite symbols. Based on this, many statistical models, which analyze correspondence between visual features and words and discover hidden semantics, have been published. These models improve the annotation and retrieval of large image databases. However, current state of the art including our previous work produces too many irrelevant keywords for images during annotation. In this paper, we propose a novel approach that augments the classical model with generic knowledge-based, WordNet. Our novel approach strives to prune irrelevant keywords by the usage of WordNet. To identify irrelevant keywords, we investigate various semantic similarity measures between keywords and finally fuse outcomes of all these measures together to make a final decision using Dempster-Shafer evidence combination. We have implemented various models to link visual tokens with keywords based on knowledge-based, WordNet and evaluated performance using precision, and recall using benchmark dataset. The results show that by augmenting knowledge-based with classical model we can improve annotation accuracy by removing irrelevant keywords.

References

[1]
Y. A. Aslandogan and C. T. Yu. Diogenes: A web search agent for content based indexing of presonal images. In Proceedings of ACM SIGIR 2000, Athens, Greece, pages 481--482, July 2000.
[2]
S. Banerjee and T. Pedersen. Extended gloss overlaps as a measure of semantic relatedness. In Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence, pages 805--810, 2003.
[3]
K. Barnard, P. D. N, de Freitas D.and Forsyth D., and B. M.Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107--1135, 2003.
[4]
D. Blei and M. Jordan. Modeling annotated data. Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 127--134, July 2003.
[5]
P. Duygulu and K. Barnard. Object recognition as machine translation: learning a lexicon for a fixed image vocabulary. In Seventh European Conference on Computer Vision (ECCV), 4:97--112, 2002.
[6]
J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. Proceedings of the 26th annual international ACM SIGIR conference on Research and development in informaion retrieval, pages 119--126, July 2003.
[7]
J.Jiang and D.Conrath. Semantic similarity based on corpus statistics and lexical taxonomy. In Procedeeings on International Conference on Research in Computational Linguistics, 1997.
[8]
F. Kang, R. Jin, and J. Y. Chai. Regularizing translation models for better automatic image annotation. CIKM'04, pages 350--359, 2004.
[9]
Kobus. http://www.cs.arizona.edu/people/kobus/research/data. 2002.
[10]
C. Leacock. Combining local context and wordnet similarity for word sense identification. In Christiane Fellbaum, editor, WordNet: A Lexical Reference System and its Application. MIT Press, Cambridge, MA., pages 265--283, 1998.
[11]
J. Li and J. Z. Wang. Automatic linguistic indexing of pictures by a statistical modeling approach. IEEE Trans. on Pattern Analysis and Machine Intelligence, 25(10), 2003.
[12]
D. Lin. Using syntatic dependency as a local context to reslove word sense ambiguity. In Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, pages 64--71, 1997.
[13]
M.Lesk. Automatic sense disambiguation machine readable dictionaries: How to tell a pine cone from an ice cream cone. In Proceedings of the 5th Annual International Conference on Systems Documentation, pages 24--26, 1986.
[14]
Y. Mori, H. Takahashi, and R. Oka. Image-to-word transformation based on dividing and vector quantizing images with words. MISRM'99 Frist International Workshop on Multimedia Intellegent Storage and Retrieval Management, 1999.
[15]
J.-Y. Pan, H.-J. Yang, C. Faloutsos, and P. Duygulu. Automatic multimedia cross-modal correlation discovery. KDD 2004, pages 653--658, August 2004.
[16]
P. Resnik. Using information content to evaluate semantic similarity in a taxonomy. In Proceedings of the 14th International Joint Conference on Artificial Intelligence, 1995.
[17]
G. Shafer. A Mathematical Theory of Evidence. Princeton University Press, 1976.
[18]
J. Shi and J. Malik. Normalized cuts and image segmentation. IEEE Conf. Computer Vision and Pattern Recognition(CVPR), pages 731--737, June 1997.
[19]
R. M. V. Lavrenko and J. Jeon. A model for learning the semantics of pictures. Proceedings of the 17th Annual Conference on Neural Information Processing Systems, 2003.
[20]
L. Wang and L. Khan. Automatic image annotation and retrieval using weighted feature selection. To appear in International Journal of Multimedia Tools and Applications by Kluwer Publisher, 2005.
[21]
Y. Jin, L. Wang and L. Khan, Improving Image Annotations using WordNet, International Workshop on Multimedia Information Systems (MIS 2005), Sorrento, Italy, page: 115-130, September, 2005.Editor's Note: This last reference was added at the author's request after official publication of the proceedings.

Cited By

View all
  • (2024)Semantic labeling of social big media using distributed online robust classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107928132(107928)Online publication date: Jun-2024
  • (2024)Comparative analysis of manual and annotations for crowd assessment and classification using artificial intelligenceData Science and Management10.1016/j.dsm.2024.04.001Online publication date: Apr-2024
  • (2024)Relevant Tag Extraction Based on Image Visual ContentApplied Intelligence10.1007/978-981-97-0827-7_25(283-295)Online publication date: 1-Mar-2024
  • Show More Cited By

Index Terms

  1. Image annotations by combining multiple evidence & wordNet

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MULTIMEDIA '05: Proceedings of the 13th annual ACM international conference on Multimedia
    November 2005
    1110 pages
    ISBN:1595930442
    DOI:10.1145/1101149
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 06 November 2005

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Dempster-Shafer rule
    2. corel dataset
    3. image annotation
    4. semantic-similarity
    5. wordNet

    Qualifiers

    • Article

    Conference

    MM05

    Acceptance Rates

    MULTIMEDIA '05 Paper Acceptance Rate 49 of 312 submissions, 16%;
    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)7
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Semantic labeling of social big media using distributed online robust classificationEngineering Applications of Artificial Intelligence10.1016/j.engappai.2024.107928132(107928)Online publication date: Jun-2024
    • (2024)Comparative analysis of manual and annotations for crowd assessment and classification using artificial intelligenceData Science and Management10.1016/j.dsm.2024.04.001Online publication date: Apr-2024
    • (2024)Relevant Tag Extraction Based on Image Visual ContentApplied Intelligence10.1007/978-981-97-0827-7_25(283-295)Online publication date: 1-Mar-2024
    • (2022)The Image Annotation Refinement in Embedding Feature Space based on Mutual InformationInternational Journal of Circuits, Systems and Signal Processing10.46300/9106.2022.16.2316(191-201)Online publication date: 10-Jan-2022
    • (2022)Semantic Image Analysis on Social Networks and Data ProcessingHandbook of Research on Foundations and Applications of Intelligent Business Analytics10.4018/978-1-7998-9016-4.ch009(189-214)Online publication date: 2022
    • (2022)Computational Methods for Integrating Vision and LanguageundefinedOnline publication date: 8-Apr-2022
    • (2021)Exploring Classification Capability of CNN FeaturesData Science10.1007/978-981-16-5940-9_21(268-282)Online publication date: 10-Sep-2021
    • (2020)A review on visual content-based and users’ tags-based image annotation: methods and techniquesMultimedia Tools and Applications10.1007/s11042-020-08862-179:29-30(21679-21741)Online publication date: 1-Aug-2020
    • (2020)Dual-Graph Regularized Sparse Low-Rank Matrix Recovery for Tag RefinementBio-inspired Computing: Theories and Applications10.1007/978-981-15-3415-7_20(243-258)Online publication date: 2-Apr-2020
    • (2020)Semantic analysis on social networks: A surveyInternational Journal of Communication Systems10.1002/dac.442433:11Online publication date: 16-Apr-2020
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media