research-article

Graph Learning on K Nearest Neighbours for Automatic Image Annotation

Authors:

Like XueAuthors Info & Claims

ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval

Pages 403 - 410

https://doi.org/10.1145/2671188.2749383

Published: 22 June 2015 Publication History

Abstract

Image annotation is an open and challenging task, especially with large label vocabulary. In this paper, we propose a novel graph learning based method for image annotation, which takes both advantages of the nearest neighbour based and the graph-based methods, by exploiting the graph learning method to propagate the labels on the graph corresponding to the K nearest neighbours of a test image. To acquire more effective graph weights for computing score for each label, besides the similarity of visual features, our method also considers the similarity of two label sets, which is computed based on the label correlation that captures the semantic information between two labels. In addition, we combine the image-to-label distance with the graph learning based score to compute the final decision value for labelling. The proposed method is evaluated on three benchmark datasets for image annotation. The result shows our method substantially outperforms the previous graph learning based methods, and our result matches the current state-of-the-art results in annotation quality.

References

[1]

K. Barnard, P. Duygulu, D. Forsyth, N. de Freitas, D. M. Blei, and M. I. Jordan. Matching words and pictures. Journal of Machine Learning Research, 3:1107--1135, 2003.

Digital Library

[2]

O. Boiman, E. Shechtman, and M. Irani. In defense of nearest-neighbor based image classification. In CVPR 2008, pages 1--8, 2008.

[3]

G. Carneiro, A. B. Chan, P. J. Moreno, and N. Vasconcelos. Supervised learning of semantic classes for image annotation and retrieval. IEEE TPAMI 2007, 29(3):394--410, 2007.

Digital Library

[4]

G. Chen, Y. Song, F. Wang, and C. Zhang. Semi-supervised multi-label learning by solving a Sylvester equation. In Proceedings of the 2008 SIAM International Conference on Data Mining, pages 410--419, 2008.

[5]

M. Chen, A. Zheng, and K. Q. Weinberger. Fast image tagging. In ICML 2013, pages 1274--1282, 2013.

[6]

C. Cusano, G. Ciocca, and R. Schettini. Image annotation using SVM. Proc. SPIE, 5304(5):330--338, 2003.

[7]

P. Duygulu, K. Barnard, J. de Freitas, and D. Forsyth. Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary. In ECCV 2002, pages 97--112. Springer Berlin Heidelberg, 2002.

Digital Library

[8]

S. Feng, R. Manmatha, and V. Lavrenko. Multiple Bernoulli relevance models for image and video annotation. In CVPR 2004, pages II-1002--II-1009 Vol.2, 2004.

Digital Library

[9]

H. Fu, Q. Zhang, and G. Qiu. Random forest for image annotation. In ECCV 2012, pages 86--99. TeX Users Group, March 2012.

Digital Library

[10]

D. Grangier and S. Bengio. A discriminative kernel-based approach to rank images from text queries. IEEE TPAMI 2008, 30(8):1371--1384, 2008.

Digital Library

[11]

M. Guillaumin, T. Mensink, J. Verbeek, and C. Schmid. TagProp: Discriminative metric learning in nearest neighbor models for image auto-annotation. In ICCV 2009, pages 309--316, 2009.

[12]

T. Hertz, A. Bar-Hillel, and D. Weinshall. Learning distance functions for image retrieval. In CVPR 2004, pages II-570--II-577 Vol.2, 2004.

Digital Library

[13]

J. Jeon, V. Lavrenko, and R. Manmatha. Automatic image annotation and retrieval using cross-media relevance models. In SIGIR 2003, pages 119--126, 2003.

Digital Library

[14]

V. Lavrenko, R. Manmatha, and J. Jeon. A model for learning the semantics of pictures. In Advances in Neural Information Processing Systems 16, pages 553--560, 2004.

[15]

X. Li, C. G. M. Snoek, and M. Worring. Learning social tag relevance by neighbor voting. IEEE TMM 2009, 11(7):1310--1322, 2009.

Digital Library

[16]

J. Liu, M. Li, Q. Liu, H. Lu, and S. Ma. Image annotation via graph learning. Pattern Recognition, 42(2):218--228, 2009.

Digital Library

[17]

J. Liu, M. Li, W.-Y. Ma, Q. Liu, and H. Lu. An adaptive graph model for automatic image annotation. In Proceedings of the 8th ACM International Workshop on Multimedia Information Retrieval, pages 61--70, 2006.

Digital Library

[18]

A. Makadia, V. Pavlovic, and S. Kumar. A new baseline for image annotation. In ECCV 2008, pages 316--329. Springer Berlin Heidelberg, 2008.

Digital Library

[19]

D. Metzler and R. Manmatha. An inference network approach to image retrieval. In Image and Video Retrieval, pages 42--50, 2004.

[20]

F. Monay and D. Gatica-Perez. PLSA-based image auto-annotation: Constraining the latent space. In ACM MM 2004, pages 348--351, 2004.

Digital Library

[21]

H. Nakayama. Linear distance metric learning for large-scale generic image recognition. PhD thesis, The University of Tokyo, 2011.

[22]

Y. Verma and C. Jawahar. Image annotation using metric learning in semantic neighbourhoods. In ECCV 2012, pages 836--849. Springer Berlin Heidelberg, 2012.

Digital Library

[23]

Y. Verma and C. Jawahar. Exploring SVM for image annotation in presence of confusing labels. In BMVC 2013, 2013.

[24]

L. von Ahn and L. Dabbish. Labeling images with a computer game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 319--326, 2004.

Digital Library

[25]

H. Wang, H. Huang, and C. Ding. Image annotation using multi-label correlated Green's function. In ICCV 2009, pages 2029--2034, 2009.

[26]

Y. Xiang, X. Zhou, T.-S. Chua, and C.-W. Ngo. A revisit of generative model for automatic image annotation using Markov Random Fields. In CVPR 2009, pages 1153--1160, 2009.

[27]

O. Yakhnenko and V. Honavar. Annotating images and image objects using a hierarchical Dirichlet process model. In ACM SIGKDD 2008, MDM '08, pages 1--7, 2008.

Digital Library

[28]

A. Yavlinsky, E. Schofield, and S. Ruger. Automated image annotation using global features and robust nonparametric density estimation. In Image and Video Retrieval, pages 507--517, 2005.

Digital Library

[29]

S. Zhang, J. Huang, Y. Huang, Y. Yu, H. Li, and D. N. Metaxas. Automatic image annotation using group sparsity. In CVPR 2010, pages 3312--3319, 2010.

[30]

D. Zhou, O. Bousquet, T. N. Lal, J. Weston, and B. Scholkopf. Learning with local and global consistency. Advances in neural information processing systems, 16(16):321--328, 2004.

Digital Library

[31]

INRIA features for image annotation and classification data sets. http://lear.inrialpes.fr/people/guillaumin/data.php.

Cited By

Muñoz JDias ZTorres R(2022)A genetic programming approach for searching on nearest neighbors graphsMultimedia Tools and Applications10.1007/s11042-022-12248-w81:16(23449-23472)Online publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1007/s11042-022-12248-w
Ghostan Khatchatoorian AJamzad M(2022)Suggesting an Integration System for Image AnnotationMultimedia Tools and Applications10.1007/s11042-021-11571-y82:6(8323-8343)Online publication date: 15-Jul-2022
https://dl.acm.org/doi/10.1007/s11042-021-11571-y
Bensaci RKhaldi BAiadi OBenchabana A(2021)Deep Convolutional Neural Network with KNN Regression for Automatic Image AnnotationApplied Sciences10.3390/app11211017611:21(10176)Online publication date: 29-Oct-2021
https://doi.org/10.3390/app112110176
Show More Cited By

Index Terms

Graph Learning on K Nearest Neighbours for Automatic Image Annotation
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
        Scene understanding
2. Information systems
  1. Information retrieval
    1. Document representation

Recommendations

Automatic image annotation via compact graph based semi-supervised learning

The insufficiency of labeled samples is major problem in automatic image annotation. However, unlabeled samples are readily available and abundant. Hence, semi-supervised learning methods, which utilize partly labeled samples and a large amount of ...
Automatic image annotation using semi-supervised generative modeling

Image annotation approaches need an annotated dataset to learn a model for the relation between images and words. Unfortunately, preparing a labeled dataset is highly time consuming and expensive. In this work, we describe the development of an ...
Multiview semi-supervised ranking for automatic image annotation
MM '13: Proceedings of the 21st ACM international conference on Multimedia

Most photo sharing sites give their users the opportunity to manually label images. The labels collected that way are usually very incomplete due to the size of the image collections: most images are not labeled according to all the categories they ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ICMR '15: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval

June 2015

700 pages

ISBN:9781450332743

DOI:10.1145/2671188

General Chairs:
Alex Hauptmann
Carnegie Mellon University, USA
,
Chong-Wah Ngo
City University of Hong Kong, China
,
Xiangyang Xue
Fudan University, China
,
Program Chairs:
Yu-Gang Jiang
Fudan University, China
,
Cees Snoek
University of Amsterdam and Qualcomm Research Netherlands
,
Nuno Vasconcelos
University of California, San Diego, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGMM: ACM Special Interest Group on Multimedia

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 June 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

National Science Foundation of China

Conference

ICMR '15

Sponsor:

SIGMM

ICMR '15: International Conference on Multimedia Retrieval

June 23 - 26, 2015

Shanghai, China

Acceptance Rates

ICMR '15 Paper Acceptance Rate 48 of 127 submissions, 38%;

Overall Acceptance Rate 254 of 830 submissions, 31%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

11
Total Citations
View Citations
210
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 18 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Muñoz JDias ZTorres R(2022)A genetic programming approach for searching on nearest neighbors graphsMultimedia Tools and Applications10.1007/s11042-022-12248-w81:16(23449-23472)Online publication date: 1-Jul-2022
https://dl.acm.org/doi/10.1007/s11042-022-12248-w
Ghostan Khatchatoorian AJamzad M(2022)Suggesting an Integration System for Image AnnotationMultimedia Tools and Applications10.1007/s11042-021-11571-y82:6(8323-8343)Online publication date: 15-Jul-2022
https://dl.acm.org/doi/10.1007/s11042-021-11571-y
Bensaci RKhaldi BAiadi OBenchabana A(2021)Deep Convolutional Neural Network with KNN Regression for Automatic Image AnnotationApplied Sciences10.3390/app11211017611:21(10176)Online publication date: 29-Oct-2021
https://doi.org/10.3390/app112110176
Banerjee AGhosh KSarkar ABhattacharjee MChatterjee S(2021)Effects of Class Imbalance Problem in Convolutional Neural Network Based Image ClassificationAdvances in Smart Communication Technology and Information Processing10.1007/978-981-15-9433-5_18(181-191)Online publication date: 16-Feb-2021
https://doi.org/10.1007/978-981-15-9433-5_18
Jian YYe MMin YTian LWang G(2020)FORF-S: A Novel Classification Technique for Class Imbalance ProblemIEEE Access10.1109/ACCESS.2020.30409788(218720-218728)Online publication date: 2020
https://doi.org/10.1109/ACCESS.2020.3040978
Ghostan Khatchatoorian AJamzad M(2020)Architecture to improve the accuracy of automatic image annotation systemsIET Computer Vision10.1049/iet-cvi.2019.050014:5(214-223)Online publication date: 23-Apr-2020
https://doi.org/10.1049/iet-cvi.2019.0500
Shao QLiu B(2019)Laplacian Eigenmaps Regularized Feature Mapping for Image Annotation2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)10.1109/SMC.2019.8913981(3901-3906)Online publication date: 6-Oct-2019
https://dl.acm.org/doi/10.1109/SMC.2019.8913981
Zhang JHe ZZhang JDai T(2019)Cograph Regularized Collective Nonnegative Matrix Factorization for Multilabel Image AnnotationIEEE Access10.1109/ACCESS.2019.29258917(88338-88356)Online publication date: 2019
https://doi.org/10.1109/ACCESS.2019.2925891
Ning ZZhou GChen ZLi Q(2018)Integration of Image Feature and Word Relevance: Toward Automatic Image Annotation in Cyber-Physical-Social SystemsIEEE Access10.1109/ACCESS.2018.28643326(44190-44198)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2864332
Zhong HYuan XChen ZZhong FLeng Y(2018)Multiple Kernel Learning Based on Weak Learner for Automatic Image AnnotationAdvances in Multimedia Information Processing – PCM 201710.1007/978-3-319-77383-4_6(56-67)Online publication date: 10-May-2018
https://doi.org/10.1007/978-3-319-77383-4_6
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten