Content-based image retrieval by clustering

ABSTRACT

In a typical content-based image retrieval (CBIR) system, query results are a set of images sorted by feature similarities with respect to the query. However, images with high feature similarities to the query may be very different from the query in terms of semantics. This is known as the semantic gap. We introduce a novel image retrieval scheme, CLUster-based rEtrieval of images by unsupervised learning (CLUE), which tackles the semantic gap problem based on a hypothesis: semantically similar images tend to be clustered in some feature space. CLUE attempts to capture semantic concepts by learning the way that images of the same semantics are similar and retrieving image clusters instead of a set of ordered images. Clustering in CLUE is dynamic. In particular, clusters formed depend on which images are retrieved in response to the query. Therefore, the clusters give the algorithm as well as the users semantic relevant clues as to where to navigate. CLUE is a general approach that can be combined with any real-valued symmetric similarity measure (metric or nonmetric). Thus it may be embedded in many current CBIR systems. Experimental results based on a database of about 60, 000 images from COREL demonstrate improved performance.

REFERENCES

• 1: Ricardo A. Baeza-Yates , Berthier Ribeiro-Neto, Modern Information Retrieval, Addison-Wesley Longman Publishing Co., Inc., Boston, MA, 1999
• 2: K. Barnard and D. Forsyth, "Learning the Semantics of Words and Pictures," Proc. 8th Int'l Conf. on Computer Vision, vol. 2, pp. 408--415, 2001.
• 3: Chad Carson , Serge Belongie , Hayit Greenspan , Jitendra Malik, Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.8, p.1026-1038, August 2002 [doi>10.1109/TPAMI.2002.1023800 ]
• 4: Yixin Chen , James Z. Wang, A Region-Based Fuzzy Feature Matching Approach to Content-Based Image Retrieval, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.24 n.9, p.1252-1267, September 2002 [doi>10.1109/TPAMI.2002.1033216 ]
• 5: I. J. Cox, M. L. Miller, T. P. Minka, T. V. Papathomas, and P. N. Yianilos, "The Bayesian Image Retrieval System, PicHunter: Theory, Implementation, and Psychophysical Experiments," IEEE Trans. Image Processing, vol. 9, no. 1, pp. 20--37, 2000.
• 6: C. Faloutsos , R. Barber , M. Flickner , J. Hafner , W. Niblack , D. Petkovic , W. Equitz, Efficient and effective querying by image content, Journal of Intelligent Information Systems, v.3 n.3-4, p.231-262, July 1994 [doi>10.1007/BF00962238]
• 7: Gene H. Golub , Charles F. Van Loan, Matrix computations (3rd ed.), Johns Hopkins University Press, Baltimore, MD, 1996
• 8: Yoram Gdalyahu , Daphna Weinshall , Michael Werman, Self-Organization in Vision: Stochastic Clustering for Image Segmentation, Perceptual Grouping, and Image Database Organization, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.10, p.1053-1074, October 2001 [doi>10.1109/34.954598 ]
• 9: Amarnath Gupta , Ramesh Jain, Visual information retrieval, Communications of the ACM, v.40 n.5, p.70-79, May 1997 [doi>10.1145/253769.253798]
• 10: Marti A. Hearst , Jan O. Pedersen, Reexamining the cluster hypothesis: scatter/gather on retrieval results, Proceedings of the 19th annual international ACM SIGIR conference on Research and development in information retrieval, p.76-84, August 18-22, 1996, Zurich, Switzerland [doi>10.1145/243199.243216]
• 11: David W. Jacobs , Daphna Weinshall , Yoram Gdalyahu, Classification with Nonmetric Distances: Image Retrieval and Class Representation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.6, p.583-600, June 2000 [doi>10.1109/34.862197 ]
• 12: Jia Li , James Z. Wang, Automatic Linguistic Indexing of Pictures by a Statistical Modeling Approach, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.25 n.9, p.1075-1088, September 2003 [doi>10.1109/TPAMI.2003.1227984]
• 13: Jia Li , James Z. Wang , Gio Wiederhold, IRM: integrated region matching for image retrieval, Proceedings of the eighth ACM international conference on Multimedia, p.147-156, October 2000, Marina del Rey, California, United States [doi>10.1145/354384.354452]
• 14: W. Y. Ma , B. S. Manjunath, NeTra: a toolbox for navigating large image databases, Proceedings of the 1997 International Conference on Image Processing (ICIP '97) 3-Volume Set-Volume 1, p.568, October 26-29, 1997
• 15: Supporting Content-based Queries over Images in MARS, Proceedings of the 1997 International Conference on Multimedia Computing and Systems (ICMCS '97), p.632, June 03-06, 1997
• 16: A. Pentland , R. W. Picard , S. Sclaroff, Photobook: content-based manipulation of image databases, International Journal of Computer Vision, v.18 n.3, p.233-254, June 1996 [doi>10.1007/BF00123143]
• 17: Y. Rui, T. S. Huang, M. Ortega, and S. Mehrotra, "Relevance Feedback: A Power Tool for Interactive Content-Based Image Retrieval," IEEE Trans. Circuits and Video Technology, vol. 8, no. 5, pp. 644--655, 1998.
• 18: Gholamhosein Sheikholeslami , Wendy Chang , Aidong Zhang, SemQuery: Semantic Clustering and Querying on Heterogeneous Features for Visual Data, IEEE Transactions on Knowledge and Data Engineering, v.14 n.5, p.988-1002, September 2002 [doi>10.1109/TKDE.2002.1033769 ]
• 19: Jianbo Shi , Jitendra Malik, Normalized Cuts and Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.22 n.8, p.888-905, August 2000 [doi>10.1109/34.868688 ]
• 20: John R. Smith , Shih-Fu Chang, VisualSEEk: a fully automated content-based image query system, Proceedings of the fourth ACM international conference on Multimedia, p.87-98, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244151]
• 21: A. Vailaya, M. A. T. Figueiredo, A. K. Jain, and H.-J. Zhang, "Image Classification for Content-Based Indexing," IEEE Trans. Image Processing, vol. 10, no. 1, pp. 117--130, 2001.
• 22: James Z. Wang , Jia Li , Gio Wiederhold, SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.23 n.9, p.947-963, September 2001 [doi>10.1109/34.955109 ]
• 23: Song Chun Zhu , Alan Yuille, Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.18 n.9, p.884-900, September 1996 [doi>10.1109/34.537343 ]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval

General Terms:
Algorithms, Design, Experimentation, Human Factors

Keywords:
content-based image retrieval, image classification, spectral graph clustering, unsupervised learning