Learning an image manifold for retrieval

ABSTRACT

We consider the problem of learning a mapping function from low-level feature space to high-level semantic space. Under the assumption that the data lie on a submanifold embedded in a high dimensional Euclidean space, we propose a relevance feedback scheme which is naturally conducted only on the image manifold in question rather than the total ambient space. While images are typically represented by feature vectors in Rn, the natural distance is often different from the distance induced by the ambient space Rn. The geodesic distances on manifold are used to measure the similarities between images. However, when the number of data points is small, it is hard to discover the intrinsic manifold structure. Based on user interactions in a relevance feedback driven query-by-example system, the intrinsic similarities between images can be accurately estimated. We then develop an algorithmic framework to approximate the optimal mapping function by a Radial Basis Function (RBF) neural network. The semantics of a new image can be inferred by the RBF neural network. Experimental results show that our approach is effective in improving the performance of content-based image retrieval systems.

REFERENCES

• 1: M. Belkin and P. Niyogi, "Laplacian eigenmaps for dimensionality reduction and data representation", Advances in Neural Information Processing Systesms, 2001.
• 2: B. Bernstein, V. de Silva, J. C. Langford, and J. B. Tenenbaum, "Graph approximations to geodesics on embedded manifolds", Technical report, Stanford University, December 2000
• 3: E. Chang, K. Goh, G. Sychay, and G. Wu, "CBSA: Content-Based Soft Annotation for Multimodal Image Retrieval Using Bayes Point Machine". IEEE Trans. on Circuits and Systems for Video Technology, vol. 13, No. 1, Jan. 2003.
• 4: X. He and P. Niyogi, "Locality Preserving Projections", in Advances in Neural Information Processing Systems 16, Vancouver, Canada, 2003.
• 5: Xiaofei He , Deng Cai , Haifeng Liu , Wei-Ying Ma, Locality preserving indexing for document representation, Proceedings of the 27th annual international ACM SIGIR conference on Research and development in information retrieval, July 25-29, 2004, Sheffield, United Kingdom [doi>10.1145/1008992.1009012]
• 6: X. He, O. King, W.-Y. Ma, M.-J. Li, and H.-J. Zhang, "Learning a semantic space from user's relevance feedback for image retrieval", IEEE Trans. on Circuit and System for Video Technology, Jan, 2003.
• 7: Xiaofei He , Shuicheng Yan , Yuxiao Hu , Hong-Jiang Zhang, Learning a Locality Preserving Subspace for Visual Recognition, Proceedings of the Ninth IEEE International Conference on Computer Vision, p.385, October 13-16, 2003
• 8: Yoshiharu Ishikawa , Ravishankar Subramanya , Christos Faloutsos, MindReader: Querying Databases Through Multiple Examples, Proceedings of the 24rd International Conference on Very Large Data Bases, p.218-227, August 24-27, 1998
• 9: Wei-Ying Ma , B. S. Manjunath, NeTra: a toolbox for navigating large image databases, Multimedia Systems, v.7 n.3, p.184-198, May 1999 [doi>10.1007/s005300050121 ]
• 10: W. Matusik, H. Pfister. M. Brand, and L. McMillan, "A data-driven reflectance model", in Proc. of SIGGRAPH, 2003.
• 11: S.T. Roweis, and L.K. Saul, "Nonlinear dimensionality reduction by locally linear embedding", Science, vol 290, 22 December 2000.
• 12: Y. Rui and T. S. Huang, "Optimizing learning in image retrieval", in Proc. of IEEE Conf. on Computer Vision and Pattern Recognition, Hilton Head, SC, June 2000.
• 13: H.S. Seung and D. Lee, "The manifold ways of perception", Science, vol 290, 22 December 2000.
• 14: John R. Smith , Shih-Fu Chang, VisualSEEk: a fully automated content-based image query system, Proceedings of the fourth ACM international conference on Multimedia, p.87-98, November 18-22, 1996, Boston, Massachusetts, United States [doi>10.1145/244130.244151]
• 15: J.B. Tenenbaum, V.D. Silva, and J.C. Langford, "A global geometric framework for nonlinear dimensionality reduction", Science, Vol 290, 22 December 2000.
• 16: K. Tieu and P. Viola, "Boosting image retrieval", in Proc. IEEE Conf. on Computer Vision and Pattern Recognitino, Hilton head, SC, June 2000.
• 17: Simon Tong , Edward Chang, Support vector machine active learning for image retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500159]
• 18: N. Vasconcelos and A. Lippman, "Learning from user feedback in image retrieval systems", Advances in Neural Information Processing Systems, Denver, Colorado, 1999.
• 19: James Z. Wang , Jia Li, Learning-based linguistic indexing of pictures with 2--d MHMMs, Proceedings of the tenth ACM international conference on Multimedia, December 01-06, 2002, Juan-les-Pins, France [doi>10.1145/641007.641104]
• 20: Xiang Sean Zhou , Thomas S. Huang, Comparing discriminating transformations and SVM for learning during multimedia retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500163]
• 21: Theory of keyblock-based image retrieval, ACM Transactions on Information Systems (TOIS), v.20 n.2, p.224-257, April 2002 [doi>10.1145/506309.506313]

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.1 Content Analysis and Indexing
          Subjects: Indexing methods

General Terms:
Algorithms, Experimentation, Management, Performance

Keywords:
dimensionality reduction, image retrieval, manifold learning, riemannian structure, semantic space