Multi-graph enabled active learning for multimodal web image retrieval

ABSTRACT

In this paper, we propose a multimodal Web image retrieval technique based on multi-graph enabled active learning. The main goal is to leverage the heterogeneous data on the Web to improve retrieval precision. Three graphs are constructed on images' content features, textual annotations and hyperlinks respectively, namely Content-Graph, Text-Graph and Link-Graph, which provide complimentary information on the images. By analyzing the three graphs, a training dataset is automatically created and transductive learning is enabled. The transductive learner is a multi-graph based classifier, which simultaneously solves the learning problem and the problem of combining heterogeneous data. This proposed approach, overall, tackles the problem of unsupervised active learning on Web graph. Although the proposed approach is discussed in the context of WWW image retrieval, it can be applied to other domains. The experimental results show the effectiveness of our approach.

REFERENCES

• 1: Barnard, K., Forsyth, D. Learning the Semantic of Words and Pictures. ICCV (2001).
• 2: Barnard K., Duygulu P., and Forsyth D. Clustering Art. Computer Vision and Pattern Recognition, pp. II:434--439, 2001.
• 3: Belkin, M., Niyogi, P., Sindhwani, V. On Manifold Regularization. AISTATS, (2005).
• 4: Deng Cai , Xiaofei He , Zhiwei Li , Wei-Ying Ma , Ji-Rong Wen, Hierarchical clustering of WWW image search results using visual, textual and link information, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027747]
• 5: Cai, D., Yu, S.P., Wen, J.-R., Ma, W.-Y. VIPS: a Vision-based Page Segmentation Algorithm. Microsoft Technical Report (MSR-TR-2003-79), (2003).
• 6: Zheng Chen , Liu Wenyin , Feng Zhang , Mingjing Li, Web mining for web image retrieval, Journal of the American Society for Information Science and Technology, v.52 n.10, p.831-839, August 2001 [doi>10.1002/asi.1132.abs]
• 7: Lise Getoor, Link mining: a new data mining challenge, ACM SIGKDD Explorations Newsletter, v.5 n.1, July 2003 [doi>10.1145/959242.959253]
• 8: Google. http://image.google.com (2005).
• 9: Jingrui He , Mingjing Li , Hong-Jiang Zhang , Hanghang Tong , Changshui Zhang, Manifold-ranking based image retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027531]
• 10: Xiaofei He , Shuicheng Yan , Yuxiao Hu , Partha Niyogi , Hong-Jiang Zhang, Face Recognition Using Laplacianfaces, IEEE Transactions on Pattern Analysis and Machine Intelligence, v.27 n.3, p.328-340, March 2005 [doi>10.1109/TPAMI.2005.55]
• 11: Jing Huang , S. Ravi Kumar , Mandar Mitra , Wei-Jing Zhu , Ramin Zabih, Image Indexing Using Color Correlograms, Proceedings of the 1997 Conference on Computer Vision and Pattern Recognition (CVPR '97), p.762, June 17-19, 1997
• 12: Thorsten Joachims, Making large-scale support vector machine learning practical, Advances in kernel methods: support vector learning, MIT Press, Cambridge, MA, 1999
• 13: Ravi Kumar , Prabhakar Raghavan , Sridhar Rajagopalan , D. Sivakumar , Andrew Tompkins , Eli Upfal, The Web as a graph, Proceedings of the nineteenth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, p.1-10, May 15-18, 2000, Dallas, Texas, United States [doi>10.1145/335168.335170]
• 14: Platt J.C., Smola A., Bartlett P., Scholkopf B., and Schuurmans D., Probabilistic Outputs for Support Vector Machines and Comparisons to Regularized Likelihood Methods, MIT Press, 1999, 61--74.
• 15: Rui, Y., Huang, T.S., Mehrotra, S. Content-Based Image Retrieval with Relevance Feedback in MARS. ICIP, (1997).
• 16: Gerard Salton , Christopher Buckley, Term-weighting approaches in automatic text retrieval, Information Processing and Management: an International Journal, v.24 n.5, p.513-523, 1988 [doi>10.1016/0306-4573(88)90021-0 ]
• 17: Gerard Salton , Michael J. McGill, Introduction to Modern Information Retrieval, McGraw-Hill, Inc., New York, NY, 1986
• 18: Simon Tong , Edward Chang, Support vector machine active learning for image retrieval, Proceedings of the ninth ACM international conference on Multimedia, September 30-October 05, 2001, Ottawa, Canada [doi>10.1145/500141.500159]
• 19: Vladimir N. Vapnik, The nature of statistical learning theory, Springer-Verlag New York, Inc., New York, NY, 1995
• 20: Xin-Jing Wang , Wei-Ying Ma , Gui-Rong Xue , Xing Li, Multi-model similarity propagation and its application for web image retrieval, Proceedings of the 12th annual ACM international conference on Multimedia, October 10-16, 2004, New York, NY, USA [doi>10.1145/1027527.1027746]
• 21: Thijs Westerveld, Probabilistic multimedia retrieval, Proceedings of the 25th annual international ACM SIGIR conference on Research and development in information retrieval, August 11-15, 2002, Tampere, Finland [doi>10.1145/564376.564482]
• 22: Yahoo!, http://images.search.yahoo.com (2005).
• 23: Zhou, D., Bousquet, O., Lal, T.N., Weston, J., and Scholkopf, B., Learning with Local and Global Consistency. NIPS 16, (2004).
• 24: Zhou, D., Schölkopf, B., and Hofmann, T. Semi-supervised Learning on Directed Graphs. NIPS 17 (2005).
• 25: Zhou, D., Weston, J., Gretton, A., Bousquet O., and Scholkopf, B. Ranking on Data Manifolds. NIPS 16, (2004).

INDEX TERMS

Primary Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Retrieval models

Additional Classification:
  H. Information Systems
  H.3 INFORMATION STORAGE AND RETRIEVAL
      H.3.3 Information Search and Retrieval
          Subjects: Search process

  I. Computing Methodologies
  I.5 PATTERN RECOGNITION
      I.5.1 Models
          Subjects: Statistical

General Terms:
Algorithms, Performance

Keywords:
active learning, graph learning, mjultimodal image retrieval