skip to main content
10.1145/1873951.1873977acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
research-article

Supervised reranking for web image search

Published: 25 October 2010 Publication History

Abstract

Visual search reranking that aims to improve the text-based image search with the help from visual content analysis has rapidly grown into a hot research topic. The interestingness of the topic stems mainly from the fact that the search reranking is an unsupervised process and therefore has the potential to scale better than its main alternative, namely the search based on offline-learned semantic concepts. However, the unsupervised nature of the reranking paradigm also makes it suffer from problems, the main of which can be identified as the difficulty to optimally determine the role of visual modality over different application scenarios. Inspired by the success of the "learning-to-rank" idea proposed in the field of information retrieval, we propose in this paper the "learning-to-rerank" paradigm, which derives the reranking function in a supervised fashion from the human-labeled training data. Although supervised learning is introduced, our approach does not suffer from scalability issues since a unified reranking model is learned that can be applied to all queries. In other words, a query-independent reranking model will be learned for all queries using query-dependent reranking features. The query-dependent reranking feature extraction is challenging since the textual query and the visual documents have different representation. In this paper, 11 lightweight reranking features are proposed by representing the textual query using visual context and pseudo relevant images from the initial search result. The experiments performed on two representative Web image datasets demonstrate that the proposed learning-to-rerank algorithm outperforms the state-of-the-art unsupervised reranking methods, which makes the learning-to-rerank paradigm a promising alternative for robust and reliable Web-scale image search.

References

[1]
http://images.bing.com/.
[2]
http://images.google.com/.
[3]
http://images.yahoo.com/.
[4]
R. Baeza-Yates and B. Ribeiro-Neto. Modern Information Retrieval. Addison Wesley, May 1999.
[5]
Z. Cao and T.-Y. Liu. Learning to rank: From pairwise approach to listwise approach. In ICML, 2007.
[6]
S.-F. Chang, J. He, Y.-G. Jiang, E. El Khoury, C.-W. Ngo, A. Yanagawa, and E. Zavesky. Columbia University/VIREO-CityU/IRIT TRECVID2008 High-Level Feature Extraction and Interactive Video Search. In NIST TRECVID Workshop, Gaithersburg, MD, November 2008.
[7]
O. Chapelle, Q. V. Le, and A. J. Smola. Large margin optimization of ranking measures. In NIPS Workshop: Machine Learning for Web Search, 2007.
[8]
J. Cui, F. Wen, and X. Tang. Real time google and live image search re-ranking. In ACM Multimedia, 2008.
[9]
A. Hauptmann, R. Yan, and W.-H. Lin. How many high-level concepts will fill the semantic gap in news video retrieval? In CIVR, 2007.
[10]
W. H. Hsu, L. S. Kennedy, and S.-F. Chang. Video search reranking via information bottleneck principle. In ACM Multimedia, 2006.
[11]
W. H. Hsu, L. S. Kennedy, and S.-F. Chang. Video search reranking through random walk over document-level context graph. In ACM Multimedia, 2007.
[12]
K. J¨arvelin and J. Kek¨al¨ainen. Ir evaluation methods for retrieving highly relevant documents. In SIGIR, 2000.
[13]
Y. Jing and S. Baluja. Visualrank: Applying pagerank to large-scale image search. IEEE Trans. on PAMI, 30(11):1877--1890, 2008.
[14]
T. Joachims. Optimizing search engines using clickthrough data. In KDD, 2002.
[15]
T. Joachims. Training linear svms in linear time. In KDD, 2006.
[16]
M. G. Kendall. A new measure of rank correlation. Biometrika, 30, 1938.
[17]
F. Korn and S. Muthukrishnan. Influence sets based on reverse nearest neighbor queries. In SIGMOD, 2000.
[18]
Y. Liu, T. Mei, X.-S. Hua, J. Tang, X. Wu, and S. Li. Learning to video search rerank via pseudo preference feedback. In ICME, 2008.
[19]
A. Natsev, A. Haubold, J. Tesic, L. Xie, and R. Yan. Semantic concept-based query expansion and re-ranking for multimedia retrieval. In ACM Multimedia, 2007.
[20]
J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman. Object retrieval with large vocabularies and fast spatial matching. In CVPR, 2007.
[21]
J. C. Platt. Fast training of support vector machines using sequential minimal optimization. In Advances in kernel methods: support vector learning. MIT Press, 1999.
[22]
S. E. Robertson and D. A. Hull. The trec-9 filtering track final report. In TREC, 2000.
[23]
F. Schroff, A. Criminisi, and A. Zisserman. Harvesting Image Databases from the Web. In ICCV, 2007.
[24]
C. Snoek, M. Worring, J. van Gemert, J.-M. Geusebroek, and A. W. M. Smeulders. The challenge problem for automated detection of 101 semantic concepts in multimedia. In ACM Multimedia, 2006.
[25]
C. G. M. Snoek and M. Worring. Concept-based video retrieval. Foundations and Trends in Information Retrieval, 4(2):215--322, 2009.
[26]
X. Tian, L. Yang, J. Wang, Y. Yang, X. Wu, and X.-S. Hua. Bayesian video search reranking. In ACM Multimedia, 2008.
[27]
X. Tian, L. Yang, X. Wu, and X.-S. Hua. Visual reranking with local learning consistency. In MMM, 2010.
[28]
R. H. van Leuken, L. Garcia, X. Olivares, and R. van Zwol. Visual diversification of image search results. In WWW, 2009.
[29]
M. Wang, L. Yang, and X.-S. Hua. Msra-mm: Bridging research and industrial societies for multimedia information retrieval. Microsoft Technical Report, 2009.
[30]
R. Yan, A. G. Hauptmann, and R. Jin. Multimedia search with pseudo-relevance feedback. In CIVR, 2003.
[31]
Y.-H. Yang and W. Hsu. Video search reranking via online ordinal reranking. In ICME, 2008.
[32]
Y. Yue and T. Joachims. Predicting diverse structural svms. In ICML, 2008.

Cited By

View all
  • (2024)Cross-Modal Remote Sensing Image–Audio Retrieval With Adaptive Learning for Aligning CorrelationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.340785762(1-13)Online publication date: 2024
  • (2024)Dual Modality Reverse Reranking (DM-RR) Based Image Retrieval FrameworkIEEE Open Journal of the Industrial Electronics Society10.1109/OJIES.2024.34359565(886-897)Online publication date: 2024
  • (2021)Benchmarking Image Retrieval Diversification Techniques for Social MediaIEEE Transactions on Multimedia10.1109/TMM.2020.298657923(677-691)Online publication date: 2021
  • Show More Cited By

Index Terms

  1. Supervised reranking for web image search

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '10: Proceedings of the 18th ACM international conference on Multimedia
    October 2010
    1836 pages
    ISBN:9781605589336
    DOI:10.1145/1873951
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 October 2010

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. learning to rerank
    2. supervised reranking
    3. visual search reranking

    Qualifiers

    • Research-article

    Conference

    MM '10
    Sponsor:
    MM '10: ACM Multimedia Conference
    October 25 - 29, 2010
    Firenze, Italy

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)10
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 18 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Cross-Modal Remote Sensing Image–Audio Retrieval With Adaptive Learning for Aligning CorrelationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.340785762(1-13)Online publication date: 2024
    • (2024)Dual Modality Reverse Reranking (DM-RR) Based Image Retrieval FrameworkIEEE Open Journal of the Industrial Electronics Society10.1109/OJIES.2024.34359565(886-897)Online publication date: 2024
    • (2021)Benchmarking Image Retrieval Diversification Techniques for Social MediaIEEE Transactions on Multimedia10.1109/TMM.2020.298657923(677-691)Online publication date: 2021
    • (2019)Matching Images and Text with Multi-modal Tensor Fusion and Re-rankingProceedings of the 27th ACM International Conference on Multimedia10.1145/3343031.3350875(12-20)Online publication date: 15-Oct-2019
    • (2018)Graph-Based Video Search Reranking with Local and Global Consistency AnalysisIEICE Transactions on Information and Systems10.1587/transinf.2017EDP7277E101.D:5(1430-1440)Online publication date: 1-May-2018
    • (2018)PSIJournal of the Association for Information Science and Technology10.1002/asi.2406869:12(1488-1501)Online publication date: 5-Dec-2018
    • (2018)MF‐Re‐Rank: A modality feature‐based Re‐Ranking model for medical image retrievalJournal of the Association for Information Science and Technology10.1002/asi.2404569:9(1095-1108)Online publication date: 7-May-2018
    • (2017)Set2Model Networks: Learning Discriminatively To Learn Generative Models2017 IEEE International Conference on Computer Vision Workshops (ICCVW)10.1109/ICCVW.2017.50(357-366)Online publication date: Oct-2017
    • (2017)Temporal localization of audio events for conflict monitoring in social media2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)10.1109/ICASSP.2017.7952426(1597-1601)Online publication date: Mar-2017
    • (2017)Set2Model networks: Learning discriminatively to learn generative modelsComputer Vision and Image Understanding10.1016/j.cviu.2017.08.001Online publication date: Aug-2017
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media