skip to main content
10.1145/1180639.1180654acmconferencesArticle/Chapter ViewAbstractPublication PagesmmConference Proceedingsconference-collections
Article

Video search reranking via information bottleneck principle

Published: 23 October 2006 Publication History

Abstract

We propose a novel and generic video/image reranking algorithm, IB reranking, which reorders results from text-only searches by discovering the salient visual patterns of relevant and irrelevant shots from the approximate relevance provided by text results. The IB reranking method, based on a rigorous Information Bottleneck (IB) principle, finds the optimal clustering of images that preserves the maximal mutual information between the search relevance and the high-dimensional low-level visual features of the images in the text search results. Evaluating the approach on the TRECVID 2003-2005 data sets shows significant improvement upon the text search baseline, with relative increases in average performance of up to 23%. The method requires no image search examples from the user, but is competitive with other state-of-the-art example-based approaches. The method is also highly generic and performs comparably with sophisticated models which are highly tuned for specific classes of queries, such as named-persons. Our experimental analysis has also confirmed the proposed reranking method works well when there exist sufficient recurrent visual patterns in the search results, as often the case in multi-source news videos.

References

[1]
A. Amir et al. IBM Research TRECVID-2005 video retrieval system. In TRECVID Workshop, Waishington DC, 2005.
[2]
Alias-i. Lingpipe named entity tagger. In http://www.alias-i.com/lingpipe/.
[3]
T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, New York, 1991.
[4]
K. M. Donald and A. F. Smeaton. A comparison of score, rank and probability-based fusion methods for video shot retrieval. In CIVR, Singapore, 2005.
[5]
R. Fergus, P. Perona, and A. Zisserman. A visual category filter for google images. In CVPR, May 2004.
[6]
S. Gordon, H. Greenspan, and J. Goldberger. Applying the information bottleneck principle to unsupervised clustering of discrete and continuous image representations. In ICCV, 2003.
[7]
A. G. Hauptmann and M. G. Christel. Successful approaches in the trec video retrieval evaluations. In ACM Multimedia 2004, New York, 2004.
[8]
W. H. Hsu and S.-F. Chang. Visual cue cluster construction via information bottleneck principle and kernel density estimation. In CIVR, Singapore, 2005.
[9]
W. H. Hsu and S.-F. Chang. Topic tracking across broadcast news videos with visual duplicates and semantic concepts. In ICIP, Atlanta, GA, USA, 2006.
[10]
J. G. Carbonell et al. Translingual information retrieval: A comparative evaluation. In International Joint Conference on Artificial Intelligence, 1997.
[11]
L. Kennedy, P. Natsev, and S.-F. Chang. Automatic discovery of query class dependent models for multimodal search. In ACM Multimedia, Singapore, November 2005.
[12]
H. Liu. Montylingua: An end-to-end natural language processor with common sense. In http://web.media.mit.edu/$\sim$hugo/montylingua.
[13]
X. Liu and W. B. Croft. Cluster-based retrieval using language models. In SIGIR, Sheffield, South Yorkshire, UK, 2004.
[14]
N. Slonim et al. Unsupervised document classification using sequential information maximization. In 25th ACM int. Conf. on Research and Development of Information Retireval, 2002.
[15]
A. Natsev, M. R. Naphade, and J. Tesic. Learning the semantics of multimedia queries and concepts from a small number of examples. In ACM Multimedia, pages 598--607, Singapore, 2005.
[16]
S. E. Robertson et al. Okapi at TREC4. In Text REtrieval Conference, pages 21--30, 1992.
[17]
S.-F. Chang et al. Columbia University TRECVID 2005 video search and high-level feature extraction. In TRECVID Workshop, Waishington DC, 2005.
[18]
D. W. Scott. Multivariate Density Estimation: Theory, Practice, and Visualization. Wiley-Interscience, 1992.
[19]
N. Slonim and N. Tishby. Agglomerative information bottleneck. In NIPS, 1999.
[20]
T.-C. Chang et al. TRECVID 2004 search and feature extraction task by NUS PRIS. In TRECVID Workshop, Waishington DC, 2004.
[21]
TRECVID. TREC Video Retrieval Evaluation. In http://www-nlpir.nist.gov/projects/trecvid/.
[22]
R. Yan, A. Hauptmann, and R. Jin. Multimedia search with pseudo-relevance feedback. In CIVR, Urbana-Champaign, IL, 2003.

Cited By

View all
  • (2024)A Survey on Information BottleneckIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.336634946:8(5325-5344)Online publication date: Aug-2024
  • (2024)Cross-Modal Remote Sensing Image–Audio Retrieval With Adaptive Learning for Aligning CorrelationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.340785762(1-13)Online publication date: 2024
  • (2024)Dual Modality Reverse Reranking (DM-RR) Based Image Retrieval FrameworkIEEE Open Journal of the Industrial Electronics Society10.1109/OJIES.2024.34359565(886-897)Online publication date: 2024
  • Show More Cited By

Index Terms

  1. Video search reranking via information bottleneck principle

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    MM '06: Proceedings of the 14th ACM international conference on Multimedia
    October 2006
    1072 pages
    ISBN:1595934472
    DOI:10.1145/1180639
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 October 2006

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. information bottleneck principle
    2. multimodal fusion
    3. video search

    Qualifiers

    • Article

    Conference

    MM06
    MM06: The 14th ACM International Conference on Multimedia 2006
    October 23 - 27, 2006
    CA, Santa Barbara, USA

    Acceptance Rates

    Overall Acceptance Rate 2,145 of 8,556 submissions, 25%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)10
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)A Survey on Information BottleneckIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.336634946:8(5325-5344)Online publication date: Aug-2024
    • (2024)Cross-Modal Remote Sensing Image–Audio Retrieval With Adaptive Learning for Aligning CorrelationIEEE Transactions on Geoscience and Remote Sensing10.1109/TGRS.2024.340785762(1-13)Online publication date: 2024
    • (2024)Dual Modality Reverse Reranking (DM-RR) Based Image Retrieval FrameworkIEEE Open Journal of the Industrial Electronics Society10.1109/OJIES.2024.34359565(886-897)Online publication date: 2024
    • (2023)Efficient algorithms for quantum information bottleneckQuantum10.22331/q-2023-03-02-9367(936)Online publication date: 2-Mar-2023
    • (2021)Information Bottleneck Theory on Convolutional Neural NetworksNeural Processing Letters10.1007/s11063-021-10445-6Online publication date: 18-Feb-2021
    • (2020)Heterogeneous-Graph-Based Video Search Reranking Using Topic RelevanceIEICE Transactions on Fundamentals of Electronics, Communications and Computer Sciences10.1587/transfun.2020SMP0023E103.A:12(1529-1540)Online publication date: 1-Dec-2020
    • (2020)Improving Supervised Phase Identification Through the Theory of Information LossesIEEE Transactions on Smart Grid10.1109/TSG.2019.295208011:3(2337-2346)Online publication date: May-2020
    • (2020)An Alphabet-Size Bound for the Information Bottleneck Function2020 IEEE International Symposium on Information Theory (ISIT)10.1109/ISIT44484.2020.9174416(2383-2388)Online publication date: Jun-2020
    • (2020)Enhancing Image Retrieval and Re-ranking Efficiency using Hybrid approach2020 International Conference on Smart Innovations in Design, Environment, Management, Planning and Computing (ICSIDEMPC)10.1109/ICSIDEMPC49020.2020.9299579(20-26)Online publication date: 30-Oct-2020
    • (2020)Hypergraph learning with collaborative representation for image search rerankingInternational Journal of Multimedia Information Retrieval10.1007/s13735-019-00191-w9:3(205-214)Online publication date: 22-Jan-2020
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media