Article

Query performance prediction in web search environments

Authors:

W. Bruce CroftAuthors Info & Claims

SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval

Pages 543 - 550

https://doi.org/10.1145/1277741.1277835

Published: 23 July 2007 Publication History

Abstract

Current prediction techniques, which are generally designed for content-based queries and are typically evaluated on relatively homogenous test collections of small sizes, face serious challenges in web search environments where collections are significantly more heterogeneous and different types of retrieval tasks exist. In this paper, we present three techniques to address these challenges. We focus on performance prediction for two types of queries in web search environments: content-based and Named-Page finding. Our evaluation is mainly performed on the GOV2 collection. In addition to evaluating our models for the two types of queries separately, we consider a more challenging and realistic situation that the two types of queries are mixed together without prior information on query types. To assist prediction under the mixed-query situation, a novel query classifier is adopted. Results show that our prediction of web query performance is substantially more accurate than the current state-of-the-art prediction techniques. Consequently, our paper provides a practical approach to performance prediction in real-world web settings.

References

[1]

Y. Zhou, W.B. Croft, Ranking Robustness: A Novel Framework to Predict Query Performance, in Proceedings of CIKM 2006.

Digital Library

[2]

D. Carmel, E. Yom-Tov, A. Darlow, D. Pelleg, What Makes a Query Difficult?, in Proceedings of SIGIR 2006.

Digital Library

[3]

C.L.A. Clarke, F. Scholer, I. Soboroff, The TREC 2005 Terabyte Track, In the Online Proceedings of 2005 TREC.

[4]

B. He and I. Ounis. Inferring query performance using pre-retrieval predictors. In proceedings of the SPIRE 2004.

[5]

S. Tomlinson. Robust, Web and Terabyte Retrieval with Hummingbird SearchServer at TREC 2004. In the Online Proceedings of 2004 TREC.

[6]

S. Cronen-Townsend, Y. Zhou, W.B. Croft, Predicting Query Performance, in Proceedings of SIGIR 2002.

Digital Library

[7]

V. Vinay, I. J.Cox, N. Mill-Frayling, K. Wood, On Ranking the Effectiveness of Searcher, in Proceedings of SIGIR 2006.

Digital Library

[8]

D. Metzler, W.B. Croft, A Markov Random Filed Model for Term Dependencies, in Proceedings of SIGIR 2005.

Digital Library

[9]

D. Metzler, T. Strohman, Y. Zhou, W.B. Croft, Indri at TREC 2005: Terabyte Track, In the Online Proceedings of 2004 TREC.

[10]

P. Ogilvie and J. Callan, Combining document representations for known-item search, in Proceedings of SIGIR 2003.

Digital Library

[11]

A. Berger, J. Lafferty, Information retrieval as statistical translation, in Proceedings of SIGIR 1999.

Digital Library

[12]

Indri search engine: http://www.lemurproject.org/indri/

[13]

I.J. Taneja: On Generalized Information Measures and Their Applications, Advances in Electronics and Electron Physics, Academic Press (USA), 76, 1989, 327--413.

[14]

S. Cronen-Townsend, Y. Zhou and Croft, W.B., "A Framework for Selective Query Expansion," in Proceedings of CIKM 2004.

Digital Library

[15]

F. Song, W.B. Croft, A general language model for information retrieval, in Proceedings of SIGIR 1999.

Digital Library

[16]

Personal email contact with Vishwa Vinay and our own experiments.

[17]

E. Yom-Tov, S. Fine, D. Carmel, A. Darlow, Learning to Estimate Query Difficulty Including Applications to Missing Content Detection and Distributed Information retrieval, in Proceedings of SIGIR 2005.

Digital Library

Cited By

Vlachou MMacdonald COosterhuis HBast HXiong C(2024)Coherence-based Query Performance Measures for Dense RetrievalProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672518(15-24)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672518
Khramtsova EZhuang SBaktashmotlagh MZuccon GHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Leveraging LLMs for Unsupervised Dense Retriever RankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657798(1307-1317)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657798
Salemi AKallumadi SZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Optimization Methods for Personalizing Large Language Models through Retrieval AugmentationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657783(752-762)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657783
Show More Cited By

Index Terms

Query performance prediction in web search environments
1. Information systems
  1. Information retrieval
    1. Information retrieval query processing

Recommendations

Query performance prediction

The prediction of query performance is an interesting and important issue in Information Retrieval (IR). Current predictors involve the use of relevance scores, which are time-consuming to compute. Therefore, current predictors are not very suitable for ...
Query Performance Prediction using Passage Information
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval

We focus on the post-retrieval query performance prediction (QPP) task. Specifically, we make a new use of passage information for this task. Using such information we derive a new mean score calibration predictor that provides a more accurate ...
Using an Inverted Index Synopsis for Query Latency and Performance Prediction

Predicting the query latency by a search engine has important benefits, for instance, in allowing the search engine to adjust its configuration to address long-running queries without unnecessarily sacrificing its effectiveness. However, for the dynamic ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '07: Proceedings of the 30th annual international ACM SIGIR conference on Research and development in information retrieval

July 2007

946 pages

ISBN:9781595935977

DOI:10.1145/1277741

General Chairs:
Wessel Kraaij
TNO, The Netherlands
,
Arjen P. de Vries
CWI, The Netherlands
,
Program Chairs:
Charles L. A. Clarke
University of Waterloo, Canada
,
Norbert Fuhr
University of Duisburg-Essen, Germany
,
Noriko Kando
National Institute of Informatics, Japan

Copyright © 2007 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 23 July 2007

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

SIGIR07

Sponsor:

SIGIR07: The 30th Annual International SIGIR Conference

July 23 - 27, 2007

Amsterdam, The Netherlands

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

181
Total Citations
View Citations
1,338
Total Downloads

Downloads (Last 12 months)63
Downloads (Last 6 weeks)6

Reflects downloads up to 09 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Vlachou MMacdonald COosterhuis HBast HXiong C(2024)Coherence-based Query Performance Measures for Dense RetrievalProceedings of the 2024 ACM SIGIR International Conference on Theory of Information Retrieval10.1145/3664190.3672518(15-24)Online publication date: 2-Aug-2024
https://dl.acm.org/doi/10.1145/3664190.3672518
Khramtsova EZhuang SBaktashmotlagh MZuccon GHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Leveraging LLMs for Unsupervised Dense Retriever RankingProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657798(1307-1317)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657798
Salemi AKallumadi SZamani HHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)Optimization Methods for Personalizing Large Language Models through Retrieval AugmentationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657783(752-762)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657783
Arabzadeh NMeng CAliannejadi MBagheri E(2024)Query Performance Prediction: From Fundamentals to Advanced TechniquesAdvances in Information Retrieval10.1007/978-3-031-56069-9_51(381-388)Online publication date: 23-Mar-2024
https://doi.org/10.1007/978-3-031-56069-9_51
Ebrahimi SKhodabakhsh MArabzadeh NBagheri E(2024)Estimating Query Performance Through Rich Contextualized Query RepresentationsAdvances in Information Retrieval10.1007/978-3-031-56066-8_6(49-58)Online publication date: 15-Mar-2024
https://doi.org/10.1007/978-3-031-56066-8_6
Saleminezhad AArabzadeh NBeheshti SBagheri E(2024)Context-Aware Query Term Difficulty Estimation for Performance PredictionAdvances in Information Retrieval10.1007/978-3-031-56066-8_4(30-39)Online publication date: 15-Mar-2024
https://doi.org/10.1007/978-3-031-56066-8_4
Chifu ADéjean SGarouani MMothe JOrtiz DUllah M(2024)Can We Predict QPP? An Approach Based on Multivariate OutliersAdvances in Information Retrieval10.1007/978-3-031-56063-7_38(458-467)Online publication date: 23-Mar-2024
https://doi.org/10.1007/978-3-031-56063-7_38
Datta SGanguly DMacAvaney SGreene D(2024)A Deep Learning Approach for Selective Relevance FeedbackAdvances in Information Retrieval10.1007/978-3-031-56060-6_13(189-204)Online publication date: 16-Mar-2024
https://doi.org/10.1007/978-3-031-56060-6_13
Chatterjee SMackie IDalton J(2024)DREQ: Document Re-ranking Using Entity-Based Query UnderstandingAdvances in Information Retrieval10.1007/978-3-031-56027-9_13(210-229)Online publication date: 24-Mar-2024
https://dl.acm.org/doi/10.1007/978-3-031-56027-9_13
Chifu AFournier S(2023)Sentiment Difficulty in Aspect-Based Sentiment AnalysisMathematics10.3390/math1122464711:22(4647)Online publication date: 14-Nov-2023
https://doi.org/10.3390/math11224647
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents