skip to main content
10.1145/1571941.1572059acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
poster

Comparing both relevance and robustness in selection of web ranking functions

Published: 19 July 2009 Publication History

Abstract

In commercial search engines, a ranking function is selected for deployment mainly by comparing the relevance measurements over candidates. In this paper we suggest to select Web ranking functions according to both their relevance and robustness to the changes that may lead to relevance degradation over time. We argue that the ranking robustness can be effectively measured by taking into account the ranking score distribution across Web pages. We then improve NDCG with two new metrics and show their superiority in terms of stability to ranking score turbulence and stability in function selection.

References

[1]
R. Bhattacharjee and A. Goel. Algorithms and incentives for robust ranking. In SODA, 2007.
[2]
K. Jarvelin and J. Kekalainen. Cumulated gain-based evaluation of ir techniques. ACM Trans. Inf. Syst.,20(4):422--446, 2002.
[3]
M. Taylor, J. Guiver, S. Robertson, and T. Minka. Softrank: optimizing non-smooth rank metrics. In WSDM 2008, pages 77--86. ACM, 2008.
[4]
Z. Zheng, K. Chen, G. Sun, and H. Zha. A regression framework for learning ranking functions using relative relevance judgments. In SIGIR, pages 287--294, 2007.
[5]
Y. Zhou and W. B. Croft. Ranking robustness: a novel framework to predict query performance. In CIKM 2006.

Cited By

View all
  • (2012)Robust ranking models via risk-sensitive optimizationProceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval10.1145/2348283.2348385(761-770)Online publication date: 12-Aug-2012

Index Terms

  1. Comparing both relevance and robustness in selection of web ranking functions

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '09: Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval
    July 2009
    896 pages
    ISBN:9781605584836
    DOI:10.1145/1571941

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 19 July 2009

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. NDCG
    2. ranking robustness
    3. web ranking

    Qualifiers

    • Poster

    Conference

    SIGIR '09
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)0
    • Downloads (Last 6 weeks)0
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2012)Robust ranking models via risk-sensitive optimizationProceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval10.1145/2348283.2348385(761-770)Online publication date: 12-Aug-2012

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media